Server-based fast remote display on client devices

• US 10,062,141 B2
• Filed: 12/23/2016
• Issued: 08/28/2018
• Est. Priority Date: 10/24/2012
• Status: Active Grant

• Current Assignee: NUBO Software
• Original Assignee: NUBO Software
• Inventors: Lifshitz, Israel, Munitz, Ron, Shoam, Meir
• Primary Examiner(s): Richer, Joni
• Application Number:
  US15/390,095
• Publication Number:
  US 20170109861A1
• Time in Patent Office: 613 Days
• Field of Search: None
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: H04L 1/0002 by adapting the transmission rate
• CPC Class Codes: H04L 47/25 with rate being modified by the source upon detecting a change of network conditions
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

A system and method for controlling a display on a client device such as a wireless communication device, e.g., a smartphone or similar device is presented. Aspects of the invention permit a more efficient and fast representation of the desired output onto the client'"'"'s display including by caching certain display data on the client, temporarily and/or permanently, and transmitting only command and control level data from the server to the client for display of the cached display data. Additional aspects of the invention reduce the volume of data sent between the client device and the server without affecting the experience of the client device user. Additional aspects of the invention provide for secure acquisition and display of data on the client device.

Claims

1. A method for controlling a display of a client device, comprising:

in a server coupled to said client device by a communication link;

determining a desired display state of said display including determining display data needed to achieve said desired display state;

generating draw commands comprising instructions for configuring said display of the client device to achieve the desired display state;

placing the draw commands in a draw command queue to send to the client device over a network communication link;

monitoring a length of the draw command queue, the length corresponding to a number of the draw commands in the draw command queue;

determining if the length of the draw command queue exceeds a threshold value; and

reducing a rate of new draw commands generated when the length of the draw command queue exceeds the threshold value.

2. The method of claim 1, further comprising maintaining the number of new commands generated when the length of the draw command queue is lower than or equal to the threshold value.

3. The method of claim 1, further comprising:

determining if a rate of draw command acknowledgements received from said client device falls below a first threshold rate of draw command acknowledgements; and

reducing said rate of said new draw commands generated when the rate of draw command acknowledgements falls below the first threshold rate of draw command acknowledgements.

4. The method of claim 3, further comprising:

determining if the rate of draw command acknowledgements received from said client device exceeds a second threshold rate of draw command acknowledgements; and

increasing said rate of said new draw commands generated when the rate of draw command acknowledgements exceeds the second threshold rate of draw command acknowledgements.

5. The method of claim 3, further comprising:

determining that a second portion of said desired display state requires new draw commands that do not exist in said client device; and

transmitting said new draw commands to said client device, said new draw commands comprising instructions for configuring said display of the client device to achieve the second portion of said desired display state.

6. The method of claim 1, further comprising:

determining that a first portion of said desired display state corresponds to a user manipulation of an object in preexisting display data stored in a memory on said client device;

sending an abbreviated code to said client device, the abbreviated code representing a sequence of draw commands stored in said memory to achieve said first portion of said desired display state; and

sending a manipulation draw command to said client device, said manipulation draw command comprising instructions corresponding to a user manipulation of said object.

7. The method of claim 6, wherein said user manipulation is one of a zoom, a translation, a rotation, or a skew of said object.

8. A server system comprising:

a processor;

a communication link interface coupling said server to a connected client device;

a data storage unit including a location storing pre-stored display data sent by said server to said client device;

a data storage unit including a location storing draw commands;

an input event generator that handles input event signals from said client device;

a draw command generator that generates draw commands;

a stream comparator that determines whether said draw commands are required to be sent to said client device;

a choreographer that determines whether a length of a draw command queue of said draw commands to be sent to said client device exceeds a threshold value; and

a window compositor that manages at least one window'"'"'s behavior at said client device.

9. The system of claim 8, wherein said choreographer generates a command to said draw command generator to reduce a number of new draw commands generated by said draw command generator by said when said length of said draw command queue exceeds said threshold value.

10. The system of claim 8, wherein said reduced number of new draw commands corresponds to a display of fewer intermediate positions or orientations of an object displayed on said client device.

11. The system of claim 8, wherein the stream comparator sends an abbreviated code and a manipulation draw command if said input event signal corresponds to a user manipulation of a pre-stored image stored on said client device, said abbreviated code representing a sequence of draw commands stored in said client device to achieve said pre-stored image, said manipulation draw command corresponding to said user manipulation of pre-stored image.

12. The system of claim 11, wherein said user manipulation is one of a zoom, a translation, a rotation, or a skew of said object.

13. A method for controlling a display of a client device, the method comprising:

in said client device;

sensing a user input corresponding to a desired display state of said display;

generating user input events corresponding to said user input;

determining if a rate of said user input events exceeds a threshold rate;

reducing a number of said user input events when said rate of said user input events exceeds said threshold rate; and

sending said reduced number of said user input events to a server when said rate of said user input events exceeds said threshold rate.

14. The method of claim 13, further comprising reducing a sampling rate of said user input events.