Synchronized graphics and region data for graphics remoting systems

• US 20050091571A1
• Filed: 10/23/2003
• Published: 04/28/2005
• Est. Priority Date: 10/23/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Leichtling, Ivan
• Granted Patent: US 7,275,212 B2
• Time in Patent Office: Days
• US Class Current: 715/203
• CPC Class Codes: G06F 21/84 output devices, e.g. displays or monitors
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2358/00 Arrangements for display data security
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/70 Media network packetisation

Abstract

Region data and graphics data for a remoting system are synchronously gathered and sent to a remote display while maintaining synchronicity between the region data and the graphics data. In one implementation, synchronized region data and synchronized graphics data are gathered into one display driver. For the remote display, incoming region data precedes corresponding synchronized graphics data so that the graphics data is displayed according to synchronous region data.

Claims

1. A method, comprising:

synchronously gathering region data for displaying a region of a server desktop remotely on a client display, wherein the region data describe a shape and a position of the region;

synchronously gathering graphics data for the region, wherein the graphics data describe visual content of the region; and

sending the region data and the graphics data to a client while maintaining synchronicity between the region data and the graphics data.

2. The method as recited in claim 1, wherein the region data and the graphics data are gathered in a single display driver.

3. The method as recited in claim 2, wherein the region data and the graphics data are gathered and stored in a format of a remoting protocol.

4. The method as recited in claim 3, wherein the region data is synchronously gathered by a display driver-level window object created to contain the shape and position information.

5. The method as recited in claim 3, wherein the graphics data is synchronously gathered by the display driver.

6. The method as recited in claim 5, wherein the display driver synchronously gathers graphics data by gathering drawing commands issued to a graphics device interface subsystem of an operating system of the server.

7. The method as recited in claim 1, wherein the sending further includes forming a sequence of region data and graphics data, wherein the region data precedes the graphics data.

8. The method as recited in claim 7, further comprising sequencing the region data to precede the graphics data using rules of a remoting protocol.

9. The method as recited in claim 8, further comprising receiving the region data and the graphics data for display on a client and displaying the graphics data according to the preceding region data.

10. The method as recited in claim 1, wherein if a bandwidth for the sending becomes too low to send the region data and the graphics data, then reducing the amount of data to send by sending no region data and sending graphics data for the entire server desktop.

11. The method as recited in claim 1, wherein if a bandwidth for the sending becomes too low to send the region data and the graphics data, then reducing the amount of data to send by sending region data for a subset of the region and by sending graphics data for the subset.

12. The method as recited in claim 11, wherein the subset has a geometry that requires less region data to describe.

13. The method as recited in claim 1, wherein if a bandwidth for the sending becomes too low to send the region data and the graphics data, then reducing the amount of data to send by surrounding the region with a larger region that requires less data to describe and enlarging the visual content of the region to fit the larger region.

14. The method as recited in claim 1, further comprising:

receiving the region data and the graphics data; and

displaying the graphics data as graphics in a region of a client desktop described by the region data.

15. A remoting synchronization engine, comprising:

a region data gathering module to synchronously gather region data describing a region of a display desktop to be remotely displayed on a client, wherein the region data describe a shape and a desktop position of the region;

a graphics data gathering module to synchronously gather graphics data, wherein the graphics data describe a visual content of the region; and

a display driver to collect the synchronously gathered region data and the synchronously gathered graphics data and to send the region data and the graphics data to the client while maintaining synchronicity between the region data and the graphics data.

16. The remoting synchronization engine as recited in claim 15, further comprising a data output scheduler associated with the display driver to send the region data and the graphics data to the client in a sequence, wherein the region data precedes the graphics data synchronized with the region data.

17. The remoting synchronization engine as recited in claim 16, further comprising a bandwidth compensator to maintain security with respect to the synchronized region data and the synchronized graphics data during a condition of low bandwidth.

18. The remoting synchronization engine as recited in claim 15, further comprising a data gathering scheduler to schedule synchronous gathering of region data and graphics data synchronized to the region data.

19. A synchronized data receiver, comprising:

a region subsystem to receive region data synchronized with graphics data and to designate a region of a client display based on the region data; and

a graphics subsystem to receive the graphics data synchronized with the region data and to display graphics in the region based on the graphics data.

20. A synchronized remoting system, comprising:

a means for producing visual content to be remotely displayed on a client;

a means for designating a visual region of the visual content;

a means for gathering region data describing geometry of the visual region, wherein gathered region data is in synchronicity with graphics data describing the visual content in the visual region;

a means for gathering the graphics data describing the visual content in the visual region, wherein gathered graphics data is in synchronicity with the region data describing the geometry of the visual region; and

a means for sending the region data and the graphics data to the client, wherein region data in synchronicity with particular graphics data precedes the particular graphics data.

21. The synchronized remoting system as recited in claim 20, further comprising:

a means for receiving the region data and the graphics data at a client; and

a means for displaying the graphics data as graphics in a region of a client desktop described by the region data.

22. A data stream structure, comprising:

region data describing geometry of a visual region to be remotely displayed;

graphics data describing visual content of the visual region, wherein the region data precedes the graphics data in the data stream structure.

23. The data stream structure as recited in claim 22, wherein the region data and the graphics data were gathered in synchronicity with each other.

24. A method, comprising:

synchronously gathering region data and graphics data for a visual region of a computing server display to be remotely displayed on a client display;

if bandwidth is sufficient for sending the region data and the graphics data to the client, then sending the region data and the graphics data to the client, wherein a region datum in synchronicity with a graphics datum is sent before the graphics datum;

if bandwidth is not sufficient for sending the region data and the graphics data to the client, then

if the client owns an entirety of information displayable on the computing server display, then sending only graphics data describing the entire visual content of the computing server display;

but

if the client does not own an entirety of information displayable on the computing server display, then

if visual content of the visual region can be truncated, then selecting a smaller visual region inscribed in the visual region and sending synchronized region data and synchronized graphics data associated with the smaller visual region, but

if the visual content of the visual region cannot be truncated, then selecting a larger visual region circumscribing the visual region, sending synchronized region data and synchronized graphics data associated with the larger visual region, and resizing visual content of the visual region to fit the larger visual region.

25. One or more computing device readable media containing instructions that are executable by a computing device to perform actions comprising:

synchronously gathering region data for displaying a visual region of a server desktop remotely on a client display, wherein the region data describe a shape and a position of the region;

synchronously gathering graphics data for the visual region to obtain synchronized region data and synchronized graphics data, wherein the synchronized graphics data describe a visual content of the visual region; and

sending the synchronized region data and the synchronized graphics data to the client while maintaining synchronicity between the region data and the graphics data.

26. The one or more computing device readable media as recited in claim 25, wherein maintaining synchronicity further comprises preceding graphics data to be sent to the client with the region data synchronized to the graphics data.

27. The one or more computing device readable media as recited in claim 25, wherein the region data and the graphics data are synchronously gathered into one display driver.