Cloning or extending a computer desktop on a wireless display surface

• US 8,839,112 B2
• Filed: 08/10/2010
• Issued: 09/16/2014
• Est. Priority Date: 08/10/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Neerudu, Srinivasa R., Porter, Nelly, Dvorak, Casey, Hassan, Amer A.
• Primary Examiner(s): HO, RUAY L
• Application Number:
  US12/854,152
• Publication Number:
  US 20120042252A1
• Time in Patent Office: 1,498 Days
• Field of Search: 715200-277, 715700-867, 345 30-111, 370/252, 705/64, 725/74
• US Class Current: 715/733
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

Techniques are provided for cloning a computer desktop to a wireless display surface. A source computer and a display in communication with a destination computer establish a wireless connection. A user console session of the source computer has a virtual display driver that corresponds to the wireless display. An application of the console session instructs the virtual display driver to render graphics to a display surface. The source computer takes this graphical information that, when executed on a processor generate an image, text, sound or input—encodes it with a remote presentation protocol. Through encoding and decoding image data, text, sound, and input of the user console session transmitted to a wireless display with a remote presentation protocol, fidelity and interactivity are improved.

Claims

1. A method for displaying images on a wireless display with high fidelity and interactivity, comprising:

establishing, by a source computer, a wireless communication channel between a user console session of the source computer and a destination computer, the destination computer being configured to display screen data on the wireless display;

determining, by the source computer, first screen data of the user console session to be displayed on the wireless display;

determining a sound of the user console session to be played contemporaneously with displaying the first screen data;

encoding, by the source computer, the first screen data with a remote presentation protocol (RPP) to identify a first time stamp, and encoding the sound to identify a second time stamp, the first time stamp and the second time stamp indicating to play the sound contemporaneously with the wireless display displaying the decoded first screen data; and

sending, by the source computer, the encoded first screen data to the destination computer from the user console session, without the encoded first screen data being transmitted through a second user session, for the destination computer to decode the encoded first screen data and the sound and display the decoded first screen data on the wireless display while contemporaneously playing the sound.

2. The method of claim 1, further comprising:

encoding second screen data of the user console session with the remote presentation protocol;

sending the encoded second screen data to the destination computer from the user console session, without the encoded second screen data being transmitted through the second user session; and

sending a priority of the second screen data to the destination computer, for the destination computer to display decoded second screen data on the wireless display before displaying the decoded first screen data on the wireless display.

3. The method of claim 2, wherein sending the priority of the second screen data to the wireless display occurs in response to:

determining that the second screen data corresponds to input to the source computer.

4. The method of claim 3, wherein the input to the source computer comprises:

a change in cursor position or a change in a shape.

5. The method of claim 1, further comprising:

determining an indication of user input of the user console session to be delivered contemporaneously with displaying the decoded first screen data;

encoding the indication of user input with the RPP; and

sending the encoded indication of user input to the destination computer from the user console session, without the encoded user input being transmitted a second user session, for the destination computer to display the indication of user input contemporaneously with the wireless display displaying the decoded first screen data.

6. The method of claim 1, further comprising:

determining a maximum decoding capability of the destination computer; and

wherein encoding the first screen data with the remote presentation protocol comprises;

encoding the first screen data based on the maximum decoding capability.

7. The method of claim 1, further comprising:

determining an amount of available processing resources of the source computer not used by a user application; and

wherein encoding the first screen data with the remote presentation protocol comprises;

encoding the first screen in real-time based on the amount of available processing resources.

8. The method of claim 1, wherein encoding the first screen data with the RPP comprises:

encoding a first portion of the first screen data with a first codec; and

encoding a second portion of the first screen data with a second codec, the second codec differing from the first codec.

9. The method of claim 8, wherein the first portion comprises a representation of text, and the second portion comprises an image.

10. The method of claim 1, wherein the first screen data comprises:

a computer desktop that is mirrored between the user console session and the display, or a computer desktop that is extended from the user console session to the display.

11. A system, comprising:

a memory bearing instructions that, upon execution by a processor, cause the system at least to;

establish a wireless communication channel between a user console session of a source computer and a destination computer, the destination computer being configured to display screen data on a wireless display;

determine first screen data of the user console session to be displayed on the wireless display;

determine a sound of the user console session to be played contemporaneously with displaying the first screen data;

encode the first screen data to identify a first time stamp, and encoding the sound to identify a second time stamp, the first time stamp and the second time stamp indicating to play the sound contemporaneously with the display displaying the decoded first screen data; and

send the encoded first screen data to the destination computer from the user console session, without the encoded first screen data being transmitted a second user session.

12. The system of claim 11, wherein the memory further bears instructions that, when executed on the processor, cause the system at least to:

encode second screen data of the user console session with a remote presentation protocol;

send the encoded second screen data to the destination computer from the user console session, without the encoded second screen data being transmitted a second user session; and

send a priority of the second screen data to the destination computer.

13. The system of claim 11, wherein the memory further bears instructions that, when executed on the processor, cause the system at least to:

determine an indication of user input of the user console session to be delivered contemporaneously with displaying the decoded first screen data;

encode the indication of user input; and

send the encoded indication of user input to the destination computer from the user console session, without the encoded indication of user input being transmitted a second user session.

14. The system of claim 11, wherein the instructions that, when executed on the processor, cause the system at least to encode the first screen data further cause the system at least to:

determine a maximum decoding capability of the destination computer; and

encode the first screen data based on the maximum decoding capability.

15. The system of claim 11, wherein the instructions that, when executed on the processor, cause the system at least to encode the first screen data further cause the system at least to:

determine an amount of available processing resources of the source computer not used by a user application; and

encode the first screen data in real-time based on the amount of available processing resources.

16. The system of claim 11, wherein the instructions that, when executed on the processor, cause the system at least to encode the first screen data further cause the system at least to:

encode a first portion of the first screen data with a first codec; and

encode a second portion of the first screen data with a second codec, the second codec differing from the first codec.

17. A computer-readable storage device, bearing computer-executable instructions that, when executed on a computer, cause the computer to perform operations comprising:

establishing a wireless communication channel between a user console session of a source computer and a destination computer, the destination computer being configured to display screen data on a wireless display;

determining first screen data of the user console session to be displayed on the wireless display;

determining a sound of the user console session to be played contemporaneously with displaying the first screen data;

encoding the first screen data to identify a first time stamp, and encoding the sound to identify a second time stamp, the first time stamp and the second time stamp indicating to play the sound contemporaneously with the display displaying the decoded first screen data; and

sending the encoded first screen data to the destination computer from the user console session, without the encoded first screen data being transmitted a second user session, such that the destination computer will decode the encoded first screen data and the sound and display the decoded first screen data on the wireless display while contemporaneously playing the sound.

18. The computer-readable storage device of claim 17, further bearing computer-executable instructions that, when executed on the computer, cause the computer to perform operations comprising:

determining an indication of user input of the user console session to be delivered contemporaneously with displaying the decoded first screen data;

encoding the indication of user input; and

sending the encoded indication of user input to the destination computer from the user console session, without the encoded user input being transmitted a second user session, such that the destination computer will display the indication of user input contemporaneously with the display displaying the decoded first screen data.

19. The computer-readable storage device of claim 17, wherein encoding the first screen data comprises:

encoding a first portion of the first screen data with a first codec; and

encoding a second portion of the first screen data with a second codec, the second codec differing from the first codec.

20. The computer-readable storage device of claim 17, wherein the wireless display comprises a display that lacks a wired communications channel between the source computer and the destination computer, the destination computer being configured to display screen data on the wireless display.