Using double buffering for screen sharing

• US 9,407,724 B2
• Filed: 05/04/2010
• Issued: 08/02/2016
• Est. Priority Date: 05/04/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Khan, Nayab, Lin, Meng, Oommen, Toji T., Singh, Jaydeep
• Primary Examiner(s): Nguyen, Phuoc
• Assistant Examiner(s): Belani, Kishin G
• Application Number:
  US12/773,025
• Publication Number:
  US 20110276900A1
• Time in Patent Office: 2,282 Days
• Field of Search: 709/204
• US Class Current: 1/1
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: H04L 65/401 wherein the services involve a main real-time session and one or more additional parallel real-time or time sensitive sessions, e.g. white board sharing or spawning of a subconference
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

Remote desktop sharing allows a user to access applications and data on a remote computer over a network. An intermediary computer can communicate with a host computer and one or more client computers to facilitate remote desktop sharing. The host can send the intermediary computer an encoded key frame and encoded updates. The intermediary computer can decode the host screen data, and apply subsequent updates to the decoded image so that the intermediary computer has a near live host screen image in a first buffer. A second buffer on the intermediary computer can store an instance of a screen encoder for each client computer that has joined the session and the most current host image that the client computer has. The encoder compares the near live screen image and the stored client computer host image to create an encoded differential screen image update to pass to the client computer.

Claims

1. A method, comprising:

receiving, at an intermediate computer, a session request from a host computer to initiate a screen sharing session;

in response to receiving the session request, establishing the requested screen sharing session on the intermediary computer;

receiving an encoded key frame from the host computer;

decoding the encoded key frame and storing the decoded key frame in a first buffer on the intermediary computer;

receiving a differential screen image update from the host computer;

decoding the differential screen image update and applying the decoded differential screen image update to the stored key frame in the first buffer to generate and store a near live host screen image in the first buffer;

receiving a communication from at least one client computer of a plurality of client computers, requesting to join the screen sharing session on the intermediary computer;

for each client computer of the plurality of client computers from whom a session join request is received,

transmitting a version of the near live host screen image in the first buffer to the client computer,

generating an instance of an encoder in a second buffer on the intermediary computer, and

storing a copy of the transmitted version of the near live host screen image in the second buffer; and

in response to receiving an update request from the at least one client computer, encoding a differential screen image update at the intermediate computer using the instance of the encoder in the second buffer corresponding to the at least one client computer and transmitting the encoded differential screen image update to the at least one client computer, the differential screen image update comprising a difference between the stored copy of the transmitted version of the near live host screen image in the second buffer and a current version of the near live host screen image stored in the first buffer.

2. The method of claim 1, further comprising:

receiving communications comprising the encoded key frame or the encoded differential screen image update from the host computer in an HTTP GET or POST command.

3. The method of claim 2, further comprising:

wherein the intermediary computer is a web server and is not a proxy server.

4. The method of claim 1, further comprising

wherein, on the intermediary computer, a constant amount of memory is used for the first buffer and wherein an amount of memory proportional to a number of client computers connected to the screen sharing session is used for the second buffer.

5. The method of claim 1, further comprising:

creating a FIFO queue on the intermediary computer and storing encoded screen image updates received from the host computer in the FIFO queue.

6. The method of claim 5, further comprising:

triggered by monitored conditions, switching between storing encoded screen image updates received from the host computer in the FIFO queue and storing the near live host screen image in the first buffer.

7. The method of claim 5, further comprising:

storing encoded screen image updates for a first group of client computers in the FIFO queue and wherein encoded screen image updates for a second group of client computers are generated by comparing the near live host screen image in the first buffer and the stored copy of the version of the host screen image for each client computer in the second group of client computers to generate encoded differential screen image updates for a second group of client computers.

8. A computer system having at least one processor and a memory containing computer-executable instructions which when executed by the processor, cause the at least one processor to:

receive, at an intermediate computer, a session request from a host computer to initiate a screen sharing session;

in response to receiving the session request, establish the requested screen sharing session on the intermediary computer;

receive an encoded key frame from the host computer;

decode the encoded key frame and storing the decoded key frame in a first buffer on the intermediary computer;

receive a differential screen image update from the host computer;

decode the differential screen image update and applying the decoded differential screen image update to the stored key frame in the first buffer to generate and store a near live host screen image in the first buffer;

receive a communication from at least one client computer of a plurality of client computers, requesting to join the screen sharing session on the intermediary computer;

for each client computer of the plurality of client computers from whom a session join request is received, transmit a version of the near live host screen image to the client computer, generate an instance of an encoder in a second buffer on the intermediary computer, and store a copy of the transmitted version of the near live host screen image in the second buffer; and

in response to receive an update request from the at least one client computer, encode a differential screen image update at the intermediate computer using the instance of the encoder in the second buffer corresponding to the at least one client computer and transmit the encoded differential screen image update to the at least one client computer, the differential screen image update comprising a difference between the stored copy of the transmitted version of the near live host screen image in the second buffer and a current version of the near live host screen image stored in the first buffer.

9. The computer system of claim 8, comprising further computer-executable instructions, which when executed cause the at least one processor to:

create a FIFO queue on the intermediary computer and store encoded screen image updates received from the host computer in the FIFO queue.

10. The computer system of claim 8, comprising further computer-executable instructions, which when executed cause the at least one processor to:

use a fixed amount of memory on the intermediary computer for the first buffer and use an amount of memory proportional to a number of client computers joining the screen sharing session for the second buffer.

11. The computer system of claim 9, comprising further computer-executable instructions, which when executed cause the at least one processor to:

store encoded screen image updates for a first group of client computers in the FIFO queue and generate encoded screen image updates for a second group of client computers by comparing the near live host screen image in the first buffer and the stored version of the host screen image for the second group of client computers.

12. The computer system of claim 8, comprising further computer-executable instructions, which when executed cause the at least one processor to:

monitor specified conditions, and in response to the monitoring, switch between storing encoded screen image updates received from the host computer in the FIFO queue and storing the near live host screen image in the first buffer.

13. The computer system of claim 12, comprising further computer-executable instructions, which when executed cause the at least one processor to:

send communications to and receive communications from the host computer and the plurality of client computers via HTTP POST and GET commands.

14. A method, comprising:

receiving, at an intermediate computer, a key frame and a sequence of differential screen image updates from a host computer during a screen sharing session;

applying the received differential screen image updates to the key frame to generate versions of a host screen image of the host computer in a first buffer;

receiving, at the intermediate computer, a join request from a client computer to join the screen sharing session;

in response to receiving the join request, transmitting a version of the host screen image in the first buffer to the client computer, generating and storing an instance of an encoder in a second buffer of the intermediary computer, and storing the transmitted version of the host screen image in the second buffer;

receiving, at the intermediate computer, an update request from the client computer; and

in response to receiving the update request, encoding a differential screen image update at the intermediate computer using the instance of the encoder in the second buffer and transmitting the encoded differential screen image update to the client computer, the differential screen image update comprising a difference between a current version of the host screen image in the first buffer and the stored version of the host screen image in the second buffer.

15. The method of claim 14 wherein applying the received differential screen image updates includes sequentially applying the received differential screen image updates to the key frame image to generate a near live screen image of the host computer.

16. The method of claim 14, further comprising:

subsequent to transmitting the version of the host screen image in the first buffer to the client computer, receiving additional differential screen image updates from the host computer;

generating an updated version of the host screen image in the first buffer based on the received additional differential screen image updates; and

wherein the differential screen image update encoded by the instance of encoder in the second buffer includes a difference between the updated version of the host screen image in the first buffer and the stored version of the host screen image in the second buffer.

17. The method of claim 16, further comprising updating the stored version of the host screen image in the second buffer based on the transmitted differential screen image update encoded by the instance of encoder in the second buffer.

18. The method of claim 14, further comprising storing the received key frame and the differential screen image updates from the host computer on the intermediate computer.

19. The method of claim 14, further comprising:

storing the received key frame and the differential screen image updates from the host computer on the intermediate computer; and

switching between storing the differential screen image updates received from the host computer and storing the versions of the host screen image in the first buffer.

20. The method of claim 14 wherein applying the received differential screen image updates is temporally independent from encoding the differential screen image update at the intermediate computer and transmitting the encoded differential screen image update to the client computer.