Composition control method for remote application delivery

• US 10,019,213 B1
• Filed: 01/20/2017
• Issued: 07/10/2018
• Est. Priority Date: 06/16/2014
• Status: Active Grant

• Current Assignee: Teradici Corporation (HP Inc.)
• Original Assignee: Teradici Corporation (HP Inc.)
• Inventors: Helter, Paul Andrew, Hobbs, David Victor
• Primary Examiner(s): Baturay, Alicia
• Application Number:
  US15/411,048
• Time in Patent Office: 536 Days
• Field of Search: 709217
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2360/18 Use of a frame buffer in a display terminal, inclusive of the display panel
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 67/01 Protocols

Abstract

A method for communicating an image stream, one embodiment comprising negotiating, between a server and client, an MTCC identifying: a window as a portion of a display of the client, and an occluded portion of the window defined by the client; generating, by the server, complying with the MTCC, a sequence of image frames each comprising pixel data in a portion of the frame identified by the occluded portion; transmitting an encoding of a first frame of the sequence not including an encoding of the pixel data in the portion of the first frame; identifying, by the server, an event associated with the sequence; and transmitting, before communication between the server and the client regarding the event, by the server, information of the event in conjunction with an encoding of a second frame of the sequence including an encoding of the pixel data in the portion of the second frame.

Claims

1. A method of synchronizing image composition of images from applications hosted on at least two nodes, comprising:

communicating, via a first session between a first node and a client, a first image rendered by the first node for display by the client;

communicating, via a second session between the first node and a second node, an application request;

communicating, responsive to the application request, (i) a second image rendered by the second node, to the client via a third session and (ii) a dimension, of the second image, determined by the second node, to the first node via the second session;

communicating to the client, via the first session, a third image rendered by the first node in response to receiving the dimension; and

delaying, by the client, displaying of the second image until i) receiving the third image by the client and ii) composing, by the client, the third image with the second image for display at the client, wherein the second image comprises a second plurality of frames, the third image comprises a third plurality of frames and wherein delaying the displaying of the second image comprises displaying select frames of the second plurality of frames and the third plurality of frames based on composition instructions from the first node.

2. The method of claim 1, wherein the first image is rendered as a desktop image, the second image is rendered as an application image, and the third image is rendered as a desktop image.

3. The method of claim 2, wherein delaying of the second image is in response to composition instructions received from the first node or the second node.

4. The method of claim 2, wherein:

the client renders and displays a graphical user interface;

the first session and the second session each comprise inter-computer communications;

the first image is an image of a desktop rendered by the first node; and

the application request comprises a request to launch an application at the second node.

5. The method of claim 4 wherein the client displays the first image and the second image as part of the graphical user interface controlled by a first user.

6. The method of claim 5, wherein the second node is a second client comprising a second graphical user interface controlled by a second user.

7. The method of claim 6, wherein the first user and the second user collaborate using the application.

8. The method of claim 7, wherein the application is under collaborative control by the first user at the client and the second user associated with the second node.

9. The method of claim 8, wherein content of the second image is rendered according to a first user input at the client and a second user input at a second client computer.

10. The method of claim 1, further comprising:

communicating to the client, via the first session, a fourth image rendered by the first node in response to a change of the dimension;

communicating to the client, via the third session, a fifth image rendered by the second node in response to receiving the change of the dimension; and

delaying, by the client, a displaying of the fourth image until i) receiving the fifth image and ii) composing the fourth image with the fifth image based on the composition instructions from the first node.

11. The method of claim 10, wherein the change in the dimension is initiated at the client.

12. The method of claim 10, wherein the change in the dimension is initiated at the second node.

13. The method of claim 1, wherein the second node and a session topology of the first, second and third sessions are selected to minimize latency of the third session.

14. The method of claim 1, wherein communicating the third image via the first session reduces processing by the first node and image transfer latency when compared to communicating the second image via the second session and the first session.

15. The method of claim 1, wherein the client, the first node and the second node are each independent computers and the first session, the second session and the third session are each Internet sessions.

16. The method of claim 1, wherein displaying the second image comprises displaying a portion of the second image based on composition instructions from at least one of the first node, the second node or the client.

17. A system for synchronizing image composition of images from applications hosted on at least two nodes, comprising:

a first node that renders a first image and a third image;

a second node that renders a second image, wherein the second node receives an application request from the first node and, in response, communicates the second image to a client and communicates a dimension of the second image to the first node; and

the client, in communication with the first node via a first session, and in communication with the second node via a second session,

wherein the client receives the first image from the first node, and then receives the second image from the second node prior to receiving the third image from the first node, wherein the first node communicates the third image in response to receiving the dimension,

wherein the client delays displaying the second image until the third image is received and a composition of the second image with the third image is displayed by the client, wherein the second image comprises a second plurality of frames, the third image comprises a third plurality of frames and wherein delaying the displaying of the second image comprises displaying select frames of the second plurality of frames and the third plurality of frames based on composition instructions from the first node.

18. The system of claim 17, wherein the first image is rendered as a desktop image, the second image is rendered as an application image, and the third image is rendered as a desktop image.