Enhanced virtual desktop infrastructures

• US 10,678,569 B2
• Filed: 03/05/2018
• Issued: 06/09/2020
• Est. Priority Date: 03/05/2018
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Mukraj, Rabindra, Maese, Ralph, Strein, Michael J., Kepler, Benjamin H.
• Primary Examiner(s): Truong, Lan Dai T
• Application Number:
  US15/911,387
• Publication Number:
  US 20190272185A1
• Time in Patent Office: 827 Days
• Field of Search: 709227, 709201, 709203, 709217, 709219
• CPC Class Codes: G06F 13/102 where the programme performs an interfacing function, e.g. device driver G06F13/105 takes precedence; scheduling within device drivers G06F9/52; contention policies within device drivers G06F9/4881
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: H04L 12/1845 broadcast or multicast in a specific location, e.g. geocast protocols for adapting network applications to user terminal location H04L67/52; services specially adapted for wireless communication networks making use of the location of users or terminals H04W4/02
• CPC Class Codes: H04L 12/1886 with traffic restrictions for efficiency improvement, e.g. involving subnets or subdomains
• CPC Class Codes: H04L 2212/00 Encapsulation of packets
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 67/01 Protocols

Abstract

Performing an operation comprising providing, by a virtual desktop infrastructure (VDI) client, access to an application executing on a host system to a first client system via a network, receiving, from the first client system, at least one internet protocol (IP) packet specifying attributes of an input provided by an input/output (I/O) device of the client system, receiving a timestamp from a time source, and storing the attributes of the input and the timestamp as metadata of a first image corresponding to a first frame of a video output of the application.

Claims

1. A method, comprising:

providing, by a virtual desktop infrastructure (VDI) host and to a first client system via a network, access to an application executing on a host system;

receiving, from the first client system, at least one internet protocol (IP) packet specifying attributes of a first input to the application, wherein the first input is provided by an input/output (I/O) device of the first client system;

receiving, from a time source, a first timestamp to associate with the first input; and

storing the attributes of the first input and the first timestamp as metadata of a first image comprising a first frame of a video output of the application, wherein the first frame corresponds to the first timestamp.

2. The method of claim 1, further comprising:

compressing a second image comprising a second frame of the video output of the application;

encrypting the compressed second image and data describing an I/O state of the application;

generating, based on a modified IP protocol, one or more IP packets encapsulating the encrypted compressed second image and data describing the I/O state of the application;

transmitting the one or more IP packets to the first client system via the network; and

storing (i) a second timestamp received from the time source to associate with the second frame and (ii) the data describing the I/O state of the application as metadata of the second image.

3. The method of claim 1, wherein access to the application is provided to a plurality of client systems including the first client system, the method further comprising:

upon determining the plurality of client systems are connected to a same local area network (LAN), transmitting the one or more IP packets as a multicast stream; and

upon determining the plurality of client systems are not connected to the same LAN, transmitting, to each client system, the one or more IP packets as a respective uniform datagram protocol (UDP) unicast stream of a plurality of synchronized UDP streams.

4. The method of claim 1, wherein access to the application is provided to a plurality of client systems including the first client system, the method further comprising:

determining, by an arbitration manager of the VDI host, that a second client system of the plurality of client systems controls inputs to the application;

receiving, by the VDI host from the second client system, at least one IP packet encapsulating an indication of a second input provided by an I/O device of the second client system;

extracting, by the VDI host from the at least one IP packet received from the second client system, the indication of the second input; and

storing (i) a second timestamp received from the time source to associate with the second input and (ii) the indication of the second input.

5. The method of claim 4, wherein the VDI host receives the at least one IP packet from the second client system as a UDP unicast stream, wherein the indication of the second input is encrypted, the method further comprising decrypting the indication of the second input.

6. The method of claim 1, wherein the attributes of the first input comprise at least one of:

(i) a type of the I/O device, (ii) a type of the first input, or (iii) a location on the video output of the application where the first input is received.

7. The method of claim 1, wherein the first input comprises one or more of:

(i) mouse input, (ii) keystroke input, (iii) speech input, or (iv) biometric input.

8. A non-transitory computer-readable medium having computer-readable program code embodied therewith, the computer-readable program code executable by a processor to perform an operation comprising:

providing, by a virtual desktop infrastructure (VDI) host and to a first client system via a network, access to an application executing on a host system;

receiving, from the first client system, at least one internet protocol (IP) packet specifying attributes of a first input to the application, wherein the first input is provided by an input/output (I/O) device of the first client system;

receiving, from a time source, a first timestamp to associate with the first input; and

storing the attributes of the first input and the first timestamp as metadata of a first image comprising a first frame of a video output of the application, wherein the first frame corresponds to the first timestamp.

9. The non-transitory computer-readable medium of claim 8, the operation further comprising:

compressing a second image comprising a second frame of the video output of the application;

encrypting the compressed second image and data describing an I/O state of the application;

generating, based on a modified IP protocol, one or more IP packets encapsulating the encrypted compressed second image and data describing the I/O state of the application;

transmitting the one or more IP packets to the first client system via the network; and

storing (i) a second timestamp received from the time source to associate with the second frame and (ii) the data describing the I/O state of the application as metadata of the second image.

10. The non-transitory computer-readable medium of claim 8, wherein access to the application is provided to a plurality of client systems including the first client system, the operation further comprising:

upon determining the plurality of client systems are connected to a same local area network (LAN), transmitting the one or more IP packets as a multicast stream; and

upon determining the plurality of client systems are not connected to the same LAN, transmitting, to each client system, the one or more IP packets as a respective uniform datagram protocol (UDP) unicast stream of a plurality of synchronized UDP streams.

11. The non-transitory computer-readable medium of claim 8, wherein access to the application is provided to a plurality of client systems including the first client system, the operation further comprising:

determining, by an arbitration manager of the VDI host, that a second client system of the plurality of client systems controls inputs to the application;

receiving, by the VDI host from the second client system, at least one IP packet encapsulating an indication of a second input provided by an I/O device of the second client system;

extracting, by the VDI host from the at least one IP packet received from the second client system, the indication of the second input; and

storing (i) a second timestamp received from the time source to associate with the second input and (ii) the indication of the second input.

12. The non-transitory computer-readable medium of claim 11, wherein the VDI host receives the at least one IP packet from the second client system as a UDP unicast stream, wherein the indication of the second input is encrypted, the operation further comprising decrypting the indication of the second input.

13. The non-transitory computer-readable storage medium of claim 8, wherein the attributes of the first input comprise at least one of:

(i) a type of the I/O device, (ii) a type of the first input, or (iii) a location on the video output of the application where the first input is received.

14. The non-transitory computer-readable medium of claim 8, wherein the first input comprises one or more of:

(i) mouse input, (ii) keystroke input, (iii) speech input, or (iv) biometric input.

15. A system, comprising

a computer processor;

and

a memory containing a program which, when executed by the computer processor, performs an operation comprising;

providing, by a virtual desktop infrastructure (VDI) host and to a first client system via a network, access to an application executing on a host system;

receiving, from the first client system, at least one internet protocol (IP) packet specifying attributes of a first input to the application, wherein the first input is provided by an input/output (I/O) device of the first client system;

receiving, from a time source, a first timestamp to associate with the first input; and

storing the attributes of the first input and the first timestamp as metadata of a first image comprising a first frame of a video output of the application, wherein the first frame corresponds to the first timestamp.

16. The system of claim 15, the operation further comprising:

compressing a second image comprising a second frame of the video output of the application;

encrypting the compressed second image and data describing an I/O state of the application;

generating, based on a modified IP protocol, one or more IP packets encapsulating the encrypted compressed second image and data describing the I/O state of the application;

transmitting the one or more IP packets to the first client system via the network; and

storing (i) a second timestamp received from the time source to associate with the second frame and (ii) the data describing the I/O state of the application as metadata of the second image.

17. The system of claim 15, wherein access to the application is provided to a plurality of client systems including the first client system, the operation further comprising:

upon determining the plurality of client systems are connected to a same local area network (LAN), transmitting the one or more IP packets as a multicast stream; and

upon determining the plurality of client systems are not connected to the same LAN, transmitting, to each client system, the one or more IP packets as a respective uniform datagram protocol (UDP) unicast stream of a plurality of synchronized UDP streams.

18. The system of claim 15, wherein access to the application is provided to a plurality of client systems including the first client system, the operation further comprising:

determining, by an arbitration manager of the VDI host, that a second client system of the plurality of client systems controls inputs to the application;

receiving, by the VDI host from the second client system, at least one IP packet encapsulating an indication of a second input provided by an I/O device of the second client system;

extracting, by the VDI host from the at least one IP packet received from the second client system, the indication of the second input; and

storing (i) a second timestamp received from the time source to associate with the second input and (ii) the indication of the second input.

19. The system of claim 18, wherein the VDI host receives the at least one IP packet from the second client system as a UDP unicast stream, wherein the indication of the second input is encrypted, the operation further comprising decrypting the indication of the second input.

20. The system of claim 15, wherein the attributes of the first input comprise at least one of:

(i) a type of the I/O device, (ii) a type of the first input, or (iii) location on the video output of the application where the first input is received, wherein the first input comprises one or more of;

(i) mouse input, (ii) keystroke input, (iii) speech input, or (iv) biometric input.