Method and system for controlling local display and remote virtual desktop from a mobile device

• US 10,432,681 B1
• Filed: 10/19/2016
• Issued: 10/01/2019
• Est. Priority Date: 04/24/2013
• Status: Active Grant

• Current Assignee: Parallels International GmbH (Corel Holdings LP)
• Original Assignee: Parallels International GmbH (Corel Holdings LP)
• Inventors: Kolomeitsev, Artem, Kiselev, Alexander, Dobrovolskiy, Nikolay N., Protassov, Stanislav S., Beloussov, Serguei M.
• Primary Examiner(s): Meky, Moustafa M
• Application Number:
  US15/298,176
• Time in Patent Office: 1,077 Days
• CPC Class Codes: H04L 65/1059 End-user terminal functionalities specially adapted for real-time communication
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal

Abstract

A computer-implemented method for transmitting and controlling images of a remotely executed application, the method including connecting a mobile computing device to a remote server; connecting a desktop computing device to the remote server and to the mobile computing device; starting an application on the remote server, wherein the application generates graphical objects controllable by commands from the mobile computing device; encoding each graphical object into a video stream; transferring the video stream to the mobile computing device and to the desktop computing device; generating the commands on mobile computing device; sending the commands to the remote server; detecting if the video stream contains results of execution of the commands; displaying the graphical objects on the local computing device; and generating an image of a substitute graphic object image if the video stream does not contain the results.

Claims

1. A method comprising:

receiving, by a processing device of a client device, a video stream from a remote server, wherein the video stream comprises a graphical object generated by an application running on the remote server and controllable by a command issued by a mobile device communicatively coupled with the client device, the command to be executed by the remote server;

detecting whether the video stream comprises a result of an execution of the command with respect to the graphical object by the remote server;

responsive to the video stream not comprising the result, generating, by the processing device of the client device, a substitute graphical object to replace the graphical object in the video stream; and

causing display of the substitute graphical object on the client device.

2. The method of claim 1, wherein the command comprises at least one of positioning an application window, sizing the application window, or switching to a different application window.

3. The method of claim 1, further comprising:

receiving the command from the mobile device; and

forwarding the command to the remote server.

4. The method of claim 1, further comprising:

receiving the command in an operating system-independent format; and

translating the command into a format corresponding to an operating system running on the client device.

5. The method of claim 1, wherein the video data stream further comprises data, embedded by a codec, indicating that the command has been executed on the remote server.

6. The method of claim 5, further comprising:

reading the data embedded by the codec in the video stream to detect whether the video stream comprises the result of the execution of the command by the remote server.

7. The method of claim 1, further comprising:

responsive to the video stream comprising the result, causing display of the graphical object on the client device.

8. A client device comprising:

a memory; and

a processing device coupled to the memory, the processing device configured to;

receive a video stream from a remote server, wherein the video stream comprises a graphical object generated by an application running on the remote server and controllable by a command issued by a mobile device communicatively coupled with the client device, the command to be executed by the remote server;

detect whether the video stream comprises a result of an execution of the command with respect to the graphical object by the remote server;

responsive to the video stream not comprising the result, generating a substitute graphical object to replace the graphical object in the video stream; and

cause display of the substitute graphical object on the client device.

9. The client device of claim 8, wherein the command comprises at least one of selecting an application, changing a display brightness, or setting a display resolution.

10. The client device of claim 8, wherein the processing device is further configured to:

receive the command from the mobile device; and

forward the command to the remote server.

11. The client device of claim 8, wherein the processing device is further configured to:

receive the command in an operating system-independent format; and

translate the command into a format corresponding to an operating system running on the client device.

12. The client device of claim 8, wherein the video data stream further comprises data, embedded by a codec, indicating that the command has been executed on the remote server.

13. The client device of claim 12, wherein the processing device is further configured to:

read the data embedded by the codec in the video stream to detect whether the video stream comprises the result of the execution of the command by the remote server.

14. The client device of claim 8, wherein the processing device is further configured to:

responsive to the video stream comprising the result, cause display of the graphical object on the client device.

15. A non-transitory computer-readable storage medium storing instructions, which when executed by a processing device of a client device, cause the processing device to perform operations comprising:

receiving, by the processing device, a video stream from a remote server, wherein the video stream comprises a graphical object generated by an application running on the remote server and controllable by a command issued by a mobile device communicatively coupled with the client device, the command to be executed by the remote server;

detecting whether the video stream comprises a result of an execution of the command with respect to the graphical object by the remote server;

responsive to the video stream not comprising the result, generating, by the processing device, a substitute graphical object to replace the graphical object in the video stream; and

causing display of the substitute graphical object on the client device.

16. The non-transitory computer-readable storage medium of claim 15, wherein the operations further comprise:

receiving the command from the mobile device; and

forwarding the command to the remote server.

17. The non-transitory computer-readable storage medium of claim 15, wherein the operations further comprise:

receiving the command in an operating system-independent format; and

translating the command into a format corresponding to an operating system running on the client device.

18. The non-transitory computer-readable storage medium of claim 15, wherein the video data stream further comprises data, embedded by a codec, indicating that the command has been executed on the remote server.

19. The non-transitory computer-readable storage medium of claim 18, further comprising:

reading the data embedded by the codec in the video stream to detect whether the video stream comprises the result of the execution of the command by the remote server.

20. The non-transitory computer-readable storage medium of claim 15, wherein the operations further comprise:

responsive to the video stream comprising the result, causing display of the graphical object on the client device.