Remote desktop video streaming alpha-channel

• US 10,404,769 B2
• Filed: 12/31/2013
• Issued: 09/03/2019
• Est. Priority Date: 12/31/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Kacmarcik, Gary, Weatherall, James, Konig, Stephen John, Pakhunov, Alex, Ulanov, Sergey Yurevich
• Primary Examiner(s): Chen, Yu
• Application Number:
  US14/145,236
• Publication Number:
  US 20150188973A1
• Time in Patent Office: 2,072 Days
• Field of Search: None
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

A method includes preparing images of a remote desktop including an open cloud-application window object corresponding to an instance of a network-hosted application executing on a network, encoding the images of the remote desktop as image frames in a video stream, and transmitting the video stream over the network to a computing device. The images the remote desktop are encoded such that when decoded, reconstructed, and displayed on a local desktop of the computing device, the open cloud-application window object corresponding to the instance of the network-hosted application executing on the network has the appearance and behavior of an open window corresponding to a natively-operating application on the local desktop of the computing device.

Claims

1. A method, comprising:

receiving, on a local computing device, a video stream including a remote desktop including an open cloud-application window corresponding to a network-hosted application, the remote desktop encoded as image frames in a video stream;

decoding the image frames in the video stream of the remote desktop;

displaying the remote desktop on a local desktop of the local computing device so that;

at least a portion of a natively-operating application on the local desktop is visible through a transparent portion of the remote desktop, wherein the natively-operating application is executing natively on the local computing device, and

the open cloud-application window has an appearance of the natively-operating application;

detecting, at a location within the transparent portion of the remote desktop, a user-input action directed to the natively-operating application; and

passing the user-input action through the transparent portion of the remote desktop to the natively-operating application.

2. The method of claim 1, wherein the remote desktop includes images having a rectangular shape.

3. The method of claim 1, wherein the remote desktop encoded as image frames in the video stream include image frames shaped so that a portion of the remote desktop, when displayed on the local desktop of the local computing device, is transparent.

4. The method of claim 3, wherein the remote desktop encoded as image frames in the video stream include images encoded so that each image pixel has at least one color channel and a separate transparency channel.

5. The method of claim 4, wherein the remote desktop encoded as image frames in the video stream include images encoded so that each image pixel has a red channel, a green channel, a blue channel, and the separate transparency channel.

6. The method of claim 4, wherein the remote desktop encoded as image frames in the video stream include values assigned to alpha channels of the pixels in the images so that a portion of the remote desktop, when displayed on the local desktop of the local computing device, is transparent.

7. The method of claim 4, wherein the remote desktop encoded as image frames in the video stream include values assigned to alpha channels of the pixels in the images so that the open cloud-application window has an appearance of a window of the natively-operating application.

8. The method of claim 1, wherein the user-input action includes one or more of a keyboard entry or a mouse click.

9. The method of claim 1, wherein the remote desktop is a first remote desktop,

the method, further comprising:

displaying a second remote desktop on the local desktop of the local computing device.

10. The method of claim 1, wherein the remote desktop has dimensions smaller than dimensions of the local desktop of the local computing device.

11. The method of claim 1, wherein the passing of the user-input action includes determining at least one of a relative geometrical position or an absolute geometrical position of the user-input action.

12. A system, comprising:

at least one server including at least one processor, at least one memory, and at least one video encoder, the server hosted on a network having a communicative link for establishing a connection to a local computing device; and

a cloud computing service module coupled to the at least one video encoder, the cloud computing service module configured to;

prepare images of a remote desktop including an open cloud-application window corresponding to a network-hosted application, the images of the remote desktop having a size conforming to a size of a local desktop of the local computing device;

encode the images of the remote desktop as image frames in a video stream signal;

add a transparency channel to the video stream signal, the transparency channel carrying values of a transparency attribute of the images of the remote desktop; and

pass a user-input action through a transparent portion of the remote desktop triggered by the transparency channel to a natively-operating application on the local desktop of the local computing device when the user-input action is directed to a visible portion of the natively-operating application, wherein the natively-operating application is executing natively on the local computing device.

13. The system of claim 12, wherein the cloud computing service is configured to encode the images in the video stream so that each image pixel has at least one color channel and a separate transparency channel.

14. The system of claim 13, wherein the cloud computing service is configured to assign values to alpha channels of the pixels in the images of the remote desktop so that the transparent portion of the remote desktop, when displayed on the local desktop of the local computing device, is transparent and the open cloud-application window has an appearance of the natively-operating application.

15. The system of claim 12, wherein the cloud computing service is configured to shape the image frames in the video stream so that the portion of the remote desktop, when displayed on the local desktop of the local computing device, is transparent such that the visible portion of the natively-operating application is visible through the transparent portion of the remote desktop.

16. The system of claim 12, wherein the portion of the remote desktop that is transparent is a background portion abutting the open cloud-application window on the remote desktop.

17. A method, comprising:

receiving, at a local computing device, encoded images of a remote desktop, the remote desktop including a transparent portion and an open cloud-application window corresponding to a network-hosted application executing on a network;

decoding the received encoded images to display, on a local desktop of the local computing device, the open cloud-application window with an appearance of an open window corresponding to a natively-operating application of the local computing device;

decoding the received encoded images to display, on the local desktop, the transparent portion of the remote desktop such that a portion of the local desktop of the local computing device is visible through the transparent portion of the remote desktop;

detecting, at a location within the transparent portion of the remote desktop, a user-input action directed to a natively-operating application; and

passing the user-input action through the transparent portion of the remote desktop to the natively-operating application, wherein the natively-operating application is executing natively on the local computing device.

18. The method of claim 17, wherein the images of the remote desktop have a rectangular shape and a size conforming to a size of the local desktop of the local computing device.

19. The method of claim 17, wherein the received encoded images of the remote desktop are encoded as image frames in a video stream, wherein the encoding includes video shaping the image frames in the video stream so that a portion of the remote desktop, when displayed on the local desktop, is transparent.

20. The method of claim 19, wherein the received encoded images have been encoded in the video stream so that each image pixel has at least one color channel and a separate transparency channel.

21. The method of claim 20, wherein the received encoded images have been encoded in the video stream so that each image pixel has a red color channel, a green color channel, a blue color channel, and the separate transparency channel.

22. The method of claim 20, wherein the received encoded images have been encoded by assigning values to alpha channels of the pixels in the images so that the transparent portion of the remote desktop, when displayed on the local desktop of the local computing device, is transparent.

23. The method of claim 20, wherein the received encoded images have been encoded by assigning values to alpha channels of the pixels in the images so that the open cloud-application window has an appearance of a window corresponding to a natively-operating application on the local desktop.