Feedback layer for native content display in virtual desktop infrastructure

• US 9,813,517 B2
• Filed: 05/29/2014
• Issued: 11/07/2017
• Est. Priority Date: 05/29/2014
• Status: Active Grant

• Current Assignee: Omnissa, LLC
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Wu, Junfei, Zhang, Yao, He, Zhibin, Teng, Shengbo, Peng, Yunquan
• Primary Examiner(s): Taylor, Nicholas
• Assistant Examiner(s): SHIU, HO T
• Application Number:
  US14/290,487
• Publication Number:
  US 20150350360A1
• Time in Patent Office: 1,258 Days
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06F 2203/0383 Remote input, i.e. interface arrangements in which the signals generated by a pointing device are transmitted to a PC at a remote location, e.g. to a PC in a LAN
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: H04L 67/08 specially adapted for terminal emulation, e.g. Telnet
• CPC Class Codes: H04L 67/289 Intermediate processing functionally located close to the data consumer application, e.g. in same machine, in same home or in same sub-network
• CPC Class Codes: H04L 67/565 Conversion or adaptation of application format or content adding application control or application functional data H04L67/561
• CPC Class Codes: H04W 4/18 Information format or content conversion, e.g. adaptation by the network of the transmitted or received information for the purpose of wireless delivery to users or terminals

Abstract

In one embodiment, a mobile computing device receives content in a desktop compatible format. The content is generated by a remote desktop executing an application. The mobile computing device determines a conversion of at least a first portion of the content to a native display format of the mobile computing device and displays the conversion of the at least the first portion of the content in the native display format on an interface. Also, the mobile computing device displays at least a second portion of the content in the desktop compatible format on the interface where the at least the second portion of the content in the desktop compatible format corresponds to the conversion of the at least the first portion of the content in the native display format of the mobile computing device.

Claims

1. A method comprising:

receiving, by a mobile computing device, an image of content in a desktop compatible format, wherein a remote desktop running on a host computer is executing an application to generate the content in the desktop compatible format;

determining, by the mobile computing device, a conversion of at least a first portion of the image of content to a native display format of the mobile computing device;

displaying, by the mobile computing device, the conversion of the at least the first portion of the image of content in the native display format on an interface;

concurrently displaying, by the mobile computing device, both the at least the first portion of the image of content in the desktop compatible format along with the conversion of the at least the first portion of the image of content in the native display format on the interface;

receiving, by the mobile computing device, a feedback input from a user of the mobile computing device indicating that the conversion of the at least the first portion of the image of content to the native display format is incorrect, wherein the feedback input is received following the concurrent display of both the at least the first portion of the image of content in the desktop compatible format along with the conversion of the at least the first portion of the image of content in the native display format on the interface; and

capturing, by the mobile computing device, information associated with the conversion of the at least the first portion of the image of content to the native display format.

2. The method of claim 1, wherein determining the conversion comprises:

receiving metadata for converting the at least the first portion of the image of content to the native display format, wherein the metadata is generated by the remote desktop; and

using the metadata to generate the conversion of the at least the first portion of the image of content to the native display format.

3. The method of claim 1, further comprising:

determining an active area on the interface; and

determining the conversion of the at least the first portion of the image of content based on the active area.

4. The method of claim 3, wherein the conversion of the at least the first portion of the image of content in the native display format includes the active area, the method further comprising:

displaying an indication on the active area in the at least the first portion of the image of content in the desktop compatible format.

5. The method of claim 1, further comprising:

sending the captured information for the conversion to a device as an indication of the incorrect conversion.

6. The method of claim 1, wherein:

the desktop compatible format comprises a first layout native to the application, and

the native display format comprises a second layout native to the mobile computing device that is different from the first layout native to the application.

7. A non-transitory computer-readable storage medium containing instructions, that when executed, control a mobile computing device to be configured for:

receiving an image of content in a desktop compatible format, wherein a remote desktop running on a host computer is executing an application to generate the content in the desktop compatible format;

determining a conversion of at least a first portion of the image of content to a native display format of the mobile computing device;

displaying the conversion of the at least the first portion of the image of content in the native display format on an interface;

concurrently displaying both the at least the first portion of the image of content in the desktop compatible format along with the conversion of the at least the first portion of the image of content in the native display format on the interface;

receiving a feedback input from a user of the mobile computing device indicating that the conversion of the at least the first portion of the image of content to the native display format is incorrect, wherein the feedback input is received following the concurrent display of both the at least the first portion of the image of content in the desktop compatible format along with the conversion of the at least the first portion of the image of content in the native display format on the interface; and

capturing information associated with the conversion of the at least the first portion of the image of content to the native display format.

8. The non-transitory computer-readable storage medium of claim 7, wherein determining the conversion comprises:

receiving metadata for converting the at least the first portion of the image of content to the native display format, wherein the metadata is generated by the remote desktop; and

using the metadata to generate the conversion of the at least the first portion of the image of content to the native display format.

9. The non-transitory computer-readable storage medium of claim 7, further comprising:

determining an active area on the interface; and

determining the conversion of the at least the first portion of the image of content based on the active area.

10. The non-transitory computer-readable storage medium of claim 9, wherein the conversion of the at least the first portion of the image of content to the native display format includes the active area, further configured for:

displaying an indication on the active area in the at least the second portion of the image of content in the desktop compatible format.

11. The non-transitory computer-readable storage medium of claim 7, further configured for:

receiving feedback input indicating that the conversion of the at least the first portion of the image of content to the native display format is incorrect; and

capturing information associated with the conversion of the at least the first portion of the image of content to the native display format.

12. The non-transitory computer-readable storage medium of claim 11, further configured for:

sending the captured information for the conversion to a device as an indication of the incorrect conversion.

13. The non-transitory computer-readable storage medium of claim 7, wherein:

the desktop compatible format comprises a first layout native to the application, and

the native display format comprises a second layout native to the mobile computing device that is different from the first layout native to the application.

14. A mobile computing device comprising:

one or more computer processors; and

a non-transitory computer-readable storage medium comprising instructions, that when executed, control the one or more computer processors to be configured for;

receiving an image of content in a desktop compatible format, wherein a remote desktop running on a host computer is executing an application to generate the content in the desktop compatible format;

determining a conversion of at least a first portion of the image of content to a native display format of the mobile computing device;

displaying the conversion of the at least the first portion of the image of content in the native display format on an interface;

concurrently displaying both the at least a second portion of the image of content in the desktop compatible format along with the conversion of the at least the first portion of the image of content in the desktop compatible format on the interface;

receiving a feedback input from a user of the mobile computing device indicating that the conversion of the at least the first portion of the image of content to the native display format is incorrect, wherein the feedback input is received following the concurrent display of both the at least the first portion of the image of content in the desktop compatible format along with the conversion of the at least the first portion of the image of content in the native display format on the interface; and

capturing information associated with the conversion of the at least the first portion of the image of content to the native display format.

15. The mobile computing device of claim 14, wherein determining the conversion comprises:

receiving metadata for converting the at least the first portion of the image of content to the native display format, wherein the metadata is generated by the remote desktop; and

using the metadata to generate the conversion of the at least the first portion of the image of content to the native display format.

16. The mobile computing device of claim 14, further configured for:

determining an active area on the interface; and

determining the conversion of the at least the first portion of the image of content based on the active area.

17. The mobile computing device of claim 16, wherein the conversion of the at least the first portion of the image of content to the native display format includes the active area, the apparatus further configured for:

displaying an indication on the active area in the at least the second portion of the image of content in the desktop compatible format.

18. The mobile computing device of claim 14, further configured for:

receiving feedback input indicating that the conversion of the at least the first portion of the image of content to the native display format is incorrect; and

capturing information associated with the conversion of the at least the first portion of the image of content to the native display format.

19. The mobile computing device of claim 18, further configured for:

sending the captured information for the conversion to a device as an indication of the incorrect conversion.

20. The mobile computing device of claim 14, wherein:

the desktop compatible format comprises a first layout native to the application, and

the native display format comprises a second layout native to the mobile computing device that is different from the first layout native to the application.