Display Management for Native User Experiences

• US 20120084663A1
• Filed: 10/05/2011
• Published: 04/05/2012
• Est. Priority Date: 10/05/2010
• Status: Active Grant

• Current Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Original Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Inventors: Momchilov, Georgy, Valicherla, Chakravarthi, Ingale, Mukund, Valicherla, Kireeti
• Granted Patent: US 9,400,585 B2
• Time in Patent Office: Days
• US Class Current: 715/744
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G09G 2340/0464 Positioning
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2370/02 Networking aspects
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications

Abstract

In some remote application execution environments, a client device (e.g., to which application output is sent and where user interaction is performed) may include native control elements for touch input such as virtual keyboards, virtual pickers and the like. When the server device detects an editable/control field of a remoted application receiving focus, the server may instruct the client to activate a native control element for user interaction. The client may determine the manner in which to modify an application display to resolve potential display conflicts. For example, a native control element may obscure or hide potions of the remoted application on the client display. Modifying the application display may include panning the display, zooming into or out of portions of the display, adapting the display for scrolling and the like. The remote computing device may also transmit values for populating a native control element.

Claims

1. A method comprising:

generating, by a client device, a display for an application executing at a remote computing device;

receiving, by the client device, an instruction from the remote computing device requesting activation of a graphical user input element native to the client device; and

determining at least one characteristic of the graphical user input element; and

modifying the application display based on the at least one characteristic of the graphical user input element.

2. The method of claim 1, wherein the instruction from the remote computing device includes a specification of a type of control element to be activated and wherein the method further comprises determining the graphical user input element native to the client device to activate based on the specified type of control element.

3. The method of claim 1, further comprising determining whether the native user input element will overlap one or more portions of the application display when the native user input element is activated.

4. The method of claim 3, wherein determining whether the graphical user input element will overlap the one or more portions of the application display is performed based on at least one of:

a size and a position of the native graphical user input element.

5. The method of claim 1, further comprising:

activating the native user input element;

detecting user interaction with the native user input element; and

transmitting a notification message to the remote computing device reporting the user interaction.

6. The method of claim 1, wherein the native user input element is a menu comprising one or more user selectable items, wherein the user interaction includes a selection of one of the one or more user selectable items and wherein the notification message includes an identification of the selected one of the one or more user selectable items.

7. The method of claim 1, further comprising:

receiving an instruction to deactivate the native user input element;

deactivating the native user input element; and

returning the application display to a state of the application display existing prior to the modification.

8. The method of claim 1, wherein modifying the application display includes scrolling the application display.

9. The method of claim 1, wherein modifying the application display includes panning the application display in a specified direction.

10. The method of claim 1, the modifying the application display includes zooming in or zooming out.

11. The method of claim 1, further comprising:

receiving, from the remote computing device, one or more selectable values in the instruction; and

populating the native user input element with the received one or more selectable values upon activation of the native user input.

12. A method comprising:

receiving, by a first computing device, a first notification indicating at least a portion of a display of a remoted application receiving focus, wherein the display of the remote application is transmitted to a second computing device remote from the first computing device;

determining a type of user control element associated with the at least a portion of the application display receiving focus; and

transmitting an instruction to the second computing device, the instruction including a request to activate a graphical user input element native to the second computing device and corresponding to the determined type of user control element.

13. The method of claim 12, further comprising:

receiving a second notification of focus leaving the remoted application; and

in response to receiving the second notification, transmitting another instruction to the second device to further modify the display of the remoted application.

14. The method of claim 13, wherein the remoted application is displayed in a first state prior to receiving the first notification and is returned to the first state after receiving the second notification.

15. The method of claim 13, wherein the other instruction further includes a request to deactivate the native user input element.

16. The method of claim 12, wherein transmitting the instructions includes transmitting at least one of:

a size and a position of the user control element associated with the at least a portion of the application display receiving focus.

17. The method of claim 12, wherein transmitting the instructions includes transmitting one or more selectable values for populating the native user input element.

18. The method of claim 12, further comprising receiving a second notification message indicating a user interaction with the native user input element.

19. The method of claim 18, wherein the user interaction includes user input into the native user input element.

20. The method of claim 18, wherein the user interaction includes a cancellation of the native user input element.

21. One or more non-transitory computer readable media storing computer readable instructions that, when executed, cause an apparatus to:

receive a first notification indicating at least a portion of a display of a remoted application receiving focus, wherein the display of the remoted application is transmitted to a computing device remote from the apparatus;

determining a type of user control element associated with the at least a portion of the application display receiving focus; and

transmitting an instruction to the second computing device, the instruction including a request to activate a graphical user input element native to the computing device and corresponding to the determined type of user control element.

22. The one or more non-transitory computer readable media of claim 21, wherein the computer readable instructions, when executed, further cause the apparatus to:

receive a second notification of focus leaving the remoted application; and

in response to receiving the second notification, transmit another instruction to the computing device to further modify the display of the remoted application.

23. The one or more non-transitory computer readable media of claim 21, wherein the remoted application is displayed in a first state prior to receiving the first notification and is returned to the first state after receiving the second notification.

24. The one or more non-transitory computer readable media of claim 21, wherein the other instruction further includes a request to deactivate the native user input element.

25. One or more non-transitory computer readable media storing computer readable instructions that, when executed, cause an apparatus to:

generate a display for an application executing at a remote computing device;

receive an instruction from the remote computing device requesting activation of a graphical user input element native to the apparatus; and

determine at least one characteristic of the graphical user input element; and

modify the application display based on the at least one characteristic of the graphical user input element.

26. The one or more non-transitory computer readable media of claim 25, wherein the instruction from the remote computing device includes a specification of a type of control element to be activated and wherein the method further comprises determining the graphical user input element native to the apparatus to activate based on the specified type of control element.

27. The one or more non-transitory computer readable media of claim 25, wherein the computer readable instructions, when executed, further cause the apparatus to determine whether the native user input element will overlap one or more portions of the application display when the native user input element is activated.

28. The one or more non-transitory computer readable media of claim 27, wherein determining whether the graphical user input element will overlap the one or more portions of the application display is performed based on at least one of:

a size and a position of the native graphical user input element.