Methods and system for locally generated gesture and transition graphics interaction with terminal control services

• US 10,353,563 B2
• Filed: 06/09/2012
• Issued: 07/16/2019
• Est. Priority Date: 06/08/2011
• Status: Active Grant

• Current Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Original Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Inventors: Vilke, Stephen D., Hinks, Paul
• Primary Examiner(s): Patel, Manglesh M
• Application Number:
  US13/492,803
• Publication Number:
  US 20120317493A1
• Time in Patent Office: 2,593 Days
• Field of Search: 715200, 715738, 715740, 715863
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

Methods and system for enabling locally generated transition graphics interaction with virtualized host computer service includes receiving a control command issued by a user to control screen data currently being rendered at the display device, at a display device of a client device. The control command is transmitted to the host computer service for processing. The host computer service processes the control command and returns updated screen data to the client device. In response to the receipt of the updated screen data, the client device identifies a desired screen effect associated with the control command. The updated screen data received from the host computer service is packaged at the client device with data associated with the desired screen effect to generate a visual response at the client device. The packaged screen effect is returned to the display device for rendering, in response to the control command.

Claims

1. A processor-implemented method for enabling locally generated transition graphics interaction with virtualized host computer service, comprising:

receiving a control command at a display device of a client device, wherein the control command involves a user interaction with the display device and is issued to control screen data rendered at the display device;

identifying a desired gesture associated with the control command, at the client device, wherein the control command is mapped to the desired gesture in a mapping table;

identifying transition graphics based on the control command and to be included so as to provide a screen transition effect for the desired gesture when rendering updated screen data;

transmitting the desired gesture associated with the control command to the host computer service for processing;

receiving, from the host computer service, updated screen data at the client device for the screen data rendered at the display device, after the desired gesture is transmitted;

packaging, at the client device responsive to receiving the updated screen data, the updated screen data received from the host computer service with transition graphics generated at the client device for the desired gesture identified, at the client device, to generate packaged screen data at the client device, the transition graphics embedded between a currently rendered screen data and updated screen data so as to provide the screen transition effect that generates a visual response between the currently rendered screen data and the updated screen data; and

returning the packaged screen data to the display device for rendering in response to the control command.

2. The method of claim 1, wherein the control command is processed to be in the form of an input command configured to assist with the selection of the updated screen data.

3. The method of claim 1, wherein the updated screen data is received as framebuffer data from the virtualized host computer service.

4. The method of claim 1, wherein identifying a desired gesture further includes,

querying the mapping table, at the client device, using the control command received at the display device of the client device;

and

selecting the desired gesture mapped to the control command from the mapping table, the desired gesture identifying data related to the transition graphics to be included during rendering of the screen update.

5. The method of claim 1, wherein identifying a desired gesture further includes,

querying the mapping table, at the host computer service, using the control command received at the display device of the client device;

and

selecting the desired gesture mapped to the control command from the mapping table, the desired gesture identifying data related to the transition graphics to be included during rendering of the screen update.

6. The method of claim 1, wherein packaging the updated screen data further includes,

retrieving current screen data associated with the screen currently being rendered at the client device from a local buffer at the client;

intercepting, by the client device, the updated screen data transmitted by the host computer service to the display device;

retrieving, at the client device, data related to the transition graphics for the desired gesture; and

packaging the updated screen data with the current screen data and the transition graphics data related to the desired gesture to generate the packaged screen data, the packaged screen data returned to the display device for rendering, wherein the transition graphics data provides a transition from the current screen data to the updated screen data when rendered at the display device.

7. A processor-implemented method for enabling locally generated gestures and transition graphics to interact with virtualized host computer service, comprising:

receiving a selection command at a display device of a client device, wherein the selection command involves a user interaction with the display device and is issued to request an update to current screen data rendered at the display device;

transmitting the selection command to the host computer service for processing;

receiving updated screen data, by the client device from the host computer service, for the screen data rendered at the display device, after the selection command is transmitted;

receiving a desired gesture to be rendered with the updated screen data from the host computer service, wherein the desired gesture corresponds to the updated screen data transmitted by the host computer service device and dictates adjustment to be made at the display device during rendering of the updated screen data, and wherein the transmitted selection command is mapped to the desired gesture in a mapping table available to the host computer service;

identifying transition graphics based on the selection command and to be included so as to provide a screen transition effect for the desired gesture when rendering updated screen data;

packaging, at the client device responsive to receiving the updated screen data, the updated screen data received from the host computer service with transition graphics generated at the client device for the desired gesture received from the host computer service to generated packaged screen data, the transition graphics embedded between a currently rendered screen data and updated screen data so as to provide the screen transition effect that generates a visual response between the currently rendered screen data and the updated screen data; and

returning the packaged screen data to the display device for rendering, in response to the selection command, the packaged screen data rendering desired gesture related data as transitions, using transition graphics generated at the client device for the desired gesture, during the rendering of the updated screen data.

8. The method of claim 7, wherein the updated screen data is received as frame buffer data.

9. The method of claim 7, further comprising:

querying the mapping table using the selection command received from the client device; and

selecting the desired gesture mapped to the selection command from the mapping table, the desired gesture identifying data related to the transition graphics to be included during rendering of the screen update.

10. The method of claim 7, wherein packaging the updated screen data further includes,

intercepting, by the client device, the updated screen data transmitted by the host computer service to the display device;

retrieving, at the client device, data related to the transition graphics for the desired gesture; and

packaging the updated screen data with the transition graphics data related to the desired gesture to generate the packaged screen data, wherein the transition graphics data provides a transition during rendering of the updated screen data at the client device.

11. The method of claim 10, further includes,

retrieving current screen data related to the screen currently being rendered at the client device from a local buffer at the client;

and

integrating the current screen data with the packaged updated screen data, the integration providing the desired transition during rendering of the updated screen data at the display device.

12. A system for enabling locally generated gestures and transition graphics to interact with virtualized host computer service, comprising:

a client device having a display device for rendering screen data, the client device executing a client-side application, the client-side application configured to,

interpret a control command obtained from the display device to identify a desired gesture associated with the control command, wherein the control command involves a user interaction with the display device, is mapped to the desired gesture in a mapping table, and controls screen data rendered at the display device;

identify transition graphics based on the control command and to be included so as to provide a screen transition effect for the desired gesture when rendering updated screen data;

transmit the desired gesture associated with the control command to a server-side application of a server for processing;

intercept the updated screen data transmitted from the server-side application to the display device;

retrieve current screen data currently being rendered at the display device;

generate, responsive to receiving the updated screen data, packaged screen data by bundling the current screen data, the updated screen data with transition graphics generated at the client device related to the desired gesture, wherein the transition graphics are embedded between a currently rendered screen data and updated screen data so as to provide the screen transition effect that generates a visual response between the currently rendered screen data and the updated screen data; and

return the packaged screen data to the display device of the client device for rendering, wherein the packaged screen data includes transition graphics providing a transition from the current screen data to the updated screen data.

13. The system of claim 12, wherein the server-side application is further configured to process the control command by converting the control command into one or more input device commands, wherein the input device commands are used in identifying the updated screen data at the host computer service.

14. The system of claim 12, further includes a first local buffer at the client device to store the screen data rendered at the display device and a second local buffer to store the updated screen data received from the server-side application.

15. The system of claim 12, wherein the mapping table is stored at the client device and is configured to store a mapping of different control commands to distinct gestures.

16. The system of claim 12, wherein the mapping table is stored at the server and is configured to store a mapping of different control commands to distinct gestures, the mapping table available to the virtualized host computer service.

17. A non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a client device to cause the client device to perform a method comprising:

receiving a control command at a display device of the client device, wherein the control command involves a user interaction with the display device, is in response to a gesture detected on the display device, and controls screen data rendered at the display device;

identifying an input command associated with the control command, wherein the control command is mapped to the input command in a mapping table;

identifying transition graphics based on the control command and to be included so as to provide a screen transition effect for the detected gesture when rendering updated screen data;

transmitting the input command to the host computer service for processing;

receiving, from the host computer service, updated screen data at the client device for the screen data rendered at the display device, after the input command is transmitted;

packaging, responsive to receiving the updated screen data, the updated screen data received from the host computer service with transition graphics generated at the client device for the detected gesture associated with the identified input command to generate packaged screen data at the client device, wherein the transition graphics embedded between a currently rendered screen data and updated screen data so as to provide the screen transition effect that generates a visual response between the currently rendered screen data and the updated screen data; and

returning the packaged screen data to the display device for rendering in response to the control command.