Hierarchical display

• US 9,423,994 B2
• Filed: 02/22/2012
• Issued: 08/23/2016
• Est. Priority Date: 02/22/2012
• Status: Active Grant

• Current Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Original Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Inventors: Suresh, Ashwin, Rivera, Juan
• Primary Examiner(s): Lazaro, David
• Assistant Examiner(s): SHAW, ROBERT A
• Application Number:
  US13/402,179
• Publication Number:
  US 20130219012A1
• Time in Patent Office: 1,644 Days
• Field of Search: 709/217, 345/418, 715/740
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/1462 with means for detecting differences between the image stored in the host and the images displayed on the remote displays
• CPC Class Codes: G09G 2320/0686 with two or more screen areas displaying information with different brightness or colours
• CPC Class Codes: G09G 2320/0693 Calibration of display systems
• CPC Class Codes: G09G 2320/103 Detection of image changes, e.g. determination of an index representative of the image change
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/0464 Positioning
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Systems and methods that provide a hierarchical display to a user based on a position of the user'"'"'s focus are described herein. The position of the user'"'"'s focus may be determined by tracking, for example, a position of the user'"'"'s eyes, a position of a mouse cursor within the display, a position of an application window within the display, or a combination thereof. Based on the position of the user'"'"'s focus, the display may be divided into multiple display regions, and the display regions may be ranked, such as by prioritizing a display region that the user is focusing on. Display quality (e.g., transmission rate, image resolution, degree of image compression, etc.) of the display regions may be determined based on the rank of the display regions, thereby providing higher display/image quality to regions based on what the user is actively looking at.

Claims

1. One or more non-transitory computer-readable media having computer-executable instructions stored thereon that, when executed by a processor, cause the processor to:

receive, from a client device, information identifying a position of a user focus within a display area, wherein the position of the user focus is determined based on an image captured by a camera associated with the client device;

determine, based on the information identifying the position of the user focus within the display area, a primary display region of the display area corresponding to the position of the user focus;

determine a secondary display region different from the primary display region;

transmit, to the client device, a frame of the primary display region; and

in response to detecting an image change in the secondary display region, prevent transmission of a frame of the secondary display region based on a determination that the secondary display region does not correspond to the position of the user focus.

2. The one or more non-transitory computer-readable media of claim 1, wherein

transmitting the frame of the primary display region comprises transmitting the frame of the primary display region with a first display quality, and wherein the one or more non-transitory computer-readable media has computer-executable instructions stored thereon that, when executed by the processor, cause the processor to:

transmit a second frame of the secondary display region with a second display quality, the first display quality being higher than the second display quality.

3. The one or more non-transitory computer-readable media of claim 1, having computer-executable instructions stored thereon that, when executed by the processor, cause the processor to:

determine a tertiary display region of the display area, wherein

the secondary display region is closer to the primary display region than the tertiary display region is to the primary display region, and wherein

a display quality of the secondary display region is higher than a display quality of the tertiary display region.

4. The one or more non-transitory computer-readable media of claim 1,

wherein preventing transmission of the frame of the secondary display region comprises discarding the frame of the secondary display region from a queue.

5. The one or more non-transitory computer-readable media of claim 1, wherein a display quality of each of the primary display region and the secondary display region comprises one or more of an image resolution, an image transmission rate, or a degree of image compression.

6. The one or more non-transitory computer-readable media of claim 1, having computer-executable instructions stored thereon that, when executed by the processor, cause the processor to:

receive, from the client device, updated information identifying a second position of the user focus indicating a change in the position of the user focus by more than a predetermined threshold; and

determine a new primary display region of the display area based on the updated information indicating the change in position of the user focus.

7. The one or more non-transitory computer-readable media of claim 1, having computer-executable instructions stored thereon that, when executed by the processor, cause the processor to:

determine, based on the image captured by the camera, a position of the user'"'"'s eyes, head, or hands; and

determine, as the position of the user'"'"'s eyes, head, or hands, the position of the user focus,

wherein determining the primary display region comprises determining the primary display region corresponding to the determined position of the user focus.

8. The one or more non-transitory computer-readable media of claim 1, wherein the position of the user focus is additionally determined based on one or more of a position of an input device indicator in the display area or a window focus in the display area.

19. The one or more non-transitory computer-readable media of claim 1, wherein transmitting the frame of the primary display region comprises transmitting the frame of the primary display region at a first image transmission rate and at a first display quality comprising one or more of a first image resolution or a first degree of image compression, wherein the one or more non-transitory computer-readable media has computer-executable instructions stored thereon that, when executed by the processor, cause the processor to:

transmit a second frame of the secondary display region at a second image transmission rate lower than the first image transmission rate and at a second display quality comprising one or more of a second image resolution lower than the first image resolution or a second degree of image compression lower than the first degree of image compression.

9. A method comprising:

receiving, from a client device, information identifying a position of a user focus in a display area, wherein the position of the user focus is determined based on an image captured by a camera associated with the client device;

determining a primary display region of the display area, the primary display region being associated with the position of the user focus in the display area;

determining a secondary display region different from the primary display region;

setting a display quality of the primary display region higher than a display quality of the secondary display region;

transmit, to the client device, a frame of the primary display region; and

in response to detecting an image change in the secondary display region, preventing transmission of a frame of the secondary display region based on a determination that the secondary display region does not correspond to the position of the user focus.

10. The method of claim 9, further comprising:

determining a tertiary display region of the display area, wherein the secondary display region is closer to the primary display region than the tertiary display region is to the primary display region; and

setting the display quality of the secondary display region higher than a display quality of the tertiary display region.

11. The method of claim 9, wherein preventing transmission of the frame of the secondary display region comprises discarding the frame of the secondary display region from a queue.

12. The method of claim 9, wherein the display quality of the primary display region comprises one or more of an image resolution, an image transmission rate, or a degree of image compression.

13. The method of claim 9, further comprising:

receiving, from the client device, updated information identifying a second position of the user focus indicating a change in the position of the user focus from the primary display region to a display region different from the primary display region; and

determining a new primary display region of the display area based on the updated information indicating the change in position of the user focus.

14. The method of claim 9, further comprising determining that the position of the user focus has changed to a new display region in the display area based on one or more of a determination that a position of an input device indicator associated with the user focus has changed to a position in the new display region or a determination that at least a portion of an application window is within the new display region.

15. The method of claim 9, further comprising determining that the position of the user focus has changed to a new display region in the display area based on a determination, based on a second image captured by the camera, that a position of the user'"'"'s eyes, head, or hands has changed from focusing on the primary display region to focusing on the new display region.

16. The method of claim 9, further comprising:

receiving, from the client device, information identifying a position of another user focus in the display area;

determining another primary display region of a display area, the another primary display region being associated with the position of the another user focus; and

setting the display quality of the another primary display region to the display quality of the primary display region.

17. The method of claim 9, wherein

transmitting the frame of the primary display region is performed in response to detecting an image change in the primary display region.

20. The method of claim 9, further comprising:

determining a second primary display region of the display area based on a second position of the user focus in the display area, wherein the second position of the user focus in the display area is determined based on one or more of a position of an input device indicator in the display area or a window focus in the display area.

18. A system, comprising:

a client device comprising;

a display screen configured to display a display area, the display area comprising a first display region and a second display region; and

a camera configured to capture images at a location of the client device; and

a server comprising;

a processor; and

memory having computer-executable instructions stored thereon that, when executed by the processor, cause the processor to;

determine, based on an image of a user captured by the camera, that a position of a user'"'"'s focus within the display area has changed from the second display region to the first display region;

assign a first transmission rate to the first display region;

assign a second transmission rate to the second display region, the second transmission rate lower than the first transmission rate;

transmit, to the client device, frames of the first display region at the first transmission rate; and

transmit, to the client device, frames of the second display region at the second transmission rate, wherein transmission of the frames of the second display region at the second transmission rate includes, in response to detecting an image change in the second display region, preventing transmission of one or more frames of the second display region based on the determining that the position of the user'"'"'s focus within the display area has changed from the second display region to the first display region.