Adaptive resolution in software applications based on dynamic eye tracking

• US 10,264,211 B2
• Filed: 03/14/2014
• Issued: 04/16/2019
• Est. Priority Date: 03/14/2014
• Status: Active Grant

• Current Assignee: Comcast Cable Communications LLC (Comcast Corporation)
• Original Assignee: Comcast Cable Communications LLC (Comcast Corporation)
• Inventors: Leech, Jonathan Alan, Pinckernell, Nicholas Adam, Monnerat, Edward David, Robinson, Chris, Johnson, Derek
• Primary Examiner(s): Natnael, Paulos M
• Application Number:
  US14/212,681
• Publication Number:
  US 20150264299A1
• Time in Patent Office: 1,859 Days
• Field of Search: 348 78, 348576, 348578, 348 51, 725 87
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G06F 2203/04805 Virtual magnifying lens, i.e. window or frame movable on top of displayed information to enlarge it for better reading or selection
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/0435 Change or adaptation of the frame rate of the video stream
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2360/08 Power processing, i.e. workload management for processors involved in display operations, such as CPUs or GPUs
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 : Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 7/0117 : involving conversion of the spatial resolution of the incoming video signal for graphics images G09G2340/0407

Abstract

Methods and systems are described for determining an image resource allocation for displaying content within a display area. An image or data capture device associated with a display device may capture an image of a space associated with the user or capture data related to other objects in the space. The viewing distance between the user and the display area (e.g., the display device) may be monitored and processed to determine and/or adjust the image resource allocation for content displayed within the display area. User movement, including eye movement, may also be monitored and processed to determine and/or adjust the image resource allocation for content displayed within the display area.

Claims

1. A method comprising:

determining, by a computing device, a first distance between a first user and a display device, wherein a display area, of the display device, comprises a plurality of regions;

receiving, by the computing device, content for display via the display device, wherein the content comprises at least;

video content; and

prioritization data for one or more frames of the video content, wherein the prioritization data prioritizes at least one region, of the plurality of regions, with respect to at least one other region of the plurality of regions;

determining that a change in eye focus of the first user exceeds a second distance threshold within the display area of the display device;

determining, based at least in part on the first distance, the prioritization data, and the change in eye focus exceeding the second distance threshold, an image resource allocation for displaying an image within the display area; and

outputting, via the display device, the image for display within the display area in accordance with the determined image resource allocation.

2. The method of claim 1, further comprising:

determining one or more images in the content to be visually emphasized to the first user.

3. The method of claim 1, further comprising:

determining, based at least in part on data obtained from one or more triggers embedded in the content, a prioritization of image characteristics for one or more regions of the plurality of regions, of the display area.

4. The method of claim 1, further comprising:

determining a plurality of users, including the first user, within a threshold distance to the display area, and

determining one or more distances between the display area of the display device and the plurality of users.

5. The method of claim 4, wherein the image resource allocation is determined based at least in part on a distance corresponding to a user, of the plurality of users, that is within a shortest distance to the display device.

6. The method of claim 1, wherein the one or more frames of the video content comprise one or more of an I-Frame, a P-Frame, or a B-Frame.

7. The method of claim 4, further comprising:

determining a second user, of the plurality of users, that is closest to a remote control device; and

adjusting, based on a distance of the second user, the image resource allocation for displaying the image within the display area.

8. A method comprising:

receiving, by a computing device, content for display via a display device, wherein the content comprises at least;

video content; and

prioritization data for one or more frames of the video content, wherein the prioritization data prioritizes an image characteristic for at least one region with respect to at least one other region of a display area of the display device;

receiving information indicating a first distance of a first user from the display area;

determining a plurality of regions of the display area;

determining that a change in eye focus of the first user exceeds a second distance threshold within the display area of the display device;

determining, based at least in part on the first distance, the prioritization data, and the change in eye focus exceeding the second distance threshold, an image resource allocation for displaying an image within the display area; and

outputting, via the display device and based on the determined image resource allocation, the image for display within the display area.

9. The method of claim 8, further comprising:

determining that the first distance of the first user satisfies a threshold distance; and

adjusting, based on the determining that the first distance of the first user satisfies the threshold distance, a first image resolution for displaying the image in a first region of the display area.

10. The method of claim 8, further comprising:

processing, by the computing device, coordinate information indicating one or more regions, of the plurality of regions, of the display area to be visually emphasized to the first user.

11. The method of claim 8, further comprising:

determining, based at least in part on the first distance and one or more triggers embedded in the received content, a first image resolution for displaying the image in a first region of the display area;

receiving, by the computing device, information indicating a second region, of the plurality of regions, of the display area; and

determining a second image resolution for displaying the image in the second region, wherein the first image resolution is higher than the second image resolution.

12. The method of claim 8, further comprising:

determining, based at least in part on the first distance and one or more triggers embedded in the received content, a first image resolution for displaying the image in a first region, of the plurality of regions, of the display area; and

adjusting the first image resolution for displaying the image in the first region in accordance with information stored in a user profile.

13. The method of claim 8, further comprising:

based on to the determining that the change in the eye focus exceeds the second distance threshold, determining, based in part on the image characteristic, a ranking associated with the plurality of regions of the display area; and

modifying the image resource allocation for displaying the image within the display area in accordance with the ranking associated with the plurality of regions of the display area.

14. The method of claim 8, further comprising:

determining one or more of a minimum resource threshold or a maximum resource threshold for adjusting a first image characteristic; and

adjusting, based on another distance of the first user from the display area, one or more of the minimum resource threshold and the maximum resource threshold.

15. The method of claim 8, further comprising:

determining one or more of a minimum resource threshold or a maximum resource threshold for adjusting a first image characteristic; and

adjusting, based on an amount of transmission resources available for outputting the image for display, one or more of the minimum resource threshold and the maximum resource threshold.

16. A method comprising:

outputting, via a display device, content for display;

determining, by a computing device, a plurality of regions of a display area of the display device;

determining, based on a distance between a first user and the display device, a sequence of image characteristics for adjusting an allocation of image resources;

determining a ranking associated with the plurality of regions based on an eye focus of the first user and at least a first image characteristic for displaying an image within the display area;

determining, based at least in part on the ranking and the sequence of image characteristics, an image resource allocation for displaying the image within the display area; and

outputting, via the display device, the image for display within the display area in accordance with the determined image resource allocation.

17. The method of claim 16, further comprising:

detecting an increase in transmission resources available for outputting the image for display within the display area; and

adjusting, based on the increase in transmission resources, a size of at least a first region, of the plurality of regions, of the display area associated with the eye focus of the first user.

18. The method of claim 16, wherein the image resource allocation is further determined based in part on one or more distances of a plurality of users, wherein the plurality of users comprises at least the first user.

19. The method of claim 16, further comprising:

adjusting, based on a new eye focus of the first user, the determined ranking; and

determining, by the computing device and based at least in part on the adjusted ranking, an updated image resource allocation for displaying the image.

20. The method of claim 16, wherein the first image characteristic comprises one or more of a rate of image adjustment, a degree of image compression, a level of image resolution, a frame rate, a level of texture resolution, or a processing power per pixel.

21. The method of claim 16, further comprising:

determining an availability of a transmission resource;

adjusting, based on the determined availability of the transmission resource, a size of a first region of the plurality of regions of the display area;

determining an updated image resource allocation for displaying the image within at least the first region of the display area; and

outputting, via the display device, the image for display within at least the first region of the display area in accordance with the updated image resource allocation.