Tracking visual gaze information for controlling content display

• US 10,242,379 B2
• Filed: 01/30/2015
• Issued: 03/26/2019
• Est. Priority Date: 01/30/2015
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Inc.
• Inventors: Ha, Eric, Rodgers, Dwight
• Primary Examiner(s): Brown, Luis A
• Assistant Examiner(s): Andrei, Radu
• Application Number:
  US14/610,859
• Publication Number:
  US 20160225012A1
• Time in Patent Office: 1,516 Days
• Field of Search: 705 1446
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06Q 30/0277 Online advertisement

Abstract

The present disclosure is directed toward tracking visual gaze information. Methods and systems of the present disclosure track a visual gaze of a user with respect to a display of a computing device. In particular, one or more embodiments track a visual gaze of a user with respect to content on a display. The methods and systems can use information about the visual gaze of the user for use in improving the presentation of content. For example, the systems and methods can increase a display size or volume of content in response to the visual gaze of the user. Still further, the methods and systems can use the visual gaze information for providing analytics regarding the viewing habits of the user.

Claims

1. A method of tracking visual gaze information for visual content, the method comprising:

tracking, by an image capture device in communication with at least one processor, a visual gaze of a user with respect to a display device of a computing device by calculating a vector from one or more eyes of the user to the display device of the computing device based on images of the one or more eyes of the user taken by the image capture device, the display device including a first content item in a primary display area of the display device and a second content item in a secondary display area of the display device;

determining, by the at least one processor, that the visual gaze of the user is on the first content item in the primary display area;

detecting, by the at least one processor, a transition of the visual gaze of the user from the first content item in the primary display area to the second content item in the secondary display area;

determining, by the at least one processor, that the visual gaze of the user with respect to the second content item meets a first predetermined threshold;

in response to the visual gaze of the user with respect to the second content item meeting the first predetermined threshold, increasing, at the computing device by the at least one processor, one or more of a display size of the second content item in the secondary display area to a first increased size or a volume associated with the second content item to a first increased volume;

determining that the visual gaze of the user remains on the second content item after meeting the first predetermined threshold until meeting a second predetermined threshold; and

switching, by the at least one processor and in response to the visual gaze of the user with respect to the second content item meeting the second predetermined threshold, a position of the first content item with a position of the second content item on the display device while the visual gaze of the user remains on the second content item by moving the first content item from the primary display area of the display device to the secondary display area of the display device and moving the second content item from the secondary display area of the display device to the primary display area of the display device.

2. The method as recited in claim 1, wherein determining that the visual gaze of the user with respect to the second content item meets the first predetermined threshold comprises determining that the visual gaze of the user remains on the second content item for a first predetermined time threshold.

3. The method as recited in claim 1, wherein determining that the visual gaze of the user with respect to the second content item meets the first predetermined threshold comprises determining that an aggregated length of time in which the visual gaze of the user is on the second content item satisfies a first aggregate predetermined time threshold within a predetermined time period.

4. The method as recited in claim 1, wherein switching the position of the first content item with the position of the second content item on the display device comprises increasing the display size of the second content item to fit to the primary display area of the display device.

5. The method as recited in claim 4, wherein switching the position of the first content item with the position of the second content item on the display device comprises decreasing a size of the first content item to fit to the secondary display area of the display.

6. The method as recited in claim 1, further comprising:

in response to the visual gaze of the user with respect to the second content item meeting the second predetermined threshold, decreasing a display size or a volume associated with one or more other content items in the display device.

7. The method as recited in claim 1, further comprising:

detecting a second transition of the visual gaze of the user from the second content item back to the first content item;

determining that the visual gaze of the user with respect to the first content item meets a third predetermined threshold; and

decreasing, by the at least one processor, one or more of the display size of the second content item or the volume associated with the second content item.

8. The method as recited in claim 1, further comprising:

tracking visual gazes of a plurality of users of the computing device;

determining that a visual gaze of a consensus of the plurality of users is on the first content item;

detecting a transition of the visual gaze of the consensus from the first content item to the second content item;

determining that the visual gaze of the consensus with respect to the second content item meets the first predetermined threshold; and

in response to the visual gaze of the consensus with respect to the second content item meeting the first predetermined threshold, increasing one or more of the display size of the second content item or the volume associated with the second content item.

9. The method as recited in claim 1, further comprising:

tracking visual gazes of a plurality of users of the computing device;

determining that a visual gaze of a first group of the plurality of users is on the first content item and a visual gaze of a second group of the plurality of users transitions to the second content item;

maintaining the first content item in a first position of the primary display area of the display device; and

repositioning the second content item in a second position of the primary display area of the display device.

10. The method as recited in claim 1, wherein increasing the display size of the second content item comprises continuously increasing the display size of the second content item on the display device while the visual gaze of the user remains on the second content item after meeting the first predetermined threshold and without the user explicitly selecting the second content item until the display size of the second content item fits to the primary display area of the display device.

11. The method as recited in claim 1, wherein increasing the display size of the second content item comprises increasing the display size of the second content item according to predetermined step sizes while the visual gaze of the user remains on the second content item after meeting the first predetermined threshold and without the user explicitly selecting the second content item until the display size of the second content item fits to the primary display area of the display device.

12. A non-transitory computer readable medium storing instructions that, when executed by at least one processor, cause a computing device to:

track, by an image capture device in communication with the at least one processor, a visual focus of a user with respect to a display device of the computing device by calculating a vector from one or more eyes of the user to the display device of the computing device based on images of the one or more eyes of the user taken by the image capture device, the display device including a first content item in a primary display area of the display device and a second content item in a secondary display area of the display device, the display including the first content item;

determine that the visual focus of the user is on the first content item in the primary display area;

determine that the visual focus of the user transitions from the first content item in the primary display area to the second content item in the secondary display area for a first predetermined threshold;

in response to the focus of the user with respect to the second content item meeting the first predetermined threshold, increase a display size of the second content item in the secondary display area to a first increased size;

determine that the visual focus of the user remains on the second content item after meeting the first predetermined threshold until meeting a second predetermined threshold; and

switch, in response to the visual focus of the user with respect to the second content item meeting the second predetermined threshold, a position of the first content item with a position of the second content item on the display device while the visual focus of the user remains on the second content item by moving the first content item from the primary display area of the display device to the secondary display area of the display device and moving the second content item from the secondary display area of the display device to the primary display area of the display device.

13. The non-transitory computer readable medium as recited in claim 12, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

identify a length of time that the visual focus of the user remains on the second content item;

assign an engagement level to the visual focus of the user on the second content item based on the length of time that the visual focus of the user remains on the second content item; and

provide an indication of the engagement level for the visual focus of the user on the second content item to an analytics tracking service.

14. The non-transitory computer readable medium as recited in claim 13, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

identify a number of times that the visual focus of the user leaves and returns to the second content item;

wherein the instructions that cause the computing device to assign the engagement level comprise instructions that cause the computing device to assign the engagement level based on the number of times that the visual focus of the user leaves and returns to the second content item.

15. The non-transitory computer readable medium as recited in claim 13, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

determine that the visual focus of the user leaves the second content item and returns to the first content item after a length of time,

wherein the instructions that cause the computing device to assign the engagement level comprise instructions that cause the computing device to assign the engagement level based on the length of time that the visual focus of the user is away from the second content item.

16. The non-transitory computer readable medium as recited in claim 13, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

detect a change in pupil dilation of the user while the visual focus of the user is on the second content item,

wherein the instructions that cause the computing device to assign the engagement level comprise instructions that cause the computing device to assign the engagement level based on the change in pupil dilation of the user.

17. The non-transitory computer readable medium as recited in claim 13, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

identify a characteristic of the second content item; and

provide a new content item on the display device based on the assigned engagement level, the new content item comprising the identified characteristic.

18. The non-transitory computer readable medium as recited in claim 12, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

determine, based on the visual focus of the user, that the focus of the user transitions among a plurality of content items;

identify a common characteristic associated with the plurality of content items; and

provide one or more new content items comprising the identified common characteristic.

19. A system for tracking visual gaze information for visual content, the system comprising:

at least one processor;

an image capture device; and

at least one non-transitory computer readable storage medium storing instructions thereon, that, when executed by the at least one processor, cause the system to;

track, by the image capture device, a visual gaze of a user with respect to a display device of a computing device by calculating a vector from one or more eyes of the user to the display device of the computing device based on images of the one or more eyes of the user taken by the image capture device, the display device including a first content item in a primary display area of the display device and a second content item in a secondary display area of the display device;

determine that the visual gaze of the user is on the first content item in the primary display area;

detect a transition of the visual gaze of the user from the first content item in the primary display area to the second content item in the secondary display area;

determine that the visual gaze of the user remains on the second content item for a first predetermined time threshold; and

in response to the visual gaze of the user remaining on the second content item for the first predetermined time threshold, and while the visual gaze of the user remains on the second content item after meeting the first predetermined time threshold;

increase, at the computing device, a display size of the second content item in the secondary display area;

decrease, at the computing device, a display size of one or more other content items in the secondary display area of the display device;

determine that the visual gaze of the user remains on the second content item after meeting the first predetermined time threshold until meeting a second predetermined time threshold; and

switch, in response to the visual gaze of the user with respect to the second content item meeting the second predetermined threshold, a position of the first content item with a position of the second content item on the display device while the visual gaze of the user remains on the second content item by moving the second content item from the primary display area of the display device to the secondary display area of the display device and moving the second content item from the secondary display area of the display device to the primary display area of the display device.

20. The system as recited in claim 19, further comprising instructions that, when executed by the at least one processor, cause the system to switch the position of the first content item, the switching comprising:

continuing to increase the display size of the second content item until the display size of the second content item fits to a size of the primary display area of the display device; and

moving the second content item to the primary area of the display device when the display size of the second content item fits to the size of the primary display area.