SELECTION USING EYE GAZE EVALUATION OVER TIME

• US 20150268821A1
• Filed: 03/20/2014
• Published: 09/24/2015
• Est. Priority Date: 03/20/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ramsby, Scott, Ambrus, Tony, Scavezze, Michael, Lee, Abby Lin, Mount, Brian, McIntyre, Ian Douglas, McMackin, Russ, Schneider, Gerhard, Miller, Quentin Simon Charles, Burns, Aaron Mackay, Doran, Katelyn Elizabeth
• Granted Patent: US 9,804,753 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

Various embodiments relating to selection of a user interface object displayed on a graphical user interface based on eye gaze are disclosed. In one embodiment, a selection input may be received. A plurality of eye gaze samples at different times within a time window may be evaluated. The time window may be selected based on a time at which the selection input is detected. A user interface object may be selected based on the plurality of eye gaze samples.

Claims

1. On a computing device, a method for selecting a user interface object displayed on a graphical user interface based on eye gaze, the method comprising:

receiving a selection input;

evaluating a plurality of eye gaze samples at different times within a time window selected based on a time at which the selection input is detected to determine a combined location of the plurality of eye gaze samples, two or more of the plurality of eye gaze samples having different eye gaze locations; and

selecting a user interface object nearest to the combined location of the plurality of eye gaze samples.

2. The method of claim 1, wherein the user interface object is selected based on having a highest weighting according to a selection rule.

3. The method of claim 1, wherein the time window includes eye gaze samples taken before and after the time at which the selection input is received.

4. The method of claim 1, further comprising:

setting a center of the time window to a time prior to detection of the selection input.

5. The method of claim 1, further comprising:

evaluating the plurality of eye gaze samples compared to a stability metric, and not including any eye gaze samples that do not meet the stability metric in evaluating the plurality of eye gaze samples.

6. The method of claim 1, further comprising:

in response to selecting the user interface object, providing visual feedback of selection of the user interface object on the graphical user interface.

7. The method of claim 1, wherein the time window is a first time window, and further comprising:

after the user interface object is selected, evaluating a different plurality of eye gaze samples in a second time window that is centered at a later time than the first time window to determine a combined location of the different plurality of eye gaze samples; and

if a different user interface object is nearest to the combined location of the different plurality of eye gaze samples, then undoing selection of the user interface object and selecting the different user interface object.

8. The method of claim 7, further comprising:

upon evaluation of the different plurality of eye gaze samples in the second time window, removing evaluated eye gaze samples in the first time window and the second time window from being eligible for future evaluations of eye gaze samples.

9. The method of claim 7, wherein the selection input is a first selection input, and further comprising:

receiving a second selection input after the first selection input is received; and

if the second selection input is detected before the different plurality of eye gaze samples are evaluated, then shortening the second time window.

10. A data-holding device holding instructions executable by a logic device to:

detect a selection input made via an input mode other than eye gaze;

evaluate a plurality of eye gaze samples at different eye gaze locations and at different times within a time window selected based on a time at which the selection input is detected to determine a combined location of the plurality of eye gaze samples; and

select a user interface object based on having a highest weighting according to a selection rule.

11. The data-holding device of claim 10, wherein the highest weighting is assigned to a user interface object that is a nearest user interface object to the combined location at which the plurality of eye gaze samples intersect with a graphical user interface.

12. The data-holding device of claim 10, wherein the time window includes eye gaze samples taken before and after the time at which the selection input is detected.

13. The data-holding device of claim 10, further holding instructions executable by the logic device to:

set a center of the time window to a time prior to detection of the selection input.

14. The data-holding device of claim 10, further holding instructions executable by the logic device to:

evaluate the plurality of eye gaze samples compared to a stability metric, and not include any eye gaze samples that do not meet the stability metric in evaluating the plurality of eye gaze samples.

15. The data-holding device of claim 10, further holding instructions executable by the logic device to:

in response to selecting the user interface object, provide visual feedback of selection of the user interface object on the graphical user interface.

16. The data-holding device of claim 10, wherein the time window is a first time window, and the data-holding device further holds instructions executable by the logic device to:

after the user interface object is selected, evaluate a different plurality of eye gaze samples in a second time window that is centered later than the first time window to determine the combined location of the different plurality of eye gaze samples; and

if a different user interface object is nearest to the combined location of the different plurality of eye gaze samples, then undo selection of the user interface object and select the different user interface object.

17. The data-holding device of claim 16, further holding instructions executable by the logic device to:

upon evaluation of the different plurality of eye gaze samples, remove evaluated eye gaze samples in the first window and the second window from being eligible for future evaluations of eye gaze samples.

18. The data-holding device of claim 16, wherein the selection input is a first selection input, and the data-holding device further holds instructions executable by the logic device to:

detect a second selection input after the first selection input is detected;

if the second selection input is detected before the different plurality of eye gaze samples are evaluated, then shorten the second time window.

19. A head-mounted display device, comprising:

a display configured to display a graphical user interface including one or more user interface objects;

an eye gaze tracking subsystem configured to detect a location at which an eye gaze of a wearer of the head-mounted display device intersects with the display;

a logic device; and

a data-holding device holding instructions executable by the logic device to detect a selection input made via an input mode other than eye gaze,

evaluate a plurality of eye gaze samples at different eye gaze locations and at different times within a time window selected based on a time at which the selection input is detected to determine a combined location of the plurality of eye gaze samples, and

select a user interface object that is a nearest user interface object to the combined location at which the plurality of eye gaze samples intersect with the graphical user interface.

20. The head-mounted display device of claim 19, wherein the time window includes eye gaze samples taken before and after the time at which the selection input is detected.