Eye-tracking enabled wearable devices

• US 10,607,075 B2
• Filed: 03/20/2018
• Issued: 03/31/2020
• Est. Priority Date: 09/24/2015
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Gustafsson, Simon, Bezugly, Alexey, Kingback, Anders, Clausen, Anders
• Primary Examiner(s): Marinelli, Patrick F
• Application Number:
  US15/926,722
• Publication Number:
  US 20180307905A1
• Time in Patent Office: 742 Days
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G09G 2340/0464 Positioning
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/34 for rolling or scrolling

Abstract

A method for mapping an input device to a virtual object in virtual space displayed on a display device is disclosed. The method may include determining, via an eye tracking device, a gaze direction of a user. The method may also include, based at least in part on the gaze direction being directed to a virtual object in virtual space displayed on a display device, modifying an action to be taken by one or more processors in response to receiving a first input from an input device. The method may further include, thereafter, in response to receiving the input from the input device, causing the action to occur, wherein the action correlates the first input to an interaction with the virtual object.

Claims

1. A computer-implemented method, comprising:

displaying an image on a display device;

determining an initial gaze point of a user on the display device;

detecting a saccade of at least one eye of the user that begins at the initial gaze point;

determining an anticipated gaze point of the user on the display device based at least in part on a speed and an amplitude of the saccade; and

increasing image quality of an area of the image associated with the anticipated gaze point by at least modifying the area of the image.

2. The computer-implemented method of claim 1, further comprising modifying a different area of the image, that is outside the area of the image, to include decreased image quality.

3. The computer-implemented method of claim 1, wherein the anticipated gaze point corresponds to an end of the saccade.

4. The computer-implemented method of claim 1, further comprising adjusting at least one of a size or a shape of the area of the image based on updated gaze points of the user.

5. The computer-implemented method of claim 1, further comprising:

determining a different gaze point of a different user on the display device;

detecting a different saccade of at least one different eye of the different user that begins at the different gaze point;

determining a different anticipated gaze point of the different user on the display device based at least in part on the different saccade; and

increasing image quality of a different area of the image associated with the different anticipated gaze point by at least modifying the different area of the image.

6. One or more non-transitory computer-readable media comprising computer-executable instructions that, when executed by one or more computer systems, cause the one or more computer systems to at least:

display an image on a display device;

determine an initial gaze point of a user on the display device;

determine an anticipated gaze point of the user on the display device based at least in part on detecting a speed and an amplitude of a saccade of at least one eye of the user, the saccade beginning at the initial gaze point; and

increasing image quality of an area of the image associated with the anticipated gaze point by at least modifying the area of the image.

7. The one or more non-transitory computer-readable media of claim 6, wherein modifying the area of the image comprises increasing one or more graphical characteristics of the image.

8. The one or more non-transitory computer-readable media of claim 7, wherein the one or more graphical characteristics comprise resolution, shading, texture- mapping, bump-mapping, fogging medium, shadows, soft shadows, reflection, transparency, translucency, refraction, diffraction, indirect illumination, caustics, anti-aliasing, or frame rate, three dimensional.

9. The one or more non-transitory computer-readable media of claim 6, wherein:

the image comprises at least one of a salient region or a salient object; and

determining the anticipated gaze point of the user on the display device is further based at least in part on the salient region or the salient object.

10. The one or more non-transitory computer-readable media of claim 6, wherein modifying the area of the image comprises modifying the area prior to a gaze point of the user arriving at the anticipated gaze point.

11. The one or more non-transitory computer-readable media of claim 6, wherein the area of the image defines a likely viewing region of the user.

12. The one or more non-transitory computer-readable media of claim 6, wherein detecting the saccade of the at least one eye of the user comprises determining an angular change in a gaze direction of the at least one eye.

13. The one or more non-transitory computer-readable media of claim 12, further comprising computer-executable instructions that, when executed by the one or more computer systems, cause the one or more computer systems to modify a different area of the image, that surrounds the area of the image, to include decreased image quality.

14. A system, comprising:

an eye tracking device for determining a gaze direction of a user; and

one or more processors configured to at least;

display an image on a display device;

receive, from the eye tracking device, information indicative of an initial gaze point of a user on the display device;

determine an anticipated gaze point of the user on the display device based at least in part on detecting a speed and an amplitude of a saccade of at least one eye of the user; and

increase image quality of an area of the image associated with the anticipated gaze point by at least modifying the area of the image.

15. The system of claim 14, wherein the saccade begins at the initial gaze point.

16. The system of claim 14, wherein:

the one or more processors are further configured to at least receive historical saccade information for a set of users; and

determining the anticipated gaze point of the user is further based at least in part on the historical saccade information.

17. The system of claim 14, wherein the anticipated gaze point corresponds to an end of the saccade.

18. The system of claim 14, wherein the one or more processors are further configured to at least modify additional areas of the image as the anticipated gaze point of the user is updated based on the saccade changing.

19. The system of claim 14, wherein modifying the area of the image associated with the anticipated gaze point comprises modifying the area of the image prior to a gaze point of the user arriving at the anticipated gaze point.