Portable eye tracking device

• US 10,389,924 B2
• Filed: 07/15/2016
• Issued: 08/20/2019
• Est. Priority Date: 09/03/2013
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Andersson, Jonas, Kingback, Anders, Gustafsson, Simon
• Primary Examiner(s): Lao, Lunyi
• Assistant Examiner(s): Suteerawongsa, Jarurat
• Application Number:
  US15/211,183
• Publication Number:
  US 20160328016A1
• Time in Patent Office: 1,131 Days
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04R 1/32 for obtaining desired directional characteristic only specially adapted for hearing aids H04R25/40
• CPC Class Codes: H04R 2201/40 Details of arrangements for obtaining desired directional characteristic by combining a number of identical transducers covered by H04R1/40 but not provided for in any of its subgroups

Abstract

A portable eye tracker device is disclosed which includes a frame, at least one optics holding member, a movement sensor, and a control unit. The frame may be a frame adapted for wearing by a user. The at least one optics holding member may include at least one illuminator configured to selectively illuminate at least a portion of at least one eye of the user, and at least one image sensor configured to capture image data representing images of at least a portion of at least one eye of the user. The movement sensor may be configured to detect movement of the frame. The control unit may be configured to control the at least one illuminator for the selective illumination of at least a portion of at least one eye of the user, receive the image data from the image sensors, and receive information from the movement sensor.

Claims

1. A portable eye tracking device comprising:

a frame adapted for wearing by a user;

a plurality of illuminators, wherein each illuminator is configured to illuminate at least a portion of at least one eye of the user upon activation;

an image sensor configured to capture image data representing images of at least a portion of at least one eye of the user; and

a control unit configured to;

analyze ambient light to identify one or more characteristics of the ambient light;

adjust one or more of the plurality of illuminators based at least in part on the one or more characteristics of the ambient light;

activate, based at least in part on the one or more characteristics of the ambient light, the one or more of the plurality of illuminators to illuminate at least a portion of at least one eye of the user, wherein the illuminating results in a glint on the at least one eye of the user;

receive the image data from the image sensor, wherein the image data represents at least one image of the glint on the at least one eye of the user illuminated by the one or more of the plurality of illuminators;

detect a contrast in the image data produced at least in part by the ambient light; and

determine a gaze direction of the user based at least in part on the glint and the contrast.

2. The portable eye tracking device of claim 1, wherein activating the one or more of the plurality of illuminators comprises:

activating only one illuminator.

3. The portable eye tracking device of claim 1, wherein:

the plurality of illuminators comprises;

a first subset of illuminators configured to illuminate a first eye of the user; and

a second subset of illuminators configured to illuminate a second eye of the user; and

activating the one or more of the plurality of illuminators comprises;

activating at least one of the first subset of illuminators; and

activating at least one of the second subset of illuminators.

4. The portable eye tracking device of claim 1, wherein:

prior to activating the one or more of the plurality of illuminators to illuminate at least a portion of at least one eye of the user, the control unit determines that the gaze direction cannot be determined from image data previously received.

5. The portable eye tracking device of claim 1:

further comprising a movement sensor configured to detect movement of the frame; and

wherein, while determining the gaze direction, the control unit is further configured to compensate for the detected movement.

6. The portable eye tracking device of claim 1, further comprising:

a positioning device configured to determine a position of the frame.

7. The portable eye tracking device of claim 1, further comprising:

a display viewable by the user.

8. The portable eye tracking device of claim 7, wherein:

information and/or graphics are displayed on the display; and

determining the gaze direction of the user comprises determining a location of a gaze on the display.

9. The portable eye tracking device of claim 8, wherein:

the information and/or graphics are changed based at least in part on the location of the gaze on the display.

10. The portable eye tracking device of claim 1, wherein the one or more characteristics of the ambient light comprise one or more of a wavelength characteristic, a frequency characteristic, a source characteristic, a proximity characteristic, or an orientation characteristic.

11. The portable eye tracking device of claim 1, wherein the control unit is further configured to, prior to activating the one or more of the plurality of illuminators and receiving the image data, adjust a setting of the image sensor based at least in part on the one or more characteristics of the ambient light.

12. The portable eye tracking device of claim 1, wherein:

the control unit is further configured to, prior to analyzing the ambient light, determine whether the gaze direction of the user can be determined based on other image data previously received; and

wherein adjusting the illuminator is responsive to determining that the gaze direction cannot be determined from the other image data.

13. The portable eye tracking device of claim 1, wherein:

the control unit is further configured to determine, based at least in part on the detected contrast, a position of a pupil and/or an iris of the at least one eye of the user; and

determining the gaze direction based at least in part on the glint and the contrast further comprises determining the gaze direction based at least in part on the glint and the determined position of the pupil and/or the iris.

14. The portable eye tracking device of claim 13, wherein the control unit is further configured to:

determine, based at least in part on a level of the ambient light, that a first accuracy for the determined position of the pupil and/or the iris is comparable to a second accuracy associated with the glint; and

activate an ambient-light-only mode of the portable eye tracking device in response to the determining that the first accuracy is comparable to the second accuracy; and

while operating in the ambient-light-only mode, determine a second gaze direction based at least in part on the contrast.

15. A method for determining a gaze direction of a user, comprising:

analyzing ambient light to identify one or more characteristics of the ambient light;

adjusting one or more of a plurality of illuminators on a frame worn by a user based at least in part on the one or more characteristics of the ambient light;

activating the one or more of the plurality of illuminators to illuminate at least a portion of at least one eye of the user based at least in part on the one or more characteristics of the ambient light, wherein the illuminating results in a glint on the at least one eye of the user;

receiving image data from an image sensor on the frame worn by the user, wherein the image data represents at least one image of the glint on the at least one eye of the user illuminated by the one or more of the plurality of illuminators;

detecting a contrast in the image data produced at least in part by the ambient light; and

determining a gaze direction of the user based at least in part on the glint and the contrast.

16. The method for determining a gaze direction of a user of claim 15, wherein activating the one or more of the plurality of illuminators comprises:

activating only one illuminator per eye of the user.

17. The method for determining a gaze direction of a user of claim 15, wherein activating the one or more of the plurality of illuminators comprises:

activating only a subset of the plurality of illuminators.

18. The method for determining a gaze direction of a user of claim 15, wherein:

the plurality of illuminators comprises;

a first subset of illuminators configured to illuminate a first eye of the user; and

a second subset of illuminators configured to illuminate a second eye of the user; and

activating the one or more of the plurality of illuminators comprises;

activating at least one of the first subset of illuminators; and

activating at least one of the second subset of illuminators.

19. The method for determining a gaze direction of a user of claim 15, further comprising:

prior to activating the one or more of the plurality of illuminators to illuminate at least a portion of at least one eye of the user, determining that the gaze direction cannot be determined from image data previously received.

20. A non-transitory machine readable medium having instructions thereon for determining a gaze direction for a user, the instructions executable by a processor for at least:

analyzing ambient light to identify one or more characteristics of the ambient light;

adjusting one or more of a plurality of illuminators on a frame worn by a user based at least in part on the one or more characteristics of the ambient light;

activating the one or more of the plurality of illuminators to illuminate at least a portion of at least one eye of the user based at least in part on the one or more characteristics of the ambient light, wherein the illuminating results in a glint on the at least one eye of the user;

receiving image data from an image sensor on the frame worn by the user, wherein the image data represents at least one image of the glint on the at least one eye of the user illuminated by the one or more of the plurality of illuminators;

detecting a contrast in the image data produced at least in part by the ambient light; and

determining a gaze direction of the user based at least in part on the glint and the contrast.

21. The non-transitory machine readable medium of claim 20, wherein activating the one or more of the plurality of illuminators comprises:

activating only one illuminator per eye of the user.

22. The non-transitory machine readable medium of claim 20, wherein activating the one or more of the plurality of illuminators comprises:

activating only a subset of the plurality of illuminators.