Eye-tracking enabled wearable devices

• US 10,635,169 B2
• Filed: 05/18/2018
• Issued: 04/28/2020
• Est. Priority Date: 09/24/2015
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Gustafsson, Simon, Torneus, Daniel Johansson, Andersson, Jonas, Kron, Johannes, Skogo, Marten, Biedert, Ralf
• Primary Examiner(s): Boddie, William
• Assistant Examiner(s): Elnafia, Saifeldin E
• Application Number:
  US15/983,772
• Publication Number:
  US 20180349698A1
• Time in Patent Office: 711 Days
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G09G 2340/0464 Positioning
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/34 for rolling or scrolling

Abstract

A system for determining a gaze direction of a user is disclosed. The system may include a first illuminator, a first profile sensor, and at least one processor. The first illuminator may be configured to illuminate an eye of a user. The first profile sensor may be configured to detect light reflected by the eye of the user. The processor(s) may be configured to determine a gaze direction of the user based at least in part on light detected by the first profile sensor.

Claims

1. A system for determining a gaze direction of a user, wherein the system comprises:

a lens;

a display device configured to be viewed through the lens,

a hot mirror;

a first illuminator configured to illuminate an eye of a user;

an area image sensor arranged behind the display device and configured to detect light reflected by the eye of the user, wherein the area image sensor has a sensitive area having a total number of pixels characterized by a total number of rows and a total number of columns; and

at least one processor configured to determine the gaze direction of the user based at least in part on light detected by the area image sensor;

wherein determining the gaze direction of the user based at least in part on light detected by the area image sensor comprises;

determining a location of a glint on the eye caused by light reflected by the eye of the user, based at least in part on a higher magnitude light level detected at one or more pixels of the area image sensor;

wherein the step of determining the gaze direction of the user based at least in part on light detected by the area image sensor further comprises the steps of;

determining magnitudes of light exposure at a subset of the total number of rows during one or more illumination periods;

determining magnitudes of light exposure at a subset of the total number of columns during one or more illumination periods;

based on the magnitudes of light exposure at the subset of the total number of rows and the magnitudes of light exposure at the subset of the total number of columns, identifying a subset of a total number of pixels of the area image sensor which include and surround the one or more pixels of the area image sensor which detected the higher magnitude light level; and

analyzing only the subset of the total number of pixels of the area image sensor during a subsequent determination of gaze direction during a subsequent illumination period.

2. A system according to claim 1, wherein the hot mirror is a coating or film arranged atop the lens.

3. A system according to claim 1, wherein said the hot mirror is applied directly onto the lens.

4. A system according to claim 1, wherein said hot mirror is in a form of a separate piece.

5. A system according to claim 1, wherein the hot mirror is arranged on a side of the lens facing the user.

6. A system according to claim 1, wherein an illuminator is arranged atop the hot mirror.

7. A system according to claim 1, wherein the illuminator is arranged atop the hot mirror via a separate piece of material such as glass or plastic.

8. A method for determining a gaze direction of a user, wherein the method comprises:

displaying, through a lens, an image to the user by using a display device;

at least in part preventing, light in the infrared spectrum emitted by the display device,

from passing the lens by using a hot mirror illuminating an eye of a user;

detecting, with an area image sensor arranged behind the display device, light reflected

by the eye of the user, wherein the area image sensor has a sensitive area having a total

number of pixels characterized by a total number of rows and a total number of columns;

determining, with at least one processor, the gaze direction of the user based at least in part on light detected by the area image sensor;

wherein determining the gaze direction of the user based at least in part on light detected by the area image sensor comprises;

determining a location of a glint on the eye caused by light reflected by the eye of the user, based at least in part on a higher magnitude light level detected at one or more pixels of the area image sensor;

wherein the step of determining the location of the glint on the eye caused by light reflected by the eye of the user, based at least in part on the higher magnitude light level detected at one or more pixels of the area image sensor further comprises the steps of;

determining magnitudes of light exposure at a subset of the total number of rows during one or more illumination periods;

determining magnitudes of light exposure at a subset of the total number of columns during one or more illumination periods;

based on the magnitudes of light exposure at the subset of the total number of rows and the magnitudes of light exposure at the subset of the total number of columns, identifying a subset of a total number of pixels of the area image sensor which include and surround the one or more pixels of the area image sensor which detected the higher magnitude light level; and

analyzing only the subset of the total number of pixels of the area image sensor during a subsequent determination of gaze direction during a subsequent illumination period.

9. A non-transitory machine readable medium comprising a computer program adapted to perform the method according to claim 8.