SYSTEMS AND METHODS FOR HIGH-RESOLUTION GAZE TRACKING

• US 20170116476A1
• Filed: 07/11/2016
• Published: 04/27/2017
• Est. Priority Date: 11/07/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Eyefluence, Inc. (Alphabet Inc.)
• Inventors: Publicover, Nelson G., Torch, William C., Spitler, Christopher N.
• Granted Patent: US 10,032,074 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 3/0025 characterised by electronic signal processing, e.g. eye models
• CPC Class Codes: A61B 3/024 for determining the visual field, e.g. perimeter types
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A system mounted within eyewear or headwear to unobtrusively produce and track reference locations on the surface of one or both eyes of an observer is provided to improve the accuracy of gaze tracking. The system utilizes multiple illumination sources and/or multiple cameras to generate and observe glints from multiple directions. The use of multiple illumination sources and cameras can compensate for the complex, three-dimensional geometry of the head and the significant anatomical variations of the head and eye region that occurs among individuals. The system continuously tracks the initial placement and any slippage of eyewear or headwear. In addition, the use of multiple illumination sources and cameras can maintain high-precision, dynamic eye tracking as an eye moves through its full physiological range. Furthermore, illumination sources placed in the normal line-of-sight of the device wearer increase the accuracy of gaze tracking by producing reference vectors that are close to the visual axis of the device wearer.

Claims

1. A system for determining gaze-tracking locations, comprising:

a device configured to be worn on a wearer'"'"'s head, the device comprising a lens disposed in front of an eye of the wearer eye when the device is worn;

a scene camera mounted on the device and positioned for viewing reference locations in the environment of the wearer;

a plurality of illumination sources mounted on the lens for illuminating the eye of the wearer;

one or more eye-tracking cameras mounted on the device and positioned for viewing the eye; and

a processor coupled to the scene camera and eye-tracking cameras for analyzing images therefrom that uses scene reference locations from scene camera images and eye-tracking locations from the one or more eye-tracking camera images to determine locations being viewed by the wearer.

2. The system of claim 1, wherein the illumination sources comprise a plurality of reflective surfaces embedded in the lens, and one or more light sources configured to direct light into the lens to be reflected from the plurality of reflective surfaces towards the eye.

3. The system of claim 1, wherein the illumination sources are essentially invisible to the wearer wearing the device.

4. The system of claim 1, wherein the illumination sources comprises a plurality of light sources on the lens and a plurality of conductive pathways on the lens coupled to the light sources, the system further comprising a controller coupled to the pathways for selectively activating the light sources.

5. The system of claim 4, wherein the plurality of light sources are sufficiently spaced apart from one another to ensure that at least one glint from a light source appears on the surface of the corneal region of the eye in images from the eye-tracking camera when the eye moves through its complete normal range of motion.

6. The system of claim 4, wherein the controller is configured to illuminate a single light source to produce a single glint, and wherein the processor is configured for isolating the single glint in the gaze-tracking camera images.

8. The method of claim 1, wherein the illumination sources comprise a plurality of reflective surfaces embedded in the lens, and one or more light sources configured to direct light into the lens to be reflected from the plurality of reflective surfaces towards the eye.

7. A method for gaze tracking, comprising:

placing a device on a wearer'"'"'s head, the device comprising a plurality of illumination sources and an eye-tracking camera oriented towards an eye of the wearer, the illumination sources disposed within a field of view of the wearer and configured to minimize interference with the wearer'"'"'s vision within the field of view; and

analyzing images of the eye acquired by the eye-tracking cameras to identify one or more glints reflected off the eye from one or more of the illumination sources and identify the location of the features of the eye relative to the one or more glints to determine locations being viewed by the wearer.

9. The method of claim 7, wherein only one of the illumination sources is activated at a time, thereby producing a single glint in the images of the eye, and wherein the location of the features of the eye is identified relative to the single glint to determine locations being viewed by the wearer.

10. The method of claim 7, wherein a plurality of the illumination sources are activated at a time, thereby producing multiple simultaneous glints in the images of the eye, and wherein the location of the features of the eye is identified relative to the multiple glints to determine locations being viewed by the wearer.

11. A system for determining gaze-tracking locations, comprising:

a device configured to be worn on a wearer'"'"'s head;

one or more illumination sources mounted on the device oriented to illuminate an eye of the wearer;

a plurality of eye-tracking cameras mounted on the device and positioned sufficiently spaced apart from one another for viewing the eye from different angles such that features of the eye appear in eye-tracking camera images from at least one of the eye-tracking cameras; and

a processor coupled to the eye-tracking cameras for analyzing images of the eye acquired by the eye-tracking cameras to identify one or more glints reflected off the eye from the one or more illumination sources and identify the location of features of the eye relative to the one or more glints to determine locations being viewed by the wearer.

12. The system of claim 11, wherein the one or more illumination sources comprise one of a LED and OLED emitter.

13. The system of claim 11, wherein the device comprises a lens mounted thereto within the line of sight of the wearer, and wherein the one or more illumination sources comprise a plurality of light sources provided on the lens to illuminate the eye.

14. The system of claim 13, wherein the plurality of light sources are provided sufficiently spaced apart from one another to ensure that at least one glint from a light source appears on the surface of the corneal region of the eye in images from the eye-tracking camera when the eye moves through its complete normal range of motion.

15. The system of claim 11, wherein the illumination source comprises a plurality of reflective surfaces embedded in a lens mounted to the device in front of the wearer'"'"'s eye, and one or more light sources configured to direct light into the lens to be reflected from the plurality of reflective surfaces towards the eye.

16. The system of claim 11, further comprising a controller coupled to one or more light sources of the illumination source to modulate the brightness levels of the one or more light sources to provide desired brightness levels within a field-of-view of the eye-tracking camera.

17. The system of claim 11, further comprising a scene camera mounted on the device and positioned for viewing reference locations in the environment of the wearer, the processor coupled to the scene camera for acquiring scene images of the field of view and correlating the location of features of the eye relative to the one or more glints to determine locations being viewed by the wearer within the scene images.

18. The system of claim 11, wherein the features of the eye comprise the eye'"'"'s pupil.