SYSTEMS, DEVICES, AND METHODS FOR LASER EYE TRACKING

• US 20170115483A1
• Filed: 10/21/2016
• Published: 04/27/2017
• Est. Priority Date: 10/23/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Thalmic Labs Inc.
• Inventors: Aleem, Idris S., Chapeskie, Jake, Vidal, Mlodie
• Granted Patent: US 9,904,051 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 26/101 with both horizontal and vertical deflecting means, e.g. raster or XY scanners colour television using laser beams scanning a display screen H04N9/3129
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 5/32 Holograms used as optical elements
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris

Abstract

Systems, devices, and methods for laser eye tracking are described. Laser eye tracking involves scanning laser light over the eye and detecting diffuse reflections of the laser light with one or more photodetector(s). While conventional camera-based eye tracking techniques rely on detecting and identifying specific reflections (i.e., Purkinje images such as the “glint”), the laser eye tracking techniques described herein detect and identify a reduction in reflection intensity due to transmission of laser light through the pupil and/or increased diffusivity of reflections from the cornea relative to reflections from the sclera. This effect is referred to herein as the “corneal shadow” effect. Laser eye tracking uses considerably less power than conventional camera-based eye tracking techniques. As a result, the laser eye tracking techniques described herein do not add significant power demands to a system and can be implemented in wearable heads-up displays without adding significant volume to the system battery.

Claims

1. A method of determining a gaze direction of a user, the method comprising:

scanning a laser light over at least a portion of an eye of the user by a scanning laser device, wherein the portion of the eye of the user includes at least a portion of a cornea of the eye of the user;

detecting reflections of the laser light from the eye by at least one photodetector;

determining a relative intensity of each reflection by a processor;

identifying, by the processor, at least one region of the eye that corresponds to a relatively low-intensity reflection;

determining, by the processor, a position of the cornea based on the at least one region of the eye that corresponds to the relatively low-intensity reflection; and

determining, by the processor, the gaze direction of the user based on the position of the cornea.

2. The method of claim 1 wherein scanning a laser light over at least a portion of an eye of the user by a scanning laser device includes scanning the laser light in two dimensions over at least a portion of the eye of the user by the scanning laser device.

3. The method of claim 1 wherein scanning a laser light over at least a portion of an eye of the user by a scanning laser device includes scanning the laser light over at least a portion of a holographic optical element by the scanning laser device and redirecting the laser light over the at least a portion of the eye of the user by the holographic optical element.

4. The method of claim 1 wherein scanning a laser light over at least a portion of an eye of the user by a scanning laser device includes scanning an infrared laser light over at least a portion of the eye of the user by the scanning laser device.

5. The method of claim 1 wherein detecting reflections of the laser light from the eye by at least one photodetector includes detecting diffuse reflections of the laser light from the eye by at least one photodetector.

6. The method of claim 1 wherein detecting reflections of the laser light from the eye by at least one photodetector includes detecting reflections of the laser light from the eye by at least two photodetectors.

7. The method of claim 1, further comprising:

mapping each reflection to a respective region of the eye by the processor.

8. The method of claim 7 wherein mapping each reflection to a respective region of the eye by the processor includes mapping at least one relatively low-intensity reflection to a region of the eye, and wherein determining, by the processor, a position of the cornea based on the at least one region of the eye that corresponds to the relatively low-intensity reflection includes identifying, by the processor, a position of the cornea based on mapping at least one relatively low-intensity reflection to a region of the eye.

9. The method of claim 7 wherein mapping each reflection to a respective region of the eye by the processor includes associating each reflection with a corresponding scan parameter of the scanning laser device by the processor.

10. The method of claim 9 wherein associating each reflection with a corresponding scan parameter of the scanning laser device by the processor includes associating, by the processor, each reflection with a corresponding scan parameter selected from the group consisting of:

a position of at least one scan mirror of the scanning laser device, respective positions of at least two scan mirrors of the scanning laser device, a projection angle of the scanning laser device, and a modulation state of the scanning laser device.

11. The method of claim 1 wherein the scanning laser device, the photodetector, and the processor are respective components of a head-mounted display that further includes a non-transitory processor-readable storage medium, and wherein the non-transitory processor-readable storage medium stores data and/or instructions that, when executed by the processor, cause the processor to:

determine the relative intensity of each reflection;

identify the at least one region of the eye that corresponds to the relatively low-intensity reflection;

determine the position of the cornea based on the at least one region of the eye that corresponds to the relatively low-intensity reflection; and

determine the gaze direction of the user based on the position of the cornea.

12. A laser eye tracker comprising:

a scanning laser device to scan a laser light over at least a portion of an eye of a user, wherein the portion of the eye of the user includes at least a portion of a cornea of the eye of the user;

at least one photodetector;

a processor communicatively coupled to both the scanning laser device and the at least one photodetector; and

a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores data and/or instructions that, when executed by the processor, cause;

the at least one photodetector to detect reflections of the laser light from the eye; and

the processor to;

determine a relative intensity of each reflection;

identify at least one region of the eye that corresponds to a relatively low-intensity reflection;

determine a position of the cornea based on the at least one region of the eye that corresponds to the relatively low-intensity reflection; and

determine a gaze direction of the user based on the position of the cornea.

13. The laser eye tracker of claim 12 wherein the scanning laser device includes an infrared laser diode, the laser light includes infrared laser light, and the at least one photodetector includes at least one infrared photodetector.

14. The laser eye tracker of claim 12 wherein the scanning laser device, the photodetector, the processor, and the non-transitory processor-readable storage medium are all components of and carried by a head-mounted display.

15. The laser eye tracker of claim 14, further comprising:

a holographic optical element carried by the head-mounted display and positioned in a field of view of the eye of the user to redirect laser light scanned by the scanning laser device towards the eye of the user.

16. The laser eye tracker of claim 12 wherein the at least one photodetector includes at least two photodetectors.

17. A wearable heads-up display comprising:

a support frame that in use is worn on a head of a user;

a scanning laser device carried by the support structure, the scanning laser device including an infrared laser diode to output an infrared light and at least one visible light laser diode to output a visible light;

a wavelength-multiplexed holographic optical element carried by the support frame and positioned within a field of view of at least one eye of the user when the support frame is worn on the head of the user, the wavelength-multiplexed holographic optical element aligned to receive both the infrared light and the visible light reflected from the scanning laser device and to redirect both the infrared light and the visible light towards the at least one eye of the user when the support frame is worn on the head of the user, wherein the wavelength-multiplexed holographic optical element includes a first hologram that is reflective of the visible light and transmissive of the infrared light and a second hologram that is reflective of the infrared light and transmissive of the visible light, and wherein the wavelength-multiplexed holographic optical element is substantially transparent to environmental light;

an infrared photodetector carried by the support frame;

a processor communicatively coupled to both the scanning laser device and the infrared photodetector; and

a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores data and/or instructions that, when executed by the processor, cause;

the scanning laser device to scan a laser light over at least a portion of an eye of the user, wherein the laser light includes infrared laser light and the portion of the eye of the user includes at least a portion of a cornea of the eye of the user;

the at least one infrared photodetector to detect reflections of the infrared laser light from the eye; and

the processor to;

determine a relative intensity of each reflection;

identify at least one region of the eye that corresponds to a relatively low-intensity reflection;

determine a position of the cornea based on the at least one region of the eye that corresponds to the relatively low-intensity reflection; and

determine a gaze direction of the user based on the position of the cornea.