Systems, devices, and methods for proximity-based eye tracking

• US 10,126,815 B2
• Filed: 12/11/2017
• Issued: 11/13/2018
• Est. Priority Date: 01/20/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Thalmic Labs Inc. (Alphabet Inc.)
• Inventors: Vidal, Melodie, Chapeskie, Jake
• Primary Examiner(s): Rayan, Mihir K
• Application Number:
  US15/837,243
• Publication Number:
  US 20180101230A1
• Time in Patent Office: 337 Days
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/10 using transmission of interrupted, pulse-modulated waves determination of distance by phase measurements G01S17/32
• CPC Class Codes: G01S 17/32 using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/46 Indirect determination of position data
• CPC Class Codes: G01S 17/58 Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 17/66 Tracking systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/88 Lidar systems specially adapted for specific applications
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

Systems, devices, and methods for proximity-based eye tracking are described. A proximity sensor positioned near the eye monitors the distance to the eye, which varies depending on the position of the corneal bulge. The corneal bulge protrudes outward from the surface of the eye and so, all other things being equal, a static proximity sensor detects a shorter distance to the eye when the cornea is directed towards the proximity sensor and a longer distance to the eye when the cornea is directed away from the proximity sensor. Optical proximity sensors that operate with infrared light are used as a non-limiting example of proximity sensors. Multiple proximity sensors may be used and processed simultaneously in order to provide a more accurate/precise determination of the gaze direction of the user. Implementations in which proximity-based eye trackers are incorporated into wearable heads-up displays are described.

Claims

1. A wearable heads-up display (“

WHUD”

) comprising;

a support frame that in use is worn on a head of a user;

a processor carried by the support frame;

a non-transitory processor-readable storage medium carried by the support frame; and

a proximity-based eye tracker carried by the support frame, wherein the proximity-based eye tracker comprises;

a first illumination source to illuminate at least a portion of an eye of a user with infrared light; and

a first photodetector to detect reflections of infrared light from the eye of the user;

and wherein the processor is communicatively coupled to at least the first photodetector and the non-transitory processor-readable storage medium stores data and/or instructions that, when executed by the processor, cause the processor to;

determine a distance between the first photodetector and the eye of the user based on reflections of infrared light from the eye of the user; and

determine a gaze direction of the user based on at least the distance between the first photodetector and the eye of the user.

2. The WHUD of claim 1 wherein the proximity-based eye tracker further comprises a second illumination source to illuminate at least a portion of the eye of the user with infrared light and a second photodetector to detect reflections of infrared light from the eye of the user, wherein:

the processor is communicatively coupled to the second photodetector; and

the non-transitory processor-readable storage medium stores data and/or instructions that, when executed by the processor, cause the processor to determine a distance between the second photodetector and the eye of the user based on reflections of infrared light from the eye of the user, and wherein the data and/or instructions that, when executed by the processor, cause the processor to determine a gaze direction of the user based on at least the distance between the first photodetector and the eye of the user cause the processor to determine the gaze direction of the user based on both the distance between the first photodetector and the eye of the user and the distance between the second photodetector and the eye of the user.

3. The WHUD of claim 1 wherein the first illumination source and the first photodetector are positioned within 1 cm of each other on the support frame.

4. The WHUD of claim 1 wherein the data and/or instructions that, when executed by the processor, cause the processor to determine a distance between the first photodetector and the eye of the user based on reflections of infrared light from the eye of the user cause the processor to determine a distance between the first photodetector and the eye of the user based on at least one property selected from a group consisting of:

intensity of reflections of infrared light from the eye of the user, power of reflections of infrared light from the eye of the user, luminance of reflections of infrared light from the eye of the user, and time of flight of reflections of infrared light from the eye of the user.

5. The WHUD of claim 1 wherein the proximity-based eye tracker further comprises a second photodetector to detect reflections of infrared light from the eye of the user, wherein:

the processor is communicatively coupled to the second photodetector; and

the non-transitory processor-readable storage medium further stores data and/or instructions that, when executed by the processor, cause the processor to;

determine a distance between the second photodetector and the eye of the user based on reflections of infrared light from the eye of the user; and

determine the gaze direction of the user based on both the distance between the first photodetector and the eye of the user and the distance between the second photodetector and the eye of the user.

6. The WHUD of claim 1, further comprising:

a first optic positioned proximate an output of the first illumination source in an optical path of infrared light emitted by the first illumination source, the first optic to shape infrared light emitted by first illumination source to a cone that illuminates the at least a portion of the eye of the user; and

a second optic positioned proximate an input of the first photodetector in an optical path of infrared light reflected from the eye of the user, the second optic to focus infrared light reflected by the at least a portion of the eye of the user on the first photodetector.

7. The WHUD of claim 1, further comprising:

a first optical filter positioned proximate the input of the first photodetector to transmit infrared light having a first wavelength through to the photodetector and block light having a wavelength other than the first wavelength from reaching the photodetector.

8. The WHUD of claim 1 wherein the data and/or instructions that, when executed by the processor, cause the processor to determine a gaze direction of the user based on at least the distance between the first photodetector and the eye of the user cause the processor to determine that the user is gazing in a direction towards the first photodetector when the distance between the first photodetector and the eye of the user is determined to be at or near a minimum value and that the user is gazing in a direction other than towards the first photodetector when the distance between the first photodetector and the eye of the user is determined to be at or near a maximum value.