SYSTEMS, DEVICES, AND METHODS FOR PROXIMITY-BASED EYE TRACKING

• US 20170205876A1
• Filed: 01/20/2017
• Published: 07/20/2017
• Est. Priority Date: 01/20/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Thalmic Labs Inc. (Alphabet Inc.)
• Inventors: Vidal, Mlodie, Chapeskie, Jake
• Granted Patent: US 10,303,246 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/10 using transmission of interrupted, pulse-modulated waves determination of distance by phase measurements G01S17/32
• CPC Class Codes: G01S 17/32 using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/46 Indirect determination of position data
• CPC Class Codes: G01S 17/58 Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 17/66 Tracking systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/88 Lidar systems specially adapted for specific applications
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

Systems, devices, and methods for proximity-based eye tracking are described. A proximity sensor positioned near the eye monitors the distance to the eye, which varies depending on the position of the corneal bulge. The corneal bulge protrudes outward from the surface of the eye and so, all other things being equal, a static proximity sensor detects a shorter distance to the eye when the cornea is directed towards the proximity sensor and a longer distance to the eye when the cornea is directed away from the proximity sensor. Optical proximity sensors that operate with infrared light are used as a non-limiting example of proximity sensors. Multiple proximity sensors may be used and processed simultaneously in order to provide a more accurate/precise determination of the gaze direction of the user. Implementations in which proximity-based eye trackers are incorporated into wearable heads-up displays are described.

Claims

1. A proximity-based eye tracker comprising:

a first illumination source to illuminate at least a portion of an eye of a user with infrared light;

a first photodetector to detect reflections of infrared light from the eye of the user;

a processor communicatively coupled to at least the first photodetector; and

a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores data and/or instructions that, when executed by the processor, cause the processor to;

determine a distance between the first photodetector and the eye of the user based on reflections of infrared light from the eye of the user; and

determine a gaze direction of the user based on at least the distance between the first photodetector and the eye of the user.

2. The proximity-based eye tracker of claim 1, further comprising:

a second illumination source to illuminate at least a portion of the eye of the user with infrared light; and

a second photodetector to detect reflections of infrared light from the eye of the user, wherein;

the processor is communicatively coupled to the second photodetector; and

the non-transitory processor-readable storage medium stores data and/or instructions that, when executed by the processor, cause the processor to;

determine a distance between the second photodetector and the eye of the user based on reflections of infrared light from the eye of the user, and wherein the data and/or instructions that, when executed by the processor, cause the processor to determine a gaze direction of the user based on at least the distance between the first photodetector and the eye of the user cause the processor to determine the gaze direction of the user based on both the distance between the first photodetector and the eye of the user and the distance between the second photodetector and the eye of the user.

3. The proximity-based eye tracker of claim 2, further comprising:

at least one additional illumination source to illuminate at least a portion of the eye of the user with infrared light; and

at least one additional photodetector to detect reflections of infrared light from the eye of the user, wherein;

the processor is communicatively coupled to the at least one additional photodetector; and

the non-transitory processor-readable storage medium stores data and/or instructions that, when executed by the processor, cause the processor to;

determine a distance between the at least one additional photodetector and the eye of the user based on reflections of infrared light from the eye of the user, and wherein the data and/or instructions that, when executed by the processor, cause the processor to determine the gaze direction of the user based on both the distance between the first photodetector and the eye of the user and the distance between the second photodetector and the eye of the user cause the processor to determine the gaze direction of the user based on;

the distance between the first photodetector and the eye of the user, the distance between the second photodetector and the eye of the user, and the distance between the at least one additional photodetector and the eye of the user.

4. The proximity-based eye tracker of claim 1, further comprising:

a support frame that in use is worn on a head of the user, wherein the first illumination source and the first photodetector are both mounted on the support frame, the first illumination source positioned to illuminate at least a portion of the eye of the user with infrared light when the support frame is worn on the head of the user and the first photodetector positioned to detect reflections of infrared light from the eye of the user when the support frame is worn on the head of the user.

5. The proximity-based eye tracker of claim 4 wherein the first illumination source and the first photodetector are positioned within 1 cm of each other on the support frame.

6. The proximity-based eye tracker of claim 1 wherein the data and/or instructions that, when executed by the processor, cause the processor to determine a distance between the first photodetector and the eye of the user based on reflections of infrared light from the eye of the user cause the processor to determine a distance between the first photodetector and the eye of the user based on at least one property selected from a group consisting of:

intensity of reflections of infrared light from the eye of the user, power of reflections of infrared light from the eye of the user, luminance of reflections of infrared light from the eye of the user, and time of flight of reflections of infrared light from the eye of the user.

7. The proximity-based eye tracker of claim 1, further comprising:

a second photodetector to detect reflections of infrared light from the eye of the user, wherein;

the processor is communicatively coupled to the second photodetector; and

the non-transitory processor-readable storage medium further stores data and/or instructions that, when executed by the processor, cause the processor to;

determine a distance between the second photodetector and the eye of the user based on reflections of infrared light from the eye of the user; and

determine the gaze direction of the user based on both the distance between the first photodetector and the eye of the user and the distance between the second photodetector and the eye of the user.

8. The proximity-based eye tracker of claim 1, further comprising:

a first optic positioned proximate an output of the first illumination source in an optical path of infrared light emitted by the first illumination source, the first optic to shape infrared light emitted by first illumination source to a cone that illuminates the at least a portion of the eye of the user; and

a second optic positioned proximate an input of the first photodetector in an optical path of infrared light reflected from the eye of the user, the second optic to focus infrared light reflected by the at least a portion of the eye of the user on the first photodetector.

9. The proximity-based eye tracker of claim 1, further comprising:

a first optical filter positioned proximate the input of the first photodetector to transmit infrared light having a first wavelength through to the photodetector and block light having a wavelength other than the first wavelength from reaching the photodetector.

10. The proximity-based eye tracker of claim 1 wherein the data and/or instructions that, when executed by the processor, cause the processor to determine a gaze direction of the user based on at least the distance between the first photodetector and the eye of the user cause the processor to determine that the user is gazing in a direction towards the first photodetector when the distance between the first photodetector and the eye of the user is determined to be at or near a minimum value and that the user is gazing in a direction other than towards the first photodetector when the distance between the first photodetector and the eye of the user is determined to be at or near a maximum value.

11. A method of determining a gaze direction of a user, the method comprising:

illuminating at least a portion of an eye of the user with infrared light by a first illumination source;

detecting reflections of infrared light from the eye of the user by a first photodetector;

determining, by a processor communicatively coupled to at least the first photodetector, a distance between the first photodetector and the eye of the user based on reflections of infrared light from the eye of the user detected by the first photodetector; and

determining, by the processor, the gaze direction of the user based on at least the distance between the first photodetector and the eye of the user.

12. The method of claim 11, further comprising:

illuminating at least a portion of the eye of the user with infrared light by a second illumination source;

detecting reflections of infrared light from the eye of the user by a second photodetector, wherein the processor is communicatively coupled to the second photodetector; and

determining, by the processor, a distance between the second photodetector and the eye of the user based on reflections of infrared light from the eye of the user detected by the second photodetector, wherein determining, by the processor, the gaze direction of the user based on at least the distance between the first photodetector and the eye of the user includes determining, by the processor, the gaze direction of the user based on both the distance between the first photodetector and the eye of the user and at least the distance between the second photodetector and the eye of the user.

13. The method of claim 12, further comprising:

illuminating at least a portion of the eye of the user with infrared light by at least one additional illumination source;

detecting reflections of infrared light from the eye of the user by at least one additional second photodetector, wherein the processor is communicatively coupled to the at least one additional photodetector; and

determining, by the processor, a distance between the at least one additional photodetector and the eye of the user based on reflections of infrared light from the eye of the user detected by the at least one additional photodetector, wherein determining, by the processor, the gaze direction of the user based on both the distance between the first photodetector and the eye of the user and at least the distance between the second photodetector and the eye of the user includes determining, by the processor, the gaze direction of the user based on;

the distance between the first photodetector and the eye of the user, the distance between the second photodetector and the eye of the user, and the distance between the at least one additional photodetector and the eye of the user.

14. The method of claim 11 wherein the processor is communicatively coupled to a non-transitory processor-readable storage medium that stores data and/or instructions that, when executed by the processor, cause the processor to:

determine the distance between the first photodetector and the eye of the user based on reflections of infrared light from the eye of the user detected by the first photodetector; and

determine the gaze direction of the user based on at least the distance between the first photodetector and the eye of the user.

15. The method of claim 11 wherein determining, by a processor communicatively coupled to at least the first photodetector, a distance between the first photodetector and the eye of the user based on reflections of infrared light from the eye of the user detected by the first photodetector includes determining, by the processor, the distance between the first photodetector and the eye of the user based on at least one property selected from a group consisting of:

an intensity of reflections of infrared light from the eye of the user detected by the first photodetector, a power of reflections of infrared light from the eye of the user, a luminance of reflections of infrared light from the eye of the user, and a time of flight of reflections of infrared light from the eye of the user.

16. The method of claim 11 wherein determining, by the processor, the gaze direction of the user based on at least the distance between the first photodetector and the eye of the user includes determining, by the processor, that the user is gazing in a direction towards the first photodetector when the distance between the first photodetector and the eye of the user is determined to be at or near a minimum value and that the user is gazing in a direction other than towards the first photodetector when the distance between the first photodetector and the eye of the user is determined to be at or near a maximum value.

17. A proximity-based eye tracker comprising:

a number X≧

1 of illumination sources, each to illuminate at least a portion of an eye of a user with infrared light;

a number Y≧

1 of photodetectors, each to detect reflections of infrared light from the eye of the user;

a processor communicatively coupled to at least each of the Y photodetectors; and

a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores data and/or instructions that, when executed by the processor, cause the processor to;

determine a respective distance between at least a subset of the Y photodetectors and the eye of the user based on reflections of infrared light from the eye of the user detected by the Y photodetectors; and

determine a gaze direction of the user based on at the respective distance between each of the at least a subset of the Y photodetectors and the eye of the user.

18. The proximity-based eye tracker of claim 17 wherein X=Y.

19. The proximity-based eye tracker of claim 17 wherein X>

Y.

20. The proximity-based eye tracker of claim 17 wherein X<

Y.

21. A wearable heads-up display comprising:

a support frame that in use is worn on a head of a user;

a processor carried by the support frame;

a non-transitory processor-readable storage medium carried by the support frame; and

a proximity-based eye tracker carried by the support frame, wherein the proximity-based eye tracker comprises;

a first illumination source to illuminate at least a portion of an eye of a user with infrared light; and

a first photodetector to detect reflections of infrared light from the eye of the user;

and wherein the processor is communicatively coupled to at least the first photodetector and the non-transitory processor-readable storage medium stores data and/or instructions that, when executed by the processor, cause the processor to;

determine a distance between the first photodetector and the eye of the user based on reflections of infrared light from the eye of the user; and

determine a gaze direction of the user based on at least the distance between the first photodetector and the eye of the user.