Dual LED usage for glint detection

• US 8,971,570 B1
• Filed: 11/04/2011
• Issued: 03/03/2015
• Est. Priority Date: 11/04/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Raffle, Hayes Solos, Wheeler, Aaron Joseph
• Primary Examiner(s): Fitzpatrick, Atiba O
• Assistant Examiner(s): Wang, Carol
• Application Number:
  US13/289,721
• Time in Patent Office: 1,215 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

A wearable computing system may include an eye-tracking system configured to track the position of an eye of a wearer of the wearable computing system. In particular, an infrared light source illuminating the eye of a wearer at a relatively high intensity may generate specular reflections off the wearer'"'"'s cornea, also called ‘glints’. The glints can be imaged with an infrared camera. When the infrared light sources are illuminated at a relatively lower intensity, determination of the pupil location is possible. Glints, in combination with the pupil location, may be used to accurately determine the gaze direction and eye rotation. The determined gaze direction could be used in various eye-tracking applications. By controlling the light sources to change intensity levels and by combining multiple images of the eye to incorporate multiple glint locations with the pupil location, eye tracking can be performed with better accuracy and with fewer light sources.

Claims

1. A method, comprising:

illuminating a viewing location during a first period of time with at least a first light source and a second light source, such that during the first period of time the first and second light sources both illuminate the viewing location and the first light source illuminates the viewing location with a higher intensity than the second light source;

acquiring a first image from light reflected from the viewing location during the first period of time using a camera, wherein the first and second light sources are located off-axis with respect to an optical axis of the camera;

illuminating the viewing location during a second period of time with at least the first light source and the second light source, such that during the second period of time the first and second light sources both illuminate the viewing location and the second light source illuminates the viewing location with a higher intensity than the first light source;

acquiring a second image from light reflected from the viewing location during the second period of time using the camera;

obtaining combined data from at least the first and the second images, wherein the combined data comprises a reference location, a first glint location, and a second glint location; and

determining a gaze direction based on the combined data.

2. The method of claim 1, wherein the first and the second light sources comprise infrared light-emitting diodes.

3. The method of claim 2, wherein the camera is an infrared camera.

4. The method of claim 3, wherein the first image is acquired by the infrared camera throughout the first period of time and the second image is acquired by the infrared camera throughout the second period of time.

5. The method of claim 1, wherein the viewing location corresponds to a position of an eye of a wearer of a head-mounted display (HMD).

6. The method of claim 5, wherein the reference location corresponds to a pupil location of the HMD wearer.

7. The method of claim 5, wherein the first and the second glint locations correspond to locations on the eye of the HMD wearer.

8. The method of claim 5, wherein the first glint location corresponds to a reflection off the cornea from the first light source and wherein the second glint location corresponds to a reflection off the cornea from the second light source.

9. The method of claim 5, wherein the gaze direction corresponds to a direction in which the HMD wearer is looking.

10. A wearable computing system, comprising:

at least a first and a second light source, wherein the first and the second light sources are configured to illuminate a viewing location during a first period of time, such that during the first period of time the first and second light sources both illuminate the viewing location and the first light source illuminates the viewing location with a higher intensity than the second light source, and wherein the first and the second light sources are configured to illuminate the viewing location during a second period of time, such that during the second period of time the first and second light sources both illuminate the viewing location and the second light source illuminates the viewing location with a higher intensity than the first light source;

a camera, wherein the camera is configured to acquire a first image from light reflected from the viewing location during the first period of time, wherein the camera is further configured to acquire a second image from light reflected from the viewing location during the second period of time, and wherein the first and second light sources are located off-axis with respect to an optical axis of the camera; and

a computer, wherein the computer is configured to obtain combined data from at least the first and the second images, wherein the combined data comprises a reference location, a first glint location, and a second glint location, and wherein the computer is further configured to determine a gaze direction based on the combined data.

11. The wearable computing system of claim 10, wherein the first and the second light sources comprise infrared light-emitting diodes.

12. The wearable computing system of claim 11, wherein the camera is an infrared camera.

13. The wearable computing system of claim 12, wherein the first image is acquired by the infrared camera throughout the first period of time and the second image is acquired by the infrared camera throughout the second period of time.

14. The wearable computing system of claim 10, wherein the viewing location corresponds to a position of an eye of a wearer of a head-mounted display (HMD).

15. The wearable computing system of claim 14, wherein the reference location corresponds to a pupil location of the HMD wearer.

16. The wearable computing system of claim 14, wherein the first and the second glint locations correspond to locations on the eye of the HMD wearer.

17. The wearable computing system of claim 14, wherein the first glint location corresponds to a reflection off the cornea from the first light source and wherein the second glint location corresponds to a reflection off the cornea from the second light source.

18. The wearable computing system of claim 14, wherein the gaze direction corresponds to a direction in which the HMD wearer is looking.

19. A non-transitory computer readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions, the functions comprising:

controlling at least a first and a second light source to illuminate a viewing location during a first period of time, such that during the first period of time the first and second light sources both illuminate the viewing location and the first light source illuminates the viewing location with a higher intensity than the second light source;

controlling a camera to acquire a first image from light reflected from the viewing location during the first period of time, wherein the first and second light sources are located off-axis with respect to an optical axis of the camera;

controlling at least the first and the second light sources to illuminate the viewing location during a second period of time, such that during the second period of time the first and second light sources both illuminate the viewing location and the second light source illuminates the viewing location with a higher intensity than the first light source;

controlling the camera to acquire a second image from light reflected from the viewing location during the second period of time;

obtaining combined data from at least the first and the second images, wherein the combined data comprises a reference location, a first glint location, and a second glint location; and

determining a gaze direction based on the combined data.

20. The non-transitory computer readable medium of claim 19, wherein the first and the second light sources comprise infrared light-emitting diodes.

21. The non-transitory computer readable medium of claim 20, wherein the camera is an infrared camera.

22. The non-transitory computer readable medium of claim 21, wherein the first image is acquired by the infrared camera throughout the first period of time and the second image is acquired by the infrared camera throughout the second period of time.

23. The non-transitory computer readable medium of claim 19, wherein the viewing location corresponds to a position of an eye of a wearer of a head-mounted display (HMD).

24. The non-transitory computer readable medium of claim 23, wherein the reference location corresponds to a pupil location of the HMD wearer.

25. The non-transitory computer readable medium of claim 23, wherein the first and the second glint locations correspond to locations on the eye of the HMD wearer.

26. The non-transitory computer readable medium of claim 23, wherein the first glint location corresponds to a reflection off the cornea from the first light source and wherein the second glint location corresponds to a reflection off the cornea from the second light source.

27. The non-transitory computer readable medium of claim 23, wherein the gaze direction corresponds to a direction in which the HMD wearer is looking.