Method and system for input detection using structured light projection

• US 8,955,973 B2
• Filed: 09/28/2012
• Issued: 02/17/2015
• Est. Priority Date: 01/06/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Raffle, Hayes Solos, Starner, Thad Eugene, Weaver, Josh, Keyes, Edward Allen
• Primary Examiner(s): Pham, Thomas K
• Assistant Examiner(s): PASKO, NICHOLAS R
• Application Number:
  US13/631,594
• Publication Number:
  US 20130176533A1
• Time in Patent Office: 872 Days
• Field of Search: 351/209, 351/210
• US Class Current: 351/209
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61F 2009/00846 Eyetracking
• CPC Class Codes: G01B 11/167 by projecting a pattern on the object
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

Exemplary methods and systems help provide for tracking an eye. An exemplary method may involve: causing the projection of a pattern onto an eye, wherein the pattern comprises at least one line, and receiving data regarding deformation of the at least one line of the pattern. The method further includes correlating the data to iris, sclera, and pupil orientation to determine a position of the eye, and causing an item on a display to move in correlation with the eye position.

Claims

1. A method comprising:

a light projector projecting a pattern of light onto an eye, wherein the pattern comprises at least one line;

receiving data regarding a first deformation of the at least one line of the pattern at a first time, wherein the first deformation of the at least one line comprises the line bowing outward when projected over the sclera or the line bowing inward when projected over the iris;

receiving data regarding a second deformation of the at least one line of the pattern at a second time that is subsequent to the first time, wherein the second deformation of the at least one line of the pattern comprises the line bowing outward relative to the first deformation of the at least one line or the line bowing inward relative to the first deformation of the at least one line; and

correlating the data to at least one of iris, sclera, and pupil orientation to determine a position of the eye.

4. The method of claim 1, wherein the pattern comprises a plurality of vertical and horizontal lines.

5. The method of claim 1, wherein at least one line is not visible over the pupil.

6. The method of claim 1, wherein correlating further comprises matching a deformed portion of the line that is bowed outward as being the portion of the line that is over the sclera.

7. The method of claim 1, wherein correlating further comprises matching a deformed portion of the line that is bowed inward as being the portion of the line that is over the iris.

8. The method of claim 1, further comprising monitoring eye movement with one or more cameras.

9. The method of claim 1, further comprising causing an item on a display to move in correlation with the eye position.

10. The method of claim 1, further comprising:

modeling glints caused by an ambient environment;

determining where the glints are relative to the eye;

predicting how the glints will change as the eye moves using data concerning the dynamics of the eye; and

combining the prediction with the correlation of the pattern and sclera, iris, and pupil data to track the eye.

2. A system comprising:

a non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer-readable medium which, when executed by at least one processor, cause the system to;

cause a light projector to project a pattern of light onto an eye, wherein the pattern comprises at least one line;

receive data regarding a first deformation of the at least one line of the pattern at a first time, wherein the first deformation of the at least one line comprises the line bowing outward when projected over the sclera or the line bowing inward when projected over the iris;

receive data regarding a second deformation of the at least one line of the pattern at a second time that is subsequent to the first time, wherein the second deformation of the at least one line of the pattern comprises the line bowing outward relative to the first deformation of the at least one line or the line bowing inward relative to the first deformation of the at least one line;

correlate the data to iris, sclera, and pupil orientation to determine the position of the eye; and

cause an item on a display to move in correlation with the eye position.

11. The system of claim 2, wherein the pattern comprises a plurality of vertical and horizontal lines.

12. The system of claim 2, wherein the at least one line is not visible over the pupil.

13. The system of claim 2, wherein correlating further comprises matching a deformed portion of the line that is bowed outward as being the portion of the line that is over the sclera.

14. The system of claim 2, wherein correlating further comprises matching a deformed portion of the line that is bowed inward as being the portion of the line that is over the iris.

15. The system of claim 2, wherein the system is a wearable computer and the display is a head-mounted display.

16. The system of claim 12, wherein the system further provides a position and an orientation to the processor.

3. A method comprising:

a light projector projecting a pattern of light onto an eye, wherein the pattern comprises at least one line;

receiving data regarding a first deformation of the at least one line of the pattern at a first time, wherein the first deformation of the at least one line comprises the line bowing outward when projected over the sclera or the line bowing inward when projected over the iris;

receiving data regarding a second deformation of the at least one line of the pattern at a second time that is subsequent to the first time, wherein the second deformation of the at least one line of the pattern comprises the line bowing outward relative to the first deformation of the at least one line or the line bowing inward relative to the first deformation of the at least one line;

correlating the data to iris, sclera, and pupil orientation, combined with eye dynamics, to determine a position of the eye; and

causing an item on a display to move in correlation with the position.