Systems and methods for tracking the eye

• US 7,747,068 B1
• Filed: 01/20/2006
• Issued: 06/29/2010
• Est. Priority Date: 01/20/2006
• Status: Active Grant

• Current Assignee: United States Of America As Represented By The Secretary Of The Army (Government of the United States of America)
• Original Assignee: United States Of America As Represented By The Secretary Of The Army (Government of the United States of America)
• Inventors: Smyth, Andrew Paul, Smyth, Christopher Charles
• Primary Examiner(s): Kim; Charles
• Assistant Examiner(s): Patel; Nirav G
• Application Number:
  US11/340,821
• Time in Patent Office: 1,621 Days
• Field of Search: 382/154, 396/51, 351/209, 351/210, 600/558, 348/E13.041, 348/E13.047
• US Class Current: 382/154
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G03B 17/00 Details of cameras or camera bodies; Accessories therefor lens hoods or caps G03B11/04
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/514 from specularities
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

Disclosed are systems and methods for tracking the eye. In one embodiment, a method for tracking the eye includes acquiring stereo images of the eye using multiple sensors, isolating internal features of the eye in the stereo images acquired from the multiple sensors, and determining an eye gaze direction relative to the isolated internal features.

Claims

1. A method for tracking an eye in eye movement space, the method comprising:

providing a means for illuminating the eye;

providing a means comprising multiple sensors for acquiring stereo images of the eye from different spatial perspectives;

acquiring stereo images of the eye using multiple sensors;

isolating internal features of the eye from the stereo images acquired from the multiple sensors;

isolating specular reflection points on a corneal surface of the eye, the specular reflection points resulting from reflections of light emitted from a plurality of light sources sequentially used to illuminate the eye;

computing a center and radius of the spherical corneal surface using the specular reflection points;

calculating three-dimensional coordinates of the isolated internal features within the eye by performing ray tracing;

matching features across stereo images that are acquired by separate sensors, wherein the matching process makes use of the distribution of the features about apparent pupil elements common to both images

wherein performing ray tracing comprises computing feature locations from intersections of stereo-matched rays, wherein the stereo-matched rays have been derived using a refractive computational process; and

determining an eye gaze direction relative to the isolated internal features.

2. The method of claim 1, wherein acquiring stereo images comprises acquiring images from sensors which further comprise light collecting optics positioned on opposite sides of a central axis of sight within the eye movement space.

3. The method of claim 1, wherein isolating internal features of the eye comprises isolating one or more of a pupil centroid, sphincteral cusps of an iris, and a capillary pattern of a retinal fundus.

4. The method of claim 1, wherein determining an eye gaze direction comprises tabulating image locations of the isolated internal features from the stereo images and estimating a visual axis of the eye from the tabulated image locations using an algorithm and calibration parameters derived from an initial calibration process.

5. The method of claim 1, wherein calculating three-dimensional coordinates of the isolated internal features comprises calculating three-dimensional coordinates of a pupil centroid, sphincteral cusps of an iris, and a capillary pattern of a retinal fundus.

6. The method of claim 1, wherein performing ray tracing comprises computing feature locations from intersections of back-traced rays and feature spheres.

7. The method of claim 1, wherein the manner in which ray tracing is performed depends upon the stereo images that are acquired.

8. A system for tracking an eye movement space, the system comprising:

means for illuminating the eye;

means for acquiring stereo images of the eye from different spatial perspectives;

means for isolating specular reflections on a corneal surface of the eye in the stereo images;

means for isolating internal features of the eye in the stereo images; and

means for determining an eye gaze direction relative to the isolated internal features, wherein the means for determining an eye gaze direction comprise a ray tracer that calculates a center and radius of the spherical corneal surface using the specular reflection points, and the coordinates of the isolated internal features of the eye relative to a computed corneal surface using ray-tracing techniques, and a stereo reconstructor that matches isolated internal features across stereo images acquired by separate sensors using a best fit feature vector approach; and

wherein the ray tracer computes feature locations from intersections of stereo-matched rays, wherein the stereo-matched rays have been derived using a refractive computational process.

9. The system of claim 8, wherein the means for acquiring stereo images comprise separate sensors which further comprise light collecting optics positioned on opposite sides of a central axis of sight within the eye movement space.

10. The system of claim 8, wherein the means for isolating specular reflections comprise an opto-electronic device that receives image signals from the means for acquiring stereo images, said opto-electronic device further comprising means for electronically representing spatially optical corneal reflection,

transmitted from said optical device.

11. The system of claim 8, wherein the means for isolating internal features of the eye comprise an image processor.

12. The system of claim 8, wherein the means for determining an eye gaze direction comprise a digital processor that estimates a visual axis from the isolated internal features using an algorithm with calibration parameters derived from an initial calibration procedure.

13. The system of claim 8, wherein the ray tracer computes feature locations from intersections of back-traced rays and feature spheres.

14. An eye gaze tracking system, comprising:

a light source that is configured to illuminate a user'"'"'s eye;

light sensors that are configured to be positioned at different locations relative to the eye and capture stereo images of the eye from different spatial perspectives;

an opto-electronic device that is configured to receive stereo images from the light sensors and isolate specular reflection points on a corneal surface of the eye that correspond to reflections of light emitted from the light source;

an image processor that is configured to isolate internal features of the eye, compute a center and radius of the spherical corneal surface using the specular reflection points, and calculate three-dimensional coordinates of the isolated internal features within the eye by performing ray tracing;

a stereo reconstructor that is configured to match internal features isolated by the image processor across stereo images acquired by separate sensors, wherein the matching process makes use of the distribution of the features about apparent pupil elements common to both images; and

a ray tracer that is configured to calculate the coordinates of the isolated internal features of the eye relative to a computed corneal surface from intersections of stereo-matched rays, wherein the stereo-matched rays have been derived using a refractive computational process.

15. The system of claim 14, wherein the light source comprises a plurality of discrete light-emitting elements, operationally connected to the said opto-electronic processor.

16. The system of claim 14, wherein the light sensors further comprise light collecting optics positioned on opposite sides of a central axis of sight within the eye movement space.

17. The system of claim 14, wherein the image processor is configured to isolate a pupil centroid, sphincteral cusps of an iris, and a capillary network of a retinal fundus of the eye.

18. The system of claim 14, wherein the ray tracer is configured to compute feature locations from intersections of back-traced rays and feature spheres.

19. The system of claim 14, wherein the ray tracer automatically computes the feature locations from intersections of stereo-matched rays if the stereo reconstructor is able to match isolated internal features across stereo images, and automatically computes feature locations from intersection of back-traced rays and feature spheres if the stereo reconstructor is not able to match isolated internal features across stereo images, such that the system dynamically alters the ray-tracing process upon the image data that is available.