Calibration-free eye gaze tracking

• US 6,578,962 B1
• Filed: 04/27/2001
• Issued: 06/17/2003
• Est. Priority Date: 04/27/2001
• Status: Expired due to Term

• Current Assignee: Tobii Technology AB
• Original Assignee: International Business Machines Corporation
• Inventors: Amir, Arnon, Russell, Gregory Fraser, Flickner, Myron Dale, Koons, David Bruce
• Primary Examiner(s): Lateef; Marvin M.
• Assistant Examiner(s): SANDERS JR, JOHN R
• Application Number:
  US09/844,682
• Publication Number:
  US 20030098954A1
• Time in Patent Office: 781 Days
• Field of Search: 351/209,210,208,200,246,206,205,211,221 396/51 345/156,157,158,973
• US Class Current: 351/209
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

A system and method for calibration-free tracking of a user'"'"'s eye gaze vector and point of regard even if substantial head movement or rotation occurs. The preferred embodiment includes two synchronized interlaced cameras, each viewing the user'"'"'s eye and having on-axis lighting that is alternately modulated. An image difference between lighted and unlighted images of the eye is used to identify a user'"'"'s pupil. A plane containing the gaze vector is defined by rotating a base plane through the angle in a camera image plane between a pupil center, a first glint, and a second glint. The intersection of two such planes (one from each camera), defines the gaze vector. The gaze position is the intersection of the gaze vector with the object being viewed by the user. Alternate embodiments are also described.

Claims

1. A method for eye gaze tracking, comprising the steps of:

focusing at least one camera upon at least one of a user'"'"'s eyes, each said camera having a focal center, an image plane, and a co-located light source emitting light toward said eye;

identifying and locating image aspects including at least one glint and a pupil image in said image plane; and

computing a gaze vector from at least one plane generated from said image aspects and camera position and orientation data.

2. The method of claim 1 wherein said method is adapted for use with an animal.

3. The method of claim 1 wherein said method is adapted for use with a person.

4. The method of claim 1 wherein said method is adapted for use with a baby.

5. The method of claim 1 comprising the further step of locating a point of regard as the intersection of said gaze vector with a predetermined surface.

7. The method of claim 1 comprising the further step of correcting said gaze vector for a foveal axis offset angle.

6. A method for eye gaze tracking, comprising the steps of:

focusing at least one camera upon at least one of a user'"'"'s eyes, each said camera having a focal center, an image plane, and a co-located light source emitting light toward said eye;

identifying and locating image aspects including at least one glint and a pupil image in said image plane;

computing a gaze vector from at least one plane generated from said image aspects and camera position and orientation data;

synchronizing scanning signals controlling each one of said at least one cameras; and

responsively alternately energizing said light sources to identify correspondences between said light sources and said glints.

8. A method for eye gaze tracking, comprising the steps of:

focusing at least one camera upon at least one of a user'"'"'s eyes, each said camera having a focal center, an image plane, and a co-located light source emitting light toward said eye;

identifying and locating image aspects including at least one glint and a pupil image in said image plane;

computing a gaze vector from at least one plane generated from said image aspects and camera position and orientation data;

determining an angle between said glint in said image plane, said focal center, and a center of said pupil image in said image plane;

finding a separation on said eye between said glint and said pupil center using said angle and a distance estimate between said eye and a point of regard;

defining a second angle between said focal center, a corneal curvature center, and said pupil center using a radius of corneal curvature to define said gaze vector; and

locating said point of regard at the intersection of said gaze vector with a predetermined surface.

9. A method for eye gaze tracking, comprising the steps of:

focusing at least one camera upon at least one of a user'"'"'s eyes, each said camera having a focal center, an image plane, and a co-located light source emitting light toward said eye;

identifying and locating image aspects including at least one glint and a pupil image in said image plane;

computing a gaze vector from at least one plane generated from said image aspects and camera position and orientation data;

defining for each one of a plurality of said at least one cameras a particular plane spanning said glint in said image plane, said focal center, and a center of said pupil image in said image plane; and

identifying an intersection line of said particular planes as said gaze vector.

10. A method for eye gaze tracking, comprising the steps of:

focusing at least one camera upon at least one of a user'"'"'s eyes, each said camera having a focal center, an image plane, and a co-located light source emitting light toward said eye;

identifying and locating image aspects including at least one glint and a pupil image in said image plane;

computing a gaze vector from at least one plane generated from said image aspects and camera position and orientation data;

for each one of a plurality of said at least one cameras, defining in said image plane an angle spanning a center of said pupil image, a first glint, and a second glint, wherein said first glint results from said light source on each said one camera and said second glint results from another light source;

defining a base plane spanning said focal center for each said one camera and said focal center for each other camera in said plurality and a point on said eye corresponding to said second glint;

for each one of said at least one cameras in said plurality, defining a particular plane by rotating said base plane through each said corresponding angle around an axis including said focal center for each said one camera and said first glint; and

identifying a line at an intersection of said planes as said gaze vector.

11. The method of claim 10 wherein said center of said pupil image lies on a line maximizing symmetry of said pupil image.

12. A method for eye gaze tracking, comprising the steps of:

focusing at least one camera upon at least one of a user'"'"'s eyes, each said camera having a focal center, an image plane, and a co-located light source emitting light toward said eye;

identifying and locating image aspects including at least one glint and a pupil image in said image plane; and

computing a line containing a point of regard on a display screen from said image aspects and camera position and orientation data.

13. The method of claim 12 wherein a position of said line on said display screen controls a graphical user interface element.

14. The method of 13 wherein said graphical user interface element is a scroll bar slider.

15. A system for eye gaze tracking, comprising:

at least one camera focusing upon at least one of a user'"'"'s eyes, each said camera having a focal center, an image plane, and a co-located light source emitting light toward said eye; and

a computer to identify and locate image aspects including at least one glint and a pupil image in said image plane, and to compute a gaze vector from at least one plane generated from said image aspects and camera position and orientation data.

16. The system of claim 15 wherein said system is adapted for use with an animal.

17. The system of claim 15 wherein said system is adapted for use with a person.

18. The system of claim 15 wherein said system is adapted for use with a baby.

19. The system of claim 15 wherein said computer locates a point of regard as the intersection of said gaze vector with a predetermined surface.

21. The system of claim 15 wherein said computer corrects said gaze vector for a foveal offset axis angle.

20. A system for eye gaze tracking, comprising:

at least one camera focusing upon at least one of a user'"'"'s eyes, each said camera having a focal center, an image plane, and a co-located light source emitting light toward said eye;

a computer to identify and locate image aspects including at least one glint and a pupil image in said image plane, and to compute a gaze vector from at least one plane generated from said image aspects and camera position and orientation data; and

a source for synchronous scan signals controlling each one of said at least one cameras and alternately energizing said light sources to identify correspondences between said light sources and said glints.

22. A system for eye gaze tracking, comprising:

at least one camera focusing upon at least one of a user'"'"'s eyes, each said camera having a focal center, an image plane, and a co-located light source emitting light toward said eye; and

a computer to identify and locate image aspects including at least one glint and a pupil image in said image plane, and to compute a gaze vector from at least one plane generated from said image aspects and camera position and orientation data, wherein said computer

(1) determines an angle between said glint in said image plane, said focal center, and a center of said pupil image in said image plane;

(2) finds a separation on said eye between said glint and said pupil center using said angle and a distance estimate between said eye and a point of regard;

(3) defines a second angle between said focal center, a corneal curvature center, and said pupil center using a radius of corneal curvature to define said gaze vector; and

(4) locates said point of regard at the intersection of said gaze vector with a predetermined surface.

23. A system for eye gaze tracking, comprising:

at least one camera focusing upon at least one of a user'"'"'s eyes, each said camera having a focal center, an image plane, and a co-located light source emitting light toward said eye; and

a computer to identify and locate image aspects including at least one glint and a pupil image in said image plane, and to compute a gaze vector from at least one plane generated from said image aspects and camera position and orientation data, wherein said computer

(1) defines for each one of a plurality of said at least one cameras a particular plane spanning said glint in said image plane, said focal center, and a center of said pupil image in said image plane; and

(2) identifies an intersection line of said planes as said gaze vector.

24. A system for eye gaze tracking, comprising:

at least one camera focusing upon at least one of a user'"'"'s eyes, each said camera having a focal center, an image plane, and a co-located light source emitting light toward said eye; and

a computer to identify and locate image aspects including at least one glint and a pupil image in said image plane, and to compute a gaze vector from at least one plane generated from said image aspects and camera position and orientation data, wherein said computer

(1) for each one of a plurality of said at least one cameras, defines in said image plane an angle spanning a center of said pupil image, a first glint, and a second glint, wherein said first glint results from said light source on each said one camera and said second glint results from another light source;

(2) defines a base plane spanning said focal centers for each said one camera and said focal center for each other camera in said plurality and a point on said eye corresponding to said second glint;

(3) for each one of said at least one cameras in said plurality, defines a particular plane by rotating said base plane through each said corresponding angle around an axis including said focal center for each said one camera and said first glint; and

(4) identifies a line at an intersection of said planes as said gaze vector.

25. The system of claim 24 wherein said computer chooses said center of said pupil image to lie on a line maximizing symmetry of said pupil image.

26. A system for eye gaze tracking comprising:

at least one camera focused upon at least one of a user'"'"'s eyes, each said camera having a focal center, an image plane, and a co-located light source emitting light toward said eye; and

a computer to identify and locate image aspects including at least one glint and a pupil image in said image plane, and to compute a line containing a point of regard on a display screen from said image aspects and camera position and orientation data.

27. The system of claim 26 wherein a position of said line on said display screen controls a graphical user interface element.

28. The system of 27 wherein said graphical user interface element is a scroll bar slider.

29. A system for eye gaze tracking comprising:

means for focusing at least one camera upon a user'"'"'s eye, each said camera having a focal center, an image plane, and a co-located light source emitting light toward said eye;

means for identifying and locating, in an image plane, image aspects including at least one glint and a pupil image;

means for computing a gaze vector from at least one plane generated from said image aspects; and

means for determining a point of regard from said image aspects and camera position and orientation data.

30. A computer program product including a program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform method steps for eye gaze tracking, said method steps comprising:

focusing at least one camera upon a user'"'"'s eye, each said camera having a focal center, an image plane, and a co-located light source emitting light toward said eye;

identifying and locating, in an image plane, image aspects including at least one glint and a pupil image;

computing a gaze vector from at least one plane generated from said image aspects; and

determining a point of regard from said image aspects and camera position and orientation data.