Method and system for accommodating pupil non-concentricity in eyetracker systems

• US 6,598,971 B2
• Filed: 11/08/2001
• Issued: 07/29/2003
• Est. Priority Date: 11/08/2001
• Status: Expired due to Term

• Current Assignee: Eyegaze, Inc.
• Original Assignee: LC Technologies Incorporated
• Inventors: Cleveland, Dixon
• Primary Examiner(s): MANUEL, GEORGE C
• Application Number:
  US09/986,368
• Publication Number:
  US 20030086057A1
• Time in Patent Office: 628 Days
• Field of Search: 351/204, 351/208, 351/209, 351/210, 351/221, 351/246, 348/16, 396/51
• US Class Current: 351/209
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

Embodiments of the present invention relate to eyetracking methods and systems that compensate for physiological variations in the location of the pupil within the eye. In one embodiment, a video eyetracker measures the location of a pupil within the eye. Using measured observable features of the eye, including pupil diameter, a pupil-location-offset is estimated with respect to a fixed point on the eyeball. The location of a fixed point within the eyeball is estimated as a function of the measured pupil location and the estimated pupil-location-offset.

Claims

1. A method to determine the location of a fixed point within an eyeball comprising the following steps:

measuring the location of a pupil;

measuring at least one observable feature of the eye related to the relative location of the pupil within the eyeball;

estimating a pupil-location-offset with respect to the fixed point on the eyeball using the at least one measured observable eye feature; and

calculating the location of the fixed point within the eyeball as a function of the measured pupil location and the estimated pupil-location offset.

2. A method according to claim 1, wherein said step of measuring observable features of the eye includes the step of measuring the pupil diameter.

3. The method according to claim 2, comprising an eyetracker calibration procedure for use on a computer system wherein variations in a user'"'"'s pupil diameter are stimulated by varying a light level in a viewed environment.

4. The method according to claim 2, comprising an eyetracker calibration procedure used in conjunction with a computer screen display wherein variations in the user'"'"'s pupil diameter are stimulated by changing the brightness of the screen display.

5. A method according to claim 2, including the steps of:

determining a prior measured pupil diameter D_past and the prior predicted pupil offset x_{pup-offset-past} and y_{pup-offset-past};

measuring the pupil diameter D_pres;

determining the change in pupil diameter dD_pup with the following relationship;

dD_pup=D_pres−

D_past;

determining whether the pupil diameter increased or decreased;

finding appropriate values for dx_pup-offset/dD_pup and dy_pup-offset/dD_pup from the pupil offset hysteresis model based on at least one of the following;

pupil-diameter increase or decrease;

the past pupil diameter D_past, and

the past pupil offset x_past and y_past;

computing changes in pupil offsets dx_pup-offset and dy_pup-offset as a function of the change in pupil diameter; and

computing a quantity for new pupil offset from the old pupil offset and the change in the pupil offset with the following relationships;

6. The method according to claim 5, comprising the step of computing changes in in pupil offsets dx_pup-offset and dy_pup-offset as a function of a change in pupil diameter, including the following steps if the pupil diameter is increasing:

 

dx_pup-offset=dx_pup-offset/dD_pup(D_pup,x,increasing)*dD_pup

7. The method of claim 6, comprising the steps of determining the location of the fixed reference point within the eyeball as expressed as the sum of the pupil location and the pupil-location offset:

8. The method of claim 2, comprising the step of estimating the pupil-center shift as a function of pupil diameter for at least two measured pupil diameter values D_pup, using the relationships:

9. The method of claim 8, comprising the step of approximating the pupil-location offsets by a pair of linear equations:

10. The method of claim 9, comprising the step of determining the location of the fixed reference point within the eyeball as expressed as the sum of the pupil location and the pupil-location offset:

11. A method according to claim 2, comprising steps for inducing hysteresis behavior of an eye'"'"'s pupil-center offset comprising the steps of:

monotonically increasing a light level from a first level to a second level;

monotonically decreasing the light level from the second level to the first level;

varying the light level in a saw tooth manner while gradually increasing and decreasing the overall light level.

12. A method according to claim 11, comprising steps for calculating the hysteresis behavior of the eye'"'"'s pupil-center offset comprising the steps of:

concurrently measuring values for pupil diameter, direction of pupil-diameter change, and pupil-center location.

13. A method according to claim 11, including the steps of:

determining a prior measured pupil diameter D_past and values for predicted pupil offset x_{pup-offset-past} and y_{pup-offset-past};

measuring the pupil diameter D_pres;

determining the change in pupil diameter dD_pup with the following relationship;

dD_pup=D_pres−

D_past;

determining whether the pupil diameter increased or decreased;

finding appropriate values for dx_pup-offsetdD_pup and dy_pup-offset/dD_pup from the pupil offset hysteresis model based on at least one of the following;

pupil-diameter increase or decrease,

the past pupil diameter D_past, and

the past pupil offset x_past and y_past;

computing changes in pupil offsets dx_pup-offset and dy_pup-offset as a function of the change in pupil diameter; and

computing a quantity for new pupil offset from the old pupil offset and the change in the pupil offset with the following relationships;

14. The method according to claim 11, comprising the step of computing changes in in pupil offsets dx_pup-offset and dy_pup-offset as a function of a change in pupil diameter, including the following steps if the pupil diameter is increasing:

15. The method of claim 14, comprising the steps of determining the location of the fixed reference point within the eyeball as expressed as the sum of the pupil location and the pupil-location offset:

16. A method according to claim 1, wherein the steps are performed iteratively and said step of measuring observable features of the eye includes the step of measuring a pupil diameter.

17. A method according to claim 16, comprising the step of measuring the iris/sclera boundary of the eye.

18. A method according to claim 16, comprising the step of recognizing and locating a feature of the eye from the group consisting of:

a blood vessel in the sclera, a feature of the iris/pupil boundary and a mark of the iris fiber pattern in proximity of the outer edge of the iris.

19. A method of using a computer system to quantify an eye'"'"'s pupil-offset behavior, comprising the following steps:

using an eyetracker system and the pupil-center-corneal reflection method to measure a first glint-pupil vector of the eye;

fixating the eye on one or more given points on a screen;

measuring at least one additional glint-pupil vector of the eye; and

quantifying measured pupil offsets as functions of measured pupil diameters.

20. The method according to claim 19, comprising the step of illuminating an eye with light, varying the light level, and monitoring the eye with a video camera.

21. A method according to claim 20, further comprising the steps of accommodation for pupil-center hysteresis by:

measuring a glint-pupil vector by including measurements of pupil-center offset changes, pupil diameter changes, and determining a pupil diameter change factor as a function of whether the pupil diameter increases or decreases; and

codifying dx_pup-offset/dD_pup and dy_pup-offset/dD_pup expressions for at least one of a pupil-diameter-increasing case and a pupil-diameter-decreasing case.

22. The method of claim 21, comprising the steps of determining the location of the fixed reference point within the eyeball as expressed as the sum of the pupil location and the pupil-location offset:

23. A method according to claim 21, wherein said step of accommodating pupil-center hysteresis is performed iteratively and includes a step of measuring a pupil diameter.

24. A method according to claim 20, comprising the step of measuring the iris/sclera boundary of the eye.

25. A method according to claim 20, comprising the step of recognizing and locating a feature of the eye from the group consisting of:

a blood vessel in the sclera, a feature of the iris/pupil boundary and a mark of the iris fiber pattern in proximity of the outer edge of the iris.

26. The method according to claim 20, comprising an eyetracker calibration procedure for use on a computer system wherein variations in a user'"'"'s pupil diameter are stimulated by varying a light level in a viewed environment.

27. The method according to claim 20, comprising an eyetracker calibration procedure used in conjunction with a computer screen display wherein variations in the user'"'"'s pupil diameter are stimulated by changing the brightness of the screen display.

28. A method according to claim 19, comprising steps for inducing hysteresis behavior of an eye'"'"'s pupil-center offset comprising the steps of:

increasing a light level from a first level to a second level;

decreasing the light level from the second level to the first level;

varying the light level in a substantially saw tooth manner while gradually increasing and decreasing the overall light level.

29. A method according to claim 28, comprising steps for mapping the hysteresis behavior of an eye'"'"'s pupil-center offset comprising the steps of:

concurrently measuring values for pupil diameter, direction of pupil-diameter change, and pupil-center location.

30. An apparatus for determining the location of a fixed point within an eyeball comprising:

a computer;

a camera coupled to the computer for observing the eye;

a means for measuring pupil location;

means for measuring at least one observable feature of the eye related to the relative location of the pupil within the eyeball;

a processor for estimating a pupil-location-offset with respect to a fixed point on the eyeball using the at least one measured observable eye feature; and

a processor for estimating the location of the fixed point within the eyeball as a function of the measured pupil location and the estimate of the pupil-location offset.

31. An apparatus according to claim 30, wherein the camera is a video camera that continuously observes the eye.

32. An apparatus according to claim 31, comprising:

a memory for storing a measured pupil diameter D_past and a prior predicted pupil offset x_{pup-offset-past} and y_{pup-offset-past};

a processor for measuring the present pupil diameter D_pres;

a processor for determining the change in pupil diameter dD_pup, wherein dD_pup=D_pres−

D_pas;

a processor for determining whether a pupil diameter increased or decreased;

a processor for finding appropriate values for dx_pup-offset/dD_pup and dy_pup-offset/dD_pup from the pupil offset hysteresis model based on;

pupil-diameter increase or decrease,

the past pupil diameter D_past, and

the past pupil offset x_past and y_past;

a processor for computing the changes in pupil offset dx_pup-offset and dy_pup-offset as a function of the change in pupil diameter, wherein if the pupil diameter is increasing the following relations are used;

33. An apparatus according to claim 32, comprising a processor for performing the steps of determining the location of the fixed reference point within the eyeball as expressed as the sum of the pupil location and the pupil-location offset using the following relationships:

34. An apparatus according to claim 32, comprising a means for performing an eyetracker calibration subroutine for use on a computer system wherein a user'"'"'s pupil diameter is measured as a light level is synchronously varied in a viewed environment.

35. An apparatus according to claim 32, further comprising an eyetracker calibration program for use in conjunction with a computer screen display wherein variations in the user'"'"'s pupil diameter are measured as the pupil is stimulated by changing the brightness of the screen display.

36. A video eyetracking system comprising:

means for measuring a pupil center location of an eyeball within a video image;

means for estimating an offset of the pupil center location with respect to a fixed reference point on the eyeball;

means for determining an offset-adjusted pupil-center using the offset estimate and the measured pupil center location; and

a processor that uses the offset-adjusted pupil-center to measure the orientation of the eyeball.

37. A system as claimed in claim 36, wherein a pupil'"'"'s pupil-center hysteresis is accommodated via:

means for including measurements of pupil-center offset changes, pupil diameter changes, and a parameter for indicating whether the pupil diameter is increasing or decreasing, and

means for codifying dx_pup-offset/dD_pup and dy_pup-offset/dD_pup expressions.

38. A system as claimed in claim 36, comprising:

a medium with one or more reference points for the observer to fixate one.

39. A system according to claim 36, comprising:

memory means for storing a plurality of measured pupil diameters D_past;

means for recalling a prior predicted pupil offset x_{pup-offset-past} and y_{pup-offset-past};

means for measuring a pupil diameter D_pres;

means for determining a change in pupil diameter dD_pup, wherein dD_pup=D_pres−

D_pas;

means for determining whether a pupil diameter has increased or decreased;

means for finding appropriate values for dx_pup-offset/dD_pup and dy_pup-offset/dD_pup from the pupil offset hysteresis model based on;

pupil-diameter increase or decrease,

the past pupil diameter D_past, and

the past pupil offset x_past and y_past;

means for computing changes in pupil offsets dx_pup-offset and dy_pup-offset as a function of a change in pupil diameter, wherein if the pupil diameter is increasing the following relations are used;

40. A system according to claim 39, comprising a means for performing an eyetracker calibration subroutine for use on a computer system wherein a user'"'"'s pupil diameter is measured as a light level is synchronously varied with the measurements.

41. A system according to claim 39, comprising an eyetracker calibration on a programmable chip for use in conjunction with a computer screen display wherein variations in the user'"'"'s pupil diameter are measured as the pupil is stimulated by changing the brightness of a screen display.

42. A system according to claim 39, comprising means for determining the location of a fixed reference point within an eyeball as expressed as the sum of the pupil location and the pupil-location offset using at least the following relationships:

43. A system according to claim 36, comprising a means for performing an eyetracker calibration subroutine for use on a computer system wherein a user'"'"'s pupil diameter is measured as a light level is synchronously varied in a viewed environment.

44. A system according to claim 36, comprising a programmable eyetracker calibration subroutine for use on a computer system in conjunction with a computer screen display wherein variations in the user'"'"'s pupil diameter are measured as the pupil is stimulated by changing the brightness of a screen display.