Calibration-free gaze tracking under natural head movement

• US 7,306,337 B2
• Filed: 02/26/2004
• Issued: 12/11/2007
• Est. Priority Date: 03/06/2003
• Status: Active Grant

• Current Assignee: Rensselaer Polytechnic Institute
• Original Assignee: Rensselaer Polytechnic Institute
• Inventors: Ji, Qiang, Zhu, Zhiwei
• Primary Examiner(s): Mack, Ricky
• Assistant Examiner(s): TRA, TUYEN Q
• Application Number:
  US10/787,359
• Publication Number:
  US 20040174496A1
• Time in Patent Office: 1,384 Days
• Field of Search: 351/209, 351/205, 351/206, 351/208, 351/210, 351/211, 351/221, 351/246, 382/103, 382/117, 382/128, 382/199, 382/288, 382/289
• US Class Current: 351/209
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris

Abstract

A method and computer system for tracking eye gaze. A camera is focused on an eye of subject viewing a gaze point on a screen while directing light toward the eye. Eye gaze data pertaining to a glint and pupil image of the eye in an image plane of the camera is sampled. Eye gaze parameters are determined from the eye gaze data. The determined eye gaze parameters include: orthogonal projections of a pupil-glint displacement vector, a ratio of a major semi-axis dimension to a minor semi-axis dimension of an ellipse that is fitted to the pupil image in the image plane, an angular orientation of the major semi-axis dimension in the image plane, and mutually orthogonal coordinates of the center of the glint in the image plane. The gaze point is estimated from the eye gaze parameters.

Claims

1. A method for tracking eye gaze, comprising the steps of:

focusing a single camera on an eye of subject viewing a gaze point on a screen while directing light toward the eye;

sampling eye gaze data pertaining to a glint and pupil image of the eye in an image plane of the single camera;

determining eye gaze parameters from the eye gaze data, wherein the eye gaze parameters include;

Δ

x, Δ

y, r, θ

, g_x, and g_y, wherein Δ

x and Δ

y are orthogonal projections of a pupil-glint displacement vector directed from the center of the pupil image to the center of the glint in the image plane, wherein r is a ratio of a major semi-axis dimension to a minor semi-axis dimension of an ellipse that is fitted to the pupil image in the image plane, wherein θ

is an angular orientation of the major semi-axis dimension in the image plane, and wherein g_x, and g_y are mutually orthogonal coordinates of the center of the glint in the image plane; and

estimating the gaze point from the eye gaze parameters.

2. The method of claim 1, wherein the gaze point is characterized by orthogonal coordinates X_SG and Y_SG in the coordinate system of the screen, and wherein the estimating step comprises estimating X_SG and Y_SG independently.

3. The method of claim 2, wherein estimating X_SG utilizes a first mapping function based on a generalized regression neural network architecture, and wherein estimating Y_SG utilizes a second mapping function based on the generalized regression neural network architecture.

4. The method of claim 3, wherein the first mapping function utilizes a first probability density function having a first Gaussian kernel characterized by a first width σ

₁, and wherein the second mapping function utilizes a second probability density function having a second Gaussian kernel characterized by a second width σ

₂.

5. The method of claim 4, wherein σ

₁=σ

₂.

6. The method of claim 4, wherein σ

₁≠

σ

₂.

7. The method of claim 3, wherein the first mapping function is calibrated with n calibration data samples, wherein the second mapping function is calibrated with the n data samples, and wherein n is at least 2.

8. The method of claim 7, wherein the generalized regression neural network architecture of the first and second mapping functions includes an input layer having 6 nodes, a hidden layer coupled to the input layer and having n nodes, a summation layer coupled to the hidden layer and having 2 nodes, and an output layer coupled to the summation layer and having 1 node.

9. The method of claim 7, wherein the method further comprises the steps of:

partitioning the screen into a two-dimensional array of screen areas; and

classifying a vector of the eye gaze parameters into a screen area of the screen areas.

10. The method of claim 7, wherein the classifying step comprises:

classifying the vector of the eye gaze parameters into a first screen area of the screen areas, using a whole classifier;

determining nearest neighbor screen areas of the first screen area; and

reclassifying the vector of the eye gaze parameters into the screen area, using a sub-classifier pertaining to the nearest neighbor screen areas.

11. The method of claim 10, wherein the method further comprises assigning the estimated gaze point to a region of the screen that has an area of about ⅛

of the area of the screen, wherein said assigning has an average accuracy of at least about 95% even if any of the following head motions of the subject occur during the focusing step;

up to 6 inches of left/right head translation, up to 6 inches of up/down head translation movement, up to ±

20 degrees of left/right head rotation, and up to ±

15 degrees of up/down head rotation.

12. The method of claim 7, wherein the method further comprises assigning the estimated gaze point to a region of the screen that has an area of about ⅛

of the area of the screen, wherein said assigning has an average accuracy of at least about 85% even if any of the following head motions of the subject occur during the focusing step;

up to 6 inches of left/right head translation, up to 6 inches of up/down head translation movement, up to ±

20 degrees of left/right head rotation, and up to ±

15 degrees of up/down head rotation.

13. The method of claim 1, wherein no camera other than the single camera is focused on the eye during the focusing step.

14. A computer system comprising a processor and a computer readable memory unit coupled to the processor, said memory unit containing instructions that when executed by the processor implement a method for tracking eye gaze, said method comprising the computer implemented steps of:

processing eye gaze data pertaining to a glint and pupil image of an eye in an image plane of a single camera, wherein the eye is comprised by a subject, and wherein the single camera is focused on the eye while the eye is viewing a gaze point on a screen and while light is directed toward the eye;

determining eye gaze parameters from the eye gaze data, wherein the eye gaze parameters include;

Δ

x, Δ

y, r, θ

, g_x, and g_y, wherein Δ

x and Δ

y are orthogonal projections of a pupil-glint displacement vector directed from the center of the pupil image to the center of the glint in the image plane, wherein r is a ratio of a major semi-axis dimension to a minor semi-axis dimension of an ellipse that is fitted to the pupil image in the image plane, wherein θ

is an angular orientation of the major semi-axis dimension in the image plane, and wherein g_x, and g_y are mutually orthogonal coordinates of the center of the glint in the image plane; and

estimating the gaze point from the eye gaze parameters.

15. The computer system of claim 14, wherein the gaze point is characterized by orthogonal coordinates X_SG and Y_SG in the coordinate system of the screen, and wherein the estimating step comprises estimating X_SG and Y_SG independently.

16. The computer system of claim 15, wherein estimating X_SG utilizes a first mapping function based on a generalized regression neural network architecture, and wherein estimating Y_SG utilizes a second mapping function based on the generalized regression neural network architecture.

17. The computer system of claim 16, wherein the first mapping function utilizes a first probability density function having a first Gaussian kernel characterized by a first width σ

₁, and wherein the second mapping function utilizes a second probability density function having a second Gaussian kernel characterized by a second width σ

₂.

18. The computer system of claim 17, wherein σ

₁=σ

₂.

19. The computer system of claim 17, wherein σ

₁≠

σ

₂.

20. The computer system of claim 16, wherein the first mapping function is calibrated with n calibration data samples, wherein the second mapping function is calibrated with the n data samples, and wherein n is at least 2.

21. The computer system of claim 20, wherein the generalized regression neural network architecture of the first and second mapping functions includes an input layer having 6 nodes, a hidden layer coupled to the input layer and having n nodes, a summation layer coupled to the hidden layer and having 2 nodes, and an output layer coupled to the summation layer and having 1 node.

22. The computer system of claim 20, wherein the method further comprises the steps of:

partitioning the screen into a two-dimensional array of screen areas; and

classifying a vector of the eye gaze parameters into a screen area of the screen areas.

23. The computer system of claim 20, wherein the classifying step comprises:

classifying the vector of the eye gaze parameters into a first screen area of the screen areas, using a whole classifier;

determining nearest neighbor screen areas of the first screen area; and

reclassifying the vector of the eye gaze parameters into the screen area, using a sub-classifier pertaining to the nearest neighbor screen areas.

24. The computer system of claim 23, wherein the method further comprises assigning the estimated gaze point to a region of the screen that has an area of about ⅛

of the area of the screen, wherein said assigning has average accuracy of at least about 95% even if any of the following head motions of the subject occur during the focusing step;

up to 6 inches of left/right head translation, up to 6 inches of up/down head translation movement, up to ±

20 degrees of left/right head rotation, and up to ±

15 degrees of up/down head rotation.

25. The computer system of claim 20, wherein the method further comprises assigning the estimated gaze point to a region of the screen that has an area of about ⅛

of the area of the screen, wherein said assigning has average accuracy of at least about 85% even if any of the following head motions of the subject occur during the focusing step;

up to 6 inches of left/right head translation, up to 6 inches of up/down head translation movement, up to ±

20 degrees of left/right head rotation, and up to ±

15 degrees of up/down head rotation.

26. The computer system of claim 14, wherein no camera other than the single camera is focused on the eye during the focusing step.