Eye tracking system and method

• US 7,653,213 B2
• Filed: 09/30/2005
• Issued: 01/26/2010
• Est. Priority Date: 03/31/2003
• Status: Active Grant

• Current Assignee: Seeing Machines LTD
• Original Assignee: Seeing Machines Pty Limited
• Inventors: Longhurst, Gavin, Rougeaux, Sebastian
• Primary Examiner(s): Tabatabai; Abolfazl
• Application Number:
  US11/241,669
• Publication Number:
  US 20060146046A1
• Time in Patent Office: 1,579 Days
• Field of Search: 382/103, 382/107, 382/117, 382/118, 382/149, 382/236, 348/169, 348/170, 348/171, 348/172, 348/208.14, 348/208.1, 348/579, 351/208, 351/209, 351/210, 351/246, 351/231, 345/418
• US Class Current: 382/103
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 3/7865 using correlation of the live video image with a stored image
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: H04N 13/302 for viewing without the aid of special glasses, i.e. using autostereoscopic displays
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

A method of tracking an expected location of a head in a computerized headtracking environment having a delayed processing requirement for locating a current head position, the method comprising the step of: utilizing previously tracked positions to estimate a likely future tracked position; outputting the likely future tracked position as the expected location of the head. Kalman filtering of the previously tracked positions can be utilized in estimating the likely future tracked position.

Claims

1. A face tracking system for predicting the future position of a face comprising:

multiple cameras for observing a user, each camera carrying out the steps of;

(a) providing a current prediction of the face position using facial features detected in a previous and current input image frame;

(b) deriving a first covariance matrix from the current prediction and a previous covariance matrix;

(c) utilizing said current prediction of the face position from step (a) and a Kalman filter to determine a corresponding projected point of the facial feature on the plane of at least one camera;

(d) deriving a Jacobian of the projected points in said step (c);

(e) deriving a residual covariance of the projected points in said step (c);

(f) deriving a suitable filter gain for said Kalman filter;

(g) deriving a suitable update coefficients for said first covariance matrix;

(h) updating said Kalman filter utilizing said filter gain.

2. The face tracking system of claim 1 further comprising each camera carrying out the step of:

(i) determining a corresponding expected eye position from the current state of the Kalman filter.

3. The face tracking system of claim 1 wherein a noise component is added to said first covariance matrix.

4. The face tracking system of claim 3 wherein said noise component includes a translational noise component and a rotational noise component.

5. The face tracking system of claim 4 wherein said residual covariance of said step (e) is utilized to tune response of the Kalman filter.

6. The face tracking system of claim 1, wherein said face tracking system further comprises an auto-stereoscopic display driven by said prediction of future position of the face.

7. A system for tracking an expected location of a head in a computerized headtracking environment having a delayed processing requirement for locating a head position, the system comprising:

a device, such that the system utilizes previously tracked positions to estimate a likely future tracked position; and

outputs the likely future tracked position as the expected location of the head to said device, wherein said likely future tracked position is utilized to control an auto-stereoscopic display for the display of images for eyes located at expected positions corresponding to said likely future tracked position, and wherein Kalman filtering of the previously tracked positions is utilized in estimating said likely future tracked position.

8. The face tracking system of claim 7 wherein utilizing previously tracked positions to estimate a likely future tracked position further includes the steps of:

(a) providing the expected location of the head using facial features detected in a previous and current tracked positions;

(b) deriving a first covariance matrix from the current prediction and a previous covariance matrix;

(c) utilising the current prediction from step (a) and a Kalman filter to determine a corresponding projected point of the facial feature on the plane of at least one camera;

(d) deriving a Jacobain of the projected points from step (c);

(e) deriving a residual covariance of the projected points from step (c);

(f) deriving a suitable filter gain for the Kalman filter;

(g) deriving a suitable update coefficients for the first covariance matrix; and

(h) updating the Kalman filter utilising the filter gain.

9. A system for providing an expected location of a head the system comprising:

video input means for providing at least one video signal of the head;

first processing means for processing the video signal so as to output a substantially continuous series of current head location data;

second processing means for processing predetermined one of the current head location data so as to output a predicted future expected head position as the expected location of the head, thereby substantially overcoming a delay in processing the head position, wherein said second processing means utilizes a Kalman filtering of the current head location data; and

an auto-stereoscopic display driven by said predicted expected location output of said head.

10. The system of claim 9 wherein said video input means includes stereo video inputs.

11. The system of claim 9 further comprising an auto-stereoscopic display driven by said predicted expected location output of said head.