Apparatus to detect and measure saccade and pupilary changes

• US 7,380,938 B2
• Filed: 03/25/2004
• Issued: 06/03/2008
• Est. Priority Date: 03/25/2003
• Status: Active Grant

• Current Assignee: Princeton Identity Inc.
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Matey, James Regis, Chmielewski, Thomas Adam Jr.
• Primary Examiner(s): Harrington; Alicia M
• Application Number:
  US10/809,471
• Publication Number:
  US 20040252277A1
• Time in Patent Office: 1,531 Days
• Field of Search: 351/209, 351/210, 351/220, 351/221
• US Class Current: 351/210
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

An eye tracking system employs a first line camera that is configured to track horizontal eye motion and a second line camera that is configured to track vertical eye motion. Output signals from the two line cameras are applied to a processor which identifies and tracks eye motion, using a correlation or edge detection algorithm on boundaries between the sclera, iris and pupil. The system includes multiple controlled light sources and the horizontal and vertical line cameras are configured to track eye motion in response to light stimulus provided by the light sources according to a programmed algorithm. Eye motion for an individual is collected and compared to a model in order to obtain a measure of fitness for the individual. The device may also be used to derive inputs to a computer system based on eye motion or gaze direction.

Claims

1. An eye tracking system comprising:

a horizontal line camera configured to capture a horizontal image of at least a portion of the eye and to provide an output signal representing the horizontal image of the eye;

a vertical line camera configured to capture a vertical image of at least a portion of the eye and to provide an output signal representing the vertical image of the eye;

a processor, coupled to the horizontal line camera and the vertical line camera to process the signals representing the horizontal and vertical images of the eye to identify and track motion of the eye; and

a beam splitter configured to reflect a portion of an image of the eye onto one of the horizontal line camera and the vertical line camera and to pass a portion of the image of the eye onto the other one of the horizontal line camera and the vertical line camera.

2. An eye tracking system comprising:

a horizontal line camera configured to capture a horizontal image of at least a portion of the eye and to provide an output signal representing the horizontal image of the eye;

a vertical line camera configured to capture a vertical image of at least a portion of the eye and to provide an output signal representing the vertical image of the eye;

a processor, coupled to the horizontal line camera and the vertical line camera to process the signals representing the horizontal and vertical images of the eye to identify and track motion of the eye; and

an optical element configured to optically focus an image of the eye onto at least one of the horizontal and vertical line cameras before the image is captured by the at least one of the horizontal line camera and the vertical line camera.

3. An eye tracking system comprising:

a horizontal line camera configured to capture a horizontal image of at least a portion of the eye and to provide an output signal representing the horizontal image of the eye;

a vertical line camera configured to capture a vertical image of at least a portion of the eye and to provide an output signal representing the vertical image of the eye;

a processor, coupled to the horizontal line camera and the vertical line camera to process the signals representing the horizontal and vertical images of the eye to identify and track motion of the eye; and

multiple controlled visible light sources coupled to the processor, wherein the processor controls the multiple light sources individually to induce eye motion, and the processor further includes a statistical analyzer for comparing the tracked eye motions of a user to eye motion data derived from other users to determine a measure of fitness for the user.

4. An eye tracking system comprising:

a horizontal line camera configured to capture a horizontal image of at least a portion of the eye and to provide an output signal representing the horizontal image of the eye;

a vertical line camera configured to capture a vertical image of at least a portion of the eye and to provide an output signal representing the vertical image of the eye;

a processor, coupled to the horizontal line camera and the vertical line camera to process the signals representing the horizontal and vertical images of the eye to identify and track motion of the eye; and

a first beam splitter configured to reflect a portion of an image of the eye onto the horizontal line camera;

a second beam splitter configured to reflect a portion of the image of the eye onto the vertical line camera;

wherein the horizontal and vertical line cameras are configured to be outside of a field of view of the eye and the first and second beam splitters are configured to pass a portion of an image of a scene in the field of view of the eye to the eye.

5. An eye tracking system comprising:

a horizontal line camera configured to capture a horizontal image of at least a portion of the eye and to provide an output signal representing the horizontal image of the eye;

a vertical line camera configured to capture a vertical image of at least a portion of the eye and to provide an output signal representing the vertical image of the eye;

a processor, coupled to the horizontal line camera and the vertical line camera to process the signals representing the horizontal and vertical images of the eye to identify and track motion of the eye; and

at least one infrared illuminator configured to illuminate the eye as images are being captured by the horizontal and vertical line cameras, the at least one infrared illuminator and the horizontal and vertical line cameras being controlled by the processor to control a rate at which sampled images are obtained from the horizontal and vertical line imagers;

wherein at least one of the infrared illuminators is configured to illuminate the eye in a direction approximately corresponding to a gaze direction of the eye, whereby the horizontal and vertical images of the illuminated eye exhibit a bright reflection of the at least one infrared illuminator in respective regions of the images corresponding to the pupil of the eye.

6. An eye tracking system comprising:

a horizontal line camera configured to capture a horizontal image of at least a portion of the eye and to provide an output signal representing the horizontal image of the eye;

a vertical line camera configured to capture a vertical image of at least a portion of the eye and to provide an output signal representing the vertical image of the eye;

a processor, coupled to the horizontal line camera and the vertical line camera to process the signals representing the horizontal and vertical images of the eye to identify and track motion of the eye; and

a visible light illuminator configured to illuminate the eye as images are being captured by the horizontal and vertical line cameras, the visible light illuminator being controlled by the processor to induce contraction of the iris of the eye; and

a pupil size monitor, in the processor, for tracking changes in size of the pupil of the eye in response to illumination of the eye in response to the visible light illuminator.

7. An eye tracking system according to claim 6, wherein the processor further includes:

a model of pupilary changes with respect to light intensity and saccadic motion;

a comparator which compares saccadic motion and pupilary change data from a user at a given test time to the model and to data derived from other users;

a fitness algorithm that identifies data corresponding to specific model parameters that are out of normal bounds as determined from the data derived from other users and, based on the identified parameters determines a metric of fitness for the user.

8. An eye tracking system for tracking eye motion of a user having first and second eyes comprising:

a horizontal line camera configured to capture a horizontal image of at least a portion of the first eye of the user and to provide an output signal representing the horizontal image of the first eye;

a vertical line camera configured to capture a vertical image of at least a portion of the second eye of the user and to provide an output signal representing the vertical image of the second eye;

a processor, coupled to the horizontal line camera and the vertical line camera to process the signals representing the horizontal and vertical images of the eye to identify and track motion of the eye.

9. An eye tracking system according to claim 8, wherein each of the images captured by the horizontal and vertical line cameras includes pixels corresponding to components of the eye including at least two of the sclera, iris and pupil, and the processor includes one of an edge detector and a matched filter to track motion of the eye by identifying corresponding boundaries between respective ones of the components of the eye in successive images provided by the horizontal line camera and the vertical line camera.

10. An eye tracking system according to claim 8 further including:

a first beam splitter configured to reflect a portion of an image of the first eye onto the horizontal line camera; and

a second beam splitter configured to reflect a portion of an image of the second eye onto the vertical line camera;

wherein the horizontal and vertical line cameras are configured to be outside of a field of view of the first and second eyes and the first and second beam splitters are configured to pass a portion of an image of a scene in the field of view of the first and second eyes to the first and second eyes.

11. An eye tracking system according to claim 8, further including:

a first optical element configured to optically focus a vertically compressed image of the first eye onto the horizontal line camera; and

a second optical element configured to optically focus a horizontally compressed image of the second eye onto the vertical line camera.

12. An eye tracking system according to claim 8 further including multiple controlled visible light sources coupled to the processor, wherein the processor controls the multiple light sources individually to induce eye motion, and the processor further includes a statistical analyzer for comparing the tracked eye motions of a user to eye motion data derived from other users to determine a measure of fitness for the user.

13. An eye tracking system according to claim 8, further including:

at least one infrared illuminator configured to illuminate the first and second eyes as images are being captured by the horizontal and vertical line cameras, the at least one infrared illuminator and the horizontal and vertical line cameras being controlled by the processor to control a rate at which sampled images are obtained from the horizontal and vertical line imagers.

14. An eye tracking system according to claim 8, wherein the at least one infrared illuminator is configured to illuminate at least one of the first and second eyes in a direction approximately corresponding to a gaze direction of the eye, whereby at least one of the horizontal and vertical images of the at least one illuminated eye exhibit a bright reflection of the at least one infrared illuminator in respective regions of the images corresponding to the at least one eye.

15. An eye tracking system according to claim 8, further including:

a visible light illuminator configured to illuminate at least one of the first and second eyes as images are being captured by the horizontal and vertical line cameras, the visible light illuminator being controlled by the processor to induce contraction of the iris of the at least one eye; and

a pupil size monitor, in the processor, for tracking changes in size of the image of the pupil of the at least one eye in response to illumination of the at least one eye in response to the visible light illuminator.

16. An eye tracking system according to claim 8, wherein the processor further includes:

a model of pupilary changes with respect to light intensity and saccadic motion;

a comparator which compares saccadic motion and pupilary change data from a user at a given test time to the model and to data derived from other users;

a fitness algorithm that identifies data corresponding to specific model parameters that are out of normal bounds as determined from the data derived from other users and, based on the identified parameters determines a metric of fitness for the user.