Eye movement detector with improved calibration and speed
First Claim
1. An eye movement detector comprising:
- a display having multiple individual areas representing different choices that can be made by an operator when viewed by an operator for a predetermined interval of time;
a camera with a lens located near said display and directed in a manner to obtain an image of an operator'"'"'s eye when the display is being viewed;
an illuminator directing an infrared source of illumination towards the position of an operator'"'"'s eye so that the reflected illumination causes the operator'"'"'s pupil to be brighter than the surrounding part of the eye due to the bright eye effect and also causes a glint reflection from the cornea of the eye;
a computer including means for analyzing said image of an illuminated operator'"'"'s eye to determine the center of the pupil and location of the glint and utilizing said information in determining the eye-gaze point and correlating said eye-gaze point with whether or not the operator is gazing at an individual area of the display;
said computer further including means for using said image of the operator'"'"'s eye in a digitized matrix of pixels that is used to make a determination of the pupil intensity threshold represented by the intensity of the pixels between the intensity of the pixels in the pupil and the pixel intensity represented by the part of the eye and face surrounding the pupil and also determining the glint intensity threshold;
means included in said computer for determining said eye-gaze point be utilizing said pupil threshold in first scanning the pixels in said matrix by a rough scan of the pixels both horizontally and vertically by periodically skipping a predetermined number of the pixels both horizontally and vertically during the scan with said number of skipped pixels in the horizontal direction and the vertical direction being less than the diameter of said pupil and with said rough scan being utilized to determine a first chord of predetermined length passing through the pupil image.
3 Assignments
0 Petitions
Accused Products
Abstract
A system for eye movement detection is disclosed that utilizes an infrared light emitting diode mounted coaxially in front of the lens of an infrared sensitive video camera for remotely making images of the eye of a computer operator. The reflected light causes bright eye effect which outlines the pupil as brighter than the rest of the eye and also causes an even bright small glint from the surface of the cornea. The computer includes graphic processing which takes a video image, digitizes it into a matrix of pixels and analyzes the matrix. Using special algorithms the analysis calibrates the system to provide a highly accurate resolution and has a quick scan technique to rapidly determine the location of the pupil'"'"'s center and the location of the glint relative to each other and with this information determines where the eye is gazing. If the eye-gaze is for a predetermined time at images in selected areas on the computer screen, the area is selected and results in actuation of other devices or the presentation of additional images on the screen. This is especially usable for handicapped persons to control their environment. Other uses include operator interfacing with workstations, cockpit controls and in industrial environments.
-
Citations
15 Claims
-
1. An eye movement detector comprising:
-
a display having multiple individual areas representing different choices that can be made by an operator when viewed by an operator for a predetermined interval of time; a camera with a lens located near said display and directed in a manner to obtain an image of an operator'"'"'s eye when the display is being viewed; an illuminator directing an infrared source of illumination towards the position of an operator'"'"'s eye so that the reflected illumination causes the operator'"'"'s pupil to be brighter than the surrounding part of the eye due to the bright eye effect and also causes a glint reflection from the cornea of the eye; a computer including means for analyzing said image of an illuminated operator'"'"'s eye to determine the center of the pupil and location of the glint and utilizing said information in determining the eye-gaze point and correlating said eye-gaze point with whether or not the operator is gazing at an individual area of the display; said computer further including means for using said image of the operator'"'"'s eye in a digitized matrix of pixels that is used to make a determination of the pupil intensity threshold represented by the intensity of the pixels between the intensity of the pixels in the pupil and the pixel intensity represented by the part of the eye and face surrounding the pupil and also determining the glint intensity threshold; means included in said computer for determining said eye-gaze point be utilizing said pupil threshold in first scanning the pixels in said matrix by a rough scan of the pixels both horizontally and vertically by periodically skipping a predetermined number of the pixels both horizontally and vertically during the scan with said number of skipped pixels in the horizontal direction and the vertical direction being less than the diameter of said pupil and with said rough scan being utilized to determine a first chord of predetermined length passing through the pupil image. - View Dependent Claims (2, 3, 4, 5)
-
-
6. A method for determining eye-gaze direction comprising the following steps:
-
placing on a display multiple individual areas representing different choices that can be made by an operator when viewed by an operator for a predetermined interval of time; obtaining an image of an operator'"'"'s eye by a camera with lens located near the display and when the display is being viewed; illuminating the operator'"'"'s eye by directing an infrared source of illumination towards the position of an operator'"'"'s eye so that the reflected illumination causes the operator'"'"'s pupil to be brighter than the surrounding part of the eye due to the bright eye effect and also causes a glint reflection from the cornea of the eye; using said image of the operator'"'"'s eye to produce a digitized matrix of pixels that is used to make a determination of the pupil intensity threshold represented by the intensity of the pixels between the intensity of the pixels and the pupil and the pixel intensity represented by the part of the eye and face surrounding the pupil and also determining the glint intensity threshold; analyzing said digitized matrix of pixels by first scanning the pixels in said matrix by a rough scan of the pixels both horizontally and vertically by periodically skipping a predetermined number of the pixels both horizontally and vertically during the scan with said number of skipped pixels in the horizontal direction and the vertical direction being less than the diameter of said pupil; utilizing said rough scan to determine a first chord of predetermined length passing through the pupil image; calculating the midpoint of said first chord of predetermined length and then determining a second chord at right angle to said first chord which passes through the midpoint of said first chord; determining a third chord at right angles to said second chord that passes through the midpoint of said second chord; and creating a region of interest in said matrix that surrounds said pupil that is fractionally larger than the length of said second chord and the length of said third chord with an approximate center of said region of interest being the center of said pupil. - View Dependent Claims (7, 8, 9)
-
-
10. An eye movement detector comprising:
-
a display having multiple individual areas representing different choices that can be made by an operator when viewed by an operator for a predetermined interval of time; a camera with a lens located near said display and directed in a manner to obtain an image of an operator'"'"'s eye when the display is being viewed; an illuminator directing an infrared source of illumination towards the position of an operator'"'"'s eye so that the reflected illumination causes the operator'"'"'s pupil to be brighter than the surrounding part of the eye due to the bright eye effect and also causes a glint reflection from the cornea of the eye; a computer including means for analyzing said image of an illuminated operator'"'"'s eye to determine the location of the center of the pupil and location of the glint and utilizing said information in determining the eye-gaze point and correlating said eye-gaze point with whether or not the operator is gazing at an individual area of the display; said computer analysis including means using said image of the operator'"'"'s eye in a digitized matrix of pixels that is used to make a determination of the location of the center of the pupil and the location of the glint; said computer further including means for calibration of the eye movement detector determined by displaying at least two calibration points in two separate areas of said display and determining the uncalibrated eye-gaze location when the operator is looking at a calibration point followed by a calculation based on the known location of the calibration look point and the uncalibrated look point of the eye to calculate the calibration coefficients for both horizontal and vertical directions using a regression function on the calibration points and utilising said calibration coefficients for accurate eye-gaze point determinations when using said eye movement detector. - View Dependent Claims (11, 12)
-
-
13. A method for determining eye-gaze direction calibration coefficients comprising the following steps:
-
placing on a display a first calibration point at a known location to be viewed by an operator for a predetermined interval of time; illuminating an operator'"'"'s eye by directing an infrared source of illumination towards the position of the operator'"'"'s eye so that the reflected illumination causes the operator'"'"'s pupil to be brighter than the surrounding part of the eye due to the bright eye effect and also causes a glint reflection from the cornea of the eye; obtaining an image in the form of a digitized matrix of pixels of an operator'"'"'s eye by a camera with lens located near the display and when the first calibration point on a display is being viewed; analyzing said digitized matrix of pixels to determine the location of the center of the pupil and the location of the glint and from said analysis determining the uncalibrated eye gaze point for said first calibration point; placing on a display a second calibration point at a known location to be viewed by the operator for a predetermined interval of time; obtaining a second image in the form of a digitized matrix of pixels of the operator'"'"'s eye by said camera when the second calibration point is being viewed; analyzing said second digitized matrix of pixels to determine the location of the center of the pupil and location of the glint and from said analysis determining the uncalibrated eye gaze point for said second calibration point; calculating the calibration coefficient on both horizontal and vertical directions using non-linear regression functions applied to said first and second uncalibrated eye gaze points when looking at the known location of said first and second calibration points; and utilizing said calibration coefficients for accurate determination of look points when using said eye movement detector. - View Dependent Claims (14, 15)
-
Specification