Method and installation for detecting and following an eye and the gaze direction thereof

• US 7,572,008 B2
• Filed: 11/21/2003
• Issued: 08/11/2009
• Est. Priority Date: 11/21/2002
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Skogo, Marten, Elvers, Gunnar, Elvesjo, John
• Primary Examiner(s): Mack; Ricky L
• Assistant Examiner(s): Greece; James R
• Application Number:
  US10/535,227
• Publication Number:
  US 20060238707A1
• Time in Patent Office: 2,090 Days
• Field of Search: 351/205, 351/206, 351/208, 396/51
• US Class Current: 351/206
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

When detecting the position and gaze direction of eyes, a photo sensor (1) and light sources (2, 3) placed around a display (1) and a calculation and control unit (6) are used. One of the light sources is placed around the sensor and includes inner and outer elements (3′; 3″). When only the inner elements are illuminated, a strong bright eye effect in a captured image is obtained, this resulting in a simple detection of the pupils and thereby a safe determination of gaze direction. When only the outer elements and the outer light sources (2) are illuminated, a determination of the distance of the eye from the photo sensor is made. After it has been possible to determine the pupils in an image, in the following captured images only those areas around the pupils are evaluated where the images of the eyes are located. Which one of the eyes that is the left eye and the right eye can be determined by following the images of the eyes and evaluating the positions thereof in successively captured images.

Claims

1. An eye detection installation comprising:

one or more light sources for emitting light in directions toward the head of a user,

a detector for receiving light from the head of a user and to repeatedly capture pictures thereof, the detector having a light sensitive surface comprising a plurality of picture elements, and

an evaluation unit connected to the detector for determining the position and/or gaze direction of an eye, the evaluation unit being arranged to determine, in a picture captured by the detector, an area in which an image of an eye or images of eyes is/are located, and, after determining the area, to control the detector to forward to the evaluation unit information about successive or following pictures that only corresponds to the determined area of the image captured by the detectors,

the detector only reading out information from a portion of the detector'"'"'s surface that corresponds to the determined area, and thereby, data that are then to be forwarded to the evaluation unit, the determined area representing less than all the picture elements of the light sensitive surface.

4. The eye detection installation according to claim 1, wherein in the case where the evaluation unit cannot from the forwarded information execute the determination, the evaluation unit is arranged to control the detector to forward for the next picture information about a larger portion of picture elements of the light sensitive surface around the previously determined area.

5. The eye detection installation according to claim 1, wherein the evaluation unit is arranged:

to decide in the determined area representing less than all the picture elements of the light sensitive surface captured by the detector whether the picture contains images of the two eyes of a user, and

in the case where the evaluation unit decides that an image of only one eye exists in the the determined area representing less than all the picture elements of the light sensitive surface, to determine that this eye has a position in said determined area that is sufficiently close to the position of the image of the eye in the previously captured image.

6. The eye detection installation according to claim 5, wherein in the case where the evaluation unit decides that an image of only one eye exists in the the determined area representing less than all the picture elements of the light sensitive surface, the position of the image of the eye in said determined area does not correspond to or is sufficiently close to the position of any eye in one or more previously captured pictures, and the position of the image of the eye in said determined area is such that the lateral distance from one edge of said determined area is smaller than but the lateral distance from the other edge is larger than a distance that corresponds to the distance between the user'"'"'s eyes, the evaluation unit is arranged to take the eye, an image of which exists in the said determined area, to be the eye that means that an image of the other eye of the user would be located outside said determined area.

7. The eye detection installation according to claim 1, wherein

at least two light sources are provided and placed at a distance from each other for emitting at least two light beams to be reflected from the cornea of an eye of a user, and

the evaluation unit is arranged to use in a captured image the positions of the images of the reflections of the light sources to determine the location of the eye in relation to the detector.

8. The eye detection installation according to claim 7, wherein the evaluation unit is arranged to determine the distance between images of the reflections of the light sources in a captured picture to determine there from the distance to the eye from the detector.

9. The eye detection installation according to claim 7, wherein at least three light sources are provided in a definite pattern, the evaluation unit arranged to determine the positions of images of the reflections of the light sources and to use all the determined positions to determine the location of the eye in relation to the detector.

10. The eye detection installation according to claim 7, wherein the light sources are divided into groups, a first group of which is arranged to emit light suited to determine, from pictures captured with illumination from only this group, the gaze direction of the eye, and a second group of which is arranged to emit light suited to determine, from pictures captured with illumination from only this group, the distance of the eye from the detector, the control unit arranged to switch either one of or both these groups on in capturing each picture.

11. The eye detection installation according to claim 7, wherein one of the light sources is arranged to emit light in a light beam coaxial with the optical axis of the detector.

12. The eye detection installation according to claim 7, wherein the light sources are divided in two groups, a first group of which is arranged to emit light that causes a bright eye effect and hence is suited to determine, from images captured with illumination from only this group, the gaze direction of the eye, and a second group of which is arranged to emit light suited to determine, from pictures captured with illumination from only this group, the distance of the eye from the detector, the control unit being arranged to activate either one or both of these groups in captured in each picture.

2. An eye detection installation comprising:

one or more light sources for emitting light in directions toward the head of a user,

a detector for receiving light from the head of a user and to repeatedly capture pictures thereof, and

an evaluation unit connected to the detector for determining the position and/or gaze direction of an eye, the evaluation unit being arranged;

to determine, in a picture captured by the detector, an area in which an image of an eye or images of eyes is/are located,

after determining the area, to control the detector to forward to the evaluation unit information about successive or following pictures that only corresponds to the determined area of the image captured by the detector,

to decide, in a current picture captured by the detector, whether the picture contains images of the two eyes of a user,

in a case where the evaluation unit decides that an image of only one eye exists in the current picture, to determine that the one eye is the same eye that has an image within a previously captured picture, provided that the image of the eye has a position in the current picture that is sufficiently close to the position of the image of the eye in the previously captured picture, and

in the case where the evaluation unit decides that an image of only one eye exists in the current picture, the position of the image of the eye in the current picture does not correspond to or is sufficiently close to the position of any eye in one or more previously captured pictures, and the position of the image of the eye in the current picture is such that the lateral distance from one edge of the current picture is smaller than, but the lateral distance from the other edge is larger than, a distance that corresponds to the distance between the user'"'"'s eyes, to take the eye, an image of which exists in the current picture, to be the eye that means that an image of the other eye of the user would be located outside the current picture.

3. An eye detection installation comprising:

one or more light sources for emitting light in directions towards the head of a user,

a detector for receiving light from the head of a user and to repeatedly capture pictures thereof, and

an evaluation unit connected to the detector to determine the position and/or gaze direction of an eye, the evaluation unit being arranged;

to determine, in a current picture captured by the detector, whether the picture contains images of the two eyes of a user,

to determine in the case where the evaluation unit decides that an image of only one eye exists in the current picture that the one eye is the same eye, an image of which exists in a previously captured picture, provided that the image of the eye has a position in the current picture that is sufficiently close to the position of the image of the eye of the previously captured image, and

in the case where the evaluation unit decides that an image of only one eye exists in the current picture, that the position of the image of the eye in the current picture does not correspond to or is sufficiently close to the position of any eye in one or more previously captured pictures and that the position of the image of the eye in the current picture is such that the lateral distance from one edge of the current picture smaller than but the lateral distance from the other edge is larger than a distance corresponding to the distance between the eyes of the user, to take that eye, an image of which exists in the current picture, to be the eye that means that an image of the other eye of the user would be located outside the current picture.