Method and installation for detecting and following an eye and the gaze direction thereof

• US 20060238707A1
• Filed: 11/21/2003
• Published: 10/26/2006
• Est. Priority Date: 11/21/2002
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Elvers, Gunnar, Elvesjo, John
• Granted Patent: US 7,572,008 B2
• Time in Patent Office: Days
• US Class Current: 351/209
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

When detecting the position and gaze direction of eyes, a photo sensor (1) and light sources (2, 3) placed around a display (1) and a calculation and control unit (6) are used. One of the light sources is placed around the sensor and includes inner and outer elements (3′; 3″). When only the inner elements are illuminated, a strong bright eye effect in a captured image is obtained, this resulting in a simple detection of the pupils and thereby a safe determination of gaze direction. When only the outer elements and the outer light sources (2) are illuminated, a determination of the distance of the eye from the photo sensor is made. After it has been possible to determine the pupils in an image, in the following captured images only those areas around the pupils are evaluated where the images of the eyes are located. Which one of the eyes that is the left eye and the right eye can be determined by following the images of the eyes and evaluating the positions thereof in successively captured images.

Claims

1. An eye detection installation comprising

one or more light sources for emitting light in directions toward the head of a user,

a detector for receiving light from the head of a user and to repeatedly capture pictures thereof, and

an evaluation unit connected to the detector for determining the position and/or gaze direction of an eye,

characterized in that the evaluation unit is arranged

to determine, in a picture captured by the detector, an area in which an image of an eye or images of eyes is/are located, and

after determining the area, to control the detector to forward to the evaluation unit information about successive or following pictures that only corresponds to the determined area of the image captured by the detector.

2. An eye detection installation according to claim 1, characterized in that the detector is arranged to only read out information from that portion of the detector surface that corresponds to the determined area and thereby the data that are to be then forwarded to the evaluation unit.

3. An eye detection installation according to claim 1, characterized in that in the case where the evaluation unit cannot from the forwarded information execute the determination, the evaluation unit is arranged to control the detector to forward for the next picture information about a larger portion of the detector around the previously determined area.

4. An eye detection installation according to claim 1, characterized in that the evaluation unit is arranged

to decide in a current picture captured by the detector whether the picture contains images of the two eyes of a user, and

in the case where the evaluation unit decides that an image of only one eye exists in the current picture, to determine that this eye is the same eye that has an image within a previously captured picture, provided that the image of the eye has a position in the current picture that is sufficiently close to the position of the image of the eye in the previously captured picture.

5. An eye detection installation according to claim 4, characterized in that in the case where the evaluation unit decides that an image of only one eye exists in the current picture, the position of the image of the eye in the current picture does not correspond to or is sufficiently close to the position of any eye in one or more previously captured pictures, and the position of the image of the eye in the current picture is such that the lateral distance from one edge of the current picture is smaller than but the lateral distance from the other edge is larger than a distance that corresponds to the distance between the user'"'"'s eyes, the evaluation unit is arranged to take the eye, an image of which exists in the current picture, to be the eye that means that an image of the other eye of the user would be located outside the current picture.

6. An eye detection installation according to claim 1, characterized in

that at least two light sources are provided and are placed at a distance from each other for emitting at least two light beams to be reflected from the cornea of an eye of a user, and

that the evaluation unit is arranged to use in a captured image the positions of the images of the reflections of the light sources to determine the location of the eye in relation to the detector.

7. An eye detection installation according to claim 6, characterized in that the evaluation unit is arranged to determine the distance between images of the reflections of the light sources in a captured picture to determine therefrom the distance of the eye from the detector.

8. An eye detection installation according to claim 6, characterized in that at least three light sources are provided in a definite pattern, the evaluation unit arranged to determine the positions of images of the reflections of the light sources and to use all the determined positions to determine the location of the eye in relation to the detector.

9. An eye detection installation according to claim 6, characterized in that the light sources are divided in two groups, a first group of which is arranged to emit light suited to determine, from pictures captured with illumination from only this group, the gaze direction of the eye, and a second group of which is arranged to emit light suited to determine, from pictures captured with illumination from only this group, the distance of the eye from the detector, the control unit arranged to switch either one of or both of these two groups on in capturing each picture.

10. An eye detection installation according to claim 6, characterized in that one of the light sources is arranged to emit light in a light beam coaxial with the optical axis of the detector.

11. An eye detection installation according to claim 6, characterized in that the light sources are divided in two groups, a first group of which is arranged to emit light that causes a bright eye effect and hence is suited to determine, from images captured with illumination from only this group, the gaze direction of the eye, and a second group of which is arranged to emit light suited to determine, from pictures captured with illumination from only this group, the distance of the eye from the detector, the control unit being arranged to activate either one or both these groups in capturing each picture.

12. An eye detection installation comprising

one or more light sources for emitting light in directions towards the head of a user,

a detector for receiving light from the head of a user and to repeatedly capture pictures thereof, and

an evaluation unit connected to the detector to determine the position and/or gaze direction of an eye, characterized in that the evaluation unit is arranged

to determine, in a current picture captured by the detector, whether the picture contains images of the two eyes of a user, and

to determine in the case where the evaluation unit decides that an image of only one eye exists in the current picture that this eye is the same eye, an image of which exists in a previously captured picture, provided that the image of the eye has a position in the current picture that is sufficiently close to the position of the image of the eye of the previously captured image.

13. An eye detection installation according to claim 12, characterized in that in the case where the evaluation unit decides that an image of only one eye exists in the current picture, that the position of the image of the eye in the current picture does not correspond to or is sufficiently close to the position of any eye in one or more previously captured pictures and that the position of the image of the eye in the current picture is such that the lateral distance from one edge of the current picture smaller than but the lateral distance from the other edge is larger than a distance corresponding to the distance between the eyes of the user, the evaluation unit is arranged to take that eye, an image of which exists in the current picture, to be the eye that means that an image of the other eye of the user would be located outside the current picture.

14. An eye detection installation comprising

at least two light sources for emitting light in directions towards the head of a user,

a detector for receiving light from the head of a user and for repeatedly capturing pictures thereof, and

an evaluation unit connected to the detector,

characterized in

that at least two of the light sources are placed at a distance from each other for emitting at least two light beams to be reflected from an eye of a user, and

that the evaluation unit is arranged to use in the captured image the positions of the images of the reflections of the light sources to determine the location of the eye in relation to the detector.

15. An eye detection installation according to claim 14, characterized in that the evaluation unit is arranged to determine the distance between images of the reflections of the light sources in the captured image to determine therefrom the distance of the eye from the detector.

16. An eye detection installation according to claim 14, characterized by at least three light sources arranged in a definite pattern, the evaluation unit arranged to determine the positions of images of the reflections of the light sources and to use all the determined positions to determine the location of the eye in relation to the detector.

17. An eye detection installation according to claim 14, characterized in that the light sources are divided in two groups, a first group of which is arranged to emit light suited to determine, from pictures captured with illumination from only this group, the gaze direction of the eye, and a second group of which is arranged to emit light suited to determine, from pictures captured with illumination from only this group, the distance of the eye from the detector, the control unit arranged to activate either one of or both of these group in capturing each picture.

18. An eye detection installation according to claim 14, characterized by at least three light sources, at least two of which are placed at an edge, in particular the upper or lower edge, of a monitor or display and one of which is placed at an opposite edge of the monitor or display, the evaluation unit arranged to determine the positions of images of the reflections of the light sources and to use the determined positions to determine the location of the eye in relation to the detector.

19. An eye detection installation according to claim 14, characterized in that one of the light sources is arranged to emit light in a light beam coaxial with the optical axis of the detector.

20. An eye detection installation according to claim 14, characterized in that the light sources are divided in two groups, a first group of which is arranged to emit light that causes a bright eye effect and hence is suited to determine, from pictures captured with illumination from only this group, the gaze direction of the eye, and a second group of which is arranged to emit light suited to determine, from images captured with illumination from only this group, the distance of the eye from the detector, the control unit arranged to activate either one of or both of these groups in capturing each picture.