Eye tracking systems

• US 20040196433A1
• Filed: 02/11/2004
• Published: 10/07/2004
• Est. Priority Date: 08/15/2001
• Status: Active Grant

• Current Assignee: Qinetiq Limited (QinetiQ Group PLC)
• Original Assignee: Qinetiq Limited (QinetiQ Group PLC)
• Inventors: Durnell, L.aurence
• Granted Patent: US 7,391,887 B2
• Time in Patent Office: Days
• US Class Current: 351/209
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

An eye tracking system for monitoring the movement of a user'"'"'s eye comprises an eye camera (2) and a scene camera (4) for supplying to interlace electronics (6) video data indicative of an image of the user'"'"'s eye and an image of the scene observed by the user. In addition the system incorporates a frame grabber (8) for digitising the video data and for separating the eye and scene data into two processing channels, and a spot location module (12) for determining from the video data the location of a reference spot formed on the user'"'"'s eye by illumination of the user'"'"'s eye by a point source of light. The spot location module (12) includes adaptive threshold sub-module for providing an indication of parts of the image produced by the eye camera (2) which have a brightness greater than a threshold value, and a spot identification sub-module for selecting a valid reference spot by comparing those parts of the image with predetermined validity criteria. The system further incorporates a pupil location module (14) for determining the location of the pupil of the user'"'"'s eye relative to the reference spot in order to determine the user'"'"'s line of gaze, and a display for indicating the user'"'"'s point of regard from the user'"'"'s line of gaze determined by the pupil and spot location modules. Such an arrangement provides a fast and versatile eye tracking system.

Claims

1. An eye tracking system for monitoring the movement of a user'"'"'s eye, the system comprising:

(a) video data input means for receiving video data produced by eye imaging means monitoring the user'"'"'s eye;

(b) spot location means for determining from the video data the location of a reference spot formed on the user'"'"'s eye by illumination of the user'"'"'s eye by a point source of light, the spot location means including adaptive threshold means for providing an indication of parts of the image produced by the eye imaging means which have a brightness greater than a threshold value, and spot identification means for selecting a valid reference spot by comparing said parts of the image with predetermined validity criteria;

(c) pupil location means for determining from the video data the location of the centre of the pupil of the user'"'"'s eye relative to the reference spot in order to determine the user'"'"'s line of gaze, the pupil location means including selection means for selecting a pupil tracking window comprising a portion of the image produced by the eye imaging means containing the pupil relative to the valid reference spot location, edge determination means for determining an edge of the pupil by selection of those parts of said image portion having a gradient greater than a threshold value, and centre determination means for location of the centre of the pupil by making reference to selected points about the edge of the pupil; and

(d) display means for indicating the user'"'"'s point of regard from the user'"'"'s line of gaze determined by the pupil and spot location means.

2. A tracking system according to claim 1, wherein the video data input means is adapted to additionally receive video data produced by scene imaging means monitoring a scene being observed by the user'"'"'s eye, and the display means is adapted to indicate the user'"'"'s point of regard relative to an image of the scene obtained from the video data.

3. A tracking system according to claim 1, wherein the adaptive threshold means is adapted to set a plurality of pixels of eye image data such that the pixels having a brightness greater than the threshold value are set to a predetermined high level and the pixels having a brightness not greater than the threshold value are set to a predetermined low level.

4. A tracking system according to claim 1, wherein the spot location means includes blob classification means for determining the location and geometry of the parts of the image indicated by the adaptive threshold means.

5. A tracking system according to claim 4, wherein the spot identification means is adapted to select valid spots by comparing the output of the blob classification means with predetermined validity criteria.

6. A tracking system according to claim 1, wherein the spot location means includes multiple spot classification means for identifying a reference spot by searching for the existence of further valid spots in search areas at defined locations relative to each reference spot indicated by the spot identification means.

7. A tracking system according to claim 1, wherein the edge determination means providies a gradient image of the edge of the pupil by convoluting said image portion in the pupil tracking window with an edge operator.

8. A tracking system according to claim 1, wherein the adaptive threshold means is adapted to provide an indication of those parts of a sub-image of the pupil produced by the eye imaging means which have a brightness greater than a threshold value.

9. A tracking system according to claim 1, wherein the pupil location means includes difference means for providing an output from which the influence of the reference spot and any related spots has been removed prior to selection of those parts of the gradient of said image portion in the pupil tracking window which have a gradient greater than the threshold value.

10. A tracking system according to claim 1, wherein the centre determination means includes triad selection means for selecting substantially randomly three super-threshold pixels to form a triad for further processing, from among a plurality of pixels of pupil image data.

11. A tracking system according to claim 10, wherein the centre determination means further includes triad processing means for determining the centre and radius of a hypothetical circle passing through each of the selected pixels.

12. A tracking system according to claim 11, wherein the centre determination means further includes accumulating means for accumulating a plurality of sets of values of the centre and radius determined by the triad processing means for a plurality of triads successively selected by the triad selection means.

13. A tracking system according to claim 12, wherein the centre determination means further includes peak determination means for determining, from the sets of values accumulated by the accumulating means, the values of the centre and radius of the pupil.

14. A tracking system according to claim 1, further comprising log file means for maintaining a log file of successive values of the user'"'"'s point of regard in the scene.

15. A tracking system according to claim 1, further including eye imaging means for monitoring the user'"'"'s eye.

16. A tracking system according to claim 1, further including scene imaging means for monitoring the scene being observed by the user'"'"'s eye.

17. A tracking system according to claim 1, further including video data recording means for recording video data produced by the eye imaging means and scene imaging means.

18. A tracking system according to claim 1, further including video data playback means for reproducing video data produced by the eye imaging means and scene imaging means.

19. An eye tracking system for monitoring the movement of a user'"'"'s eye, the system comprising:

(a) video data input means for receiving video data produced by eye imaging means monitoring the user'"'"'s eye and scene imaging means monitoring a scene being observed by the user'"'"'s eye;

(b) spot location means for determining from the video data the location of at least one reference spot formed on the user'"'"'s eye by illumination of the user'"'"'s eye by a point source of light;

(c) pupil location means for determining the location of the pupil of the user'"'"'s eye relative to the reference spot in order to determine the user'"'"'s line of gaze; and

(d) scene overlay means for indicating the user'"'"'s point of regard in the scene from the user'"'"'s line of gaze determined by the pupil and spot location means relative to an image of the scene obtained from the video data.

20. A tracking system according to claim 19, wherein the spot location means includes adaptive threshold means for providing an indication a of those parts of the image produced by the eye imaging means which have a brightness greater than a threshold value.

21. A tracking system according to claim 20, wherein the spot location means includes spot identification means for selecting a valid reference spot by comparing said parts of the image with predetermined validity criteria.

22. and 23 (canceled)