Eye tracking system

• US 6,394,602 B1
• Filed: 12/14/2000
• Issued: 05/28/2002
• Est. Priority Date: 06/16/1998
• Status: Expired due to Term

• Current Assignee: Leica Instruments Singapore Pte Ltd (Danaher Corporation)
• Original Assignee: Leica Microsystems (Schweiz) AG
• Inventors: Morrison, Euan, Green, Alan Edward
• Primary Examiner(s): MANUEL, GEORGE C
• Application Number:
  US09/719,612
• Time in Patent Office: 530 Days
• Field of Search: 351/206,209,210,214,221 359/630 382/117
• US Class Current: 351/206
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G03B 2213/025 Sightline detection
• CPC Class Codes: G06V 40/14 Vascular patterns
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

An optical instrument, such as a microscope or a camera, is provided for forming a viewable image of an object. The optical instrument comprises an objective lens for forming the viewable image at an image plane, an eye sensor for sensing the direction of gaze of a user viewing the viewable image and means for controlling a controllable function of the optical instrument in dependence upon the sensed direction of gaze. The eye sensor comprises a sensor lens for focusing light reflected from the retina of the user, an imaging transducer located at a plane which is conjugate to the image plane with respect to the sensor lens, for generating an electrical image sign of a portion of the user'"'"'s retina, a memory for storing retinal image information of the user and circuitry for comparing the retinal image signal generated by the imaging transducer with the stored retinal image information to generate gaze information indicative of the direction of gaze of the user.

Claims

1. A microscope for forming a viewable image of an object, comprising:

an objective lens for forming a viewable image of the object at an image plane;

an eye sensor for sensing a direction of gaze of a user viewing the viewable image; and

means for adjusting a controllable function of said microscope in dependence upon the sensed direction of gaze;

characterised in that said eye sensor comprises;

(i) a sensor lens for focusing light reflected from the retina of the user;

(ii) an imaging transducer located at a plane which when the user views said viewable image, is commonly conjugate with said image plane to the retina of the user'"'"'s eye, for generating an electrical image signal of a portion of the user'"'"'s retina;

(iii) a memory for storing retinal image information of the user; and

(iv) comparing means for comparing signals representative of the retinal image signal generated by said imaging transducer with said stored retinal image information to generate gaze information indicative of the direction of gaze of the user.

2. The microscope according to claim 1 further comprising an eyepiece through which the user can view said viewable image, and wherein said eyepiece comprises a lens system for increasing an apparent field of view of the microscope.

3. The microscope according to claim 1 or 2, wherein said sensor lens and said imaging transducer are located off an optical axis of said microscope.

4. The microscope according to claim 3, further comprising a beam splitter for reflecting the light reflected from the retina onto said sensor lens and said imaging transducer.

5. The microscope according to claim 2, further comprising an illumination source for illuminating the user'"'"'s eye and an illumination lens for focusing the light from the illumination source.

6. The microscope according to claim 5, wherein said illumination lens is operable, in use, to focus tie light from the illumination source onto the cornea of the user'"'"'s eye.

7. The microscope according to claim 5, wherein said illumination source is operable to generate light having a wavelength in the range of about 800 nm to about 950 nm.

8. The microscope according to claim 7, wherein said illumination source comprises an 850 nm LED.

9. The microscope according to any of claims 5 to 8, wherein said illumination source is located at an angle to an optical axis of the microscope.

10. The microscope according to claim 9, further comprising a beam splitter located on the optical axis of the microscope for reflecting the light from the illumination source into the eye.

11. The microscope according to claim 5, wherein said illumination source is operable to illuminate the eye through the eyepiece.

12. The microscope according to claim 11, wherein said eyepiece comprises an anti-reflective coating for preventing light from the source from being reflected from the eyepiece into the imaging transducer.

13. The microscope according to claim 5, wherein said illumination source comprises a plurality of light emitting portions which are operable to illuminate the user'"'"'s eye at different angles.

14. The microscope according to claim 13, further comprising control means for selectively causing one or more of said light emitting portions to emit light.

15. The microscope according to claim 14, wherein said control means selects said one or more portions in dependence upon the direction of gaze of the user.

16. The microscope according to claim 14, wherein said control means selects said one or more portions in dependence upon the retinal image signal output by said imaging transducer.

17. The microscope according to claim 5, wherein said imaging transducer is operable to output a retinal image signal at a frequency between ten and fifty times per second.

18. The microscope according to claim 17, wherein the illumination source is operable to send light in pulses which are synchronised with said imaging transducer frequency.

19. The microscope according to claim 2, further comprising a half silvered minor in an intermediate image plane in order to reflect a retinal image trough the eyepiece.

20. The microscope according to claim 2, wherein said microscope includes two stereo beam paths each having an eyepiece associated therewith wherein said controllable function is an adjustment of a distance between the eyepieces to accommodate the interpupillary distance of a user'"'"'s eyes.

21. The microscope according to claim 1, wherein said imaging transducer comprises a two-dimensional CCD sensor.

22. The microscope according to claim 1, further comprising means for processing the retinal image signal to highlight characteristic features in the retinal image.

23. The microscope according to claim 22, wherein said characteristic features include blood vessels.

24. The microscope according to claim 22 or 23 wherein said processing means comprises edge detection means for detecting edges in the retinal image.

25. The microscope according to claim 1, wherein said stored retinal image information comprises a map of the user'"'"'s retina.

26. The microscope according to claim 25, wherein said retinal map is generated in advance during a calibration routine, in which the user looks at different preselected locations in the field of view of the microscope and in which the imaging transducer generates a retinal image for each of the preselected locations, from which the retinal map for the user is determined.

27. The microscope according to claim 1, wherein said controllable function is an automatic focusing of the objective lens so that a point in the field of view which the user is looking at is in focus.

28. The microscope according to claim 1, wherein said controllable function is the content of a display which is optically superimposed on the field of view of the microscope.

29. The microscope according to claim 1, wherein said imaging transducer is located at a plane conjugate to said image plane with respect to said sensor lens.

30. The microscope according to claim 1, wherein said controllable function is movement of the microscope over the object being viewed.

31. A microscope for forming a viewable image of an object, comprising:

an objective lens for forming a viewable image of the object at an image plane;

an eye sensor for sensing a direction of gaze of a user viewing the viewable image; and

means for adjusting a controllable function of said microscope in dependence upon the sensed direction of gaze;

characterised in that said eye sensor comprises;

(i) a sensor lens for focusing light reflected from the retina of the user;

an imaging transducer located at a plant conjugate to said image plane with respect to said sensor lens for generating an electrical image signal of a portion of the user'"'"'s retina;

(iii) a memory for storing retinal image information of the user; and

(iv) comparing means for comparing signals representative of the retinal image signal generated by said imaging transducer with said stored retinal image information to generate gaze information indicative of the direction of gaze of the user'"'"'s eye.

32. A microscope for forming a viewable image of an object, comprising:

an objective lens for forming a viewable image of the object at an image plane;

an eyepiece through which a user can view said viewable image;

an eye sensor for sensing a direction of gaze of said user viewing said image through said eyepiece; and

means for controlling a controllable function of said microscope in dependence upon the sensed direction of gaze;

characterised in that said eye sensor comprises;

(i) a sensor lens for focusing light reflected from the retina of the user;

(ii) an imaging transducer located at a plane which, when the user views said viewable image, is commonly conjugate with said image plane to the retina of the user'"'"'s eye, for generating an electrical image signal of the portion of the user'"'"'s retina which can be seen through said eyepiece;

(iii) a memory for storing retinal image information; and

(iv) comparing means for comparing signals representative of the retinal image signal generated by said imaging transducer with said stored retinal image information to generate gaze information indicative of the direction of gaze of the user.

33. A microscope for forming a viewable image of an object, comprising:

an objective lens for forming a viewable image of the object at an image plane;

an eye sensor for sensing a direction of gaze of a user viewing the viewable image; and

means for adjusting a controllable function of said microscope in dependence upon the sensed direction of gaze;

characterised in that said eye sensor comprises;

(i) a sensor lens for focusing light reflected from the retina of the user;

(ii) an imaging transducer located at a plane which, when the user views said viewable image, is commonly conjugate with said image plane to the retina of the user'"'"'s eye, for generating an electrical image signal of the pardon of the user'"'"'s retina which is imaged on said transducer by said sensor lens;

(iii) a memory for storing retinal image information of the user; and

(iv) comparing means for comparing signals representative of the retinal image signal generated by said imaging transducer with said stored retinal image information to generate gaze information indicative of the direction of gaze of the user.

34. A method of operating a microscope for forming viewable image of an object, comprising the steps of:

providing an objective lens for forming a viewable image of the object at an image plane;

sensing a direction of gaze of a user viewing the viewable image; and

controlling a controllable function of the microscope in dependence upon the sensed direction of gaze;

characterized in that said sensing step comprises the steps of;

(i) using a sensor lens to focus light reflected from the retina of the user

(ii) providing an imaging transducer located at a plane which, when the user views said viewable image, is commonly conjugate with said image plant to the retina of the user'"'"'s eye, for generating an electrical signal of a portion of the user'"'"'s retina; and

(iii) comparing signals representative of the retina image signal generated by the imaging transducer with stored retinal image information to generate gaze information indicative of the direction of gaze of the user.

35. An optical instrument for forming a viewable image of an object, comprising:

an objective lens for forming a viewable image of the object at in image plane;

an eye sensor for sensing a direction of gaze of a user viewing the viewable image;

means for adjusting a controllable function of said optical instrument in dependence upon the sensed direction of gaze; and

an eyepiece through which the user can view said viewable image, said eyepiece comprising a lens system for increasing an apparent field of view of the optical instrument;

characterised in that said eye sensor comprises;

(i) an illumination source for illuminating the user'"'"'s eye and an illumination lens for focusing the light from the illumination source, wherein said illumination source comprises a plurality of light emitting portions which are operable to illuminate the uses eye at different angles;

(ii) control means for selectively causing one or more of said light emitting portions to emit light in dependence upon the direction of gaze of the user;

(iii) a sensor lens for focusing light reflected from the retina of the user,

(iv) an imaging transducer located, at a plane which, when the user views said viewable image, is commonly conjugate with said image plane to the retina of the uses eye, for generating an electrical image signal of a portion of the user'"'"'s retina;

(v) a memory for storing retinal i image information of the user; and

(vi) comparing means for comparing signals representative of the retina image signal generated by said imaging transducer with said stored retina image information to generate gaze information indicative of the direction of gaze of the user.

36. An optical instrument for forming a viewable image of an object, comprising:

an objective lens for forming a viewable image of the object at an image plane;

an eye sensor for sensing a direction of gaze of a user viewing the viewable image; and

means for adjusting a controllable function of said optical instrument in dependence upon the sensed direction of gaze;

characterized in that said eye sensor comprises;

(i) a sensor lens for focusing light reflected from the retina of the user;

(ii) an imaging transducer located at a plane which, when the user views said viewable image, is commonly conjugate with said image plane to the retina of the user'"'"'s eye, for generating in electrical image signal of a portion of the user'"'"'s retina;

(iii) means for processing the retinal image signal to highlight characteristic features in the retinal image, wherein said processing means comprises edge detection means for detecting edges in the retinal image;

(iv) a memory for storing retinal image information of the user; and

(v) comparing means for comparing signals representative of the retinal image signal generated by said imaging transducer with said stoked retinal image information to generate gaze information indicative of the direction of gaze of the user.

37. The instrument according to claim 36, wherein said characteristic features include blood vessels.

38. An optical instrument for forming a viewable image of an object, comprising:

an objective lens for forming a viewable image of the object at an image plant;

an eye sensor for sensing a direction of gaze of a user viewing the viewable image; and

means for adjusting a controllable function of said optical instrument in dependence upon the sensed direction of gaze;

characterised in that said eye sensor comprises;

(i) a sensor lens for focusing light reflected from the retina of the user;

(ii) an imaging transducer located at a plane which, when the user views said viewable image, is commonly conjugate with said image plane to the retina of the user'"'"'s eye, for generating an electrical image signal of a portion of the user'"'"'s retina;

(iii) a memory for storing retinal image information of the user, wherein said stored retinal image information comprises a map of the user'"'"'s retina generated in advance during a calibration routine, in which the user looks at different preselected locations in the field of view of the instrument and in which the imaging transducer generates a retinal image for each of the preselected locations, from which the retinal map for the user is determined; and

(iv) comparing means for comparing signals representative of the retinal image signal generated by said imaging transducer with said stored retinal image information to generate gaze information indicative of the direction of gaze of the user.

39. An optical instrument for forming a viewable image of an object, comprising:

an objective lens for forming a viewable image of the object at an image plane;

an eye sensor for sensing a direction of gaze of a user viewing the viewable image; and

means for adjusting a controllable function of said optical instrument in dependence upon the sensed direction of gaze;

characterised in that said eye sensor comprises;

(i) a sensor lens for focusing light reflected from the retina of the user;

(ii) an imaging transducer located at a plane which, when the user views said viewable image, is commonly conjugate with said imago plane to the retina of the user'"'"'s eye, for generating an electrical image signal of a portion of the user'"'"'s retina;

(iii) a memory for storing retina image information of the user, and

(iv) comparing means for comparing signals representative of the retinal image signal generated by said imaging transducer with said stared retinal image information to generate gaze information indicative of the direction of gaze of the user; and

said optical instrument being farther characterised in that said controllable function is an automatic focusing of the objective lens so that a point in the field of view which the user is looking at is in focus.

40. The optical instrument according to claim 39, wherein said controllable function includes an automatic movement of the optical instrument over the object being viewed.

41. At optical instrument for forming a viewable image of an object, comprising;

an objective lens for forming a viewable image of the object at an image plane

an eye sensor for sensing a direction of gaze of a user viewing the viewable image;

means for adjusting a controllable function of said optical instrument in dependence upon the sensed direction of gaze; and

a of eyepieces through which the user can view said viewable image along stereo beam paths;

characterised in that said eye sensor comprises;

(i) a sensor lens for focusing light reflected from the retina of the user;

(ii) an imaging transducer located at a plane which, when the user views said viewable image, is commonly conjugate with said image plane to the retina of the user'"'"'s eye, for generating an electrical image signal of a portion of the user'"'"'s retina;

(iii) a memory for storing retinal image information of the user; and

(iv) comparing means for comparing signals representative of the retinal image signal generated by said imaging transducer with said stored retinal image information to generate gaze information indicative of the direction of gaze of the user; and

said optical instrument being further characterised in that said controllable function is an adjustment of a distance between the pair of eyepieces to accommodate the interpupillary distance of a user'"'"'s eyes.

42. An optical instrument for forming a viewable image of an object, comprising:

an objective lens for forming a viewable image of the object at an image plane;

an eye sensor for sensing a direction of gaze of a user viewing the viewable image; and

means for adjusting a controllable function of said optical instrument in dependence upon the sensed direction of gaze;

characterised in that said eye sensor comprises;

(i) a sensor lens for focusing light reflected from the retina of the user;

(ii) an imaging transducer located at a plane which, when the user views said viewable image, is commonly conjugate with said image plane to the retina of the user'"'"'s eye, for generating an electrical image signal of a portion of the user'"'"'s retina;

(iii) a memory for storing retinal image information of the user; and

(iv) comparing means for comparing signals representative of the retinal image signal generated by said imaging transducer with said stored retinal image information to generate gaze information indicative of the direction of gaze of the user; and

said optical instrument being further characterised in that said controllable function is the content of a display which is optically superimposed on the field of view of the instrument.