Virtual retinal display with eye tracking

• US 6,154,321 A
• Filed: 03/30/1999
• Issued: 11/28/2000
• Est. Priority Date: 01/20/1998
• Status: Expired due to Term

• Current Assignee: University of Washington
• Original Assignee: University of Washington
• Inventors: Johnston, Richard S., Melville, Charles D.
• Primary Examiner(s): MACK, RICKY LEVERN
• Application Number:
  US09/281,768
• Time in Patent Office: 609 Days
• Field of Search: 359/629, 359/630, 359/633, 345/7, 345/8, 351/206, 351/209, 351/211, 351/246, 382/117
• US Class Current: 359/630
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 7/287 including a sight line detecting device
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G09G 3/001 using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background slide projectors per se G03B23/00 = 42 HP

Abstract

Light emitted from a virtual retinal display light source passes through a beamsplitter to a scanning subsystem and on to an eyepiece and the viewer'"'"'s eye. Some of the light is reflected from the viewer'"'"'s eye passing back along the same path. Such light however is deflected at the beamsplitter toward a photodetector. The reflected light is detected and correlated to the display scanner'"'"'s position. The content of the reflected light and the scanner position for such sample is used to generate a map of the viewer'"'"'s retina. Such map includes `landmarks` such as the viewer'"'"'s optic nerve, fovea, and blood vessels. The map of the viewer'"'"'s retina is stored and used for purposes of viewer identification. The viewer'"'"'s fovea position is monitored to track where the viewer is looking.

Claims

1. A method for determining viewing orientation of the viewer'"'"'s eye with a virtual retinal display, the method comprising the steps of:

generating visible light;

receiving the generated visible light at an input of a lensing system, the lensing system defining an optical path from the input toward an output, the lensing system output adapted for transmitting light to and receiving light from a viewer'"'"'s eye;

receiving, at the lensing system, output light reflected from the viewer'"'"'s eye;

directing a portion of the light reflected from the viewer'"'"'s eye toward an optical detector;

generating an electrical signal corresponding to the directed portion of light; and

analyzing the electrical signal over time to identify select characteristics of the viewer'"'"'s eye, the select characteristics being indicative of the viewing orientation.

2. The method of claim 1, in which the select characteristics are retinal characteristics.

3. The method of claim 1, in which the select characteristics are iris characteristics.

4. The method of claim 1, in which the select characteristics are capillary structures of the viewer'"'"'s eye.

5. The method of claim 1, wherein the select characteristics correspond to a fovea of the viewer'"'"'s eye.

6. The method of claim 1, further comprising the step of:

generating a plurality of signals indicative of a map of the viewer'"'"'s eye based upon a group of said electrical signals produced over time.

7. The method of claim 1, further comprising the step of:

generating a virtual image output to the viewer'"'"'s eye through the lensing system; and

overlaying a display object within the virtual image as a function of the viewing orientation.

8. The method of claim 1, further comprising the step of

deflecting the generated light along a raster pattern;

and

concurrently deflecting the reflected light received from the viewer'"'"'s eye.

9. A method for identifying a viewer'"'"'s eye orientation with a virtual retinal display, the method comprising the steps of:

receiving an image data signal at the display to define image content of an image to be scanned upon a viewer'"'"'s retina;

generating light modulated as a function of the image data signal;

deflecting the light along a raster pattern onto the viewer'"'"'s eye with a light scanner;

receiving returning light reflected from the viewer'"'"'s eye at the light scanner,

deflecting the returning light;

directing at least a portion of the deflected light toward an optical detector;

generating a first signal at the optical detector in response to detection of the returning light; and

identifying a viewer'"'"'s eye orientation based upon relative location of a select data pattern within the first signal.

10. The method of claim 9, in which the select data pattern corresponds to retinal characteristics of the viewer'"'"'s eye.

11. The method of claim 9, in which the select data pattern corresponds to iris characteristics of the viewer'"'"'s eye.

12. The method of claim 9, in which the select data pattern corresponds to capillary structures of the viewer'"'"'s eye.

13. The method of claim 9, wherein the select data pattern corresponds to a fovea of the viewer'"'"'s eye.

14. The method of claim 9, further comprising the step of positioning a display object within the virtual image as a function of the viewer'"'"'s eye orientation.

15. A method for generating a map of a viewer'"'"'s retina with a virtual retinal display, the method comprising the steps of:

generating light;

receiving the generated light at an input of a lensing system, the lensing system defining a first optical path from the input toward an output, the lensing system output adapted for transmitting light to and receiving light from a viewer'"'"'s eye;

receiving light reflected from the viewer'"'"'s eye at a beamsplitter located in the optical path;

redirecting with the beamsplitter the reflected light along an alternative optical path;

detecting the redirected light with an optical detector positioned in the alternative optical path, the optical detector responsive to the redirected light to produce an electrical signal corresponding to the redirected light;

determining viewing orientation of the viewer'"'"'s eye using the electrical signal;

generating a virtual image output to the viewer'"'"'s eye through the lensing system; and

overlaying a virtual display object within the virtual image responsive to the determined eye orientation.

16. A method for generating a map of a viewer'"'"'s iris characteristics with a virtual retinal display, the method comprising the steps of:

receiving an image data signal at the display to define image content of an image to be scanned upon a viewer'"'"'s retina;

generating light modulated as a function of the image data signal;

deflecting the light along a raster pattern with a light scanner;

receiving returning light reflected from the viewer'"'"'s eye at the light scanner,

deflecting the returning light toward a beamsplitter;

directing said deflected returning light with the beamsplitter toward an optical detector;

generating a first signal at the optical detector in response to detection of the returning light; and

correlating respective samples of the first signal to corresponding timing positions within the raster pattern, wherein the correlated samples define the map of the viewer'"'"'s iris characteristics.

17. The method of claim 16, further comprising the steps of:

storing the map as a first map; and

comparing a second map to the first map to determine whether the first map and second map correspond to a same eye.

18. The method of claim 17, further comprising the step of identifying a viewer'"'"'s eye position over time based upon relative location of a select data pattern within the first signal, wherein the select data pattern corresponds to a fovea of the viewer'"'"'s eye.

19. The method of claim 16, further comprising the step of overlaying a virtual display object within the virtual image as a function of the viewer'"'"'s eye position.

20. A method for generating a map of a viewer'"'"'s eye capillary structures with a virtual retinal display, the method comprising the steps of:

receiving an image data signal at the display to define image content of an image to be scanned upon a viewer'"'"'s retina;

generating light modulated as a function of the image data signal;

deflecting the light along a raster pattern with a light scanner;

receiving returning light reflected from the viewer'"'"'s eye at the light scanner,

deflecting the returning light toward a beamsplitter;

directing said deflected returning light with the beamsplitter toward an optical detector;

generating a first signal at the optical detector in response to detection of the returning light; and

correlating respective samples of the first signal to corresponding timing positions within the raster pattern, wherein the correlated samples define the map of the viewer'"'"'s eye capillary structures.

21. The method of claim 20, further comprising the steps of:

storing the map as a first map; and

comparing a second map to the first map to determine whether the first map and second map correspond to a same eye.

22. The method of claim 20, further comprising the step of identifying a viewer'"'"'s eye position over time based upon relative location of a select data pattern within the first signal, wherein the select data pattern corresponds to a fovea of the viewer'"'"'s eye.

23. The method of claim 20, further comprising the step of overlaying a virtual display object within the virtual image as a function of the viewer'"'"'s eye position.

24. A display apparatus, comprising:

an image source for generating light;

a lensing system having an input aligned to the image source for receiving the generated light, the lensing system having an output adapted for transmitting light to and receiving light from a viewer'"'"'s eye;

an optical detector, wherein the lensing system defines a first optical path from the input to the output and a second optical path from output to the optical detector, and wherein the optical detector is responsive to light along the second optical path to produce an electrical signal corresponding to said light along the second optical path;

a memory device storing data representing a reference level; and

a processor coupled to the detector and to the memory device, the processor being responsive to the electrical signal and the data to produce a signal indicative of a relationship between the electrical signal and the reference level.

25. The display apparatus of claim 24, in which over time the processor produces a first group of indicative signals which are stored in the memory device and correspond to a map of the viewer'"'"'s eye.

26. The display apparatus of claim 25, in which the map is a first map, and wherein over time the processor produces a second group of indicative signals which correspond to a second map of the viewer'"'"'s eye, the processor comparing the second map to the first map to determine whether the first map and second map correspond to a common eye.

27. The display apparatus of claim 26, in which the processor comprises means for identifying a viewer'"'"'s eye orientation based upon relative location of a select data pattern within the electrical signal.

28. The display apparatus of claim 27, in which the processor comprises means for overlaying a virtual display object within the virtual image as a function of the viewer'"'"'s eye orientation.

29. The display apparatus of claim 24, further comprising a scanner along the first and second optical paths, the scanner deflecting the generated light along a raster pattern, the scanner concurrently deflecting the light received from the viewer'"'"'s eye.