Virtual retinal display with eye tracking

• US 5,982,555 A
• Filed: 01/20/1998
• Issued: 11/09/1999
• Est. Priority Date: 01/20/1998
• Status: Expired due to Term

• Current Assignee: Washington University In St Louis
• Original Assignee: University of Washington
• Inventors: Johnston, Richard S., Melville, Charles D.
• Primary Examiner(s): Epps, Georgia
• Assistant Examiner(s): MACK, RICKY LEVERN
• Application Number:
  US09/008,918
• Time in Patent Office: 658 Days
• Field of Search: 359/629, 359/630, 359/631, 359/633, 345/7, 345/8, 351/206, 351/209, 351/211, 351/246, 382/117
• US Class Current: 359/630
• CPC Class Codes: A61B 5/1171 based on the shapes or appearances of their bodies or parts thereof
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 7/287 including a sight line detecting device
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G09G 3/001 using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background slide projectors per se G03B23/00 = 42 HP

Abstract

Light emitted from a virtual retinal display light source passes through a beamsplitter to a scanning subsystem and on to an eyepiece and the viewer'"'"'s eye. Some of the light is reflected from the viewer'"'"'s eye passing back along the same path. Such light however is deflected at the beamsplitter toward a photodetector. The reflected light is detected and correlated to the display scanner'"'"'s position. The content of the reflected light and the scanner position for such sample is used to generate a map of the viewer'"'"'s retina. Such map includes `landmarks` such as the viewer'"'"'s optic nerve, fovea, and blood vessels. The map of the viewer'"'"'s retina is stored and used for purposes of viewer identification. The viewer'"'"'s fovea position is monitored to track where the viewer is looking

Claims

1. A method for generating a map of a viewer'"'"'s retina with a virtual retinal display, the method comprising the steps of:

generating light;

receiving the generated light at an input of a lensing system, the lensing system defining a first optical path from the input toward an output, the lensing system output adapted for transmitting light to the and receiving light from a viewer'"'"'s eye, the lensing system including a beam deflector and a beam splitter located along the optical path;

receiving light reflected from the viewer'"'"'s eye;

deflecting the generated light along a raster pattern with the beam deflector;

concurrently deflecting the reflected light received from the viewer'"'"'s eye with the beam deflector;

redirecting with a beamsplitter the deflected reflected light along an alternative optical path;

detecting the redirected light with an optical detector responsive to the redirected light to produce an electrical signal corresponding to the redirected light.

2. A display appartus, comprising:

an image source for generating light;

a lensing system having an input aligned to the image source for receiving the generated light, the lensing system having an output adapted for transmitting light to and receiving light from a viewer'"'"'s eye, the lensing system defining an optical path from the input to the output;

a beamsplitter in the optical path which redirects light received from the viewer'"'"'s eye along an alternative optical path;

an optical detector positioned in the alternative optical path and responsive to the redirected light to produce an electrical signal corresponding to the redirected light;

a memory device storing data representing a reference level; and

a processor coupled to the detector and to the memory device, the processor being responsive to the electrical signal and the data to produce a signal indicative of a relationship between the electrical signal and the reference level.

3. A display apparatus comprising:

an image source for generating modulated light based upon a sequence of modulated values;

a lensing system having an input aligned to the image source for receiving the generated modulated light, the lensing system having an output adapted for transmitting modulated light to and receiving corresponding reflected light from a viewer'"'"'s eye, the lensing system defining an optical path from the input to the output;

a beamsplitter in the optical path which redirects light received from the viewer'"'"'s eye along an alternative optical path;

an optical detector positioned in the alternative optical path and responsive to the redirected light to produce an electrical signal corresponding to the redirected light; and

a processor coupled to the detector, the processor being responsive to the electrical signal and the sequence of modulated values to determine a viewing orientation of the viewer'"'"'s eye.

4. The display apparatus of claim 3, in which over time the processor produces a first group of indicative signals which are stored in the memory device and correspond to a map of the viewer'"'"'s eye.

5. The display apparatus of claim 4, in which the map is a first map, and wherein over time the processor produces a second group of indicative signals which correspond to a second map of the viewer'"'"'s eye, the processor comparing the second map to the first map to determine whether the first map and second map correspond to a common eye.

6. The display apparatus of claim 3, in which the processor comprises means for positioning a display object within the virtual image as a function of the viewer'"'"'s eye position.

7. A display apparatus, comprising:

an image source for generating light;

a lensing system having an input aligned to the image source for receiving the generated light, the lensing system having an output adapted for transmitting light to and receiving light from a viewer'"'"'s eye, the lensing system defining an optical path from the input to the output;

a beamsplitter in the optical path which redirects light received from the viewer'"'"'s eye along an alternative optical path;

an optial detector positioned in the alternative optical path and responsive to the redirected light to produce an electrical signal corresponding to the redirected light; and

a scanner along the optical path, the scanner deflecting the generated light along a raster pattern, the scanner concurrently deflecting the light received from the viewer'"'"'s eye.

8. A virtual retinal display apparatus with eye tracking, the apparatus receiving an image data signal for generating a virtual image upon a viewer'"'"'s eye, the apparatus comprising:

an image light source which modulates image light as a function of the image data signal to output modulated image light which defines the virtual image in a sequence of display pixels;

a scanner receiving the modulated visible light, the scanner deflecting the received visible light along a raster pattern, the scanner receiving a synchronization signal correlating to timing of a portion of the raster pattern;

a photodetector;

wherein the scanner receives returning light reflected from the viewer'"'"'s eye, the scanner deflecting the returning light;

a beamsplitter receiving the deflected returning light from the scanner, the beamsplitter directing said deflected returning light toward the photodetector, the photodetector generating a first signal in response to detection of the returning light; and

a processor receiving the first signal and the synchronization signal, for correlating samples of the first signal to a timing position within the synchronization signal.

9. The apparatus of claim 8, in which the processor comprises means for generating a map of the viewer'"'"'s eye based upon the first signal and the synchronization signal.

10. The apparatus of claim 9, in which the processor comprises means for storing the map as a first map, and means for comparing a second map to the first map to determine whether the first map and second map correspond to a same eye.

11. The apparatus of claim 8, in which the processor comprises means for identifying a viewer'"'"'s eye position over time based upon the first signal and the synchronization signal.

12. The apparatus of claim 11, in which the processor comprises means for positioning a display object within the virtual image as a function of the viewer'"'"'s eye position.

13. A method for generating a map of a viewer'"'"'s retina with a virtual retinal display, the method comprising the steps of:

receiving an image data signal at the display to define image content of an image to be scanned upon a viewer'"'"'s retina;

generating light modulated as a function of the image data signal;

deflecting the light along a raster pattern with a light scanner;

receiving returning light reflected from the viewer'"'"'s eye at the light scanner,

deflecting the returning light toward a beamsplitter;

directing said deflected returning light with the beamsplitter toward an optical detector;

generating a first signal at the optical detector in response to detection of the returning light; and

correlating respective samples of the first signal to corresponding timing positions within the raster pattern, wherein the correlated samples define the map of the viewer'"'"'s retina.

14. The method of claim 13, further comprising the steps of:

storing the map as a first map; and

comparing a second map to the first map to determine whether the first map and second map correspond to a same eye.

15. The method of claim 13, further comprising the step of identifying a viewer'"'"'s eye position over time based upon relative location of a select data pattern within the first signal, wherein the select data pattern corresponds to a fovea of the viewer'"'"'s eye.

16. The method of claim 13, further comprising the step of positioning a display object within the virtual image as a function of the viewer'"'"'s eye position.