Virtual retinal display with eye tracking

• US 6,285,505 B1
• Filed: 11/24/2000
• Issued: 09/04/2001
• Est. Priority Date: 01/20/1998
• Status: Expired due to Term

• Current Assignee: University of Washington
• Original Assignee: University of Washington
• Inventors: Johnston, Richard S., Melville, Charles D.
• Primary Examiner(s): MACK, RICKY LEVERN
• Application Number:
  US09/721,795
• Time in Patent Office: 284 Days
• Field of Search: 359/629, 359/630, 359/633, 345/7, 345/8, 351/206, 351/209, 351/211, 351/246, 382/117
• US Class Current: 359/630
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 7/287 including a sight line detecting device
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G09G 3/001 using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background slide projectors per se G03B23/00 = 42 HP

Abstract

Light emitted from a virtual retinal display light source passes through a beamsplitter to a scanning subsystem and on to an eyepiece and the viewer'"'"'s eye. Some of the light is reflected from the viewer'"'"'s eye passing back along the same path. Such light however is deflected at the beamsplitter toward a photodetector. The reflected light is detected and correlated to the display scanner'"'"'s position. The content of the reflected light and the scanner position for such sample is used to generate a map of the viewer'"'"'s retina. Such map includes ‘landmarks’ such as the viewer'"'"'s optic nerve, fovea, and blood vessels. The map of the viewer'"'"'s retina is stored and used for purposes of viewer identification. The viewer'"'"'s fovea position is monitored to track where the viewer is looking

Claims

1. A method for identifying a viewer'"'"'s eye orientation with a virtual retinal display, the method comprising the steps of:

generating a first light signal;

modulating a second light signal;

deflecting the first light signal and second light signal along a raster pattern onto the viewer'"'"'s eye with a light scanner;

receiving returning light reflected from the viewer'"'"'s eye at the light scanner,

deflecting the returning light with the light scanner;

directing at least a portion of the deflected returning light toward an optical detector;

generating a first signal at the optical detector in response to detection of the returning light from the second light signal; and

identifying a viewer'"'"'s eye orientation based upon relative location of a select data pattern within the first signal.

2. The method of claim 1, in which the select data pattern corresponds to retinal characteristics of the viewer'"'"'s eye.

3. The method of claim 1, in which the select data pattern corresponds to iris characteristics of the viewer'"'"'s eye.

4. The method of claim 1, in which the select data pattern corresponds to capillary structures of the viewer'"'"'s eye.

5. The method of claim 1, wherein the select data pattern corresponds to a fovea of the viewer'"'"'s eye.

6. The method of claim 1, further comprising the step of positioning a display object within the virtual image as a function of the viewer'"'"'s eye orientation.

7. A method for generating a map of a viewer'"'"'s iris characteristics with a virtual retinal display, the method comprising the steps of:

generating a first light signal;

modulating a second light signal;

deflecting the first light signal and second light signal along a raster pattern with a light scanner;

receiving returning light reflected from the viewer'"'"'s eye at the light scanner, deflecting the returning light with the optical scanner toward a beamsplitter;

directing said deflected returning light with the beamsplitter toward an optical detector;

generating a first signal at the optical detector in response to detection of the returning light for the second light signal; and

correlating respective samples of the first signal to corresponding timing positions within the raster pattern, wherein the correlated samples define the map of the viewer'"'"'s iris characteristics.

8. The method of claim 7, further comprising the steps of:

storing the map as a first map; and

comparing a second map to the first map to determine whether the first map and second map correspond to a same eye.

9. The method of claim 8, further comprising the step of identifying a viewer'"'"'s eye position over time based upon relative location of a select data pattern within the first signal, wherein the select data pattern corresponds to a fovea of the viewer'"'"'s eye.

10. The method of claim 7, further comprising the step of overlaying a virtual display object within the virtual image as a function of the viewer'"'"'s eye position.

11. A method for generating a map of a viewer'"'"'s eye capillary structures with a virtual retinal display, the method comprising the steps of:

receiving an image data signal at the display to define image content of an image to be scanned upon a viewer'"'"'s retina;

generating light modulated as a function of the image data signal;

deflecting the light along a raster pattern with a light scanner;

receiving returning light reflected from the viewer'"'"'s eye at the light scanner;

deflecting the returning light with the light scanner toward a beamsplitter;

directing said deflected returning light with the beamsplitter toward an optical detector;

generating a first signal at the optical detector in response to detection of the returning light; and

correlating respective samples of the first signal to corresponding timing positions within the raster pattern, wherein the correlated samples define the map of the viewer'"'"'s eye capillary structures.

12. The method of claim 11, further comprising the steps of:

storing the map as a first map; and

comparing a second map to the first map to determine whether the first map and second map correspond to a same eye.

13. The method of claim 11, further comprising the step of identifying a viewer'"'"'s eye position over time based upon relative location of a select data pattern within the first signal, wherein the select data pattern corresponds to a fovea of the viewer'"'"'s eye.

14. The method of claim 11, further comprising the step of overlaying a virtual display object within the virtual image as a function of the viewer'"'"'s eye position.

15. A method for controlling a pointer within a virtual image area based upon a viewer'"'"'s eye orientation, comprising the steps of:

receiving an image data signal to be scanned upon a viewer'"'"'s retina;

generating light modulated as a function of the image data signal;

deflecting the light along a raster pattern onto the viewer'"'"'s eye with a light scanner to define the virtual image area;

receiving returning light reflected from the viewer'"'"'s eye at the light scanner;

deflecting the returning light with the light scanner;

directing at least a portion of the deflected returning light toward an optical detector;

generating a first signal at the optical detector in response to detection of the returning light;

identifying a viewer'"'"'s eye orientation based upon relative location of a select data pattern within the first signal;

defining a location of the pointer within the virtual image area based upon the eye orientation; and

including image content of the pointer within the image data signal at the defined location.

16. The method of claim 15, in which a viewer implements a clicking operation at a select location within the virtual image area, the method further comprising the step of:

detecting a blink of the viewer'"'"'s eye based upon the first signal; and

implementing the clicking operation in response to a detected blink at the defined location.

17. The method of claim 16, in which the clicking operation is for any one function of the following group of functions:

opening a pull down menu; and

selecting a menu item.

18. The method of claim 15, in which the image data signal comprises a sequence of image frames, wherein the step of deflecting the light along a raster pattern is for each one of a sequence of image frames, and wherein the step of including image content of the pointer comprises including image content of the pointer within the image data signal at the defined location with not more than a two frame lag time.

19. The method of claim 15, in which the image data signal comprises a sequence of image frames, wherein the step of deflecting the light along a raster pattern is for each one of a sequence of image frames, and wherein the step of including image content of the pointer comprises including image content of the pointer within the image data signal at the defined location with a one frame lag time.

20. A method for identifying a viewer of a retinal display coupled to a computer system, comprising the steps of:

generating light;

receiving the generated light at an input of a lensing system, the lensing system defining a first optical path from the input toward an output, the lensing system output adapted for transmitting light to and receiving light from a viewer'"'"'s eye;

receiving light reflected from a first viewer'"'"'s eye at a beamsplitter located in the optical path;

redirecting with the beamsplitter the reflected light along an alternative optical path;

detecting the redirected light with an optical detector positioned in the alternative optical path, the optical detector responsive to the redirected light to produce an electrical signal corresponding to the redirected light;

determining viewing orientation of the first viewer'"'"'s eye using the electrical signal;

generating a plurality of signals indicative of a first map of the first viewer'"'"'s eye based upon a group of said electrical signals produced over time;

storing the first map in the computer system as an identification of the first viewer;

generating a plurality of signals indicative of a second map of a second viewer'"'"'s eye based upon a second group of said electrical signals produced over time; and

comparing the second map to the first map to determine whether the second viewer is identified to be the first viewer.

21. The method of claim 20 for controlling access to the computer system, further comprising the steps of:

storing a set of computer system access privileges for the first viewer; and

when the second viewer is identified as the first viewer, implementing the set of access privileges corresponding to said first viewer.