Methods that integrate visible light eye tracking in scanning laser projection displays

• US 10,180,578 B2
• Filed: 11/09/2017
• Issued: 01/15/2019
• Est. Priority Date: 05/28/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: North Inc. (Alphabet Inc.)
• Inventors: Alexander, Stefan, Chapeskie, Jake, Holland, Lloyd Frederick, Mahon, Thomas
• Primary Examiner(s): Ritchie, Darlene M
• Application Number:
  US15/807,856
• Publication Number:
  US 20180067323A1
• Time in Patent Office: 432 Days
• CPC Class Codes: G02B 2027/0107 with optical power
• CPC Class Codes: G02B 2027/0109 comprising details concerning the making of holograms
• CPC Class Codes: G02B 2027/015 involving arrangement aiming to get less bulky devices
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 26/0833 the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD G02B26/0825 takes precedence; micromechanical devices in general B81B
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0103 comprising holographic elements
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G02B 5/32 Holograms used as optical elements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: H04N 9/3129 scanning a light beam on the display screen scanning a light beam on a screen in displays other than projection devices G09G3/02; scanning systems in general G02B26/10; projectors using laser light sources in general H04N9/3161
• CPC Class Codes: H04N 9/3161 using laser light sources using laser beams scanning the display screen H04N9/3129

Abstract

Systems, devices, and methods that integrate eye tracking capability into scanning laser projector (“SLP”)-based wearable heads-up displays are described. At least one narrow waveband laser diode is used in an SLP to define one or more portion(s) of a visible image. At least one corresponding narrow waveband photodetector is aligned to detect reflections of the portion(s) of the image from features of the eye. A holographic optical element (“HOE”) may be used to combine the image and environmental light into the user'"'"'s “field of view.” Three narrow waveband photodetectors each responsive to a respective one of three narrow wavebands output by the RGB laser diodes of an RGB SLP are aligned to detect reflections of a projected RGB image from features of the eye.

Claims

1. A method of operating a laser projector to project an image to an eye of a user and to track the eye of the user, the method comprising:

outputting visible laser light by a laser module, wherein the laser module includes at least a first laser diode and a second laser diode, and outputting visible laser light by the laser module includes outputting a first visible laser light in a first narrow waveband from the first laser diode of the laser module and outputting a second visible laser light in a second narrow waveband from the second laser diode, the second narrow waveband different from the first narrow waveband, wherein the first visible laser light is representative of at least a first portion of a displayed image and the second visible laser light is representative of at least a second portion of the displayed image;

controllably and reflectively scanning the first visible laser light and the second visible laser light by a scan mirror;

redirecting the visible laser light towards the eye of the user by a wavelength-multiplexed HOE comprising a first hologram that is responsive to light in the first narrow waveband and unresponsive to light that is outside the first narrow waveband and a second hologram that is responsive to light in the second narrow waveband and unresponsive to light that is outside the second narrow waveband, wherein redirecting the visible laser light towards the eye of the user by the wavelength-multiplexed HOE includes redirecting the first portion of the displayed image towards the eye of the user by the first hologram of the wavelength-multiplexed HOE and redirecting the second portion of the displayed image towards the eye of the user by the second hologram of the wavelength-multiplexed HOE, and wherein the wavelength-multiplexed HOE is substantially transparent to environmental light;

detecting a reflection of at least a portion of the visible laser light from the eye of the user by at least a first narrow waveband photodetector and a second narrow waveband photodetector, wherein the first narrow waveband photodetector is responsive to light in the first narrow waveband and substantially unresponsive to light that is outside of the first narrow waveband, the second narrow waveband photodetector is responsive to light in the second narrow waveband and substantially unresponsive to light that is outside of the second narrow waveband, and wherein detecting a reflection of the at least a portion of the visible laser light from the eye of the user by at least a first narrow waveband photodetector and a second narrow waveband photodetector includes detecting a reflection of the first portion of the displayed image by the first narrow waveband photodetector and detecting a reflection of the second portion of the displayed image from the eye of the user by the second narrow waveband photodetector; and

determining a position of at least one feature of the eye based on at least the reflection of the first portion of the displayed image from the eye of the user detected by the first narrow waveband photodetector and the reflection of the second portion of the displayed image from the eye of the user detected by the second narrow waveband photodetector.

2. The method of claim 1, wherein redirecting the visible laser light towards the eye of the user by a wavelength-multiplexed HOE includes converging, by the wavelength-multiplexed HOE, the first visible laser light and the second visible laser light towards a first exit pupil at the eye of the user.

3. The method of claim 1 wherein:

the laser module includes a third laser diode and outputting visible light by a laser module further includes outputting a third visible laser light in a third narrow waveband by the third laser diode of the laser module, the third narrow waveband different from both the first narrow waveband and the second narrow waveband, and wherein the third visible laser light is representative of at least a third portion of the displayed image;

controllably and reflectively scanning the visible laser light by a scan mirror further includes controllably and reflectively scanning the third portion of the displayed image by the scan mirror;

the wavelength-multiplexed HOE further comprises a third hologram that is responsive to light in the third narrow waveband and unresponsive to light that is outside the third narrow waveband and redirecting the visible laser light towards the eye of the user by the wavelength-multiplexed HOE further includes redirecting the third portion of the displayed image towards the eye of the user by the third hologram of the wavelength-multiplexed HOE;

detecting a reflection of at least a portion of the visible laser light from the eye of the user by at least a first narrow waveband photodetector further includes detecting a reflection of the third portion of the displayed image from the eye of the user by a third narrow waveband photodetector, wherein the third narrow waveband photodetector is responsive to light in the third narrow waveband and substantially unresponsive to light that is outside of the third narrow waveband; and

wherein determining a position of at least one feature of the eye further includes determining a position of at least one feature of the eye based on the reflection of the third portion of the displayed image from the eye of the user detected by the third narrow waveband photodetector.

4. The method of claim 3 wherein:

the first laser diode is a red laser diode, outputting a first visible laser light in a first narrow waveband by the first laser diode of the laser module includes outputting a red laser light by the red laser diode, and the first portion of the displayed image is a red portion of the displayed image;

the second laser diode is a green laser diode, outputting a second visible laser light in a second narrow waveband by the second laser diode of the laser module includes outputting a green laser light by the green laser diode, and the second portion of the displayed image is a green portion of the displayed image; and

the third laser diode is a blue laser diode, outputting a third visible laser light in a third narrow waveband by the third laser diode of the laser module includes outputting a blue laser light by the blue laser diode, and the third portion of the displayed image is a blue portion of the displayed image.