Display with integrated visible light eye tracking

• US 10,073,268 B2
• Filed: 11/09/2017
• Issued: 09/11/2018
• Est. Priority Date: 05/28/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Thalmic Labs Inc. (Alphabet Inc.)
• Inventors: Alexander, Stefan, Chapeskie, Jake, Holland, Lloyd Frederick, Mahon, Thomas
• Primary Examiner(s): Ritchie, Darlene M
• Application Number:
  US15/807,849
• Publication Number:
  US 20180067322A1
• Time in Patent Office: 306 Days
• CPC Class Codes: G02B 2027/0107 with optical power
• CPC Class Codes: G02B 2027/0109 comprising details concerning the making of holograms
• CPC Class Codes: G02B 2027/015 involving arrangement aiming to get less bulky devices
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 26/0833 the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD G02B26/0825 takes precedence; micromechanical devices in general B81B
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0103 comprising holographic elements
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G02B 5/32 Holograms used as optical elements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: H04N 9/3129 scanning a light beam on the display screen scanning a light beam on a screen in displays other than projection devices G09G3/02; scanning systems in general G02B26/10; projectors using laser light sources in general H04N9/3161
• CPC Class Codes: H04N 9/3161 using laser light sources using laser beams scanning the display screen H04N9/3129

Abstract

Systems, devices, and methods that integrate eye tracking capability into scanning laser projector (“SLP”)-based wearable heads-up displays are described. At least one narrow waveband laser diode is used in an SLP to define one or more portion(s) of a visible image. At least one corresponding narrow waveband photodetector is aligned to detect reflections of the portion(s) of the image from features of the eye. A holographic optical element (“HOE”) may be used to combine the image and environmental light into the user'"'"'s “field of view.” Three narrow waveband photodetectors each responsive to a respective one of three narrow wavebands output by the RGB laser diodes of an RGB SLP are aligned to detect reflections of a projected RGB image from features of the eye.

Claims

1. A wearable heads-up display (“

WHUD”

) comprising;

a support frame that in use is worn on a head of a user;

a laser module carried by the support frame, the laser module including a first laser diode to output a first visible laser light in a first narrow waveband, the first visible laser light representative of at least a first portion of a displayed image, and a second laser diode to output a second visible laser light in a second narrow waveband, the second narrow waveband different from the first narrow waveband, wherein the second visible laser light is representative of at least a second portion of the displayed image;

a scan mirror carried by the support frame and aligned with an output of the laser module to receive the first visible laser light and the second visible laser light and to controllably reflect the first visible laser light and the second visible laser light;

a wavelength-multiplexed holographic optical element (“

HOE”

) carried by the support frame and positioned within a field of view of and in opposing relation to at least one eye of the user when the support frame is worn on the head of the user, the HOE aligned to receive the first visible laser light and the second visible laser light reflected from the scan mirror and to redirect the first visible laser light and the second visible laser light towards the at least one eye of the user when the support frame is worn on the head of the user, wherein the HOE includes a first hologram that is responsive to the first visible laser light in the first narrow waveband and unresponsive to the second visible laser light in the second narrow waveband and a second hologram that is responsive to the second visible laser light in the second narrow waveband and unresponsive to the first visible laser light in the first narrow waveband, and wherein the HOE is substantially transparent to environmental light; and

a first narrow waveband photodetector carried by the support frame and aligned to receive at least a portion of the first visible laser light that is reflected from the at least one eye of the user when the support frame is worn on the head of the user, wherein the first narrow waveband photodetector is responsive to the first visible laser light in the first narrow waveband and unresponsive to light that is outside of the first narrow waveband, wherein the first narrow waveband photodetector is positioned proximate the HOE and aligned such that the first narrow waveband photodetector directly receives the at least a portion of the first visible laser light that is reflected from the at least one eye of the user; and

a second narrow waveband photodetector carried by the support frame and aligned to receive at least a portion of the second visible laser light that is reflected from the at least one eye of the user when the support frame is worn on the head of the user, wherein the second narrow waveband photodetector is responsive to the second visible laser light in the second narrow waveband and unresponsive to light that is outside of the second narrow waveband.

2. The WHUD of claim 1 wherein the support frame has a general shape and appearance of a pair of eyeglasses.

3. The WHUD of claim 1 wherein:

the laser module includes a third laser diode to output a third visible laser light in a third narrow waveband, the third narrow waveband different from both the first narrow waveband and the second narrow waveband, wherein the third visible laser light is representative of at least a third portion of the displayed image;

the first hologram is unresponsive to third visible laser light in the third narrow waveband;

the second hologram is unresponsive to the third visible laser light in the third narrow waveband;

the wavelength-multiplexed HOE includes a third hologram that is responsive to the third visible laser light in the third narrow waveband and unresponsive to both the first visible laser light in the first narrow waveband and the second visible laser light in the second narrow waveband; and

the WHUD further comprises a third narrow waveband photodetector carried by the support frame and aligned to receive at least a portion of the third visible laser light that is reflected from the at least one eye of the user when the support frame is worn on the head of the user, wherein the third narrow waveband photodetector is responsive to the third visible laser light in the third narrow waveband and unresponsive to light that is outside of the third narrow waveband.

4. The WHUD of claim 3 wherein:

the first laser diode is a red laser diode and the first narrow waveband corresponds to a first range of wavelengths that are visible as red to the eye of the user;

the second laser diode is a green laser diode and the second narrow waveband corresponds to a second range of wavelengths that are visible as green to the eye of the user; and

the third laser diode is a blue laser diode and the third narrow waveband corresponds to a third range of wavelengths that are visible as blue to the eye of the user.

5. The WHUD of claim 3 wherein the HOE comprises at least three distinct layers of holographic material, a first layer of holographic material that includes the first hologram, a second layer of holographic material that includes the second hologram, and a third layer of holographic material that includes the third hologram.

6. The WHUD of claim 3 wherein the HOE comprises a single volume of holographic material that includes all three of the first hologram, the second hologram, and the third hologram.

7. The WHUD of claim 3 wherein the first hologram applies a first optical power to the first visible laser light in the first narrow waveband, the second hologram applies the same first optical power to the second visible light in the second narrow waveband, and the third hologram applies the same first optical power to the third visible laser light in the third narrow waveband.

8. The WHUD of claim 1 wherein the HOE is aligned to receive the first visible laser light and the second visible laser light reflected from the scan mirror and to converge both the first visible laser light and the second visible laser light towards a first exit pupil at the at least one eye of the user.

9. The WHUD of claim 8 wherein the first hologram is oriented to converge the first visible laser light in the first narrow waveband to the first exit pupil at the at least one eye of the user and the second hologram is oriented to converge the second visible laser light in the second narrow waveband to the first exit pupil at the at least one eye of the user.

10. The WHUD of claim 8 wherein each laser light output provided by the laser module is representative of the respective first and second portions of the displayed image in the first exit pupil at the eye of the user.