SYSTEMS, DEVICES, AND METHODS THAT INTEGRATE EYE TRACKING AND SCANNING LASER PROJECTION IN WEARABLE HEADS-UP DISPLAYS

• US 20160349514A1
• Filed: 05/27/2016
• Published: 12/01/2016
• Est. Priority Date: 05/28/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Thalmic Labs Inc. (Alphabet Inc.)
• Inventors: Alexander, Stefan, Chapeskie, Jake, Holland, Lloyd Frederick, Mahon, Thomas
• Granted Patent: US 10,078,219 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0107 with optical power
• CPC Class Codes: G02B 2027/0109 comprising details concerning the making of holograms
• CPC Class Codes: G02B 2027/015 involving arrangement aiming to get less bulky devices
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 26/0833 the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD G02B26/0825 takes precedence; micromechanical devices in general B81B
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0103 comprising holographic elements
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G02B 5/32 Holograms used as optical elements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: H04N 9/3129 scanning a light beam on the display screen scanning a light beam on a screen in displays other than projection devices G09G3/02; scanning systems in general G02B26/10; projectors using laser light sources in general H04N9/3161
• CPC Class Codes: H04N 9/3161 using laser light sources using laser beams scanning the display screen H04N9/3129

Abstract

Systems, devices, and methods that integrate eye tracking capability into scanning laser projector (“SLP”)-based wearable heads-up displays are described. An infrared laser diode is added to an RGB SLP and an infrared photodetector is aligned to detect reflections of the infrared light from features of the eye. A holographic optical element (“HOE”) may be used to combine visible light, infrared light, and environmental light into the user'"'"'s “field of view.” The HOE may be heterogeneous and multiplexed to apply positive optical power to the visible light and zero or negative optical power to the infrared light.

Claims

1. A laser projector with an integrated eye tracker, comprising:

a laser module including an infrared laser diode to output an infrared light and at least one visible light laser diode to output a visible light;

a scan mirror aligned with an output of the laser module to receive both the infrared light and the visible light and to controllably reflect both the infrared light and the visible light;

a wavelength-multiplexed holographic optical element (“

HOE”

) aligned to receive both the infrared light and the visible light reflected from the scan mirror and to redirect both the infrared light and the visible light towards an eye of a user, wherein the wavelength-multiplexed HOE includes a first hologram that is responsive to the visible light and unresponsive to the infrared light and a second hologram that is responsive to the infrared light and unresponsive to the visible light; and

an infrared detector aligned to receive at least a portion of infrared light reflected from the eye of the user.

2. The laser projector of claim 1 wherein the wavelength-multiplexed HOE comprises at least two distinct layers of holographic material, a first layer of holographic material that includes the first hologram and a second layer of holographic material that includes the second hologram.

3. The laser projector of claim 1 wherein the wavelength-multiplexed HOE comprises a single volume of holographic material that includes both the first hologram and the second hologram.

4. The laser projector of claim 1 wherein the at least one visible light laser diode in the laser module includes at least one visible light laser diode selected from the group consisting of:

a red laser diode, a green laser diode, a blue laser diode, and any combination of a red laser diode, a green laser diode, and/or a blue laser diode.

5. The laser projector of claim 1 wherein the first hologram applies a first optical power to the visible light and the second hologram applies a second optical power to the infrared light, the second optical power different from the first optical power.

6. The laser projector of claim 5 wherein the first optical power is a positive optical power and the first optical power is greater than the second optical power.

7. The laser projector of claim 6 wherein the second optical power is less than or equal to zero.

8. The laser projector of claim 1, further comprising:

a support frame that has a general shape and appearance of a pair of eyeglasses, wherein the laser module, the scan mirror, the wavelength-multiplexed HOE, and the infrared detector are all carried by the support frame, and wherein the wavelength-multiplexed HOE is substantially transparent to environmental light and positioned in a field of view of at least one eye of the user when the support frame is worn on a head of the user.

9. A wearable heads-up display (“

WHUD”

) comprising;

a support frame that in use is worn on a head of a user;

a laser module carried by the support frame, the laser module including an infrared laser diode to output an infrared light and at least one visible light laser diode to output a visible light;

a scan mirror carried by the support frame and aligned with an output of the laser module to receive both the infrared light and the visible light output by the laser module, the scan mirror to controllably reflect both the infrared light and the visible light;

a wavelength-multiplexed HOE carried by the support frame and positioned within a field of view of at least one eye of the user when the support frame is worn on the head of the user, the wavelength-multiplexed HOE aligned to receive both the infrared light and the visible light reflected from the scan mirror and to redirect both the infrared light and the visible light towards the at least one eye of the user when the support frame is worn on the head of the user, wherein the wavelength-multiplexed HOE includes a first hologram that is responsive to the visible light and unresponsive to the infrared light and a second hologram that is responsive to the infrared light and unresponsive to the visible light, and wherein the wavelength-multiplexed HOE is substantially transparent to environmental light; and

an infrared detector carried by the support frame and aligned to receive at least a portion of infrared light reflected from the at least one eye of the user when the support frame is worn on the head of the user.

10. The WHUD of claim 9 wherein the support frame has a general shape and appearance of a pair of eyeglasses.

11. The WHUD of claim 9 wherein the wavelength-multiplexed HOE comprises at least two distinct layers of holographic material, a first layer of holographic material that includes the first hologram and a second layer of holographic material that includes the second hologram.

12. The WHUD of claim 9 wherein the wavelength-multiplexed HOE comprises a single volume of holographic material that includes both the first hologram and the second hologram.

13. The WHUD of claim 9 wherein the at least one visible light laser diode in the laser module includes at least one visible light laser diode selected from the group consisting of:

a red laser diode, a green laser diode, a blue laser diode, and any combination of a red laser diode, a green laser diode, and/or a blue laser diode.

14. The WHUD of claim 9 wherein the first hologram applies a first optical power to the visible light and the second hologram applies a second optical power to the infrared light, the second optical power different from the first optical power.

15. The WHUD of claim 14 wherein the first optical power is a positive optical power and the first optical power is greater than the second optical power.

16. The WHUD of claim 15 wherein the second optical power is less than or equal to zero.

17. A method of operating a laser projector to project an image to an eye of a user and to track the eye of the user, the method comprising:

outputting visible light by at least a first laser diode of the laser projector, the visible light representative of at least a portion of the image;

outputting infrared light by an infrared laser diode of the laser projector;

controllably and reflectively scanning both the visible light and the infrared light by a scan mirror of the laser projector;

redirecting both the visible light and the infrared light towards the eye of the user by a wavelength-multiplexed HOE;

detecting a reflection of at least a portion of the infrared light from the eye of the user by an infrared photodetector; and

determining a position of at least one feature of the eye based on the reflection of at least a portion of the infrared light from the eye of the user detected by the infrared photodetector.

18. The method of claim 17 wherein redirecting both the visible light and the infrared light towards the eye of the user by a wavelength-multiplexed HOE includes:

applying a first optical power to the visible light by a first hologram of the wavelength-multiplexed HOE; and

applying a second optical power to the infrared light by a second hologram of the wavelength-multiplexed HOE, the second optical power different from the first optical power.

19. The method of claim 18 wherein applying a first optical power to the visible light by a first hologram of the wavelength-multiplexed HOE includes applying a first positive optical power to the visible light by the first hologram of the wavelength-multiplexed HOE, and wherein applying a second optical power to the infrared light by a second hologram of the wavelength-multiplexed HOE includes applying a second optical power that is less than the first optical power to the infrared light by the second hologram of the wavelength-multiplexed HOE.

20. The method of claim 19 wherein applying a second optical power that is less than the first optical power to the infrared light by the second hologram of the wavelength-multiplexed HOE includes applying a second optical power that is less than or equal to zero to the infrared light by the second hologram of the wavelength-multiplexed HOE.

21. The method of claim 19 wherein determining a position of at least one feature of the eye based on the reflection of the infrared light from the eye of the user detected by the infrared photodetector includes determining the position of at least one feature of the eye based on the reflection of the infrared light from the eye of the user by a processor.

22. The method of claim 19 wherein outputting visible light by at least a first laser diode of the laser projector includes at least one of:

outputting red light by a red laser diode of the laser projector;

outputting green light by a green laser diode of the laser projector; and

/or

outputting blue light by a blue laser diode of the laser projector.