NEAR EYE DIFFRACTIVE HOLOGRAPHIC PROJECTION METHOD

• US 20170371163A1
• Filed: 06/27/2017
• Published: 12/28/2017
• Est. Priority Date: 06/27/2016
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: DAQRI, LLC
• Inventors: Ries, Ryan
• Granted Patent: US 10,444,509 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/0127 comprising devices increasing the depth of field
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 26/0833 the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD G02B26/0825 takes precedence; micromechanical devices in general B81B
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G03H 1/2205 using downstream optical component
• CPC Class Codes: G03H 1/2294 Addressing the hologram to an active spatial light modulator
• CPC Class Codes: G03H 1/2645 Multiplexing processes, e.g. aperture, shift, or wavefront multiplexing
• CPC Class Codes: G03H 2001/2221 Screen having complex surface, e.g. a structured object
• CPC Class Codes: G03H 2001/2242 Multiple viewing windows
• CPC Class Codes: G03H 2001/2284 Superimposing the holobject with other visual information
• CPC Class Codes: G03H 2225/21 Acousto-optic SLM [AO-SLM]
• CPC Class Codes: G03H 2225/24 Having movable pixels, e.g. microelectromechanical systems [MEMS]
• CPC Class Codes: G03H 2227/02 Handheld portable device, e.g. holographic camera, mobile holographic display
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

An augmented reality display device (such as a head mounted device) includes a partially transparent and partially reflective lens, a laser light source, a radio frequency source, a display controller, an acousto-optical modulator, and a microelectromechanical (MEMS) device. The laser light source generates light. The radio frequency (RF) source generates a RF signal. The display controller generates a synchronization signal. The acousto-optical modulator receives at least a portion of the light, modulates the light based on the RF signal, and provides modulated light. The MEMS device receives the synchronization signal from the display controller and reflects the modulated light towards the partially transparent and partially reflective lens. The MEMS device determines a direction in which the modulated light reflects based on the synchronization signal and the partially transparent and partially reflective lens reflecting the modulated laser light towards an eye of a user of the augmented realty display device.

Claims

1. An augmented reality display device comprising:

a partially transparent and partially reflective lens;

a laser light source configured to generate light;

a radio frequency source configured to generate a radio frequency (RF) signal;

a display controller configured to generate a synchronization signal;

an acousto-optical modulator configured to receive at least a portion of the light, modulate the light based on the radio frequency signal, and provide modulated light; and

a microelectromechanical (MEMS) device configured to receive the synchronization signal from the display controller and reflect the modulated light towards the partially transparent reflective lens, the MEMS device determining a direction in which the modulated light reflects based on the synchronization signal and the partially transparent and partially reflective lens reflecting the modulated laser light towards an eye of a user of the augmented realty display device.

2. The augmented reality display device of claim 1, wherein the partially transparent reflective lens comprises a parabolic lens including a concave side, a convex side, and a partially reflective layer applied to the concave side, wherein the parabolic lens forms a focus point located about the eye of the user.

3. The augmented reality display device of claim 1, wherein the partially transparent and partially reflective lens combines an external light reflected off a physical object with the modulated laser light at the eye of the user

4. The augmented reality display device of claim 1, further comprising:

one or more processors comprising an augmented reality application configured to generate augmented reality content and to send a data signal of the augmented reality content to the display controller.

5. The augmented reality display device of claim 4, wherein the augmented reality application comprises:

a recognition module configured to identify a physical object imaged by a camera of the augmented reality display device;

a rendering module configured to retrieve the augmented reality content associated with the physical object; and

a dynamic depth encoder configured to identify a depth of the physical object and encode a depth of the augmented reality content into the RF signal, the depth of the augmented reality content being coded based on the depth of the physical object.

6. The augmented reality display device of claim 5, wherein the display controller comprises:

a graphics processing unit (GPU) configured to render an image of the augmented reality content and generate a GPU output signal corresponding to the rendered image;

an RF source configured to generate the RF signal;

an RF mixer configured to mix the GPU output signal with the RF signal; and

a laser source configured to generate the laser light based on the mixed GPU output signal.

7. The augmented reality display device of claim 1, wherein the display controller comprises:

a graphics processing unit (GPU) configured to render an image of an augmented reality content and generate a GPU output signal corresponding to the rendered image, the augmented reality content independent of data from a camera of the augmented reality display device;

an RF source configured to generate the RF signal;

an RF mixer configured to mix the GPU output signal with the RF signal; and

a laser source configured to generate the laser light based on the mixed GPU output signal.

8. The augmented reality display device of claim 1, wherein the display controller comprises:

a graphics processing unit (GPU) configured to render an image of an augmented reality content and generate a GPU output signal corresponding to the rendered image, a depth of the rendered image independent of data from a camera of the augmented reality display device;

an RF source configured to generate the RF signal;

an RF mixer configured to mix the GPU output signal with the RF signal; and

a laser source configured to generate the laser light based on the mixed GPU output signal.

9. The augmented reality display device of claim 1, wherein the MEMS device is synchronized with the display controller via a reference timing, the reference timing further synchronizing the display controller with the acousto-optical modulator.

10. The augmented reality display device of claim 1, wherein the MEMS device reflects and spreads a single beam of the modulated laser light in different directions based on the RF signal towards the partially transparent reflective lens.

11. A method of forming a holographic image at a head mounted device, the method comprising:

generating a radiofrequency (RF) signal, a laser light, and a synchronization signal at a display controller of the head mounted device;

receiving the RF signal and the laser light at an acousto-optical modulator;

modulating the laser light based on the RF signal at the acousto-optical modulator;

projecting the modulated laser light to a microelectromechanical system (MEMS) device;

providing the synchronization signal and the modulated laser light to the MEMS device; and

directing the MEMS device to reflect the modulated laser light in a direction based on the synchronization signal towards a partially transparent and partially reflective lens of the head mounted device, the partially transparent reflective lens reflecting the modulated laser light towards an eye of a user of the head mounted device.

12. The method of claim 11, wherein the partially transparent and partially reflective lens comprises a parabolic lens including a concave side, a convex side, and a reflective layer applied to the concave side, wherein the parabolic lens forms a focus point located about the eye of the user.

13. The method of claim 11, wherein the partially transparent and partially reflective lens combines an external light reflected off a physical object with the modulated laser light at the eye of the user

14. The method of claim 11, further comprising:

generating an augmented reality content with an augmented reality application implemented in one or more processors in the head mounted device;

sending data signal of the augmented reality content to the display controller.

15. The method of claim 14, further comprising:

identifying a physical object imaged by a camera of the augmented reality display device;

retrieving the augmented reality content associated with the physical object;

identifying a depth of the physical object; and

encoding a depth of the augmented reality content into the RF signal, the depth of the augmented reality content being coded based on the depth of the physical object.

16. The method of claim 15, further comprising:

rendering an image of the augmented reality content with a Graphical Processing Unit (GPU);

generating a GPU output signal corresponding to the rendered image;

generating the RF signal with an RF source;

mixing the GPU output signal with the RF signal using an RF mixer; and

generating the laser light based on the mixed GPU output signal.

17. The method of claim 11, further comprising:

rendering an image of an augmented reality content with a GPU;

generating a GPU output signal corresponding to the rendered image, the augmented reality content independent of data from a camera of the augmented reality display device;

generating the RF signal with an RF source;

mixing the GPU output signal with the RF signal using an RF mixer; and

generating the laser light based on the mixed GPU output signal.

18. The method of claim 11, further comprising:

rendering an image of an augmented reality content with a GPU;

generating a GPU output signal corresponding to the rendered image, a depth of the rendered image independent of data from a camera of the augmented reality display device;

generating the RF signal with an RF source;

mixing the GPU output signal with the RF signal using an RF mixer; and

generating the laser light based on the mixed GPU output signal.

19. The method of claim 11, further comprising:

synchronizing the MEMS device with the display controller via a reference timing; and

synchronizing the display controller with the acousto-optical modulator using the reference timing.

20. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that, when executed by a computer, cause the computer to:

generate an RF signal, a laser light, and a synchronization signal at a display controller of the head mounted device;

receive the RF signal and the laser light at an acousto-optical modulator;

modulate the laser light based on the RF signal at the acousto-optical modulator;

project the modulated laser light to a microelectromechanical system (MEMS) device;

provide the synchronization signal and the modulated laser light to the MEMS device; and

direct the MEMS device to reflect the modulated laser light in a direction based on the synchronization signal towards a partially transparent and partially reflective lens of the head mounted device, the partially transparent reflective lens reflecting the modulated laser light towards an eye of a user of the head mounted device.