Methods and systems for displaying stereoscopy with a freeform optical system with addressable focus for virtual and augmented reality

• US 10,156,722 B2
• Filed: 05/29/2015
• Issued: 12/18/2018
• Est. Priority Date: 12/24/2010
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Gao, Chunyu, Hua, Hong
• Primary Examiner(s): Pasko, Nicholas R
• Application Number:
  US14/726,396
• Publication Number:
  US 20160011419A1
• Time in Patent Office: 1,299 Days
• Field of Search: 359630-633, 349 11
• CPC Class Codes: G02B 2027/011 comprising device for correcting geometrical aberrations, distortion
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0123 comprising devices increasing the field of view
• CPC Class Codes: G02B 2027/0127 comprising devices increasing the depth of field
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0147 comprising a device modifying the resolution of the displayed image
• CPC Class Codes: G02B 26/0833 the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD G02B26/0825 takes precedence; micromechanical devices in general B81B
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 3/12 Fluid-filled or evacuated lenses

Abstract

Several unique hardware configurations and methods for freeform optical display systems are disclosed. A freeform display system includes primary freeform optical element(s) and secondary freeform optical element(s) in tiled arrangements to expand the horizontal field of view (FOV) or the vertical field of view. The system may include a variable focusing system that produces intermediate pupil and changes the focal distance of a single focal plane or switches among multiple focal planes for rendering objects in focus while resolving accommodation-convergence conflict. The system may map light samples to appropriate light rays in physical space and use a cluster of projectors to project the mapped light rays to produce the light field of the virtual display content. Methods for making tiled freeform optical display systems and methods for producing virtual content with variable focus freeform optics and rendering light fields are also disclosed.

Claims

1. A tiled freeform optical display system, comprising:

an image source transmitting light rays;

a primary freeform optical element devised to include a first aspect ratio and to provide an increased first field of view in a first direction based at least in part upon the first aspect ratio, wherein

the increased first field of view is larger than a first corresponding field of view provided by the primary freeform optical element in a second direction;

a secondary freeform optical element devised to include a second aspect ratio and to provide an increased second field of view in the second direction based at least in part upon the second aspect ratio and coupled directly with no intervening elements or indirectly with one or more intervening elements to the primary freeform optical element along the first direction or the second direction in a first tiled arrangement to form a tiled freeform optical display device that expands an aggregated field of view of the tiled freeform optical display system in a plurality of directions for displaying virtual display content with at least a portion of the light rays from the image source, wherein

the increased second field of view is larger than a second corresponding field of view provided by the secondary freeform optical element in the first direction;

at least one coupling lens interposed between the image source and the primary freeform optical element and configured to relay the light rays from the image source to the primary freeform optical element and the secondary freeform optical element and to correct optical aberrations for the light rays; and

a variable focusing element coupled to the image source and the tiled freeform optical display device and comprising a deformable optical element that is configured to change a variable focal distance of a single focal plane based at least in part upon the aggregated field of view of the tiled freeform optical display system and virtual depth of virtual display content to render the virtual display content on the single focal plane in focus with the variable focus distance.

2. The tiled freeform optical display system of claim 1, further comprising:

a compensation lens disposed with a gap between a surface of at least a portion of the compensation lens and an opposing surface of at least a part of the primary freeform optical element or the secondary freeform optical element to improve quality of light rays emitted or reflected from a real-world scene through at least a part of the primary freeform optical element or the secondary freeform optical element at least by reducing optical aberrations of the tiled freeform optical display system.

3. The tiled freeform optical display system of claim 1, wherein the primary freeform optical element includes a freeform wedge coupled directly with no intervening elements or indirectly with one or more intervening elements to one or more relay lens, and the variable focusing element includes a liquid lens to form an intermediate pupil.

4. The tiled freeform optical display system of claim 3, wherein the primary freeform optical element is further coupled directly with no intervening elements or indirectly with one or more first intervening elements to a folding reflector that receives the first light rays from the variable focusing element.

5. The tiled freeform optical display system of claim 1, wherein the variable focusing element is further configured to change the variable focal distance of the single focal plane based at least in part upon color, radiance, or depth information extracted from the light rays to render the virtual display content on the single focal plane in focus with the variable focus distance.

6. The tiled freeform optical display system of claim 1, further comprising a bundle of projectors, wherein a total number of projectors in the bundle is determined based at least in part upon a desired or required resolution of the tiled freeform optical display system.

7. The tiled freeform optical display system of claim 6, wherein the variable focusing element includes a coupling lens group, two or more beam splitters operatively coupled directly with no intervening elements or indirectly with one or more first intervening elements to the deformable membrane mirror.

8. The tiled freeform optical display system of claim 1, wherein the primary freeform optical element and the secondary freeform optical element present two fields of view that are identical.

9. The tiled freeform optical display system of claim 1, wherein

the primary freeform optical element is devised to provide a first primary vertical field of view,

the secondary freeform optical element is devised to provide a secondary vertical field of view based in part upon the first primary vertical field of view, and

the tiled freeform optical display device is devised to provide the aggregated field of view based at least in part upon the first primary vertical field of view and a predetermined aspect ratio for a field of view provided by the primary freeform optical element.

10. The tiled freeform optical display system of claim 1, wherein

the primary freeform optical element is devised to provide a primary vertical field of view,

the secondary freeform optical element is devised to provide a secondary horizontal field of view, and

the tiled freeform optical display device is configured to arrange the primary freeform display element and the secondary freeform display element in an arrangement that is devised to provide the aggregated field of view based at least in part upon both the primary vertical field of view and the secondary horizontal field of view.

11. The tiled freeform optical display system of claim 1, the variable focus element comprising:

a deformable optical component and configured to render a perception of varying depths at least by switching among a plurality of focal planes each corresponding to a respective focal distance to render the virtual display content in focus.

12. The tiled freeform optical display system of claim 11, the variable focus element further configured to resolve an accommodation-convergence conflict by switching among at least some of the plurality of focal planes.

13. The tiled freeform optical display system of claim 1, the variable focus element configured to address an accommodation cue and a convergence cue to a changing focal distance, without tying the accommodation cue or the convergence cue to a fixed focal distance.

14. The tiled freeform optical display system of claim 1, further comprising:

a beam splitter operatively coupled to the variable focus element; and

one or more actuators operatively coupled to the variable focus element, wherein

the variable focus element comprises a deformable optical component includes a deformable membrane mirror, a mechanical-deformation-based lens, a deformable mirror, an electrowetting lens, or an optical component including a plurality of fluids with different refractive indices.

15. The tiled freeform optical display system of claim 14, wherein

the deformable optical component is configured to reflect light rays to the beam splitter, and

the beam splitter is further configured to relay the light rays into the primary or the secondary freeform optical element.

16. The tiled freeform optical display system of claim 15, wherein

the variable focusing element is further configured to define at least the focal plane corresponding to the focal distance and to move the focal distance relative to an observer'"'"'s eye to create a three-dimensional perception in rendering the virtual display content.

17. The tiled freeform optical display system of claim 16, wherein the deformable optical component is configured to control wavefronts of the light rays, to correct optical aberrations, or to alter optical power of the deformable optical component to alter the focal distance by moving an intermediate focus plane.

18. The tiled freeform optical display system of claim 1, wherein the variable focus element is further configured to control the focal distance to produce the virtual display content on continuously changing focal planes.

19. A freeform optical display mechanism for generating stereoscopic images, comprising:

one or more coupling lenses configured to receive light beams of the stereoscopic images transmitted from an image source and to correct optical aberrations for the light beams;

a primary display device comprising a primary freeform optical element configured to receive the light beams from the one or more coupling lens and to define a primary field of view in a first direction, wherein

the primary freeform optical element comprises a freeform surface that further includes at least a first surface and a second surface, wherein the first surface is configured with a semi-transparent coating that is selected based at least in part upon a reflective efficiency of the first surface, and the second surface is configured with a dielectric coating that is selected based at least in part upon a total internal reflection condition;

a secondary display device defining a secondary field of view in a second direction and comprising a secondary freeform optical element;

at least one compensation lens comprising one or more refractive surfaces and configured to overlap at least a first portion of the freeform surface of the primary freeform optical element and at least a second portion of a second freeform optical element; and

a tiled display device including the primary display device and the secondary display device in a tiled arrangement and the at least one compensation lens to expand an aggregated field of view of the tiled display device in a plurality of directions.

20. The freeform optical display mechanism of claim 19, wherein the secondary display device and the primary display device are integrated into a first tiled arrangement to form at least a portion of the tiled display device that expands a temple horizontal field of view of the primary display device.

21. The freeform optical display mechanism of claim 19, further comprising:

at least one freeform compensation lens attached to the primary freeform optical element or the second freeform optical element and configured to reduce optical aberrations of the tiled freeform optical display system; and

a variable focusing mechanism including a variable focusing element and optically coupled directly with no intervening elements or indirectly with one or more intervening elements to the primary display device of the freeform optical display mechanism to adjust at least one focal distance for rendering virtual display content in focus.

22. The freeform optical display mechanism of claim 21, further comprising:

a control circuit for optical power operatively coupled directly with no intervening elements or indirectly with one or more intervening elements to a deformable mirror in the variable focusing mechanism of the freeform optical display mechanism to change the at least one focal distance.

23. The freeform optical display mechanism of claim 21, the at least one freeform compensation lens comprising:

one or more freeform refractive surfaces, wherein

at least a part of the at least one freeform compensation lens overlaps at least a portion of the primary display device of the freeform optical display mechanism with an air gap between the at least one compensation lens and the portion.

24. The freeform optical display mechanism of claim 23, further comprising:

one or more freeform refractive surfaces operatively coupled to the primary freeform optical element or the secondary freeform optical element and configured to improve light rays emitted or reflected from a real-world object and perceived by at least one eye of a user through the freeform optical display mechanism.

25. The freeform optical display mechanism of claim 19, further comprising a variable focusing mechanism that includes a variable focusing element and is to switch among a plurality of focal distances corresponding to a plurality of focal planes of the freeform optical display mechanism.

26. The freeform optical display mechanism of claim 19, wherein the primary display device and / or the secondary display device includes addressable focus functionality to control a focal distance of a single focal plane for stereoscopic images produced by the freeform optical display mechanism.

27. The freeform optical display mechanism of claim 19, wherein the primary display device and / or the secondary display device includes addressable focus functionality to switch among a plurality of focal planes for the stereoscopic images produced by the freeform optical display mechanism.

28. The freeform optical display mechanism of claim 19, further comprising:

a first arrangement that arranges the primary display device and the secondary display device adjacent to each other to expand a horizontal field of view or a diagonal field of view of the freeform optical display mechanism.

29. The freeform optical display mechanism of claim 19, further comprising:

a second arrangement that that arranges the primary display device atop the secondary display device or the secondary display device atop the primary display device to expand a vertical field of view or a diagonal field of view of the freeform optical display mechanism.

30. The freeform optical display mechanism of claim 19, further comprising:

a third arrangement that approximately or exactly matching the secondary field of view of the secondary display device with the primary field of view of the primary display device expanding a vertical field of view or a diagonal field of view of the freeform optical display mechanism.

31. The freeform optical display mechanism of claim 19, further comprising:

one or more other secondary display devices operatively coupled directly with no intervening elements or indirectly with one or more intervening elements to the primary display device to further expand the field of view of the freeform optical display mechanism.

32. The freeform optical display mechanism of claim 19, further comprising:

an optical coupling optical mechanism operatively coupled directly with no intervening elements or indirectly with one or more intervening elements to the primary display device and one or more image sources to improve quality of stereoscopic images produced by the freeform optical display mechanism.

33. The freeform optical display mechanism of claim 19, further comprising at least one compensation lens that further comprises:

one or more freeform refractive surfaces that are attached to at least one exterior surface of the primary freeform optical element or the secondary freeform optical element.