Apparatus for optical see-through head mounted display with mutual occlusion and opaqueness control capability

• US 9,547,174 B2
• Filed: 04/05/2013
• Issued: 01/17/2017
• Est. Priority Date: 04/05/2012
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Gao, Chunyu, Lin, Yuxiang, Hua, Hong
• Primary Examiner(s): Won, Bumsuk
• Assistant Examiner(s): Washington, Tamara Y
• Application Number:
  US13/857,656
• Publication Number:
  US 20140177023A1
• Time in Patent Office: 1,383 Days
• US Class Current: 1/1
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0145 creating an intermediate image
• CPC Class Codes: G02B 2027/015 involving arrangement aiming to get less bulky devices
• CPC Class Codes: G02B 25/001 Eyepieces
• CPC Class Codes: G02B 26/00 Optical devices or arrangements for the control of light using movable or deformable optical elements control of light by modification of the optical properties of the media of the elements involved therein G02F1/00
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/1066 for enhancing image performance, like resolution, pixel numbers, dual magnifications or dynamic range, by tiling, slicing or overlapping fields of view
• CPC Class Codes: G02B 27/144 using partially transparent surfaces without spectral selectivity G02B27/147 takes precedence
• CPC Class Codes: G02B 27/283 used for beam splitting or combining
• CPC Class Codes: G02B 5/04 Prisms
• CPC Class Codes: G03B 37/02 with scanning movement of lens or cameras
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

The present invention comprises a compact optical see-through head-mounted display capable of combining, a see-through image path with a virtual image path such that the opaqueness of the see-through image path can be modulated and the virtual image occludes parts of the see-through image and vice versa.

Claims

1. A compact optical see-through head-mounted display, capable of combining a see-through path with a virtual view path such that the opaqueness of a see-through view can be modulated and a virtual view occludes parts of the see-through view and vice versa, the display comprising:

a. a microdisplay for generating an image to be viewed by a user, the microdisplay having a virtual view path associated therewith;

b. a spatial light modulator for modifying light from an external scene in the real world to block portions of the see-through view that are to be occluded, the spatial light modulator having a see-through path associated therewith;

c. an objective optics configured to receive the incoming light from the external scene and to focus the light upon the spatial light modulator;

d. a beamsplitter configured to merge a virtual image from a microdisplay and a modulated see-through image of an external scene passing from a spatial light modulator, producing a combined image;

e. an eyepiece configured to magnify the combined image;

f. an exit pupil configured to face the eyepiece, the exit pupil whereupon the user observes a combined view of the virtual and see-through views in which the virtual view occludes portions of the see-through view; and

g. a plurality of reactive surfaces configured to fold the virtual view path and the see-through path into two layers,

wherein a first reflective surface is disposed upon a front layer of the display oriented to reflect the light from the external scene, wherein the objective optics is disposed upon the front layer of the display, wherein a second reflective surface is disposed upon the front layer of the display oriented to reflect the light into the spatial light modulator, where the spatial light modulator is disposed at or near an intermediate image plane of the see-through path, in optical communication with the objective optics and the eyepiece through the beamsplitter along the see-through path, where the microdisplay is disposed at a focal plane of the eyepiece along the virtual view path, in optical communication with the eyepiece through the beamsplitter, where the, beam splitter is disposed such that the see-through path is merged with the virtual view path and the light from both the see-through path and the virtual view path is directed to the eyepiece, wherein the eyepiece is disposed upon a back layer of the display, wherein a third reflective surface is disposed upon the back layer of the display oriented to reflect the light from the eyepiece into the exit pupil;

whereupon the objective optics receives light of the external scene, whereupon the objective optics focus the light of the external scene and forms a see-through image upon the spatial light modulator, whereupon the spatial light modulator modifies the see-through image to remove portions of the image that are to be occluded, whereupon the microdisplay projects a virtual image to the beam splitter, whereupon the spatial light modulator transmits the modulated see-through image to the beam splitter, whereupon the beam splitter merges the virtual and see-through images producing a combined image in which the virtual image occludes portions of the see-through image, whereupon the beam splitter projects the combined image to the eyepiece, whereupon the eyepiece projects the combined image to the exit pupil, whereupon the user observes the combined image, in which the virtual image occludes portions of the external scene.

2. The display of claim 1, where the spatial light modulator is a transmission type spatial light modulator, where the spatial light modulator is disposed in front of the beam splitter, whereupon the light from the objective optics passes through the spatial light modulator before reaching the beam splitter, wherein the opaqueness of the spatial light modulator is controlled to block light from portions of the external scene.

3. The display of claim 1, where the spatial light modulator is a reflection type spatial light modulator, where the spatial light modulator is disposed behind the beam splitter, whereupon light from the objective optics passes through the beam splitter and is reflected back from the spatial light modulator to the beam splitter, wherein a reflectivity of the spatial light modulator is controlled to reflect only light from parts of the external scene not to be occluded.

4. The display of claim 1, where an intermediate image is formed at one or more points in the see-through path, where the spatial light modulator is disposed at or near one of the intermediate image planes.

5. The display of claim 1, where one or more of the first, second, and third reflective surfaces are stand-alone surfaces with optical power to fold the see-through and virtual view paths and focus the light.

6. The display of claim 1, where one or more of the first, second, and third reflective surfaces are freeform surfaces.

7. The display of claim 1, where the first and/or second reflective surfaces in the front layer are contained within the objective optics.

8. The display of claim 1, where the third reflective surface in the back layer is contained within the eyepiece.

9. The display of claim 1, where the objective optics is a freeform prism formed by a plurality of reflective and refractive surfaces to image the external scene into the spatial light modulator.

10. The display of claim 1, where the eyepiece is a freeform prism formed by a plurality of reflective and refractive surfaces to magnify the virtual image and the modulated see-through image.

11. The display of claim 9, where the first and/or the second reflective surface in the front layer is contained within the objective optics.

12. The display of claim 10, where the third reflective surface in the back layer is contained within the eyepiece.

13. The display of claim 1, where an even number of intermediate images are formed along the see-through path to invert the see-through view so as to maintain parity between the external scene and the see-through view presented to a viewer.

14. The display of claim 1, where one of the first, second, and third reflective surfaces is replaced by a roof mirror in order to revert the see-through view so as to maintain parity between the external scene and the see-through view presented to a viewer.

15. The display of claim 1, where both the eyepiece and the objective optics have identical optical structure.

16. The display of claim 15, where both the eyepiece and the objective optics are freeform prisms of identical shape.

17. The display of claim 1 where the beamsplitter is disposed upon the front layer.

18. The display of claim 1, where one or more diffractive optical element plates are placed in an optical path to correct chromatic aberrations.

19. The display of claim 1, wherein the objective optics is a one-reflection prism comprising three optical surfaces:

a first refractive surface, a third reflective surface and a second refractive surface, and wherein the eyepiece is a one-reflection prism comprising three optical surfaces;

a third refractive surface, a fourth reflective surface and a fourth refractive surface, where the second reflective surface is contained within the objective optics and the third reflective surface is contained within the eyepiece, where a roof mirror replaces the first reflective surface to invert the see-through view, and a reflection-type spatial light modulator is used to modulate the light from the external scene.

20. The display of claim 19, whereupon the incoming light from external scene reflected by the roof mirror, enters the objective optics through the first refractive surface, then is reflected by the third reflective surface and exits the objective optics through the second refractive surface and forms an intermediate image at its focal plane on the spatial light modulator, whereupon the spatial light modulator modulates the light in the see-through path to block the light to be occluded, whereupon the spatial light modulator reflects the modulated light into the beam splitter, whereupon the light from the microdisplay enters the beam splitter, whereupon the beamsplitter merges the modulated light in the see-through path with the light in the virtual view path and folds toward the eyepiece for viewing, whereupon the light from the beamsplitter enters the eyepiece through the fourth refractive surface, that is reflected by the fourth reflective surface and exits the eyepiece through the third refractive surface and reaches the exit pupil, where a viewer'"'"'s eye is aligned to see a combined view of a virtual view and a modulated see-through view.

21. The display of claim 1, where the objective optics is a four-reflection freeform prism comprising six optical surfaces:

refractive surface, reflective surfaces and refractive surface, where the eyepiece is a two-reflection prism comprising four optical surfaces;

refractive surface, reflective surface, reflective surface and refractive surface, where the first reflective surface and the second reflective surfaces are contained within the objective optics and the third reflective surface is contained within the eyepiece, where the objective optics forms an intermediate image inside the objective optics, where an even number of intermediate images are formed along the see-through path to invert the see-through view, where a reflection-type spatial light modulator is used to modulate the light from the external scene.

22. The display of claim 21, whereupon the incoming light from an external scene enters objective optics through the refractive surface, then is consecutively reflected by the reflective surfaces, and exits the objective optics through the refractive surface, whereupon the incoming light forms an intermediate image at its focal plane on the spatial light modulator, whereupon the spatial light modulator modulates the light in the see-through path to block the light to be occluded, whereupon the spatial light modulator reflects the modulated light into the beam splitter, whereupon the light from the microdisplay enters the beam splitter, whereupon the beamsplitter merges the modulated light in the see-through path with the light in the virtual view path and folds toward the eyepiece for viewing, whereupon the light from the beam splitter enters the eyepiece through the refractive surface, then is consecutively reflected by the reflective surfaces and exits the eyepiece through the refractive surface Si and reaches the exit pupil, where the viewer'"'"'s eye is aligned to see a combined view of a virtual view and a modulated see-through view.

23. The display of claim 21, where the refractive surface and reflective surface are the same physical surfaces and possess the same set of surface prescriptions.

24. The display of claim 1, where the objective optics is a one-reflection prism comprising three optical surfaces:

refractive surface, reflective surface and refractive surface, where the eyepiece is a two-reflection prism comprising four optical surfaces;

refractive surface, reflective surface, reflective surface and refractive surface, where the first reflective surface is contained within the objective optics and the third reflective surface is contained within the eyepiece, where a roof mirror replaces the second reflective surface to invert the see-through view, where a transmission-type spatial light modulator is used to modulate the light from the external scene.

25. The display of claim 24, whereupon incoming light from an external scene enters the objective optics through the refractive surface, then is reflected by the reflective surface and exits the objective optics through the refractive surface and is folded by the mirror toward the back layer and forms an intermediate image at its focal plane on the spatial light modulator, whereupon the spatial light modulator modulates the light in the see-through path to block the light to be occluded, whereupon the spatial light modulator transmits the modulated light into the beam splitter, whereupon the light from the microdisplay enters the beam splitter, whereupon the beamsplitter merges the modulated light in the see-through path with the light in the virtual view path and folds toward the eyepiece for viewing, whereupon the light from the beam splitter enters the eyepiece through the refractive surface, then is consecutively reflected by reflective surfaces, and exits the eyepiece through the refractive surface and reaches exit pupil, where the viewer'"'"'s eye is aligned to see a combined view of a virtual view and a modulated see-through view.

26. The system of claim 24, where surface and reflective surface are the same physical surfaces and possess the same set of surface prescriptions.

27. The display of claim 1, where the objective optics is a three-reflection prism comprising five optical freeform surfaces:

refractive surface, reflective surface and refractive surface, where the eyepiece is a two-reflection prism comprising four optical surfaces;

refractive surface, reflective surface, reflective surface and refractive surface, where the first reflective surface and the second reflective surface are contained within the objective optics and the third reflective surface is contained within the eyepiece, where the objective optics forms an intermediate image inside the objective optics, where an even number of intermediate images are formed along the see-through path to invert the see-through view, where a transmission-type spatial light modulator is used to modulate the light from the external scene.

28. The display of claim 27, whereupon the incoming light from an external scene enters objective optics through the refractive surface, is consecutively reflected by reflective surfaces, and exits the objective optics through the refractive surface whereupon the incoming light forms an intermediate image at its focal plane on the spatial light modulator, whereupon the spatial light modulator modulates the light in the see-through path to block the light to be occluded, whereupon the spatial light modulator transmits the modulated light into the beam splitter, whereupon the light from the microdisplay enters the beam splitter, whereupon the beamsplitter merges the modulated light in the see-through path with the light in the virtual view path and folds toward the eyepiece for viewing, whereupon the light from the beam splitter enters the eyepiece through the refractive surface, then is consecutively reflected by the reflective surfaces, and exits the eyepiece through the refractive surface Si and reaches the exit pupil, where the viewer'"'"'s eye is aligned to see a combined view of a virtual view and a modulated see-through view.

29. There system of claim 27, where the refractive surface and the reflective surface may the same physical surfaces and possess the same set of surface prescriptions.

30. There system of claim 27, where the refractive surface and the reflective surface are the same physical surfaces and possess the same set of surface prescriptions.

31. The display of claim 1, where the objective optics is a two-reflection prism comprising our optical freeform surfaces:

refractive surface, reflective surface, reflective surface and refractive surface, where the eyepiece is a two-reflection prism comprising four optical surfaces;

refractive surface, reflective surface, reflective surface and refractive surface, where the first reflective surface is contained within the objective optics and the third reflective surface is contained within the eyepiece, where the second reflective surface as the minor along with a first beamsplitter and a relay lens is configured to fold the see-through path and create a secondary intermediate image, where an even number of intermediate images are formed along the see-through path to invert the see-through view, where a reflection-type spatial light modulator is used to modulate the tight from the external scene.

32. The display of claim 31, whereupon the incoming light from an external scene enters the objective optics through refractive surface, consecutively reflected by the reflective surfaces, and exits the objective optics through the refractive surface, whereupon the incoming light is reflected by the first beamsplitter onto the mirror, where it forms an intermediate image, whereupon the mirror reflects the light from the front layer onto the relay lens, whereupon the relay lens forms another intermediate image upon the spatial light modulator whereupon the spatial tight modulator modulates the light in the see-through path to block the light to be occluded, whereupon the spatial light modulator reflects the modulated light into the second beam splitter, whereupon the light from the microdisplay enters the second beam splitter, whereupon the second beam splitter merges the modulated light in the see-through path with the light in the virtual view path and folds toward the eyepiece for viewing, whereupon the light from the beam splitter enters the eyepiece through refractive surface, then is consecutively reflected by the reflective surfaces, and exits the eyepiece through the refractive surface Si and reaches exit pupil, where the viewer'"'"'s eye is aligned to see a combined view of a virtual view and a modulated see-through view.

33. The display of claim 31, where the positions of the mirror and the spatial light modulator are interchangeable.

34. The display of claim 19, where one or more of the see-through and virtual view surfaces of the objective optics is an aspherical surface with or without rotational symmetry.

35. The display of claim 19, where one or more of the see-through and virtual view surfaces of the eyepiece prism is an aspherical surface with or without rotational symmetry.

36. The display of claim 1, where the beamsplitter is in form of a cube or a plate and could be a non-polarized beamsplitter or a polarized beamsplitter.

37. The display according to claim 1, where said compact optical see-through head mounted display is one of a monocular and a binocular.