Combining at least one variable focus element with a plurality of stacked waveguides for augmented or virtual reality display

• US 9,915,824 B2
• Filed: 05/06/2015
• Issued: 03/13/2018
• Est. Priority Date: 11/27/2013
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Schowengerdt, Brian T., Watson, Mathew D.
• Primary Examiner(s): Wu, Xiao
• Assistant Examiner(s): Lhymn, Sarah
• Application Number:
  US14/705,804
• Publication Number:
  US 20160109707A1
• Time in Patent Office: 1,042 Days
• Field of Search: 345619, 345632, 345633, 359 1, 359 10, 359 11, 359 15, 359 16, 359 19, 359 25, 359 26, 359 34, 3591961, 3591971, 3592011, 3592012, 3592041, 3592042, 3592043, 3592044, 3592071, 3592072, 3592074, 3592075, 3592077, 3592078, 3592082, 3592101, 3592112, 3592113, 3592121, 3592122, 3592161, 3592172, 3592174, 3592211, 359362, 359364, 359365, 359366
• CPC Class Codes: G02B 17/08 Catadioptric systems used in non-imaging applications G02B19/00
• CPC Class Codes: G02B 2006/0098 for scanning
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0123 comprising devices increasing the field of view
• CPC Class Codes: G02B 2027/0125 Field-of-view increase by wavefront division
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0163 Electric or electronic control thereof
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 26/0808 by means of one or more diffracting elements
• CPC Class Codes: G02B 27/0087 Phased arrays
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G02B 27/4205 having a diffractive optical element [DOE] contributing to image formation, e.g. whereby modulation transfer function MTF or optical aberrations are relevant
• CPC Class Codes: G02B 3/0006 Arrays G02B3/02, G02B5/188 take precedence
• CPC Class Codes: G02B 3/0037 characterized by the distribution or form of lenses
• CPC Class Codes: G02B 30/24 involving temporal multiplexing, e.g. using sequentially activated left and right shutters
• CPC Class Codes: G02B 5/005 : Diaphragms for cameras G03B9/02
• CPC Class Codes: G02B 5/1814 : structurally combined with one or more further optical elements, e.g. lenses, mirrors, prisms or other diffraction gratings G02B5/189 takes precedence
• CPC Class Codes: G02B 5/1828 : having means for producing variable diffraction controlling the direction of light by means of one or more diffracting elements G02B26/0808; acousto-optical elements G02F1/11, G02F1/33; electro- or magneto-optical diffraction G02F1/292, G02F1/2955
• CPC Class Codes: G02B 5/20 : Filters polarising elements G02B5/30
• CPC Class Codes: G02B 6/00 : Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings
• CPC Class Codes: G02B 6/02042 : Multicore optical fibres
• CPC Class Codes: G02B 6/06 : the relative position of the fibres being the same at both ends, e.g. for transporting images
• CPC Class Codes: G02B 6/32 : having lens focusing means positioned between opposed fibre ends with lens being an integral part of the single fibre end G02B6/262
• CPC Class Codes: G02B 6/34 : utilising prism or grating G02B6/293 takes precedence
• CPC Class Codes: G02B 6/40 : having fibre bundle mating means
• CPC Class Codes: G02C 2202/16 : Laminated or compound lenses
• CPC Class Codes: G02C 7/049 : Contact lenses having special fitting or structural features achieved by special materials or material structures
• CPC Class Codes: G02C 7/104 : having spectral characteristics for purposes other than sun-protection
• CPC Class Codes: G02F 1/0105 : Illuminating devices
• CPC Class Codes: G02F 1/1334 : based on polymer dispersed liquid crystals, e.g. microencapsulated liquid crystals
• CPC Class Codes: G02F 1/17 : based on variable-absorption elements not provided for in groups G02F1/015 - G02F1/169
• CPC Class Codes: G02F 1/29 : for the control of the position or the direction of light beams, i.e. deflection
• CPC Class Codes: G02F 1/292 : by controlled diffraction or phased-array beam steering controlled diffraction for optical switching G02F1/31
• CPC Class Codes: G02F 1/294 : Variable focal length devices
• CPC Class Codes: G02F 1/31 : Digital deflection, i.e. optical switchingG02F1/33 takes precedence
• CPC Class Codes: G06F 3/011 : Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 : Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 : Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/1423 : controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06T 13/40 : of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 15/00 : 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 15/10 : Geometric effects
• CPC Class Codes: G06T 15/506 : Illumination models
• CPC Class Codes: G06T 17/10 : Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 19/003 : Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 : Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10021 : Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30201 : Face
• CPC Class Codes: G06T 2213/08 : Animation software package
• CPC Class Codes: G06T 5/50 : using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 : Denoising; Smoothing
• CPC Class Codes: G06T 5/73 : Deblurring; Sharpening
• CPC Class Codes: G06T 7/50 : Depth or shape recovery
• CPC Class Codes: G06T 7/73 : using feature-based methods
• CPC Class Codes: G06V 40/193 : Preprocessing; Feature extraction
• CPC Class Codes: G09G 5/003 : Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092
• CPC Class Codes: G09G 5/02 : characterised by the way in which colour is displayed details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607
• CPC Class Codes: G09G 5/10 : Intensity circuits
• CPC Class Codes: G09G 5/18 : Timing circuits for raster scan displays specially adapted for television H04N ; synchronisation between the display unit and other display units, videodisc player G09G5/12
• CPC Class Codes: H04N 13/117 : the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/286 : having separate monoscopic and stereoscopic modes
• CPC Class Codes: H04N 13/341 : using temporal multiplexing
• CPC Class Codes: H04N 13/344 : with head-mounted left-right displays
• CPC Class Codes: H04N 13/361 : Reproducing mixed stereoscopic images; Reproducing mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background
• CPC Class Codes: H04N 13/363 : using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 13/366 : using viewer tracking
• CPC Class Codes: H04N 13/383 : for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 2013/0085 : Motion estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/00 : Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 5/21 : Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
• CPC Class Codes: H04N 5/74 : Projection arrangements for image reproduction, e.g. using eidophor

Abstract

Configurations are disclosed for presenting virtual reality and augmented reality experiences to users. The system may comprise an image-generating source to provide one or more frames of image data in a time-sequential manner, a light modulator configured to transmit light associated with the one or more frames of image data, a substrate to direct image information to a user'"'"'s eye, wherein the substrate houses a plurality of reflectors, a first reflector of the plurality of reflectors to reflect transmitted light associated with a first frame of image data at a first angle to the user'"'"'s eye, and a second reflector to reflect transmitted light associated with a second frame of the image data at a second angle to the user'"'"'s eye.

Claims

1. A system for displaying virtual content to a user, comprising:

a plurality of waveguides to receive light rays associated with image data and to transmit the light rays toward at least one eye of a user, wherein the plurality of waveguides are stacked about a direction pointing to the at least one eye of the user;

a first lens coupled to a first waveguide of the plurality of waveguides to modify light rays transmitted from the first waveguide, thereby delivering first light rays having a first wavefront curvature and a first bit depth to the at least one eye of the user;

a second lens coupled to a second waveguide of the plurality of waveguides to modify light rays transmitted from the second waveguide, thereby delivering second light rays having a second wavefront curvature and a second bit depth to the at least one eye of the user, wherein

the first bit depth is lower than the second bit depth; and

a combined volumetric display subsystem including the first lens, the first waveguide, the second lens, and the second waveguide and configured to project, to the at least one eye of the user, light rays that have a variable wavefront curvature and a higher bit depth than the first bit depth and a higher frame rate than a lower frame rate at which the first light rays or the second light rays are presented.

2. The system of claim 1, wherein the first wavefront curvature is different from the second wavefront curvature.

3. The system of claim 1, further comprising a third waveguide of the plurality of waveguides to deliver collimated light to the at least one eye of the user, such that the user perceives the image data as coming from an optical infinity plane and producing three-dimensional perception with a combination of at least the first light rays and the second light rays respectively sent to the at least one eye of the user.

4. The system of claim 1, wherein the second waveguide is configured to transmit collimated light to the first lens.

5. The system of claim 1, further comprising a compensating lens layer to compensate for an aggregate power of the lenses stacked in the direction facing the at least one eye of the user, wherein the compensating lens layer is stacked farthest from the at least one eye of the user.

6. The system of claim 1, wherein the first waveguide comprises a plurality of reflectors configurable to reflect the first light rays injected into the first waveguide toward the at least one eye of the user.

7. The system of claim 1, wherein at least one of the first waveguide and the second waveguide is electro-active.

8. The system of claim 1, wherein at least one of the first waveguide and the second waveguide includes diffractive elements that are switchable.

9. The system of claim 1, wherein the first light rays having the first wavefront curvature and the second light rays having the second wavefront curvature are delivered simultaneously to the at least one eye of the user.

10. The system of claim 9, wherein the second wavefront curvature corresponds to a margin of the first wavefront curvature, thereby providing a focal range in which the user accommodates.

11. The system of claim 1, wherein the first light rays having the first wavefront curvature and the second light rays having the second wavefront curvature are delivered sequentially to the at least one eye of the user.

12. The system of claim 1, wherein the system is further configured to track, at an accommodation module, an accommodation of the at least one eye of the user, wherein a variable focus element varies focal distances of light patterns based at least in part on the accommodation of the at least one eye of the user.

13. The system of claim 1, wherein the system is further configured to:

deliver, with the first waveguide and the first lens, first collimated light to the at least one eye of the user to produce visual perception at an optical infinity focal plane for the user; and

deliver, with the second waveguide and the second lens, second collimated light to the first lens coupled to the first waveguide before transmitting the second collimated light to the at least one eye of the user to produce the visual perception at a separate focal plane closer to the user than the optical infinity focal plane.

14. The system of claim 13, wherein the second lens is configured to create a convex wavefront curvature for the second wavefront curvature.

15. The system of claim 14, the system further comprising:

a third lens coupled to a third waveguide of the plurality of waveguides to modify light rays transmitted from the third waveguide, thereby delivering third light rays having a third wavefront curvature indicative of a third focal distance when the third light rays are sent to the at least one eye of the user; and

the system is further configured to deliver, with the third waveguide and the third lens, third collimated light to the second lens coupled to the second waveguide before transmitting the third collimated light to the at least one eye of the user via the first waveguide to produce the visual perception at a different focal plane closer to the user than the separate focal plane.

16. The system of claim 1, the system is further configured to:

identify diffractive optical elements operatively coupled to the first focal distance and the second focal distance, respectively; and

switch the diffractive optical elements at a first rate in synchronization with a frame rate at which the virtual content is displayed to the user.

17. The system of claim 1, wherein the system is further configured to:

identify diffractive optical elements operatively coupled to the first focal distance and the second focal distance, respectively; and

switch a focus state of the diffractive optical elements on a pixel-by-pixel basis at a second rate when a set of pixels for the virtual content is scanned into the at least one eye of the user, wherein the second rate at which the focus state is switched is faster than a frame rate at which the virtual content is displayed to the user.

18. The system of claim 1, wherein the system is further configured to:

identify diffractive optical elements operatively coupled to the first focal distance and the second focal distance, respectively; and

switch a focus state of the diffractive optical elements on a line-by-line basis at a third rate to adjust the focus state from a first line to a second line in displaying the virtual content to the at least one eye of the user, wherein the third rate at which the focus state is switched is slower than a second switch rate at which the focus state of the diffractive optical elements is switched on a pixel-by-pixel basis.

19. The system of claim 1, wherein the system is further configured to:

identify a higher-frame-rate and lower persistence display that corresponds to a lower bit depth;

identify a lower-frame-rate and higher persistence display that corresponds to a higher bit depth than the lower bit depth of the higher-frame-rate and lower persistence display; and

present a plurality of two-dimensional image slices in a frame sequential manner with a higher-frame-rate and higher bit depth display at least by combining the higher-frame-rate and lower persistence display and the lower-frame-rate and higher persistence display.

20. The system of claim 19, wherein the system is further configured to:

create a visual perception for the virtual content at least by projecting the plurality of two-dimensional image slices to the at least one eye of the user in a temporal sequence while varying focal distances in the virtual content.