Beam angle sensor in virtual/augmented reality system

• US 10,359,634 B2
• Filed: 07/09/2018
• Issued: 07/23/2019
• Est. Priority Date: 01/12/2016
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Yeoh, Ivan L., Edwin, Lionel Ernest, Tekolste, Robert Dale
• Primary Examiner(s): Mushambo, Martin
• Application Number:
  US16/030,713
• Publication Number:
  US 20180341112A1
• Time in Patent Office: 379 Days
• Field of Search: None
• CPC Class Codes: G01B 11/272 using photoelectric detection means
• CPC Class Codes: G02B 2027/011 comprising device for correcting geometrical aberrations, distortion
• CPC Class Codes: G02B 2027/0125 Field-of-view increase by wavefront division
• CPC Class Codes: G02B 26/103 having movable or deformable optical fibres, light guides or waveguides as scanning elements light guides per se G02B6/00
• CPC Class Codes: G02B 27/0081 with means for altering, e.g. enlarging, the entrance or exit pupil
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G02B 6/0013 Means for improving the coupling-in of light from the light source into the light guide coupling light into light guides in general G02B6/42
• CPC Class Codes: G02B 6/0016 Grooves, prisms, gratings, scattering particles or rough surfaces
• CPC Class Codes: G02B 6/34 utilising prism or grating G02B6/293 takes precedence
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 15/50 Lighting effects
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A virtual image generation system for use by an end user comprises a projection subsystem configured for generating a collimated light beam, and a display configured emitting light rays in response to the collimated light beam to display a pixel of an image frame to the end user. The pixel has a location encoded with angles of the emitted light rays. The virtual image generation system further comprises a sensing assembly configured for sensing at least one parameter indicative of at least one of the emitted light ray angles, and a control subsystem configured for generating image data defining a location of the pixel, and controlling an angle of the light beam relative to the display based on the defined location of the pixel and the sensed parameter(s).

Claims

1. A display system comprising:

a projection assembly configured to emit light;

a sensing assembly configured to measure at least one parameter of light incident thereon;

a waveguide apparatus;

an in-coupling optical element optically coupled to the projection assembly and waveguide apparatus, the in-coupling optical element and configured to in-couple light from the projection assembly into the waveguide apparatus;

a first out-coupling optical element disposed upon the waveguide apparatus and configured to direct light from the projection assembly guided within the waveguide apparatus out of the waveguide apparatus to an eye of a user;

a second out-coupling optical element disposed upon the waveguide apparatus and optically coupled to the sensing assembly, the second out-coupling optical element configured to direct light from the projection assembly guided within the waveguide apparatus to the sensing assembly;

a control subsystem in communication with the projection assembly and the sensing assembly, the control subsystem configured to;

cause the projection assembly to emit light corresponding to a pixel of an image frame;

receive measurement data from the sensing assembly indicating at least one parameter of light corresponding to the pixel of the image frame;

determine, based on the measurement data received from the sensing assembly, an angle at which light corresponding to the pixel of the image frame is incident the sensing assembly; and

control operation of the projection assembly based at least in part on the determined angle at which light corresponding to the pixel of the image frame is incident the sensing assembly.

2. The display system of claim 1, wherein the second out-coupling optical element is configured to split light from the projection assembly guided within the waveguide apparatus into multiple light rays that exit the waveguide apparatus.

3. The display system of claim 2, wherein the second out-coupling optical element comprises a linear diffraction pattern configured to split light from the projection assembly guided within the waveguide apparatus into multiple substantially-parallel light rays that exit the waveguide apparatus.

4. The display system of claim 1, wherein the second out-coupling optical element is positioned outside of a field of view (FOV) of the user.

5. The display system of claim 1, wherein the control subsystem is configured to control operation of the projection assembly based further on a position of the pixel within the image frame.

6. The display system of claim 5, wherein the control subsystem is configured to compute a discrepancy between the determined angle and an angle corresponding to the position of the pixel within the image frame, and wherein the control subsystem being configured to control operation of the projection assembly based further on the position of the pixel within the image frame comprises the control subsystem being configured to control operation of the projection assembly based on the computed discrepancy.

7. The display system of claim 1, wherein the control subsystem is configured to control operation of the projection assembly based further on a color of the pixel of the image frame.

8. The display system of claim 1, wherein the control subsystem is configured to control operation of the projection assembly based further on an intensity of the pixel of the image frame.

9. The display system of claim 1, wherein the control subsystem being configured to control operation of the projection assembly based at least in part on the determined angle comprises the control subsystem being configured to generate one or more additional image frames based at least in part on the determined angle and cause the projection assembly to emit light corresponding to pixels of the one or more additional image frames generated by the control subsystem.

10. The display system of claim 1, wherein the control subsystem being configured to control operation of the projection assembly based at least in part on the determined angle comprises the control subsystem being configured to control an angle at which the projection assembly emits light relative to the in-coupling optical element based at least in part on the determined angle.

11. The display system of claim 1 further comprising a frame structure configured to be worn about a head of the user, wherein the frame structure carries the waveguide apparatus.

12. The display system of claim 1, wherein the sensing assembly being configured to measure at least one parameter of light incident thereon comprises the sensing assembly being configured to measure an intensity of light incident thereon.

13. The display system of claim 1, wherein the sensing assembly comprises at least one photo-intensity sensor and at least one angle selective layer configured to transmit light incident thereon to the at least one photo-intensity sensor.

14. The display system of claim 13, wherein the sensing assembly further comprises at least one lens configured to transmit an orthogonal intensity component of light incident thereon to the at least one angle sensitive layer.

15. A display system comprising:

a projection assembly configured to emit light;

a sensing assembly optically coupled to the projection assembly and configured to measure at least one parameter of light incident thereon;

a control subsystem in communication with the projection assembly and the sensing assembly, the control subsystem configured to;

cause the projection assembly to emit light corresponding to a pixel at a particular position within an image frame;

receive measurement data from the sensing assembly indicating at least one parameter of light corresponding to the pixel of the image frame;

determine, based on the measurement data received from the sensing assembly, an angle at which light corresponding to the pixel of the image frame is incident the sensing assembly;

compute a discrepancy between the determined angle and an angle corresponding to the particular position of the pixel within the image frame; and

control operation of the projection assembly based at least in part on the computed discrepancy.

16. The display system of claim 15 further comprising a frame structure configured to be worn about a head of a user.

17. The display system of claim 16 further comprising a waveguide apparatus carried by the frame structure and configured to be positioned within a field of view (FOV) of the user, wherein the waveguide apparatus is optically coupled to the projection assembly.

18. The display system of claim 17, wherein the sensing assembly is optically coupled to the waveguide apparatus, and wherein the sensing assembly being optically coupled to the projection assembly comprises the sensing assembly being optically coupled to the projection assembly at least in part by way of the waveguide apparatus.

19. The display system of claim 17 further comprising an in-coupling optical element optically coupled to the projection assembly and waveguide apparatus, the in-coupling optical element and configured to in-couple light from the projection assembly into the waveguide apparatus.

20. The display system of claim 19, wherein the control subsystem being configured to control operation of the projection assembly based at least in part on the determined angle comprises the control subsystem being configured to control an angle at which the projection assembly emits light relative to the in-coupling optical element based at least in part on the determined angle.