Beam angle sensor in virtual/augmented reality system

• US 10,042,166 B2
• Filed: 01/12/2017
• Issued: 08/07/2018
• Est. Priority Date: 01/12/2016
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Yeoh, Ivan L., Edwin, Lionel Ernest, TeKolste, Robert Dale
• Primary Examiner(s): Mushambo, Martin
• Application Number:
  US15/405,146
• Publication Number:
  US 20170199384A1
• Time in Patent Office: 572 Days
• Field of Search: None
• CPC Class Codes: G01B 11/272 using photoelectric detection means
• CPC Class Codes: G02B 2027/011 comprising device for correcting geometrical aberrations, distortion
• CPC Class Codes: G02B 2027/0125 Field-of-view increase by wavefront division
• CPC Class Codes: G02B 26/103 having movable or deformable optical fibres, light guides or waveguides as scanning elements light guides per se G02B6/00
• CPC Class Codes: G02B 27/0081 with means for altering, e.g. enlarging, the entrance or exit pupil
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G02B 6/0013 Means for improving the coupling-in of light from the light source into the light guide coupling light into light guides in general G02B6/42
• CPC Class Codes: G02B 6/0016 Grooves, prisms, gratings, scattering particles or rough surfaces
• CPC Class Codes: G02B 6/34 utilising prism or grating G02B6/293 takes precedence
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 15/50 Lighting effects
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A virtual image generation system for use by an end user comprises a projection subsystem configured for generating a collimated light beam, and a display configured emitting light rays in response to the collimated light beam to display a pixel of an image frame to the end user. The pixel has a location encoded with angles of the emitted light rays. The virtual image generation system further comprises a sensing assembly configured for sensing at least one parameter indicative of at least one of the emitted light ray angles, and a control subsystem configured for generating image data defining a location of the pixel, and controlling an angle of the light beam relative to the display based on the defined location of the pixel and the sensed parameter(s).

Claims

1. A display subsystem for a virtual image generation system used by an end user, comprising:

a waveguide apparatus;

an imaging element configured for emitting light;

a collimation element configured for collimating the light from the imaging element into a light beam;

an in-coupling element (ICE) configured for directing the collimated light beam from the collimation element down the waveguide apparatus, such that a plurality of light rays exit the waveguide apparatus to display a pixel of an image frame to the end user, the pixel having a location encoded with angles of the plurality of exiting light rays; and

a sensing assembly configured for sensing at least one parameter indicative of the exiting light ray angles, wherein the at least one sensed parameter comprises an intensity of at least one light ray representative of the plurality of exiting light rays.

2. The display subsystem of claim 1, wherein the imaging element comprises:

at least one light source configured for generating the light;

an optical fiber configured for emitting the light; and

a mechanical drive assembly to which the optical fiber is mounted, the mechanical drive assembly configured for displacing the optical fiber in accordance with a scan pattern.

3. The display subsystem of claim 2, wherein the mechanical drive assembly comprises a piezoelectric element to which the optical fiber is mounted, and drive electronics configured for conveying electrical signals to the piezoelectric element, thereby causing the optical fiber to vibrate in accordance with the scan pattern.

4. The display subsystem of claim 1, wherein the waveguide apparatus is configured for being positioned in front of the eyes of the end user.

5. The display subsystem of claim 1, wherein the waveguide apparatus has a partially transparent display surface configured for being positioned in the field of view between the eyes of the end user and an ambient environment.

6. The display subsystem of claim 1, further comprising a frame structure configured for being worn by the end user, the frame structure carrying the waveguide apparatus.

7. The display subsystem of claim 1, wherein the waveguide apparatus comprises:

a planar optical waveguide, wherein the ICE is configured for optically coupling the collimated light beam from the imaging element into the planar optical waveguide as an in-coupled light beam;

an orthogonal pupil expansion (OPE) element associated with the planar optical waveguide for splitting the in-coupled light beam into a plurality of orthogonal light beams; and

an exit pupil expansion (EPE) element associated with the planar optical waveguide for splitting the plurality of orthogonal light beams into the light rays that exit the planar optical waveguide.

8. The display subsystem of claim 7, wherein the planar optical waveguide comprises a single substrate formed of a single pane of optically transparent material.

9. The display subsystem of claim 1, wherein the at least one sensed parameter is indicative of the exiting light ray angles projected in at least one plane that is orthogonal to a plane that coincides with the exterior surface of the waveguide apparatus.

10. The display subsystem of claim 9, wherein the at least one plane comprises two planes that are orthogonal to each other.

11. The display subsystem of claim 1, wherein the at least one representative light ray is different from the plurality of exiting light rays.

12. The display subsystem of claim 11, wherein the at least one representative light ray exits the waveguide apparatus at a different location from the plurality of exiting light rays outside of a field of view of the end user.

13. The display subsystem of claim 1, wherein the plurality of exiting light rays comprises the at least one representative light ray.

14. The display subsystem of claim 1, wherein the sensing assembly comprises at least one angle sensor, each of which includes a photo-intensity sensor and an angle selective layer mounted between the waveguide apparatus and the photo-intensity sensor.

15. The display subsystem of claim 14, wherein the at least one angle sensor comprises a pair of orthogonal sensors respectively configured for sensing first and second orthogonal intensity components of the at least one representative light ray.

16. The display subsystem of claim 15, wherein the pair of orthogonal sensors respectively comprises first and second cylindrical lenses configured for respectively passing the first and second orthogonal intensity components of the at least one representative light ray.

17. The display subsystem of claim 15, wherein the pair of orthogonal sensors respectively comprises first and second diffractive optical elements configured for respectively passing the first and second orthogonal intensity components of the at least one representative light ray.

18. The display subsystem of claim 15, wherein the pair of orthogonal sensors respectively comprises first and second polarization elements configured for respectively polarizing each of the at least one representative light ray into orthogonally polarized light rays.

19. The display subsystem of claim 15, wherein the angle selective layers are strained in orthogonal orientations.

20. The display subsystem of claim 15, wherein the at least one sensed parameter comprises an absolute intensity of the at least one representative light ray, such that the first and second orthogonal intensity components can be normalized.

21. The display of subsystem of claim 20, wherein the sensing assembly comprises another photo-intensity sensor configured for measuring the absolute intensity of the at least one representative light ray.

22. The display subsystem of claim 15, wherein the at least one sensed parameter is indicative of relative angles of the plurality of exiting light rays.

23. The display subsystem of claim 22, wherein the at least one sensed parameter further comprises a quadrant in which the collimated light beam is projected in a plane.

24. The display subsystem of claim 23, wherein the sensing assembly comprises a plurality of sensors spaced apart in the quadrants of a reference plane.

25. The display subsystem of claim 23, wherein the sensing assembly comprises a quadrant position sensing detector (PSD).

26. A virtual image generation system, comprising:

the display subsystem of claim 1; and

a control subsystem configured for generating image data defining a location of the pixel, and controlling an angle of the light beam relative to the ICE based on the defined location of the pixel and the at least one sensed parameter.

27. The virtual image generation system of claim 26, further comprising memory storing a three-dimensional scene, wherein the control subsystem is configured for rendering a plurality of synthetic image frames of the three-dimensional scene, and the display subsystem is configured for sequentially displaying the plurality of image frames to the end user.

28. The virtual image generation system of claim 26, wherein the control subsystem comprises a graphics processing unit (GPU).

29. A virtual image generation system for use by an end user, comprising:

a projection subsystem configured for generating a collimated light beam;

a display configured emitting light rays in response to the collimated light beam to display a pixel of an image frame to the end user, the pixel having a location encoded with angles of the emitted light rays;

a sensing assembly configured for sensing at least one parameter indicative of at least one of the emitted light ray angles; and

a control subsystem configured for generating image data defining a location of the pixel, and controlling an angle of the light beam relative to the display based on the defined location of the pixel and the at least one sensed parameter, wherein the at least one sensed parameter comprises an intensity of at least one light ray representative of the plurality of exiting light rays.

30. The virtual image generation system of claim 29, wherein the projection subsystem comprises:

at least one light source configured for generating light;

an optical fiber configured for emitting the light;

a collimation element configured for collimating the light from the optical fiber into the collimated light beam; and

a mechanical drive assembly to which the optical fiber is mounted, the mechanical drive assembly configured for displacing the optical fiber in accordance with a scan pattern.

31. The virtual image generation system of claim 29, wherein the mechanical drive assembly comprises a piezoelectric element to which the optical fiber is mounted, and drive electronics configured for conveying electrical signals to the piezoelectric element, thereby causing the optical fiber to vibrate in accordance with the scan pattern.

32. The virtual image generation system of claim 29, wherein the display is configured for displaying the pixel of the image frame selectively at one of a plurality of different focal points to the end user.

33. The virtual image generation system of claim 29, wherein the display is configured for being positioned in front of the eyes of the end user.

34. The virtual image generation system of claim 29, wherein the display has a partially transparent display surface configured for being positioned in the field of view between the eyes of the end user and an ambient environment.

35. The virtual image generation system of claim 29, further comprising a frame structure configured for being worn by the end user, the frame structure carrying the display.

36. The virtual image generation system of claim 29, further comprising memory storing a three-dimensional scene, wherein the control subsystem is configured for rendering a plurality of synthetic image frames of the three-dimensional scene, and the display is configured for sequentially displaying the plurality of image frames to the end user.

37. The virtual image generation system of claim 29, wherein the control subsystem comprises a graphics processing unit (GPU).

38. The virtual image generation system of claim 29, wherein the waveguide apparatus comprises:

a planar optical waveguide, wherein the ICE is configured for optically coupling the collimated light beam from the imaging element into the planar optical waveguide as an in-coupled light beam;

an orthogonal pupil expansion (OPE) element associated with the planar optical waveguide for splitting the in-coupled light beam into a plurality of orthogonal light beams; and

an exit pupil expansion (EPE) element associated with the planar optical waveguide for splitting the plurality of orthogonal light beams into the light rays that are emitted from the planar optical waveguide.

39. The virtual image generation system of claim 38, wherein the planar optical waveguide comprises a single substrate formed of a single pane of optically transparent material.

40. The virtual image generation system of claim 29, wherein the at least one sensed parameter is indicative of the emitted light ray angles projected in at least one plane that is orthogonal to a plane that coincides with the exterior surface of the waveguide apparatus.

41. The virtual image generation system of claim 40, wherein the at least one plane comprises two planes that are orthogonal to each other.

42. The virtual image generation system of claim 29, wherein the at least one representative light ray is different from the emitted light rays.

43. The virtual image generation system of claim 42, wherein the at least one representative light ray is emitted by the waveguide apparatus at a different location from the plurality of exiting light rays outside of a field of view of the end user.

44. The virtual image generation system of claim 29, wherein the plurality of emitted light rays comprises the at least one representative light ray.

45. The virtual image generation system of claim 29, wherein the sensing assembly comprises at least one angle sensor, each of which includes a photo-intensity sensor and an angle selective layer mounted between the waveguide apparatus and the photo-intensity sensor.

46. The virtual image generation system of claim 45, wherein the at least one angle sensor comprises a pair of orthogonal sensors respectively configured for sensing first and second orthogonal intensity components of the at least one representative light ray.

47. The virtual image generation system of claim 46, wherein the pair of orthogonal sensors respectively comprises first and second cylindrical lenses configured for respectively passing the first and second orthogonal intensity components of the at least one representative light ray.

48. The virtual image generation system of claim 46, wherein the pair of orthogonal sensors respectively comprises first and second diffractive optical elements configured for respectively passing the first and second orthogonal intensity components of the at least one representative light ray.

49. The virtual image generation system of claim 46, wherein the pair of orthogonal sensors respectively comprises first and second polarization elements configured for respectively polarizing each of the at least one representative light ray into orthogonally polarized light rays.

50. The virtual image generation system of claim 46, wherein the angle selective layers are strained in orthogonal orientations.

51. The virtual image generation system of claim 46, wherein the at least one sensed parameter comprises an absolute intensity of the at least one representative light ray, such that the first and second orthogonal intensity components can be normalized.

52. The virtual image generation system of claim 51, wherein the sensing assembly further comprises another photo-intensity sensor configured for measuring the absolute intensity of the at least one representative light ray.

53. The virtual image generation system of claim 46, wherein the at least one sensed parameter is indicative of relative angles of the emitted light rays.

54. The virtual image generation system of claim 53, wherein the at least one sensed parameter further comprises a quadrant in which the collimated light beam is projected in a plane.

55. The virtual image generation system of claim 54, wherein the sensing assembly comprises a plurality of sensors spaced apart in the quadrants of a reference plane.

56. The virtual image generation system of claim 54, wherein the sensing assembly comprises a quadrant position sensing detector (PSD).