Eye imaging with an off-axis imager

• US 10,466,478 B2
• Filed: 09/21/2016
• Issued: 11/05/2019
• Est. Priority Date: 09/23/2015
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Klug, Michael Anthony, Kaehler, Adrian
• Primary Examiner(s): Lei, Jie
• Application Number:
  US15/271,802
• Publication Number:
  US 20170082858A1
• Time in Patent Office: 1,140 Days
• Field of Search: 359350, 359 13, 359630, 359631, 359633, 345 7, 345 8
• CPC Class Codes: A61B 3/10 Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 3/12 for looking at the eye fundus, e.g. ophthalmoscopes A61B3/13 takes precedence
• CPC Class Codes: A61B 3/14 Arrangements specially adapted for eye photography apparatus or arrangements for taking photographs per se G03B
• CPC Class Codes: A61B 5/1122 of movement trajectories
• CPC Class Codes: A61B 5/117 Identification of persons methods or arrangements for recognising patterns, e.g. fingerprints, G06F18/00, G06V40/00; identification of persons by analysing their voice or speech G10L17/00
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/742 using visual displays A61B5/339 takes precedence
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features

Abstract

Examples of an imaging system for use with a head mounted display (HMD) are disclosed. The imaging system can include a forward-facing imaging camera and a surface of a display of the HMD can include an off-axis diffractive optical element (DOE) or hot mirror configured to reflect light to the imaging camera. The DOE or hot mirror can be segmented. The imaging system can be used for eye tracking, biometric identification, multiscopic reconstruction of the three-dimensional shape of the eye, etc.

Claims

1. A display system comprising:

an optical element configured to display an image to a user, the optical element configured to be positioned forward an eye of the user;

a forward-facing imager configured to be mounted rearward the eye of the user and configured to provide a depth of field, for imaging an eye region of the user, that is approximately twice as large as a backward-facing imager; and

a reflective element disposed in or on the optical element, the reflective element configured to reflect toward the forward-facing imager infrared light received from the eye of the user.

2. The display system of claim 1, wherein the optical element comprises a light field display.

3. The display system of claim 1, wherein the reflective element comprises a hot mirror, an off-axis diffractive optical element (DOE), an off-axis holographic mirror (OAHM), or an off-axis volumetric diffractive optical element (OAVDOE).

4. The display system of claim 1, wherein the reflective element comprises a plurality of segments having different optical power or different reflection angle.

5. The display system of claim 4, wherein each of the plurality of segments is configured as a virtual camera imaging the eye at a correspondingly different angular location.

6. The display system of claim 5, wherein at least one of the plurality of segments is configured to generate the respective virtual camera at infinity.

7. The display system of claim 4, wherein to image the eye, the imager uses an upper segment when the user is looking upward and a lower segment when the user is looking downward.

8. The display system of claim 4, wherein to image the eye the display system selects a segment of the plurality of segments that has the least occlusion by eyelashes or eyelids of the user.

9. The display system of claim 1, further comprising:

non-transitory memory configured to store images of the eye of the user obtained by the forward-facing imager; and

a hardware processor in communication with the non-transitory memory, the hardware processor programmed to;

access the images of the eye; and

perform one or more of the following;

track the eye of the user;

extract biometric information associated with the eye of the user;

reconstruct a shape of a portion of the eye of the user;

estimate an accommodation state of the eye of the user;

or

image a retina, an iris, or other element of the eye of the user.

10. The display system of claim 9, wherein the hardware processor is programmed to estimate an accommodation state of the eye of the user.

11. The display system of claim 10, wherein to estimate the accommodation state of the eye of the user, the hardware processor is programmed to estimate distortion of an apparent location of the pupil of the eye of the user.

12. The display system of claim 9, wherein the hardware processor is programmed to reconstruct a shape of a portion of the eye of the user.

13. The display system of claim 12, wherein the hardware processor is programmed to utilize the shape of the portion of the eye of the user to estimate orientation of the eye.

14. The display system of claim 13, wherein to estimate orientation of the eye, the hardware processor is programmed to estimate location of a bulge of the cornea of the eye.

15. A head mounted display system comprising:

a frame configured to support the display system according to claim 1 such that the optical element is positioned forward a first eye of the user.

16. The head mounted display system of claim 15, wherein the frame supports a second display system according to claim 1 such that the optical element of the second display system is positioned forward a second eye of the user.