Corneal sphere tracking for generating an eye model

• US 10,109,067 B2
• Filed: 03/10/2017
• Issued: 10/23/2018
• Est. Priority Date: 03/11/2016
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Oculus VR, LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Inventors: Hunt, Warren Andrew, Cavin, Robert Dale
• Primary Examiner(s): Abdi, Amara
• Application Number:
  US15/456,358
• Publication Number:
  US 20170263006A1
• Time in Patent Office: 592 Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1637 Details related to the display arrangement, including those related to the mounting of the display in the housing
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/514 from specularities
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

A head mounted display (HMD) comprises an eye tracking system configured to enable eye-tracking using light. The eye tracking system comprises two or more illumination sources positioned relative to one another and an optical detector in order to capture. The optical detector is configured to capture images of the cornea based on one or more reflections. The eye tracking unit is configured to generate a model of the user'"'"'s eye. The generated eye model is used to determine eye tracking information such as gaze direction as the user glances at different objects in the HMD.

Claims

1. A head mounted display (HMD) comprising:

an eye tracking system including;

two or more illumination sources configured to illuminate a user'"'"'s eye,

a detector configured to capture one or more images of the user'"'"'s eye including light reflected from a cornea of the user'"'"'s eye, and

a control module configured to;

generate a model of the eye based on the captured one or more images of the user'"'"'s eye the generated model comprising a first sphere that approximates a portion of a scleral surface of the eye and a second sphere that approximates the cornea, and

perform an optical action based in part on the generated model of the eye.

2. The HMD of claim 1, wherein the two or more illumination sources are positioned relative to one another such that the detector is able to capture one or more images of the two or more illumination sources based on the light reflected from cornea.

3. The HMD of claim 1, wherein the two or more illumination sources are part of the electronic display.

4. The HMD of claim 1, wherein each of the two or more illumination sources are configured to emit light of a different wavelength.

5. The HMD of claim 1, wherein light emitted by each of the two or more illumination sources is modulated at a different frequency and a different amplitude.

6. The HMD of claim 1, wherein the two or more illumination sources form a ring pattern.

7. The HMD of claim 1, wherein the detector is an optical sensor that can capture a plurality of images of the user'"'"'s eye.

8. The HMD of claim 1, wherein the control module is further configured to:

receive the one or more captured images of the user'"'"'s eye, the images including one or more corneal reflections;

generate the model of the eye, the generated model of the eye including a radius and an origin associated with the first sphere and a radius and an origin associated with the second sphere; and

store the generated model in a database associated with the HMD.

9. The HMD of claim 8, wherein the control module is further configured to model corneal motion as a rotation about the origin associated with the first sphere.

10. The HMD of claim 1, wherein the control module is further configured to:

retrieve a learning model of a reference eye, the learning model including a relationship between one or more movements of the reference eye and the corneal reflections associated with the reference eye;

capture an image of the user'"'"'s eye, the one or more images including a corneal reflections; and

extrapolate one or more parameter values associated with the eye model based on a comparison between the one or more corneal reflections of the reference eye and the corneal reflections associated with the captured image.

11. The HMD of claim 1, wherein an optical action is selected from a group consisting of:

determining a user'"'"'s vergence angle, determining a user'"'"'s accommodation depth, identifying the user, determining an eye'"'"'s torsional state, or some combination thereof.

12. A head mounted display (HMD) comprising:

an electronic display configured to display images to the user;

an optics block configured to magnify light received by the optics block from the electronic display; and

an eye tracking system including;

two or more illumination sources configured to illuminate a surface of an eye of the user,

a detector configured to capture one or more images of the user'"'"'s eye including light reflected from the cornea of the user'"'"'s eye, and

a control module configured to;

generate a model of the eye based on the captured one or more images of the user'"'"'s eye, the generated model comprising a first sphere that approximates a portion of a scleral surface of the eye and a second sphere that approximates the cornea, and

perform an optical action based in part on the generated model of the eye.

13. The HMD of claim 12, wherein the two or more illumination sources are positioned relative to one another such that the detector is able to capture one or more images of the two or more illumination sources based on the light reflected from cornea.

14. The HMD of claim 12, wherein the two or more illumination sources are part of the electronic display.

15. The HMD of claim 12 , wherein the control module is further configured to:

receive the one or more captured images of the user'"'"'s eye, the images including one or more corneal reflections;

generate the model of the eye, the generated model of the eye including a radius and an origin associated with the first sphere and a radius and an origin associated with the second sphere; and

store the generated model in a database associated with the HMD.

16. The HMD of claim 15, wherein the control module is further configured to model corneal motion as a rotation about the origin associated with the first sphere.

17. the HMD of claim 12, wherein the control module is further configured to:

retrieve a learning model of a reference eye, the learning model including a relationship between one or more movements of the reference eye and the corneal reflections associated with the reference eye;

capture an image of the user'"'"'s eye, the one or more images including a corneal reflections; and

extrapolate one or more parameter values associated with the eye model based on a comparison between the one or more corneal reflections of the reference eye and the corneal reflections associated with the captured image.

18. The HMD of claim 12, wherein an optical action is selected from a group consisting of:

determining a user'"'"'s vergence angle, determining a user'"'"'s accommodation depth, identifying the user, determining an eye'"'"'s torsional state, or some combination thereof.

19. A method comprising:

illuminating a user'"'"'s eye with light from two or more illumination sources;

capturing, at a detector, one or more images of the illuminated user'"'"'s eye, the one or more images including light reflected from a cornea of the user'"'"'s eye;

generating a model of the user'"'"'s eye based on the captured one or more images of the user'"'"'s eye, the generated model comprises a first sphere that approximates a portion of a scleral surface of the eye and a second sphere that approximates the cornea; and

performing an optical action based in part on the generated model of the eye.

20. The method of claim 19, wherein the two or more illumination sources are positioned relative to one another such that the detector is able to capture one or more images of the two or more illumination sources based on the light reflected from cornea.

21. The method of claim 19, wherein the two or more illumination sources are part of the electronic display.

22. The method of claim 19, wherein illuminating a user'"'"'s eye with light from two or more illumination sources comprises illuminating the user'"'"'s eye with at least two different wavelengths of light.

23. The method of claim 19, wherein light emitted by each of the two or more illumination sources is modulated at a different frequency and a different amplitude.

24. The method of claim 19, wherein the two or more illumination sources form a ring pattern.

25. The method of claim 19, wherein the detector is an optical sensor that can capture a plurality of images of the user'"'"'s eye.

26. The method of claim 19 further comprising:

receiving the one or more captured images of the user'"'"'s eye, the images including one or more corneal reflections;

generating the model of the eye, the generated model of the eye including a radius and an origin associated with the first sphere and a radius and an origin associated with the second sphere; and

storing the generated model in a database associated with the HMD.

27. The method of claim 26 further comprising:

modeling corneal motion as a rotation about the origin associated with the first sphere.

28. The method of claim 19 further comprising:

retrieving a learning model of a reference eye, the learning model including a relationship between one or more movements of the reference eye and the corneal reflections associated with the reference eye;

capturing an image of the user'"'"'s eye, the one or more images including a corneal reflections; and

extrapolating one or more parameter values associated with the eye model based on a comparison between the one or more corneal reflections of the reference eye and the corneal reflections associated with the captured image.

29. The method of claim 19, wherein an optical action is selected from a group consisting of:

determining a user'"'"'s vergence angle, determining a user'"'"'s accommodation depth, identifying the user, determining an eye'"'"'s torsional state, or some combination thereof.