ULTRASOUND/RADAR FOR EYE TRACKING

• US 20170261610A1
• Filed: 03/10/2017
• Published: 09/14/2017
• Est. Priority Date: 03/11/2016
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Oculus VR, LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Inventors: Scally, Brian Michael, Perek, David R.
• Granted Patent: US 10,908,279 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 13/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S13/46
• CPC Class Codes: G01S 13/50 Systems of measurement based on relative movement of target
• CPC Class Codes: G01S 13/66 Radar-tracking systems; Analogous systems
• CPC Class Codes: G01S 13/89 for mapping or imaging
• CPC Class Codes: G01S 15/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S15/46
• CPC Class Codes: G01S 15/50 Systems of measurement, based on relative movement of the target
• CPC Class Codes: G01S 15/66 Sonar tracking systems
• CPC Class Codes: G01S 15/89 for mapping or imaging
• CPC Class Codes: G01S 2013/0245 Radar with phased array antenna
• CPC Class Codes: G01S 7/539 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Abstract

An eye tracking unit that includes one or more transmitters that transmit a signal (e.g., a radar signal or an ultrasonic sound) at an eye, one or more receivers that receive a reflection of the signal generated by interaction of the signal with the eye, and an eye orientation estimation module that estimates an orientation of the eye based on the reflected signal received by the one or more ultrasonic receivers and based on a model of the eye. The eye tracking unit may be part of a head-mounted display (HMD) that includes a display element configured to display content to a user wearing the HMD. The model of the eye may be trained by displaying a visual indicator on the electronic element and detecting a reflected signal corresponding to the eye looking at the visual indicator.

Claims

1. An eye tracking unit comprising:

one or more ultrasonic transmitters configured to transmit ultrasonic sound towards an eye;

one or more ultrasonic receivers configured to receive an echo generated by interaction of the ultrasonic sound with the eye; and

an eye orientation estimation module configured to estimate an orientation of the eye based on the echo received by the one or more ultrasonic receivers and based on a model of the eye.

2. The eye tracking unit of claim 1, wherein the eye tracking unit is a component of a head-mounted display (HMD), the HMD comprising:

a display element configured to display content to a user wearing the HMD; and

an optics block configured to direct light from the display element to an exit pupil of the HMD.

3. The eye tracking unit of claim 1, wherein each of the one or more ultrasonic transmitters is an ultrasonic transceiver that operates as an ultrasonic receiver to receive the echo.

4. The eye tracking unit of claim 1, further comprising a calibration module, the calibration module configured to:

control a display element to display a visual indicator at a position on the display element;

receive, from the one or more ultrasonic receivers, a calibration echo corresponding to a calibration ultrasonic sound transmitted by the one or more ultrasonic transmitters; and

train the model of the eye of the user based on the captured second image and based on the position of the visual indicator.

5. The eye tracking unit of claim 1, wherein the eye orientation estimation module is configured to:

estimate a direction of a pupillary axis of the eye; and

estimate a direction of a foveal axis of the eye based on the estimated direction of the pupillary axis and based on an offset between the pupillary axis and the foveal axis.

6. The eye tracking unit of claim 1, wherein the eye orientation estimation module is configured to estimate the orientation of the eye based in part on detecting the position of a feature of the eye based on the echo and the model of the eye, the feature selected from a group consisting of the curvature of the cornea, the cornea-sclera interface, iris features beneath the cornea, and the vertex of the cornea.

7. The eye tracking unit of claim 1, wherein the eye orientation estimation module is configured to estimate the orientation of the eye based in part on determining a yaw, pitch, and roll rotation for the eye and a 3-dimensional translation vector for the eye.

8. The eye tracking unit of claim 1, wherein the eye orientation module determines velocity information for the eye based Doppler frequency shifts in the echo, and wherein the eye orientation module estimates the orientation of the eye based on the velocity information.

9. The eye tracking unit of claim 1, wherein the eye orientation module determines one or more cross-sectional scans for the eye based on the received echo, and wherein the orientation module estimates the orientation of the eye based on the one or more cross-sectional scans.

10. The eye tracking unit of claim 1, wherein one or more transmitters include multiple transmitters that operate as a phased array.

11. An eye tracking unit comprising:

a first set of ultrasonic transmitters configured to transmit a first ultrasonic sound towards a first eye;

a second set of ultrasonic transmitters configured to transmit a second ultrasonic sound towards a second eye;

a first set of ultrasonic receivers configured to receive a first echo generated by interaction of the first ultrasonic sound with the first eye;

a second set of ultrasonic receivers configured to receive a second echo generated by interaction of the second ultrasonic sound with the second eye; and

an eye orientation estimation module configured to;

estimate a first orientation of the first eye based on the first echo received by the first set of ultrasonic receivers and based on an eye model, and

estimate a second orientation of the second eye based on the second echo received by the second set of ultrasonic receivers and based on the eye model.

12. The eye tracking unit of claim 11, wherein the eye tracking unit is a component of a head-mounted display (HMD), the HMD configured to determine an inter-pupillary distance of the user using the first orientation and the second orientation.

13. The eye tracking unit of claim 11, wherein the eye tracking unit is a component of a head-mounted display (HMD), the HMD configured to determine a gaze direction of the user using the first orientation and the second orientation.

14. The eye tracking unit of claim 11, further comprising a calibration module, the calibration module configured to:

control a display element to display a visual indicator at a position on the electronic element;

receive, from the first set of ultrasonic transmitters, a calibration echo corresponding to a calibration ultrasonic sound transmitted by the first set of ultrasonic transmitters; and

train the eye model based on the received calibration echo and based on the position of the visual indicator.

15. The eye tracking unit of claim 11, wherein the eye orientation estimation module is further configured to:

estimate a direction of a pupillary axis of the first eye; and

estimate a direction of a foveal axis of the first eye based on the estimated direction of the pupillary axis and based on an offset between the pupillary axis and the foveal axis.

16. The eye tracking unit of claim 11, wherein the eye orientation estimation module is configured to estimate the orientation of the first eye based in part on detecting the position of a feature of the first eye based on the first echo and the eye model, the feature selected from a group consisting of the curvature of the cornea, the cornea-sclera interface, iris features beneath the cornea, and the vertex of the cornea.

17. The eye tracking unit of claim 11, wherein the eye orientation estimation module is configured to estimate the first orientation and the second orientation based in part on determining, for each of the first eye and the second eye, a respective yaw, pitch, and roll rotation and a respective translation vector for the eye.

18. The eye tracking unit of claim 1, wherein the eye orientation module determines velocity information for the first eye based Doppler frequency shifts in the first echo, and wherein the eye orientation module estimates the orientation of the first eye based on the velocity information.

19. The eye tracking unit of claim 1, wherein the eye orientation module is further configured to:

determine one or more first cross-sectional scans for the first eye based on the received first echo;

determine one or more second cross-sectional scans for the second eye based on the received second echo;

estimate the first orientation of the first eye based on the one or more first cross-sectional scans; and

estimate the second orientation of the second eye based on the one or more second cross-sectional scans.

20. The eye tracking unit of claim 11, wherein each of the first set of ultrasonic transmitters is an ultrasonic transceiver that operates as one of the first set of ultrasonic receivers to receive the first echo, and wherein each of the second set of ultrasonic transmitters is an ultrasonic transceiver that operates as one of the second set of ultrasonic receivers to receive the second echo.