MULTI-PERSPECTIVE EYE-TRACKING FOR VR/AR SYSTEMS

• US 20190129174A1
• Filed: 12/05/2017
• Published: 05/02/2019
• Est. Priority Date: 10/31/2017
• Status: Abandoned Application

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Perreault, John D.
• Application Number:
  US15/832,275
• Publication Number:
  US 20190129174A1
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0961 Lens arrays lens arrays per se G02B3/0006
• CPC Class Codes: G02B 27/143 using macroscopically faceted or segmented reflective surfaces
• CPC Class Codes: G02B 3/0006 Arrays G02B3/02, G02B5/188 take precedence
• CPC Class Codes: G02B 3/08 with discontinuous faces, e.g. Fresnel lens diffractive Fresnel lenses G02B5/1876
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 2213/001 Constructional or mechanical details

Abstract

A display system, such as a head mounted display, tracks a pose of a user'"'"'s eye based on multiple perspectives of the eye captured via a segmented optics array, such as a lenslet array or Fresnel lens. The display system reflects light (e.g., infra-red light) off each segment of the segmented optics array, and captures an image based on the reflected light. Because of the segmented optics, the captured image represents multiple concurrent perspectives of the user'"'"'s eye. The display system analyzes the different perspectives and selects a perspective, or combination of perspectives, and based on the selected perspective or combination, identifies a pose of the user'"'"'s eye.

Claims

1. A method comprising:

capturing, at an image capturing device, a plurality of concurrent perspectives of an eye, the plurality of concurrent perspectives depicting the eye from different angles, based on light reflected from a segmented optics array;

identifying a perspective of the plurality of concurrent perspectives that is predicted to accurately identify a position of the eye; and

identifying a position of the eye based on the dentified perspective.

2. The method of claim 1, wherein capturing the plurality of concurrent perspectives comprises capturing the plurality of concurrent perspectives in a single image at the image capturing device.

3. The method of claim 1, wherein the segmented optics array comprises an array of lenslets.

4. The method of claim 3, wherein each of the plurality of concurrent perspectives corresponds to a different lenslet of the array of lenslets.

5. The method of claim 1, wherein the segmented optics array comprises a Fresnel lens.

6. The method of claim 1, wherein capturing the plurality of concurrent perspectives comprises:

reflecting non-visible light off the segmented optics array, the segmented optics array configured to pass through visible light; and

identifying the position of the eye based on the reflected non-visible light.

7. The method of claim 1, further comprising:

generating an image for display based on the position of the eye; and

displaying the image via the segmented optics array.

8. At a head-mounted display (HMD) system, a method comprising:

displaying a first image via a segmented optics array of the HMD;

capturing a second image based on light reflected from the segmented optics array, the second image comprising a plurality of perspectives of an eye, the plurality of perspectives depicting the eye from different angles;

identifying a perspective of the plurality of perspectives that is predicted to accurately identify a position of the eye; and

identifying a position of the eye based on the identified perspective.

9. The method of claim 8, wherein the segmented optics array comprises a plurality of lenslets.

10. The method of claim 8, wherein the segmented optics array comprises a Fresnel lens.

11. The method of claim 8, wherein identifying the position of the eye comprises:

providing the second image to a neural network, the neural network trained based on a training set of captured eye images; and

identifying the position of the eye at the neural network.

12. The method of claim 8, further comprising:

generating a third image based on the identified position of the eye; and

displaying the third image at the HMD via the segmented optics array.

13. The method of claim 8, wherein the light reflected from the segmented optics array is non-visible light.

14. A head-mounted display (HMD) system, comprising:

a segmented optics array;

an image capturing device to capture a plurality of concurrent perspectives of an eye based on light reflected from the segmented optics array, the plurality of concurrent perspectives depicting the eye from different angles; and

an eye-tracking module to select a perspective that is predicted to accurately identify a position of the eye and identify the position of the eye based on the selected perspective.

15. The HMD system of claim 14, wherein the image capturing device is configured to capture the plurality of concurrent perspectives in a single image.

16. The HMD system of claim 14, wherein the segmented optics array comprises an array of lenslets.

17. The HMD system of claim 16, wherein each of the plurality of concurrent perspectives corresponds to a different lenslet of the array of lenslets.

18. The HMD system of claim 14, wherein the segmented optics array comprises a Fresnel lens.

19. The HMD system of claim 14, wherein:

the segmented optics array comprises a reflective coating configured to reflect non-visible light and to pass through visible light; and

wherein the image capturing device is configured to capture the plurality of concurrent perspectives based on the reflected non-visible light.

20. The HMD system of claim 14, further comprising:

a display configured to display an image via the segmented optics array, the image based on the position of the eye.