Integrated eye tracking and display system

• US 8,998,414 B2
• Filed: 09/26/2011
• Issued: 04/07/2015
• Est. Priority Date: 09/26/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bohn, David D.
• Primary Examiner(s): Sugarman, Scott J
• Assistant Examiner(s): Choudhury, Mustak
• Application Number:
  US13/245,700
• Publication Number:
  US 20130077049A1
• Time in Patent Office: 1,289 Days
• Field of Search: 351/210, 351/206, 351/209, 345/8, 348/E13.045, 348/E13.047, 348/E13.066
• US Class Current: 351/210
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 5/20 Filters polarising elements G02B5/30

Abstract

Technology is provided for an integrated eye tracking and display system for a see-through, near-eye, mixed reality display device. Image data and IR illumination for eye tracking are optically coupled into a respective see-through, planar waveguide positioned to be seen through by each eye in a respective display optical system of the display device. The respective planar waveguide comprises one or more wavelength selective filters positioned to be co-axial with an optical axis of the respective display optical system. The wavelength selective filters direct IR and visible illumination out of the planar waveguide in the direction of the respective eye and direct IR reflections, including reflections from the eye, into the planar waveguide. The reflections are optically coupled out of the waveguide to an IR sensor which generates eye tracking data based on the reflections.

Claims

1. An integrated eye tracking and display system for a near-eye, mixed reality display device comprising:

a display optical system positioned to be seen through by an eye by the near-eye, mixed reality display device, the display optical system having an optical axis and a planar waveguide;

one or more wavelength selective filters positioned in the planar waveguide in co-axial alignment with the optical axis of the display optical system for directing infrared and visible illumination out of the planar waveguide in the direction of the eye and for directing infrared reflections from the eye into the planar waveguide; and

an integrated array of optical elements including light sources interspersed with one or more infrared sensors, the light sources being for transmitting infrared and visible illumination, the light sources being optically coupled to the planar waveguide for directing their illumination into the planar waveguide, and the one or more infrared sensors of the integrated array being optically coupled to the planar waveguide for receiving infrared reflections directed from the one or more wavelength selective filters.

2. The system of claim 1 wherein the one or more wavelength selective filters comprises a bidirectional infrared wavelength selective filter.

3. The system of claim 1 wherein the integrated array of optical elements includes a linear array of lines of the light sources which are scanned by a scanning mirror for coupling the visible and infrared illumination into the planar waveguide.

4. The system of claim 3 wherein infrared reflections of the eye are optically coupled in a reverse optical path from at least one of the one or more wavelength selective filters in the planar waveguide to the scanning mirror and to the one or more infrared sensors in the integrated array of optical elements.

5. The system of claim 4 wherein the scanning mirror is a two-dimensional scanning mirror.

6. The system of claim 3 wherein the scanning mirror is implemented using micro electromechanical system technology.

7. The system of claim 1 wherein the one or more wavelength selective filters further comprise an active grating being modulated for being selective for different wavelengths in the visible and infrared spectrums at separate time intervals.

8. The system of claim 1 wherein the integrated array of optical elements including light sources interspersed with one or more infrared sensors further comprises an integrated array of lines of the light sources interspersed with lines of infrared sensors.

9. A method for processing visible and infrared wavelengths for image display and eye tracking in a display optical system of a near-eye, mixed reality display device comprising:

optically coupling visible and infrared illumination from an integrated array of optical elements including visible and infrared light sources interspersed with one or more infrared sensors into a planar waveguide positioned in the display optical system to be seen through by an eye in a first direction of an optical path of the planar waveguide;

directing the visible and infrared illumination toward an eye out of the planar waveguide by one or more wavelength selective filters co-axially aligned with an optical axis of the display optical system;

directing infrared reflections from the eye by the one or more wavelength selective filters into the planar waveguide in a second direction of the same optical path;

optically coupling the infrared reflections from the planar waveguide to the one or more infrared sensors of the integrated array of optical elements; and

storing data generated by the one or more infrared sensors as eye tracking data.

10. The method of claim 9 further comprises:

transmitting the infrared and visible illumination at separate time intervals;

wherein the one or more wavelength selective filters co-axially aligned with the optical axis of the display optical system includes an active grating; and

modulating a wavelength selectivity of the active grating between a visible spectrum and an infrared spectrum in timing with the separate time intervals.

11. The method of claim 9

wherein optically coupling visible and infrared illumination into the planar waveguide in the first direction of the optical path further comprises a first wavelength selective filter passing through the visible and infrared illumination in the first direction into the planar waveguide;

and

wherein optically coupling the infrared reflections from the planar waveguide to the one or more infrared sensors of the integrated array of optical elements further comprises the first wavelength selective filter passing through only infrared reflections received from the planar waveguide to the one or more infrared sensors.

12. The method of claim 9 wherein the integrated array of optical elements including visible and infrared light sources interspersed with one or more infrared sensors further comprises an integrated array of lines of the light sources interspersed with lines of infrared sensors.