INTEGRATED EYE TRACKING AND DISPLAY SYSTEM

• US 20130077049A1
• Filed: 09/26/2011
• Published: 03/28/2013
• Est. Priority Date: 09/26/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bohn, David D.
• Granted Patent: US 8,998,414 B2
• Time in Patent Office: Days
• US Class Current: 351/210
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 5/20 Filters polarising elements G02B5/30

Abstract

Technology is provided for an integrated eye tracking and display system for a see-through, near-eye, mixed reality display device. Image data and IR illumination for eye tracking are optically coupled into a respective see-through, planar waveguide positioned to be seen through by each eye in a respective display optical system of the display device. The respective planar waveguide comprises one or more wavelength selective filters positioned to be co-axial with an optical axis of the respective display optical system. The wavelength selective filters direct IR and visible illumination out of the planar waveguide in the direction of the respective eye and direct IR reflections, including reflections from the eye, into the planar waveguide. The reflections are optically coupled out of the waveguide to an IR sensor which generates eye tracking data based on the reflections.

Claims

1. An integrated eye tracking and display system for a see-through, near-eye, mixed reality display device comprising:

a display optical system for each eye, the display optical system having an optical axis and a see-through, planar waveguide positioned to be seen through by the respective eye;

one or more wavelength selective filters positioned in the respective see-through, planar waveguide in co-axial alignment with the optical axis of the respective display optical system for directing infrared and visible illumination out of the respective planar waveguide and for directing infrared reflections into the planar waveguide;

an infrared illumination source being positioned for having its infrared illumination optically coupled into the planar waveguide;

an infrared sensor being optically coupled to the planar waveguide for receiving the infrared reflections directed from the wavelength selective filters; and

an image generation unit optically coupled for transmitting visible illumination into the planar waveguide.

2. The system of claim 1 wherein the one or more wavelength selective filters include one or more gratings and the infrared sensor is optically coupled to the planar waveguide by an infrared wavelength selective grating.

3. The system of claim 2 wherein at least one of the gratings include a diffraction grating.

4. The system of claim 1 wherein the one or more wavelength selective filters include stacked gratings and at least one of the stacked gratings is selective for one or more visible spectrum wavelengths and at least one of the stacked gratings is selective for a wavelength in the infrared spectrum.

5. The system of claim 1 wherein at least one of the one or more wavelength selective filters is bi-directional.

6. The system of claim 1 wherein the one or more wavelength selective filters include a hot mirror.

7. The system of claim 1 wherein the image generation unit includes a micro display unit.

8. The system of claim 1 wherein the image generation unit comprises

a light source being modulated for transmitting illumination at different wavelengths at separate time intervals;

the infrared illumination source also comprises the light source and the different wavelengths include wavelengths in both the visible and infrared spectrums; and

a two-dimensional scanning mirror optically coupled to the light source for receiving the illumination and optically coupled to an optical path into the planar waveguide for directing the illumination into the planar waveguide.

9. The system of claim 1 further comprises:

the image generation unit comprises light sources for transmitting visible illumination;

the infrared illumination source comprises a light source for transmitting infrared illumination;

a beam combiner positioned to receive the visible and infrared illumination and combine the visible and infrared illumination into a combined beam; and

a two-dimensional scanning mirror optically coupled to the beam combiner for receiving the combined beam and the scanning mirror being optically coupled to an optical path into the planar waveguide for directing the combined beam into the planar waveguide.

10. An integrated eye tracking and display system for a see-through, near-eye, mixed reality display device comprising:

a display optical system for each eye, the display optical system having an optical axis and a see-through, planar waveguide positioned to be seen through by the respective eye;

one or more wavelength selective filters positioned in the respective see-through, planar waveguide in co-axial alignment with the optical axis of the respective display optical system for directing infrared and visible illumination out of the respective planar waveguide and for directing infrared reflections into the planar waveguide;

an array of optical elements including light sources for transmitting infrared and visible illumination, the light sources being optically coupled to the planar waveguide for directing their illumination into the planar waveguide; and

an infrared sensor being optically coupled to the planar waveguide for receiving infrared reflections directed from the wavelength selective filters.

11. The system of claim 10 wherein the infrared sensor comprises one or more sensors in the array of optical elements interspersed with the array of light sources.

12. The system of claim 11 wherein the one or more wavelength selective filters comprises a bidirectional infrared wavelength selective filter.

13. The system of claim 10 wherein the array of light sources is a linear array of lines of light sources which are scanned by a scanning mirror for coupling the visible and infrared illumination into the planar waveguide.

14. The system of claim 11 wherein infrared reflections of the respective eye are optically coupled in a reverse optical path from at least one of the one or more wavelength selective filter in the planar waveguide to the scanning mirror and to the one or more sensors in the array of optical elements.

15. The system of claim 14 wherein the scanning mirror is a two-dimensional scanning mirror.

16. The system of claim 13 wherein the scanning mirror is implemented using microelectromechanical system technology.

17. The system of claim 10 wherein the one or more wavelength selective filters further comprise an active grating being modulated for being selective for different wavelengths in the visible and infrared spectrums at separate time intervals.

18. A method for processing visible and infrared wavelengths for image display and eye tracking in an optical path of a see-through, planar waveguide positioned to be seen through in a display optical system of a see-through, near-eye, mixed reality display device comprising:

optically coupling visible and infrared illumination into the planar waveguide in a first direction of an optical path;

directing the visible and infrared illumination toward the eye out of the planar waveguide by one or more wavelength selective filters co-axially aligned with an optical axis of the display optical system;

directing infrared reflections from the eye by the one or more wavelength selective filters into the planar waveguide in a second direction of the same optical path;

optically coupling the infrared reflections from the planar waveguide to an infrared sensor; and

storing data generated by the infrared sensor as eye tracking data.

19. The method of claim 18 further comprises:

transmitting the infrared and visible illumination at separate time intervals;

wherein the one or more wavelength selective filters co-axially aligned with the optical axis of the display optical system includes an active grating; and

modulating a wavelength selectivity of the active grating between a visible spectrum and an infrared spectrum in timing with the separate time intervals

20. The method of claim 18 wherein the infrared sensor is a plurality of infrared sensing elements interspersed in an integrated array also including visible and IR light sources for generating the visible and infrared illumination;

wherein optically coupling visible and infrared illumination into the planar waveguide in the first direction of the optical path further comprises a first wavelength selective filter passing through the visible and infrared illumination in the first direction into the planar waveguide; and

wherein optically coupling the infrared reflections from the planar waveguide to the infrared sensor further comprises the first wavelength selective filter passing through only infrared reflections received from the planar waveguide to the plurality of infrared sensing elements.