Eye-tracking system using a freeform prism and gaze-detection light

• US 10,228,561 B2
• Filed: 06/25/2013
• Issued: 03/12/2019
• Est. Priority Date: 06/25/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Robbins, Steve, McEldowney, Scott, Lou, Xinye, Nister, David, Steedly, Drew, Miller, Quentin Simon Charles, Bohn, David D, Terrell, Jr., James Peele, Goris, Andrew C., Ackerman, Nathan
• Primary Examiner(s): Suh, Joseph
• Application Number:
  US13/927,025
• Publication Number:
  US 20140375790A1
• Time in Patent Office: 2,086 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 5/04 Prisms

Abstract

An example see-through head-mounted display system includes a freeform prism and a display device configured to emit display light through the freeform prism to an eye of a user. The see-through head-mounted display system may also include an imaging device configured to receive gaze-detection light reflected from the eye and directed through the freeform prism.

Claims

1. A see-through head-mounted display system comprising:

an illumination prism including a display-facing side, an eye-facing side, a camera-facing side between the display-facing side and the eye-facing side, and an interior beam splitter;

a freeform prism;

a display device configured to emit display light toward the display-facing side of the illumination prism and through the freeform prism to an eye of a user, the illumination prism positioned to receive the display light from the display device at the display-facing side and to transmit the display light from the display device through the beam splitter and the eye-facing side to the freeform prism;

an imaging device configured to receive gaze-detection light reflected from the eye and directed through the freeform prism and the illumination prism, the illumination prism positioned to receive, at the eye-facing side, the gaze-detection light as reflected from the eye and to reflect the gaze-detection light at the beam splitter through the camera-facing side to the imaging device, the illumination prism positioned optically intermediate the display device and the freeform prism, optically intermediate the imaging device and the freeform prism, and optically separated from the freeform prism; and

a plurality of glint sources positioned on different sides of the imaging device and configured to emit the gaze-detection light toward the camera-facing side of the illumination prism, and wherein the beam splitter is positioned to reflect the gaze-detection light from the plurality of glint sources toward the eye-facing side of the illumination prism and the freeform prism.

2. The see-through head-mounted display system of claim 1, further comprising a frame configured to position the freeform prism at a viewing location relative to the eye.

3. The see-through head-mounted display system of claim 1, wherein the display device comprises an illumination light source and a liquid crystal on silicon (LCoS) micro-display, the illumination light source configured to direct illumination light toward an illumination-facing side of the illumination prism and through the illumination prism to the LCoS, the illumination prism positioned to reflect the illumination light at the beam splitter through the display-facing side to the LCoS.

4. The see-through head-mounted display system of claim 1, wherein the display device comprises an illumination light source and a microdisplay, the illumination light source configured to direct illumination light through an illumination-facing side of the illumination prism to the microdisplay, the illumination-facing side being between the display-facing side and the eye-facing side, and opposite of the camera-facing side.

5. The see-through head-mounted display system of claim 1, further comprising a user-facing compensator and an outward-facing compensator, the freeform prism positioned between the user-facing compensator and the outward-facing compensator, and an eye-side surface of the user-facing compensator being substantially parallel to an environment-side surface of the outward-facing compensator.

6. The see-through head-mounted display system of claim 5, wherein the user-facing compensator and the outward-facing compensator each have a substantially similar index of refraction as the freeform prism.

7. The see-through head-mounted display system of claim 1, wherein the freeform prism comprises at least three surfaces, each surface being non-planar and non-spherical.

8. The see-through head-mounted display system of claim 7, wherein the at least three surfaces includes a user-facing total internal reflective surface.

9. The see-through head-mounted display system of claim 8, wherein the at least three surfaces further includes an outward-facing surface opposite the user-facing total internal reflective surface, the outward-facing surface having a coating that is more reflective for infrared light than for visible light, and where the coating is partially reflective for visible light.

10. The see-through head-mounted display system of claim 1, wherein an angle between a visual axis of the eye and an optical axis of the imaging device is less than 90 degrees.

11. A see-through head-mounted display system comprising:

a freeform prism having positive optical power;

a display device configured to emit display light through the freeform prism to an eye of a user;

a plurality of glint sources configured to emit gaze-detection light through the freeform prism to the eye of the user;

an imaging device positioned in between the plurality of glint sources and configured to receive the gaze-detection light reflected from the eye and directed through the freeform prism; and

an illumination prism including a beam splitter, the illumination prism positioned to;

receive the display light from the display device at a first side and pass the display light to the freeform prism via transmission of the display light through the beam splitter,

receive the emitted gaze-detection light from the plurality of glint sources at a second side and pass the emitted gaze-detection light to the freeform prism via reflection of the gaze-detection light by the beam splitter,

receive, at a third side of the illumination prism, return gaze-detection light comprising the emitted gaze-detection light as reflected from the eye and directed through the freeform prism, and

reflect, at the beam splitter, the return gaze-detection light through the second side to the imaging device, the second side of the illumination prism being positioned in between and adjacent to each of the first side of the illumination prism and the third side of the illumination prism, the first side of the illumination prism being opposite of the third side of the illumination prism.

12. The see-through head-mounted display system of claim 11, wherein the plurality of glint sources comprises at least one infrared light source.

13. The see-through head-mounted display system of claim 12, wherein the beam splitter is configured to transmit visible light and reflect the gaze-detection light from the plurality of glint sources.

14. The see-through head-mounted display system of claim 13, wherein the plurality of glint sources and the imaging device are each positioned adjacent the second side of the illumination prism, the freeform prism is positioned adjacent the third side of the illumination prism, and the display device.

15. A method of performing gaze-detection with a see-through head-mounted display system, the method comprising:

emitting display light from a display device through a first side of an illumination prism including a beam splitter positioned to pass the display light to a freeform prism having positive optical power and then from the freeform prism to an eye of a user, the display device being illuminated via illumination light received at a fourth side of the illumination prism and directed to the display device, the fourth side being adjacent to the first side;

emitting gaze-detection light from a plurality of glint sources positioned on different sides of an image sensor toward a second side of the illumination prism, wherein the beam splitter is positioned to reflect the gaze-detection light from the plurality of glint sources toward the third side of the illumination prism and into the freeform prism, and where the freeform prism is positioned to direct the gaze-detection light to the eye;

receiving the gaze-detection light reflected from the eye and directed back through the freeform prism to the third side of the illumination prism, and reflected by the beam splitter through the second side of the illumination prism to an image sensor, the second side being in between and adjacent to each of the first side of the illumination prism and the third side of the illumination prism, and the second side being opposite of the fourth side of the illumination prism; and

determining gaze direction from the gaze-detection light reflected from the eye and directed to the image sensor.

16. The method of claim 15, further comprising storing the gaze direction and information associated with the gaze-detection light for use in future gaze direction determinations.