Head mounted display with eyetracking capability

• US 6,433,760 B1
• Filed: 12/23/1999
• Issued: 08/13/2002
• Est. Priority Date: 01/14/1999
• Status: Expired due to Term

• Current Assignee: University of Central Florida Research Foundation Inc. (State University System of Florida)
• Original Assignee: University of Central Florida (State University System of Florida)
• Inventors: Vaissie, Laurent, Rolland, Jannick
• Primary Examiner(s): Liang, Regina
• Application Number:
  US09/470,778
• Time in Patent Office: 964 Days
• Field of Search: 345/7, 345/8, 345/9, 359/13, 359/630, 359/631, 250/221, 340/980
• US Class Current: 345/8
• CPC Class Codes: G01S 17/875 for determining attitude
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 2027/0198 System for aligning or maintaining alignment of an image in a predetermined direction
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

The eye-tracking system is based on the reflection of four light emitting diodes (LED)s at the cornea of user'"'"'s eve. The LEDs emit infrared light at 900 nanometers and the virtual images formed behind the cornea as well as a near infrared image of the pupil are displayed on a charged couple device (CCD) sensor. The optical system used to display virtual environments is also used to conjugate the virtual images of the LEDs to the CCD sensor. This optimizes the integration of eyetracking system into the head mounted device (HMD). The four LEDs are laid out around the imaging (optical) system and their beam (rays) impinge directly on the eye by reflection on the hot mirror. Then the light reflected by the cornea is reflected again by the hot mirror, goes through the optical system and the cold mirror to be imaged on the sensor. The whole eye is illuminated by near infrared light and the contrast between the dark pupil and the bright iris on the CCD sensor allows knowledge of the location of the center of the entrance pupil. The processing of the location of the cornea images and the center of the entrance pupil leads to determine the sight direction.

Claims

1. A head mounted optical eye displaying and tracking apparatus for a user comprising in combination:

(c) optical system means for displaying virtual images to a user'"'"'s eye;

(d) means for near infrared tracking at least one of the user'"'"'s eyes to determine sight direction, the tracking means including LEDs(light emitting diodes) positioned adjacent and uniformally about a perimeter of a lens;

the LEDs for beaming light to reflect off a cornea of the user'"'"'s eye to determine sight direction of the user'"'"'s eye; and

(c) means for mounting the optical system and the tracking system on a user'"'"'s head, the mounting means including optical components common to both the optical system and the tracking system for operating along a common optical path, the optical components having only a beam splitter positioned in front of the user'"'"'s eye to be used for the displaying of the virtual image in front of the user'"'"'s eye and for the near infrared tracking of the user'"'"'s eye, and the optical components including the lens being common to both the optical system and the tracking means with the lens not being positioned in front of the user'"'"'s eye.

2. The apparatus of claim 1, wherein said optical system means includes:

means to create a virtual image.

3. The apparatus of claim 1, wherein said optical system means includes:

means to form a virtual image to the user'"'"'s eyes that appears virtually in front of the user'"'"'s eyes.

4. The apparatus of claim 1, further comprising:

means to detect reflected infrared light from the user'"'"'s eyes to determine location of gaze direction of the eyes.

5. The apparatus of claim 4, further comprising:

means to focus the reflected light to the tracking sensor.

6. The apparatus of claim 1, wherein the LEDs include:

four LEDs positioned adjacent to and uniformly about the perimeter of the lens.

7. The apparatus of claim 1, wherein the common beam splitter for the optical system means and the tracking means includes:

a common mirror having the property of reflecting approximately 50% of the visible light and approximately 100% of the near infrared light.

8. The apparatus of claim 1, wherein the optical system means and the tracking means include:

a common cold mirror which reflects visible light and transmits an infrared beam.

9. The apparatus of claim 1, wherein the common beamsplitter of the optical system means and tracking means includes:

a common wavelength dependent beamsplitter.

10. The apparatus of claim 1, wherein the common optical components of the optical system means and the tracking means include:

a common cold mirror, the common lens and the common beamsplitter.

11. A method for eye-tracking a viewer'"'"'s eye response to a virtual image with common optical components in a head mounted display comprising the steps of:

displaying a virtual image from a display means, to appear in front of the viewer'"'"'s eyes;

beaming infrared radiation beams from an eye-tracking means to the viewer'"'"'s eyes;

reflecting light from the beams off the corneas of the viewer'"'"'s eyes;

tracking the viewer'"'"'s eyes responsive to said virtual image and said reflecting light, thereby detecting sight directional change by means invisible to the viewer; and

using common optical components for the displaying of the virtual image and the beaming of the infrared radiation and the tracking of the viewer'"'"'s eyes along a common optical path, the common optical components include a common beamsplitter, wherein only the common beamsplitter is positioned in front of the viewer'"'"'s eyes.

12. The method according to claim 11, wherein the beaming step further includes:

beaming infrared radiation from a plurality of LED'"'"'s.

13. The method according to claim 12, wherein the beaming step includes:

beaming near infrared radiation from four LEDs positioned equidistant from about a common lens to each of the viewer'"'"'s eyes.

14. The method of claim 11, wherein the common beam splitter reflects approximately 50% of the visible light and reflects approximately 100% of the near infrared light.

15. The method of claim 11, wherein using the common optical components further includes the step of:

using a common mirror.

16. The method of claim 15, wherein the common mirror includes:

using a cold mirror that reflects visible light and transmits infrared light.

17. The method of claim 11, wherein the step of using the common optical components further includes the step of:

using a common mirror and a common lens.