Sensory integrated data interface

• US 5,751,260 A
• Filed: 04/03/1995
• Issued: 05/12/1998
• Est. Priority Date: 01/10/1992
• Status: Expired due to Term

• Current Assignee: the united states of america as represented by the secretary of the navy
• Original Assignee: the united states of america as represented by the secretary of the navy
• Inventors: Nappi, Bruce, Miller, James E., Schroeder, William
• Primary Examiner(s): BRIER, JEFFERY A
• Application Number:
  US08/416,591
• Time in Patent Office: 1,135 Days
• Field of Search: 340/705-707, 340/709, 340/825.19, 351/210, 381/43, 358/103, 358/104, 382/2, 434/112, 345/8, 345/156-158, 345/169
• US Class Current: 345/8
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

A system for interfacing between a user and a computer includes a microvideo display monitor, which may be mounted in a position which is substantially fixed relative to a given eye of the user, and a video input device which obtains an image of the given eye. The video input device is maintained in a fixed spatial relationship with the microvideo display monitor. From the image of the user'"'"'s eye obtained by the video input device, the location on the display on which this eye is focused may be determined. A computer which controls the generation of selected displays on the microvideo display monitor responds to the eye focus location to perform predetermined functions. A voice or speech recognition unit may also be used to provide inputs to the computer. The computer may then perform predetermined functions in response to any combination of speech, and eye location data. The system may also be used for identification of the user and to determine whether the user is impaired.

Claims

1. A hands free, sensory integrated data interface for interacting with a computer, said interface comprising:

a headset to be worn by a user;

means for interacting with said user attached to said headset, said interacting means communicating with said computer;

said interacting means comprising a microvisual display monitor mounted to said headset and positionable in front of a user'"'"'s eye;

said microvisual display monitor receiving information to be displayed from said computer;

said interacting means further comprising means for tracking the position of said user'"'"'s eye and for identifying that portion of said displayed information upon which said user'"'"'s eye is focused and means for transmitting commands to said computer based upon that portion of said displayed information upon which said user'"'"'s eye is focused;

said interacting means further comprising first means attached to said headset for receiving an oral command from said user and for transmitting said oral command to said computer via said transmitting means;

said computer being programmed to perform at least one function in response to said oral command;

an earphone attached to said headset; and

said computer having means for generating synthesized voice prompts for said user and for transmitting said voice prompts to said earphone via said transmitting means.

2. The hands free interface of claim 1 wherein:

said tracking means comprises an eye tracker which includes means for obtaining an image of the user'"'"'s eye and means for determining a position on the microvideo display monitor where the user'"'"'s eye is focused; and

said image obtaining means being arranged in a fixed spatial relationship with said microvisual display monitor, whereby said user'"'"'s eye may be used to control the position of a cursor displayed on said monitor.

3. The hands free interface of claim 2 wherein said determining means comprises means for determining a duration of time during which the user'"'"'s eye is directed at said position on said microvideo display monitor and means for generating said commands if said duration exceeds a predetermined threshold.

4. The hands free interface of claim 1 wherein said computer has a second means for receiving said oral command and means for automatically identifying said user from said received oral command.

5. The hands free interface of claim 2 wherein said computer includes means for automatically identifying said user from said image of said user'"'"'s eye.

6. The hands free interface of claim 1 wherein said computer includes means for automatically determining if the user is impaired.

7. The hands free interface of claim 1 wherein said computer is at a location remote from the user and said transmitting means includes a transceiver for bilateral communication with the computer.

8. The hands free interface of claim 1 further comprising:

a housing mounted to said headset;

said housing enclosing said microvisual display monitor, said eye position tracking means and said first means for receiving said oral command; and

said housing maintaining a fixed spatial relationship between the microvisual display monitor and the eye position tracking means.

9. The hands free interface of claim 8 wherein said eye position tracking means comprises an infrared illuminator for illuminating the user'"'"'s eye, a video chip for sending signals to said transmitting means, and a lens for focusing reflections from the user'"'"'s eye into the video chip.

10. The hands free interface of claim 8 wherein said housing supports a dichroic mirror which allows the user to look directly at the microvisual display monitor without interference from the eye position tracking means.

11. The hands free interface of claim 10 wherein said housing supports a lens between said dichroic mirror and said microvisual display monitor in order to enhance the picture obtained by said monitor.

12. The hands free interface of claim 1 further comprising means for calibrating and placing said microvisual display monitor and said eye position tracking means into a fixed spatial relationship by calibrating the eye position tracking means with an X-Y coordinate system of said display monitor and for maintaining said display monitor and said eye position tracking means in said fixed spatial relationship.

13. The hands free interface of claim 12 wherein said calibrating and placing means comprises means for displaying at least two points on said display monitor and for instructing the user to look at said points.

14. The hands free interface of claim 1 wherein said microvisual display monitor displays said information as a single image.