System and method for controlling host system interface with point-of-interest data

• US 6,243,076 B1
• Filed: 09/01/1998
• Issued: 06/05/2001
• Est. Priority Date: 09/01/1998
• Status: Expired due to Term

• Current Assignee: Synthetic Environments, Inc., Tobii Technology AB
• Original Assignee: Synthetic Environments, Inc.
• Inventors: Hatfield, Franz
• Primary Examiner(s): CHOW, DOON Y
• Application Number:
  US09/145,159
• Time in Patent Office: 1,008 Days
• Field of Search: 345/156, 345/157, 345/158, 345/173, 345/121, 345/123, 345/126, 345/333, 345/334, 345/347, 345/348, 345/473, 345/474, 345/977, 345/7
• US Class Current: 345/156
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: Y10S 715/977 Dynamic icon, e.g. animated or live action

Abstract

A system and method are disclosed for a user interface control that utilizes one or more graphical objects (“comets”), which can be operated solely by the eye gaze of a user. The disclosed system and method enables users to access and control, for example, computer displays without use of hands, voice or other forms of input. Each comet may be implemented as an animated graphic object that is mapped to one or more computer functions (a command set) by a computer or software application programmer. A computer-based system may create one or more comets on a display device and place them in motion. Eye tracking data may be collected over a period of time based on the detected point-of-gaze of a user'"'"'s eye. If the user is determined to be visually tacking a particular comet, the comet will become selected and the computer or host system may execute the command set that has been mapped to the selected comet. The disclosed system and method may also be adapted to determine a user'"'"'s point-of-interest based on the movement of an input device (such as a conventional mouse, trackball, data glove, etc.) and to select a comet based on a correspondence between the movement of the comet and the movement of the input device.

Claims

1. An eye gaze interaction system for controlling an interface of a host system, said eye gaze interaction system comprising:

a display screen that displays at least one graphical object, each said graphical object being displayed in motion on said display screen, each said graphical object being associated with at least one predetermined action to be performed by said host system;

an eye tracking system that detects eye point-of-gaze data of a user'"'"'s eye over a predetermined time period;

means for detecting a correspondence between a direction of movement of said graphical object on said display screen and a direction of movement of the user'"'"'s eye based on said eye point-of-gaze data detected by said eye tracking system; and

means for causing said host system to perform said predetermined action when said detecting means detects a correspondence between the direction of movement of said graphical object and the direction of movement of the user'"'"'s eye.

2. An eye gaze interaction system according to claim 1, wherein said detecting means comprises means for determining a distance between the direction of movement of said graphical object on said display screen and the direction of movement of the user'"'"'s eye, said detecting means detecting that a correspondence exists between the direction of movement of said graphical object and the direction of movement of the user'"'"'s eye when the distance determined by said determining means is within a predetermined tolerance ε

.

3. An eye gaze interaction system according to claim 1, wherein said graphical object is displayed in motion on said display screen so as to follow a predefined path on said display screen.

4. An eye gaze interaction system according to claim 1, further comprising means for removing each said graphical object from said display screen if a correspondence between the direction of movement of said graphical object and the direction of movement of the user'"'"'s eye is not detected by said detecting means.

5. An eye gaze interaction system according to claim 1, wherein a plurality of graphical objects are displayed in motion on said display screen, each of said plurality of graphical objects being associated with a predetermined action to be performed by said host system.

6. An eye gaze interaction system according to claim 5, wherein said detecting means detects whether a correspondence exists between a direction of movement of any one of said plurality of graphical objects on said display screen and a direction of movement of the user'"'"'s eye based on said eye point-of-gaze data detected by said eye tracking system, such that when said detecting means detects that a direction of movement of a selected one of said plurality of graphical objects corresponds to the direction of movement of the user'"'"'s eye, said causing means causes said host system to perform a predetermined action associated with said selected one of said plurality of graphical objects.

7. A method for eye gaze control for selecting a graphical object displayed in motion on a display screen, said method comprising:

displaying at least one graphical object in motion on said display screen;

collecting eye point-of-gaze data of a user'"'"'s eye over a predetermined time period;

detecting a correspondence between a direction of movement of said graphical object on said display screen and a direction of movement of the user'"'"'s eye based on said eye point-of-gaze data; and

selecting said graphical object on said display screen when a correspondence is detected between the direction of movement of said graphical object and the direction of movement of the user'"'"'s eye.

8. A method for eye gaze control according to claim 7, wherein each said graphical object is associated with a predetermined action to be performed by a host system, said method further comprising causing said host system to perform said predetermined action when said graphical object is selected based on the detected correspondence between the direction of movement of said graphical object and the direction of movement of the user'"'"'s eye.

9. A method for eye gaze control according to claim 7, wherein said method further comprises determining a distance between the direction of movement of said graphical object on said display screen and the direction of movement of the user'"'"'s eye, and detecting that a correspondence exists between the direction of movement of said graphical object and the direction of movement of the user'"'"'s eye when the distance is within a predetermined tolerance ε

.

10. A method for eye gaze control according to claim 7, wherein said method further comprises displaying said graphical object on said display screen so as to follow a predefined path.

11. A method for eye gaze control according to claim 7, said method further comprising removing said graphical object from said display screen if a correspondence between the direction of movement of said graphical object and the direction of movement of the user'"'"'s eye is not detected.

12. A method for eye gaze control according to claim 7, said method further comprising displaying a plurality of graphical objects in motion on said display screen, each of said plurality of graphical objects being associated with a predetermined action to be performed by a host system.

13. A method for eye gaze control according to claim 12, said method further comprising detecting whether a correspondence exists between a direction of movement of any one of said plurality of graphical objects on said display screen and a direction of movement of the user'"'"'s eye based on said eye point-of-gaze data, and selecting the first of said plurality of graphical objects that is detected as having a direction of movement that corresponds to the movement of the user'"'"'s eye.

14. A computer program embodied on computer readable medium for eye gaze interaction and control of a host system, comprising:

first computer readable program code means for causing a display screen of said host system to display at least one graphical object, each said graphical object being displayed in motion on said display screen, each said graphical object being associated with a predetermined action to be performed by said host system;

second computer readable program code means for collecting, from an eye tracking system, eye direction data of a user'"'"'s eye over a predetermined time period;

third computer readable program code means for detecting a correspondence between a direction of movement of said graphical object on said display screen and a direction of movement of the user'"'"'s eye based on said eye direction data; and

fourth computer readable program code means for causing said host system to perform said predetermined action when a correspondence is detected between the direction of movement of said graphical object and the direction of movement of the user'"'"'s eye.

15. A computer program embodied on a computer readable medium according to claim 14, wherein said third computer readable program code means comprises a source code segment that determines a distance between the direction of movement of said graphical object on said display screen and the direction of movement of the user'"'"'s eye, said third computer readable program code means detecting that a correspondence exists between the direction of movement of said graphical object and the direction of movement of the user'"'"'s eye when the distance is determined to be within a predetermined tolerance ε

.

16. A computer program embodied on a computer readable medium according to claim 14, wherein said first computer readable program code means causes said graphical object to be displayed in motion on said display screen so as to follow a predefined path on said display screen.

17. A computer program embodied on a computer readable medium according to claim 14, further comprising fifth computer readable program code means for causing said graphical object to be removed from said display screen if a correspondence between the direction of movement of said graphical object and the direction of movement of the user'"'"'s eye is not detected.

18. A computer program embodied on a computer readable medium according to claim 14, wherein said first computer readable program code means causes a plurality of graphical objects to be displayed in motion on said display screen, each of said plurality of graphical objects being associated with a predetermined action to be performed by said host system.

19. A computer program embodied on a computer readable medium according to claim 18, wherein said third computer readable program code means detects whether a correspondence exists between a direction of movement of any one of said plurality of graphical objects on said display screen and a direction of movement of the user'"'"'s eye based on said eye direction data, such that when a direction of movement of a selected one of said plurality of graphical objects is detected to correspond to the direction of movement of the users eye, said fourth computer readable program code means causes said host system to perform a predetermined action associated with said selected one of said plurality of graphical objects.

20. A computer program embodied on a computer readable medium according to claim 14, wherein said eye direction data collected by said second computer readable program code means from said eye tracking system comprises eye point-of-gaze data.

21. A method for selecting a graphical object displayed in motion on a display screen, said method comprising:

displaying at least one graphical object in motion on said display screen;

collecting point-of-interest data from a movable input device controlled by a user over a predetermined time period;

detecting a correspondence between a direction of movement of said graphical object on said display screen and a direction of movement of said input device based on said point-of-interest data; and

selecting said graphical object on said display screen when a correspondence is detected between the direction of movement of said graphical object and the direction of movement of said input device.

22. A method for selecting a graphical object according to claim 21, wherein each said graphical object is associated with a predetermined action to be performed by a host system, said method further comprising causing said host system to perform said predetermined action when said graphical object is selected based on the detected correspondence between the direction of movement of said graphical object and the direction of movement of said input device.

23. A method for selecting a graphical object according to claim 21, wherein said method further comprises determining a distance between the direction of movement of said graphical object on said display screen and the direction of movement of said input device, and detecting that a correspondence exists between the direction of movement of said graphical object and the direction of movement of said input device when the distance is within a predetermined tolerance ε

.

24. A method for selecting a graphical object according to claim 21, said method further comprising removing said graphical object from said display screen if a correspondence between the direction of movement of said graphical object and the direction of movement of said input device is not detected.