Arrangement, method and computer program for controlling a computer apparatus based on eye-tracking

• US 8,185,845 B2
• Filed: 05/24/2005
• Issued: 05/22/2012
• Est. Priority Date: 06/18/2004
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Bjorklund, Christoffer, Eskilsson, Henrik, Skogo, Marten, Jacobson, Magnus
• Primary Examiner(s): Queler, Adam M
• Assistant Examiner(s): LONG, ANDREA NATAE
• Application Number:
  US11/570,840
• Publication Number:
  US 20070164990A1
• Time in Patent Office: 2,555 Days
• Field of Search: 715/763, 715/638, 351/209, 345/158
• US Class Current: 715/863
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A computer based eye-tracking solution is disclosed. A computer apparatus is associated with one or more graphical displays (GUI components) that may be manipulated based on user-generated commands. An event engine is adapted to receive an eye-tracking data signal that describes a user'"'"'s point of regard on the display. Based on the signal, the event engine is adapted to produce a set of non-cursor controlling event output signals, which influence the GUI-components. Each non-cursor controlling event output signal describes a particular aspect of the user'"'"'s ocular activity in respect of the display. Initially, the proposed event engine receives a control signal request from each of the GUI-components. The control signal request defines a sub-set of the set of non-cursor controlling event output signals which is required by the particular GUI-component. The event engine delivers non-cursor controlling event output signals to the GUI-components in accordance with each respective control signal request.

Claims

1. An arrangement for controlling a computer apparatus associated with a graphical display, the display presenting a representation of a plurality of GUI-components that are configured to be manipulated based on user-generated commands, the arrangement comprising;

a display;

an event engine configured to receive an eye-tracking data signal describing a user'"'"'s point of regard on the display, and at least based on the eye-tracking data signal, configured to produce a set of noncursor controlling event output signals influencing the GUI-components, each of the noncursor controlling event output signals describing a different aspect of the user'"'"'s ocular activity in respect of the display, wherein the event engine is configured to;

receive a respective control signal request from each of the GUI-components, each respective control signal request defining a sub-set of the set of non-cursor controlling event output signals which is required by a corresponding one of the GUI-components to operate as intended,

produce the set of non-cursor controlling event output signals which is actually requested by the respective control signal request from each of the GUI-components, and

deliver the set of non-cursor controlling event output signals to the GUI-components in accordance with each respective control signal request.

2. The arrangement according to claim 1, wherein the computer apparatus is configured to receive a cursor control signal, and control a graphical pointer on the display in response to the cursor control signal.

3. The arrangement according to claim 1, wherein at least one of the GUI-components is configured to generate at least one respective output control signal upon a user manipulation of the component.

4. The arrangement according to claim 1, wherein the event engine is configured to produce at least a first signal of the non-cursor controlling event output signals based on a dynamic development of the eye-tracking data signal.

5. The arrangement according to claim 4, wherein the first signal represents a particular gaze pattern over the display.

6. The arrangement according to claim 5, wherein at least one of the GUI-components is configured to interpret the first signal as an estimated intention of the user, and trigger a user manipulation of the component in response to the estimated intention.

7. The arrangement according to claim 5, wherein at least one of the GUI-components is configured to interpret the first signal as an estimated attention level of the user, and trigger a user manipulation of the component in response to the estimated attention level.

8. The arrangement according to claim 5, wherein at least one of the GUI-components is configured to interpret the first signal as a state-of-mind parameter of the user, and trigger a user manipulation of the component in response to the state-of-mind parameter.

9. The arrangement according to claim 1, wherein the event engine is configured to receive at least one auxiliary input signal and produce the set of non-cursor controlling event output signals on the further basis of the at least one auxiliary input signal.

10. The arrangement according to claim 1, wherein the at least one auxiliary input signal is originated from at least one of a button, a switch, a speech signal, a movement pattern of an input member, a gesture pattern, a facial expression and an EEG-signal.

11. A method of controlling a computer apparatus associated with a graphical display, the display representing a plurality of GUI-components that are configured to be manipulated based on user-generated commands, the method comprising:

receiving at an event engine an eye-tracking data signal which describes a user'"'"'s point of regard on the display,

receiving at the event engine a respective control signal request from each of the GUI-components, the respective control signal request defining a sub-set of a set of noncursor controlling event output signals, the sub-set of the set of noncursor controlling event output signals being required by a corresponding one of the GUI-components to operate as intended,

producing the set of non-cursor controlling event output signals at least based on the eye-tracking data signal, the set of noncursor controlling event output signals influencing the GUI-components, each of the non-cursor controlling event output signals describing a different aspect of the user'"'"'s ocular activity in respect of the display, the set of non-cursor controlling event output signals being actually requested by the respective control signal request from each of the GUI-components, and

delivering from the event engine the set of non-cursor controlling event output signals to the GUI-components in accordance with each respective control signal request.

12. The method according to claim 11, further comprising receiving a cursor control signal, and controlling a graphical pointer on the display in response to the cursor control signal.

13. The method according to claim 11, further comprising generating at least one output control signal by at least one of the GUI-components upon a user manipulation of the at least one of the GUI-components.

14. The method according to claim 11, further comprising producing at least a first signal of the non-cursor controlling event output signals based on a dynamic development of the eye-tracking data signal.

15. The method according to claim 14, wherein the first signal represents a particular gaze pattern over the display.

16. The method according to claim 15, further comprising interpreting the first signal by at least one of the GUI-components as an estimated intention of the user, and triggering a user manipulation of the at least one of the GUI-components in response to the estimated intention.

17. The method according to claim 15, further comprising interpreting the first signal by at least one of the GUI-components as an estimated attention level of the user, and triggering a user manipulation of the at least one of the GUI-components in response to the estimated attention level.

18. The method according to claim 15, further comprising interpreting the first signal by at least one of the GUI-components as a state-of-mind parameter of the users, and triggering a user manipulation of the at least one of the GUI-components in response to the state-of-mind parameter.

19. The method according to claim 11, further comprising:

receiving at least one auxiliary input signal, and producing the set of non-cursor controlling event output signals on the further basis of the at least one auxiliary input signal.

20. The method according to claim 11, wherein the at least one auxiliary input signal being originated from at least one of a button, a switch, a speech signal, a movement pattern of an input member, a gesture pattern, a facial expression and an EEG-signal.

21. A computer program embodied into the internal memory of a computer, comprising software for executing the method of claim 11 when said program is run on the computer.

22. A non-transitory computer readable medium, having a program recorded thereon for controlling a computer apparatus associated with a graphical display, the display representing a plurality of GUI-components that are to be manipulated based on user-generated commands, where the program operates as an event engine where the program includes computer readable program code portions stored therein, the computer readable program code portions comprising:

a first program code portion configured to receive an eye-tracking data signal which describes a user'"'"'s point of regard on the display;

a second program code portion configured to receive a respective control signal request from each of the GUI-components, the respective control signal request defining a sub-set of a set of noncursor controlling event output signals, the sub-set of the set of noncursor controlling event output signals being required by a corresponding one of the GUI-components to operate as intended;

a third program code portion configured to produce the set of non-cursor controlling event output signals at least based on the eye-tracking data signal, the set of noncursor controlling event output signals influencing the GUI-components, each of the non-cursor controlling event output signals describing a different aspect of the user'"'"'s ocular activity in respect of the display, the set of non-cursor controlling event output signals being actually requested by the respective control signal request from each of the GUI-components; and

a fourth program code portion configured to deliver the set of non-cursor controlling event output signals to the GUI-components in accordance with each respective control signal request.