Arrangement, method and computer program for controlling a computer apparatus based on eye-tracking

• US 10,203,758 B2
• Filed: 08/06/2013
• Issued: 02/12/2019
• Est. Priority Date: 06/18/2004
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Bjorklund, Christoffer, Eskilsson, Henrik, Jacobson, Magnus, Skogo, Marten
• Primary Examiner(s): Long, Andrea N
• Application Number:
  US13/960,530
• Publication Number:
  US 20130326431A1
• Time in Patent Office: 2,016 Days
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A computer apparatus is associated with a graphical display presenting at least one GUI-component adapted to be manipulated based on user-generated commands. An event engine is adapted to receive an eye-tracking data signal that describes a user'"'"'s point of regard on the display. Based on the signal, the event engine produces a set of non-cursor controlling event output signals, which influence the at least one GUI-component. Each non-cursor controlling event output signal describes a particular aspect of the user'"'"'s ocular activity in respect of the display. Initially, the event engine receives a control signal request from each of the at least one GUI-component. The control signal request defines a sub-set of the set of non-cursor controlling event output signals which is required by the particular GUI-component. The event engine delivers non-cursor controlling event output signals to the at least one GUI-component in accordance with each respective control signal request.

Claims

1. An arrangement comprising a hardware implementation or a non-transitory computer-readable medium storing executable instructions for controlling a computer apparatus associated with a graphical display, the graphical display displaying a representation of a plurality of graphical user interface (GUI)-components, the arrangement comprising:

a plurality of applications, wherein each application has a GUI-component presentable on the display and uses a different subset of non-cursor controlling event signals in association with controlling the GUI-component; and

an event engine configured to;

receive an eye-tracking data signal describing a user'"'"'s point of regard on the display;

receive an auxiliary input signal;

produce a set of non-cursor controlling event signals based on the eye-tracking data signal and the auxiliary input signal; and

upon receiving a request from an application of the plurality of applications having a GUI-component, respond with a subset of the non-cursor controlling event signals used by the application, wherein the subset causes a manipulation of the GUI-component based on the subset of the non-cursor controlling event signals,

wherein the manipulation comprises generating one or more pop-up windows in response to the subset of non-cursor controlling event signals indicating an estimated state-of-mind parameter reflecting a degree to which the user is at least one of focused, concentrated, distracted, tired, unfocused or confused.

2. The arrangement according to claim 1, wherein the manipulation of the representation includes changing a number of and contents of the GUI-components, wherein the number of and contents of the GUI-components is altered in response to the estimated state-of-mind of the user.

3. The arrangement according to claim 1, wherein items on the display that have not been observed by the user are highlighted.

4. A method for controlling a computer apparatus associated with a graphical display, the graphical display displaying a representation of a plurality of graphical user interface (GUI)-components, the method comprising:

receiving, by an event engine, an auxiliary input signal and an eye-tracking data signal that describes a user'"'"'s point of regard on the display;

producing, by the event engine, a set of non-cursor controlling event signals based on the eye-tracking data signal and the auxiliary input signal; and

upon receiving a request from an application of a plurality of applications of the computer apparatus, responding, by the event engine, with a subset of the non-cursor controlling event signals used by the application, wherein the subset causes a manipulation of a GUI-component of the application based on the subset of the non-cursor controlling event signals,

wherein the manipulation comprises at least generating one or more pop-up windows in response to the subset of non-cursor controlling event signals indicating an estimated state-of-mind parameter reflecting a degree to which the user is at least one of focused, concentrated, distracted, tired, unfocused or confused.

5. The method according to claim 4, wherein the manipulation of the representation includes changing a number of and contents of the GUI-components, and wherein the number of and contents of the GUI-components is altered in response to the estimated state-of-mind of the user.

6. The method according to claim 4, further comprising highlighting items on the display that have not been observed by the user.

7. A non-transitory computer readable medium containing a computer program product for controlling a computer apparatus associated with a graphical display, the graphical display displaying a representation of a plurality of graphical user interface (GUI)-components, the computer program product comprising computer-executable instructions for:

receiving, by an event engine, an auxiliary input signal and an eye-tracking data signal that describes a user'"'"'s point of regard on the display;

producing, by the event engine, a set of non-cursor controlling event signals based on the eye-tracking data signal and the auxiliary input; and

upon receiving a request from an application of a plurality of applications of the computer apparatus, responding, by the event engine, with a subset of the non-cursor controlling event signals used by the application, wherein the subset causes a manipulation of a GUI-component of the application based on the subset of the non-cursor controlling event signals,

wherein the manipulation comprises at least generating one or more pop-up windows in response to the subset of non-cursor controlling event signals indicating an estimated state-of-mind parameter reflecting a degree to which the user is at least one of focused, concentrated, distracted, tired, unfocused or confused.

8. The non-transitory computer readable medium according to claim 7, wherein the computer program product further comprises computer-executable instructions for altering a number of and contents of the GUI-components in response to the estimated state-of-mind of the user.

9. The non-transitory computer readable medium according to claim 7, wherein the computer program product further comprises computer-executable instructions for highlighting items on the display that have not been observed by the user.