Assisting user selection of graphical user interface elements

• US 6,323,884 B1
• Filed: 06/21/1999
• Issued: 11/27/2001
• Est. Priority Date: 03/31/1999
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bird, Colin Leonard, Chapman, Sydney George
• Primary Examiner(s): Jankus, Almis R.
• Assistant Examiner(s): CUNNINGHAM, GREGORY F
• Application Number:
  US09/337,605
• Time in Patent Office: 890 Days
• Field of Search: 345/145, 345/146, 345/358, 345/810, 345/856, 345/863
• US Class Current: 715/810
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

Provided are a method and a software-implemented mechanism for assisting a user of a graphical user interface (GUI) with the task of positioning an input device pointer (or otherwise indicating a position selection) when selecting a GUI element such as a button or other icon. The software identifies those GUI elements which are valid selections for the current application state and so are potentially the next element that the user will interact with. Then a predefined set of characteristics for the identified GUI elements are compared to predict which of these GUI elements will be the next GUI element that the user will interact with. The result of this prediction is indicated by moving the pointer to the position within the GUI of the predicted GUI element or by adding emphasis to the predicted GUI element. The user then decides whether this GUI element is required.

Claims

1. A computer program product comprising computer readable program code stored on a computer readable recording medium for assisting a user of computing apparatus in the task of selection of a GUI element within a computer program by controlling the operation of a computing apparatus to execute processes:

to identify those GUI elements within a currently displayed GUI which correspond to valid user interactions for the current state of the computer program which includes the GUI, and hence are potentially the next element that the user will interact with;

to evaluate a predefined set of characteristics for the identified GUI elements to predict which of these GUI elements will be the next element that the user will interact with, wherein the set of characteristics include;

distance measurements between a user indicated position within the GUI and the position of each GUI element; and

weightings which are associated with individual GUI elements and which indicate a likelihood of selection of the respective individual GUI elements for said current program state; and

to visually indicate the result of the prediction.

2. A computer program product according to claim 1, wherein the evaluation of characteristics to predict a next GUI element is initiated when a user indicated position within the GUI is within a predefined distance of one of the identified GUI elements.

3. A computer program product according to claim 1, wherein the user indication of a position within the GUI comprises focusing of the eye, as measured by an eye tracking mechanism within the computing apparatus.

4. A computer program product according to claim 1, wherein the user indication of a position within the GUI comprises user-controlled positioning of an input device pointer.

5. A computer program product according to claim 1, wherein the predefined set of characteristics include a parameter identifying whether the identified GUI elements are within a user indicated region of the GUI, the user indication of a region comprising an initial user-controlled movement of a selection pointer and the direction of movement being calculated by the program code.

6. A computer program product according to claim 1, wherein the computer program controls the computing apparatus to execute processes to record statistics of user interaction sequences and the weightings are generated from the statistics of previous user interaction sequences.

7. A computer program product according to claim 1, wherein visually indicating the result of the prediction includes moving an input device pointer to overlay the predicted GUI element.

8. A computer program product according to claim 7, wherein the movement of the input device pointer is a progressive movement to the position of the predicted GUI element from the position of the input device pointer when the prediction was initiated.

9. A computer program product according to claim 1, wherein visually indicating the result of the prediction includes adding emphasis to the predicted GUI element.

10. A computer program product according to claim 1, wherein the step of visually indicating the result of the prediction is only performed if the prediction step predicts interaction with a particular GUI element with at least a predefined minimum probability.

11. A computer program product according to claim 1, wherein the computer program is triggered to perform its processes by opening of a new window within the GUI, the identification of potential next GUI elements then being limited to GUI elements within the new window.

12. A computer program product comprising computer readable program code, recorded on a computer readable recording medium, the program code including a service component for providing run-time services to application programs to assist a user of computing apparatus with the task of selection of a GUI element within an application program by controlling the operation of a computing apparatus to execute processes

to identify those GUI elements within a currently displayed GUI which correspond to valid user interactions for the current state of the application program which includes the GUI, and hence are potentially the next element that the user will interact with;

to evaluate a predefined set of characteristics for the identified GUI elements to predict which of these GUI elements will be the next element that the user will interact with, wherein the set of characteristics include;

distance measurements between a user indicated position within the GUI and the position of each GUI element; and

weightings which are associated with individual GUI elements and which indicate a likelihood of selection of the respective individual GUI elements for the current application program state; and

to visually indicate the result of the prediction.

13. A method for predicting and indicating to a user of computing apparatus a prediction of a next GUI element within a computer program with which the user will interact, the method including the steps of:

identifying those GUI elements within a currently displayed GUI which correspond to valid user interactions for the current state of the computer program which includes the GUI, and hence are potentially the next GUI element that the user will interact with;

evaluating a predefined set of characteristics for the identified GUI elements to predict which of these GUI elements will be the next element that the user will interact with, wherein the set of characteristics include;

distance measurements between a user indicated position within the GUI and the position of each GUI element; and

weightings which are associated with individual GUI elements and which indicate a likelihood of selection of the respective individual GUI elements for the current program state; and

visually indicating the result of the prediction.

14. A computer program product comprising computer readable program code recorded on a computer readable recording medium, the program code including:

means for identifying GUI elements within a currently displayed GUI which correspond to valid user interactions for the current state of the computer program which includes the GUI, and hence are potentially the next GUI element that the user will interact with;

a prediction unit for evaluating a predefined set of characteristics for the identified GUI elements to predict which of the identified GUI elements will be the next element that the user will interact with, wherein the set of characteristics include;

distance measurements between a user indicated position within the GUI and the position of each GUI element; and

weightings which are associated with individual GUI elements and which indicate a likelihood of selection of the respective individual GUI elements for the current state of the computer program which includes the GUI; and

means for visually indicating the result of the prediction.

15. A method, implemented in a computing apparatus including eye tracking apparatus for identifying user interaction with a GUI of a computer program, for assisting a user with the task of selecting a GUI element, the method including the steps of:

identifying those GUI elements within a currently displayed GUI which correspond to valid user interactions for the current state of the computer program which includes the GUI, and hence are potentially the next GUI element that the user will interact with;

in response to identification by the eye tracking apparatus of a user indication of a position within the GUI, determining whether the indicated position is within a predefined distance of one of the identified GUI elements and, if within said predefined distance, evaluating a predefined set of characteristics for the identified GUI elements to predict which of these GUI elements will be the next element that the user will interact with, wherein the set of characteristics include;

distance measurements between a user indicated position within the GUI and the position of each identified GUI element; and

weightings which are associated with individual GUI elements and which indicate a likelihood of selection of the respective individual GUI elements for said current program state; and

visually indicating the result of the prediction including adding emphasis to the predicted GUI element.

16. An interactive data processing apparatus including a GUI and eye tracking apparatus for identifying user interaction with the GUI, the data processing apparatus including computer program code for assisting a user with the task of selecting a GUI element, the program code including:

means for identifying GUI elements within a currently displayed GUI which correspond to valid user interactions for the current state of the computer program which includes the GUI, and hence are potentially the next GUI element that the user will interact with;

a prediction unit for evaluating a predefined set of characteristics for the identified GUI elements to predict which of the identified GUI elements will be the next element that the user will interact with, wherein the set of characteristics include;

distance measurements between a user indicated position within the GUI and the position of each identified GUI element; and

weightings which are associated with individual GUI elements and which indicate a likelihood of selection of the respective individual GUI elements for said current program state; and

means for visually indicating the result of the prediction.

17. A data processing apparatus including means for implementing a method for predicting and indicating to a user of computing apparatus a prediction of a next GUI element within a computer program with which the user will interact, the apparatus including:

means for identifying those GUI elements within a currently displayed GUI which correspond to valid user interactions for the current state of the computer program which includes the GUI, and hence are potentially the next GUI element that the user will interact with;

means for evaluating a predefined set of characteristics for the identified GUI elements to predict which of these GUI elements will be the next element that the user will interact with, wherein the set of characteristics include;

distance measurements between a user indicated position within the GUI and the position of each GUI element; and

weightings which are associated with individual GUI elements and which indicate a likelihood of selection of the respective individual GUI elements for the current program state; and

means for visually indicating the result of the prediction.