System and method for selecting and activating a target object using a combination of eye gaze and key presses

• US 20050243054A1
• Filed: 04/29/2004
• Published: 11/03/2005
• Est. Priority Date: 08/25/2003
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Beymer, David, Zhai, Shumin, Farrell, Stephen P.
• Granted Patent: US 9,274,598 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: A61B 3/0033 characterised by user input arrangements
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves

Abstract

A pointing method uses eye-gaze as an implicit control channel. A user looks in a natural fashion at a target object as a button on a graphical user interface and then presses a manual selection key. Once the selection key is pressed, the present method uses probability analysis to determine a most probable target object and to highlight it. If the highlighted object is the target object, the user can select it by pressing a key such as the selection key once again. If the highlighted object is not the target object, the user can select another target object using additional keys to navigate to the intended target object.

Claims

1. A pointing method for selecting a target object from a plurality of objects, comprising:

displaying the plurality of objects on a display;

tracking a position of a natural look of a user while viewing the display;

concurrently monitoring a press of a manual selection key;

selecting the target object on the display based on a coincidence of the press of the manual selection key and a sensed user'"'"'s natural look to a region proximate to the target object;

if needed, changing a selected object using a navigation key; and

activating the selected target object with an activation key.

2. The method of claim 1, wherein the display comprises any of a computer monitor or a television display.

3. The method of claim 1, further comprising using a probability analysis to determine a most probable target object among the plurality of objects.

4. The method of claim 3, wherein the probability analysis comprises a history of a graphical user interface object.

5. The method of claim 1, wherein the target object is a graphical user interface object.

6. The method of claim 1, wherein selecting the target object comprises capturing the position of the natural look upon a user pressing the manual selection key.

7. The method of claim 1, wherein selecting the target object comprises highlighting the target object.

8. The method of claim 7, wherein highlighting the target object is implemented by means of a bounding box.

9. The method of claim 7, wherein highlighting the target object is implemented by changing a color of the target object.

10. The method of claim 7, wherein highlighting the target object is implemented by placing a cursor adjacent to the target object.

11. The method of claim 7, wherein highlighting the target object is implemented by pointing an arrow to the target object.

12. The method of claim 1, wherein selecting the target object comprises selecting a candidate object and then manually adjusting a selection from the candidate object to the target object.

13. The method of claim 12, further comprising a user pressing a navigation key to move the indicator toward another probable target object.

14. The method of claim 13, further comprising highlighting said another probable target upon selection by the user.

15. The method of claim 1, wherein the manual selection key and the activation key are the same key.

16. The method of claim 1, wherein selecting the target object is implemented by means of a combined press of two keys.

17. The method of claim 16, wherein the two keys are a control key and a tab key.

18. The method of claim 17, wherein changing the selected object comprises navigation through a plurality of objects using the tab key.

19. The method of claim 18, wherein activating the target object comprises releasing the control key.

20. The method of claim 1, further comprising determining whether the most probable target object comprises a text field.

21. The method of claim 19, further comprising continuing with text entry if the position of the natural look is at or near a caret in the text field;

and

moving the caret to the position of the natural look if the position of the natural look is not near the caret in the text field.

22. The method of claim 1, wherein the manual selection key is a dedicated key on a keyboard.

23. The method of claim 1, wherein the manual selection key is a user-defined key on a keyboard.

24. The method of claim 1, wherein the manual selection key is any key on a keyboard.

25. The method of claim 1, wherein the activation key is a dedicated key on a keyboard.

26. The method of claim 1, wherein the activation key is a user-defined key on a keyboard.

27. The method of claim 1, wherein the activation key is any key on a keyboard.

28. A pointing apparatus for selecting a target object from a plurality of objects, comprising:

a display for displaying the plurality of objects;

a gaze tracking module that tracks a position of a natural look of a user while viewing the display;

a user input module that concurrently monitors a press of a manual selection key;

the user input module further selects the target object on the display based on a coincidence of the press of the manual selection key and a sensed user'"'"'s natural look to a region proximate to the target object;

a navigation key for changing a selected object, if needed; and

an activation key for activating the selected target object.

29. The pointing apparatus of claim 28, wherein the display comprises any of a computer monitor or a television display.

30. The pointing apparatus of claim 28, wherein the user input module uses a probability analysis to determine a most probable target object among the plurality of objects.

31. The pointing apparatus of claim 30, wherein the probability analysis comprises a history of a graphical user interface object.

32. The pointing apparatus of claim 28, wherein the target object is a graphical user interface object.

33. The pointing apparatus of claim 28, wherein the user input module selects the target object by capturing a position of the natural look upon the user pressing the manual selection key.

34. A computer program product having a set of instruction codes for selecting a target object from a plurality of objects, comprising:

a first set of instruction codes for tracking a position of a natural look of a user while viewing the plurality of objects;

a second set of instruction codes for concurrently monitoring a press of a manual selection key;

a third set of instruction codes for selecting the target object based on a coincidence of the press of the manual selection key and a sensed user'"'"'s natural look to a region proximate to the target object; and

an activation key for activating the selected target object.

35. The computer program product of claim 34, wherein the third set of instruction codes selects the target object by capturing a position of the natural look upon the user pressing the manual selection key.

36. The computer program product of claim 34, wherein the third set of instruction codes uses a probability analysis to determine a most probable target object among the plurality of objects.

37. The computer program product of claim 36, wherein the probability analysis comprises a history of a graphical user interface object.

38. The computer program product of claim 34, wherein the target object is a graphical user interface object.

39. The computer program produce of claim 34, further comprising a fourth set of instruction codes for changing a selected object, if needed.