Gaze-Assisted Computer Interface

• US 20120272179A1
• Filed: 04/21/2011
• Published: 10/25/2012
• Est. Priority Date: 04/21/2011
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Stafford, Jeffrey Roger
• Granted Patent: US 8,793,620 B2
• Time in Patent Office: Days
• US Class Current: 715/781
• CPC Class Codes: G06F 2203/04801 Cursor retrieval aid, i.e. visual aspect modification, blinking, colour changes, enlargement or other visual cues, for helping user do find the cursor in graphical user interfaces
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G09G 2320/0693 Calibration of display systems
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/08 Cursor circuits
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Methods, systems, and computer programs for interfacing a user with a Graphical User Interface (GUI) are provided. One method includes an operation for identifying the point of gaze (POG) of the user. The initiation of a physical action by the user, to move a position of a cursor on a display, is detected, where the cursor defines a focus area associated with a computer program executing the GUI. Further, the method includes an operation for determining if the distance between the current position of the cursor and the POG is greater than a threshold distance. The cursor is moved from the current position to a region proximate to the POG in response to the determination of the POG and to the detection of the initiation of the physical action.

Claims

1. A method for interfacing a user with a computer program executing in a computing device, the method comprising:

identifying a point of gaze (POG) of the user;

detecting initiation of a physical action by the user to move a position of a cursor on a display, the cursor defining a focus area associated with the computer program;

determining if a distance between a current position of the cursor and the POG is greater than a threshold distance; and

moving the cursor from the current position to a region proximate to the POG in response to the determining and the detecting.

2. The method as recited in claim 1, wherein the physical action is one of moving a mouse, pressing a key, pressing a button, touching a screen, touching a touchpad, making a gesture, or speaking.

3. The method as recited in claim 1, wherein determining the distance further includes:

determining a value of the distance measured in screen pixels between the current position and the POG.

4. The method as recited in claim 1, wherein determining the distance further includes:

determining if the current position and the POG are situated in different windows in the GUI.

5. The method as recited in claim 1, wherein the cursor is one of a mouse cursor, a text cursor, a field selected cursor, or a page cursor.

6. The method as recited in claim 1, wherein moving the cursor further includes:

removing the cursor from the current position and placing the cursor on the POG.

7. The method as recited in claim 1, wherein moving the cursor further includes one or more of:

placing the cursor on a window where the POG is located;

placing the cursor on a display where the POG is located;

placing the cursor on a GUI object proximate to the POG;

or

placing the cursor on an entry field proximate to the POG.

8. The method as recited in claim 1, further including:

changing a sensitivity of a mouse that controls the cursor based on the distance between the cursor and the POG.

9. The method as recited in claim 1, further including:

tracking a position of a head of the user.

10. The method as recited in claim 1, further including:

detecting an input from the user that triggers immediate gaze-assisted navigation; and

moving the cursor to the POG after detecting the input.

11. A system with a graphical user interface (GUI), the system comprising:

a processor executing a computer program that provides the GUI, the GUI being rendered in one or more displays;

an input device operable to move a position of a cursor on the one or more displays when a physical action is initiated by the user on the input device; and

a gaze detection module operable to identify a point of gaze (POG) of the user, wherein the computer program determines if a distance between a current position of the cursor and the POG is greater than a threshold distance, and wherein the computer program moves the cursor from the current position to a region proximate to the POG in response to the determination of the distance and detection of the physical action.

12. The system as recited in claim 11, wherein the gaze detection module uses one of analysis of corneal reflection, or analysis of an user image taken with a camera.

13. The system as recited in claim 12, wherein the gaze detection module analyzes a plurality of user images to determine a location of one of more physical features of the user.

14. The system as recited in claim 11, wherein the GUI provides a hint to the user when the POG is placed on an object in the GUI.

15. The system as recited in claim 11, further including:

a voice recognition module operable to execute a voice command based on the POG.

16. A non-transitory computer readable medium having program instructions that when executed by a computer implement a method for interfacing a user with a computer program, said method comprising:

identifying a point of gaze (POG) of the user;

detecting initiation of a physical action by the user to move a position of a cursor on a display, the cursor defining a focus area associated with the computer program;

determining if a distance between a current position of the cursor and the POG is greater than a threshold distance; and

moving the cursor from the current position to a region proximate to the POG in response to the determining and the detecting.

17. The non-transitory computer readable medium as recited in claim 16, said method further including one or more of:

moving a window of the GUI;

pressing a button of the GUI;

opening the window of the GUI;

closing the window of the GUI;

opening a menu of the GUI;

selection an option from the menu;

or

hiding windows of the GUI other than the window where the POG is located.

18. The non-transitory computer readable medium as recited in claim 16, wherein the computer is a game console, wherein the computer program is a computer game, and wherein the computer game determines a reaction of the computer game when the user presses a button on a controller, the reaction being based on the POG.

19. The non-transitory computer readable medium as recited in claim 16, said method further including:

detecting that the user is reading; and

scrolling content of a window when the POG is near an edge of the window.

20. The non-transitory computer readable medium as recited in claim 16, said method further including:

determining that the POG is outside the display;

detecting initiation of another physical action by the user; and

scrolling content of a window based on the POG.