Gaze-assisted computer interface

• US 9,971,401 B2
• Filed: 07/28/2014
• Issued: 05/15/2018
• Est. Priority Date: 04/21/2011
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Stafford, Jeffrey Roger
• Primary Examiner(s): Xia, Xuyang
• Application Number:
  US14/444,946
• Publication Number:
  US 20140333535A1
• Time in Patent Office: 1,387 Days
• CPC Class Codes: G06F 2203/04801 Cursor retrieval aid, i.e. visual aspect modification, blinking, colour changes, enlargement or other visual cues, for helping user do find the cursor in graphical user interfaces
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G09G 2320/0693 Calibration of display systems
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/08 Cursor circuits
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Methods, systems, and computer programs for interfacing a user with a Graphical User Interface (GUI) are provided. One method includes operations for identifying a point of gaze (POG) of a user, and for detecting initiation of an action by the user to move a position of a selector for selecting objects presented on a graphical user interface (GUI). In addition, the method includes an operation for determining the distance between the current position of the selector and the POG. The displacement speed of the selector is adjusted based on the distance between the current position of the selector and the POG, where the displacement speed is reduced as the distance between the current position of the selector and the POG becomes smaller.

Claims

1. A computer-implemented method comprising:

identifying a point of gaze (POG) of a user on an object in a graphical user interface of a computer;

detecting initiation of an action by the user using an input device to move a position of a selector for selecting the object presented on the graphical user interface while the POG of the user is on the object;

moving the selector by the user using the input device to the object if a current position of the selector is determined to be within a defined area around the POG of the user; and

if the current position of the selector is determined to be outside of the defined area around the POG of the user, moving the selector, by the computer, in a direction that is toward the POG and placing the selector on the POG of the user, wherein a displacement speed of moving the selector is adjusted based on a distance between the current position of the selector and the POG of the user, wherein the displacement speed is reduced as the distance between the current position of the selector and the POG of the user becomes smaller.

2. The method as recited in claim 1, wherein the distance between the current position of the selector and the POG is measured as one or more of a number of pixels, an angle, and a physical distance.

3. The method as recited in claim 1, wherein the selector is a text entry selector indicating where text is entered within a text field or document.

4. The method as recited in claim 1, wherein the selector is a mouse cursor, wherein the action by the user is a motion of a mouse.

5. The method as recited in claim 1, wherein the selector is an avatar in a game, wherein the avatar moves within a virtual world.

6. The method as recited in claim 1, wherein the action is one of moving a mouse, pressing a key, pressing a button, touching a screen, touching a touchpad, making a gesture, or speaking.

7. The method as recited in claim 1, further including:

detecting a first object selected with the selector; and

manipulating the object in response to a user input.

8. The method as recited in claim 7, wherein the selector is an avatar in a game, wherein manipulating the object includes picking up the object by the avatar.

9. The method as recited in claim 1, wherein adjusting the displacement speed further includes:

determining if the distance between the current position of the selector and the POG is greater than a threshold distance; and

adjusting the displacement speed when the distance between the current position of the selector and the POG is greater than the threshold distance.

10. The method as recited in claim 1, wherein adjusting the displacement speed further includes:

determining if the POG and the selector are in a same window of graphical user interface; and

adjusting the displacement speed when the POG and the selector are not in the same window.

11. A computer-implemented method comprising:

detecting a plurality of entry fields in a web page by a user on a graphical user interface of a computer;

identifying a point of gaze (POG) of the user on an area that contains one or more entry fields from the plurality of entry fields in the web page;

detecting an input by the user using an input device for moving a selector to select one of the one or more entry fields from the plurality of entry fields for data entry while the POG of the user is on the one or more entry fields;

moving the selector by the user using the input device if a current position of the selector is determined to be within a defined area around the POG of the user; and

if the current position of the selector is determined to be outside of the defined area around the POG of the user, placing the selector, by the computer, on an entry field from the one or more entry fields of the plurality of entry fields that is closest to the POG of the user when the user initiates the input, wherein the placing of the selector involves a movement of the selector from the current position to the entry field from the one or more entry fields of the plurality of entry fields at a speed of placing the selector that is based on a determined distance between the current position of the selector and the entry field from the one or more entry fields of the plurality of entry fields that is closest to the POG of the user, the speed being reduced as the distance between the current position of the selector and the entry field from the one or more entry fields of the plurality of entry fields becomes smaller.

12. The method as recited in claim 11, wherein the input is a keyboard input, wherein the selector is placed in the entry filed that is closest to the POG when the keyboard input is received.

13. The method as recited in claim 11, wherein the determined distance is measured as one or more of a number of pixels, an angle, and a physical distance.

14. A non-transitory computer-readable storage medium storing a computer program, the non-transitory computer-readable storage medium comprising:

identifying a point of gaze (POG) of a user on an object in a graphical user interface of a computer;

detecting initiation of an action by the user using an input device to move a position of a selector for selecting the object presented on the graphical user interface while the POG of the user is on the object;

moving the selector by the user using the input device to the object if a current position of the selector is determined to be within a defined area around the POG of the user; and

if the current position of the selector is determined to be outside of the defined area around the POG of the user,

moving the selector, by the computer, in a direction that is toward the POG and placing the selector on the POG of the user, wherein a displacement speed of moving the selector is adjusted based on a distance between the current position of the selector and the POG of the user, wherein the displacement speed is reduced as the distance between the current position of the selector and the POG of the user becomes smaller.

15. The non-transitory computer-readable storage medium as recited in claim 14, further including:

determining a first window on the user interface where the POG is detected.

16. The non-transitory computer-readable storage medium as recited in claim 15, further including:

providing a transparent appearance to windows of the graphical user interface other than the first window.

17. The non-transitory computer-readable storage medium as recited in claim 16, further including:

making the first window as having a window focus of the graphical user interface, wherein the window focus enables user interaction with a program associated with the window focus while disabling interaction with windows not having the window focus.

18. The non-transitory computer-readable storage medium as recited in claim 14, further including:

rearranging, on the graphical user interface, windows that are open based on the POG.

19. The non-transitory computer-readable storage medium as recited in claim 14, further including:

magnifying, after moving the selector, an area around the selector.

20. The non-transitory computer-readable storage medium as recited in claim 14, wherein the distance between the current position of the selector and the POG is measured as one or more of a number of pixels, an angle, and a physical distance.