Sustained Eye Gaze for Determining Intent to Interact

• US 20130304479A1
• Filed: 05/08/2012
• Published: 11/14/2013
• Est. Priority Date: 05/08/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Teller, Eric, Aminzade, Daniel
• Granted Patent: US 9,423,870 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Methods and systems for determining intent in voice and gesture interfaces are described. An example method includes determining that a gaze direction is in a direction of a gaze target, and determining whether a predetermined time period has elapsed while the gaze direction is in the direction of the gaze target. The method may also include providing an indication that the predetermined time period has elapsed when the predetermined time period has elapsed. According to the method, a voice or gesture command that is received after the predetermined time period has elapsed may be determined to be an input for a computing system. Additional example systems and methods are described herein.

Claims

1. A method comprising:

determining that a gaze direction is in a direction of a gaze target;

determining whether a predetermined time period has elapsed while the gaze direction is in the direction of the gaze target;

providing an indication that the predetermined time period has elapsed when the predetermined time period has elapsed; and

determining a voice command that is received after the predetermined time period has elapsed is an input for a computing system.

2. The method of claim 1, further comprising:

determining a direction to a source of the voice command; and

determining the voice command is an input for the computing system when the direction to the source of the voice command matches a direction to an origin of the gaze direction.

3. The method of claim 2, further comprising:

determining the voice command is not an input for the computing system when the direction to the source of the voice command matches a direction to a position other than the origin of the gaze direction.

4. The method of claim 1, further comprising:

determining the voice command is an input for the computing system when the voice command is received within a second predetermined time period of when the indication is provided.

5. The method of claim 1, wherein the indication is a visual indication that is provided at a location of the gaze target.

6. The method of claim 1, wherein the gaze target is a location of a gaze tracking device configured to determine the gaze direction.

7. The method of claim 1, wherein the gaze target is a location that is separate from a location of a gaze tracking device configured to determine the gaze direction.

8. The method of claim 1, wherein the indication is an audible indication.

9. The method of claim 1, further comprising providing a first indication when the gaze direction is in the direction of the gaze target.

10. The method of claim 1, wherein determining that a gaze direction is in a direction of a gaze target comprises:

determining an expected angle from an origin of the gaze direction to a location of the gaze target; and

determining that the gaze direction is in the direction of the gaze target based on a comparison of the expected angle and the gaze direction.

11. The method of claim 10, further comprising:

determining that the gaze direction is within a threshold of the expected angle; and

providing feedback indicating a relationship between the gaze direction and the expected angle, wherein the feedback includes information associated with a change in the gaze direction to align the gaze direction with the expected angle.

12. A non-transitory computer-readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

determining that a gaze direction is in a direction of a gaze target;

determining whether a predetermined time period has elapsed while the gaze direction is in the direction of the gaze target;

providing an indication that the predetermined time period has elapsed when the predetermined time period has elapsed; and

determining a voice command that is received after the predetermined time period has elapsed is an input for a computing system.

13. The non-transitory computer-readable medium of claim 12, further comprising instructions executable by the computing device to perform functions comprising:

determining a direction to a source of the voice command; and

determining the voice command is an input for the computing system when the direction to the source of the voice command matches a direction to an origin of the gaze direction.

14. The non-transitory computer-readable medium of claim 12, wherein the indication is a visual indication that is provided at a location of the gaze target.

15. The non-transitory computer-readable medium of claim 12, further comprising instructions executable by the computing device to perform functions comprising providing a first indication when the gaze direction is in the direction of the gaze target.

16. A system comprising:

a gaze tracking component configured to determine a gaze direction;

a voice input component configured to receive a voice command;

an indicator component; and

instructions stored in a memory and executable by a processor to;

determine that the gaze direction is in a direction of a gaze target;

determine whether a predetermined time period has elapsed while the gaze direction is in the direction of the gaze target;

cause the indicator component to provide an indication that the predetermined time period has elapsed when the predetermined time period has elapsed; and

determine a voice command that is received after the predetermined time period has elapsed is an input for the system.

17. The system of claim 16, further comprising:

an audio localization component configured to determine a direction to a source of the voice command; and

wherein the instructions further comprises instructions executable by the processor to;

determine the received voice command is an input for the system when the voice command is received after the predetermined time period has elapsed and the direction to the source of the voice command matches a direction to an origin of the gaze direction.

18. The system of claim 17, wherein the audio localization component comprises a microphone array.

19. The system of claim 16, wherein the indicator component comprises one or more light-emitting diodes coupled to the gaze target.

20. The system of claim 16, further comprising instructions stored in the memory and executable by the processor to cause the indicator component to provide a first indication that the gaze direction is in a direction of the gaze target when the gaze direction is in the direction of the gaze target.

21. A method comprising:

determining that a gaze direction is in a direction of a gaze target;

determining whether a predetermined time period has elapsed while the gaze direction is in the direction of the gaze target;

providing an indication that the predetermined time period has elapsed when the predetermined time period has elapsed; and

determining a gesture that is received after the predetermined time period has elapsed is an input for a computing system.

22. The method of claim 21, further comprising determining the gesture is an input for the computing system when the gesture is received within a second predetermined time period of when the indication is provided.

23. The method of claim 21, further comprising providing a first indication when the gaze direction is in the direction of the gaze target.

24. The method of claim 21, wherein determining that a gaze direction is in a direction of a gaze target comprises:

determining an expected angle from an origin of the gaze direction to a location of the gaze target; and

determining that the gaze direction is in the direction of the gaze target based on a comparison of the expected angle and the gaze direction.

25. A non-transitory computer-readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

determining that a gaze direction is in a direction of a gaze target;

determining whether a predetermined time period has elapsed while the gaze direction is in the direction of the gaze target;

providing an indication that the predetermined time period has elapsed when the predetermined time period has elapsed; and

determining a gesture that is received after the predetermined time period has elapsed is an input for a computing system.

26. The non-transitory computer-readable medium of claim 25, further comprising instructions executable by the computing device to cause the computing device to perform functions comprising determining the gesture is an input for the computing system when the gesture is received within a second predetermined time period of when the indication is provided.

27. The non-transitory computer-readable medium of claim 25, further comprising instructions executable by the computing device to cause the computing device to perform functions comprising providing a first indication when the gaze direction is in the direction of the gaze target.

28. The non-transitory computer-readable medium of claim 25, wherein determining that a gaze direction is in a direction of a gaze target comprises:

determining an expected angle from an origin of the gaze direction to a location of the gaze target; and

determining that the gaze direction is in the direction of the gaze target based on a comparison of the expected angle and the gaze direction.