Input determination method

• US 9,939,896 B2
• Filed: 07/12/2016
• Issued: 04/10/2018
• Est. Priority Date: 05/08/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Teller, Eric, Aminzade, Daniel
• Primary Examiner(s): JACKSON, JAKIEDA R
• Application Number:
  US15/208,561
• Publication Number:
  US 20160320838A1
• Time in Patent Office: 637 Days
• Field of Search: 704275
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Methods and systems for determining intent in voice and gesture interfaces are described. An example method includes determining that a gaze direction is in a direction of a gaze target, and determining whether a predetermined time period has elapsed while the gaze direction is in the direction of the gaze target. The method may also include providing an indication that the predetermined time period has elapsed when the predetermined time period has elapsed. According to the method, a voice or gesture command that is received after the predetermined time period has elapsed may be determined to be an input for a computing system. Additional example systems and methods are described herein.

Claims

1. A method comprising:

operating a voice-controlled device with voice control of the voice-controlled device deactivated, wherein the voice-controlled device comprises a gaze tracking component, a light-emitting electronic component, a processor, and a microphone;

determining, using the gaze tracking component and while the voice control of the voice-controlled device is deactivated, that a gaze direction is in a direction of a gaze target, wherein the gaze target comprises a portion of the voice-controlled device in which the gaze tracking component and the light-emitting electronic component are positioned;

responsive to determining that the gaze direction is in the direction of the gaze target and while the voice control is deactivated, transmitting by the processor, a first signal to the light-emitting electronic component that causes the light-emitting electronic component to output a first visual indication that;

the gaze direction is in the direction of the gaze target, the voice control is deactivated, and maintaining the gaze direction in the direction of the gaze target for the predetermined time period will activate the voice control;

responsive to determining that the gaze direction is in the direction of the gaze target, determining, using the gaze tracking component of the voice-controlled device and while the voice control is deactivated, that the predetermined time period has elapsed while the gaze direction is in the direction of the gaze target; and

responsive to determining that the predetermined time period has elapsed while the gaze direction is in the direction of the gaze target;

activating the voice control of the voice-controlled device, wherein activating the voice control comprises configuring the voice-controlled device such that voice commands that are subsequently received by the microphone are interpreted as inputs for the voice-controlled device, and

transmitting, by the processor, a second signal to the light-emitting electronic component that causes the light-emitting electronic component to output a second visual indication that the voice control is activated; and

processing a voice command that is received by the microphone after activating the voice control.

2. The method of claim 1, wherein activating the voice control comprises switching the microphone from an off-state to an on-state.

3. The method of claim 1, further comprising:

determining, using audio localization, a voice direction; and

determining that the voice command is an input for the voice-controlled device based on the voice direction matching the gaze direction.

4. The method of claim 1, wherein the gaze direction corresponds to a gaze of a user, and wherein the voice-controlled device is located remotely from the user.

5. The method of claim 1, wherein determining, using the gaze tracking component, that the gaze direction is in the direction of the gaze target comprises:

determining, using a camera, that the gaze direction is in the direction of the gaze target.

6. The method of claim 5, wherein the camera is positioned on the voice-controlled device such that a user of the voice-controlled device is within a field of view of the camera when the voice-controlled device is in use, and wherein the light-emitting electronic component is positioned on the device such that the first visual indication and the second visual indication are perceivable by the user when the user is within the field of view of the camera.

7. A method comprising:

operating a voice-controlled device with voice control of the voice-controlled device deactivated, wherein the voice-controlled device comprises a processor and a microphone;

determining, using a gaze tracking component and while the voice control of the voice-controlled device is deactivated, that a gaze direction is in a direction of a gaze target;

responsive to determining that the gaze direction is in the direction of the gaze target and while the voice control is deactivated, transmitting, by the processor, to a light-emitting electronic component a first signal that causes the light-emitting electronic component to output a first visual indication that indicates that;

the gaze direction is in the direction of the gaze target, the voice control is deactivated, and maintaining the gaze direction in the direction of the gaze target for a predetermined time period will activate the voice control;

after providing the first visual indication and while the voice control is deactivated, determining, using the gaze tracking component, that the predetermined time period has elapsed while the gaze direction is in the direction of the gaze target;

responsive to determining that the predetermined time period has elapsed while the gaze direction is in the direction of the gaze target;

activating the voice control of the voice-controlled device, wherein activating the voice control comprises configuring the voice-controlled device such that voice commands that are subsequently received by the microphone are interpreted as inputs for the voice-controlled device, and

transmitting, by the processor, a second signal to the light-emitting electronic component that causes the light-emitting electronic component to output a second visual indication that indicates that the voice control is activated; and

processing a voice command that is received by the microphone after activating the voice control.

8. The method of claim 7, wherein the gaze direction corresponds to a gaze of a user, and wherein the gaze tracking component is located remotely from the user.

9. The method of claim 7, wherein the gaze tracking component is a component of the voice-controlled device.

10. The method of claim 9, wherein the gaze direction corresponds to a gaze of a user, and wherein the voice-controlled device is located remotely from the user.

11. The method of claim 7, wherein the gaze target is the gaze tracking component.

12. The method of claim 7, wherein the gaze target is a physical target that is separate from the gaze tracking component and located at a known position with respect to the gaze tracking component.

13. The method of claim 7, wherein the gaze target is a virtual target that is located at a known position with respect to the gaze tracking component.

14. The method of claim 7, further comprising continuously providing the first visual indication as feedback while the gaze direction is in the direction of the gaze target until determining that the predetermined time period has elapsed.

15. A voice-controlled device comprising:

a microphone;

one or more processors; and

instructions stored in a memory and executable by the processor to;

operate a voice-controlled device with voice control of the voice-controlled device deactivated,

determine, using a gaze tracking component and while the voice control of the voice-controlled device is deactivated, whether a gaze direction is in a direction of a gaze target,

upon determining that the gaze direction is in the direction of the gaze target and while the voice control is deactivated, transmit a first signal to a light-emitting electronic component that causes the light-emitting electronic component to output a first visual indication that;

the gaze direction is in the direction of the gaze target, the voice control is deactivated, and maintaining the gaze direction in the direction of the gaze target for a predetermined time period will activate the voice control,

upon determining that the predetermined time period has elapsed while the gaze direction is in the direction of the gaze target, (i) activate the voice control, wherein activating the voice control comprises configuring the voice-controlled device such that voice commands that are subsequently received by the microphone are interpreted as inputs for the voice-controlled device and (ii) transmit a second signal to the light-emitting electronic component that causes the light-emitting electronic component to output a second visual indication that indicates that the voice control is activated, and

processing a voice command that is received by the microphone after activating the voice control.

16. The voice-controlled device of claim 15, wherein the gaze tracking component is a component of the voice-controlled device, and wherein the gaze target comprises a portion of the voice-controlled device in which the gaze tracking component and the light-emitting electronic component are positioned.

17. The voice-controlled device of claim 15, wherein the gaze target is the gaze tracking component.