Visual indication of a recognized voice-initiated action

• US 9,430,186 B2
• Filed: 04/01/2014
• Issued: 08/30/2016
• Est. Priority Date: 03/17/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Faaborg, Alexander, Sonoda, Gustavo, Kaplan, Joshua Robin
• Primary Examiner(s): Tan, Alvin
• Assistant Examiner(s): Yi, Rinna
• Application Number:
  US14/242,427
• Publication Number:
  US 20150261496A1
• Time in Patent Office: 882 Days
• Field of Search: 715/706, 715/709, 715/727, 715/728, 715/861, 715/762, 715/763, 715/771, 704/231, 704/233, 704/235, 704/260, 704/275, 704/246, 704/270, 704/270.1, 704/E11.011
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00

Abstract

A computing device is described that outputs, for display, an initial speech recognition graphical user interface (GUI) having at least one element. The computing device receives audio data and determines, based on the audio data, a voice-initiated action. Responsive to determining the voice-initiated action, the computing device outputs, for display, an updated speech recognition GUI having an animation of a change in a position of the at least one element to indicate that the voice-initiated action has been determined.

Claims

1. A method comprising:

while receiving initial audio data indicating an initial portion of voice input, outputting, by a computing device and for display, an initial speech recognition graphical user interface (GUI) including at least one element, wherein the at least one element is output for display in a first visual format to indicate the computing device is receiving the initial audio data;

generating, by the computing device, based at least in part on the initial audio data, a transcription the initial audio data; and

while receiving additional audio data indicating a second portion of the voice input;

determining, by the computing device and based at least in part on a comparison of at least one a word from the transcription to a preconfigured set of actions, a voice-initiated action associated with the initial portion of the voice input; and

responsive to determining the voice-initiated action and prior to executing the voice-initiated action;

outputting, by the computing device and for display, a first updated speech recognition GUI including an animation of a change in a position of the at least one element from the initial speech recognition GUI, wherein the animation of the change in the position indicates that the voice-initiated action has been determined based on the initial audio data;

after outputting the first update speech recognition GUI, outputting, by the computing device and for display, a second updated speech recognition GUI including the at least one element from the initial speech recognition GUI, the at least one element from the initial speech recognition GUI being displayed in a second visual format, different from the first visual format, to further indicate that the voice-initiated action has been determined from the initial audio data; and

executing the voice-initiated action based on the initial audio data and the additional audio data.

2. The method of claim 1, wherein:

outputting the initial speech recognition GUI for display comprises outputting, by the computing device and for display, the at least one element at an initial location of the initial speech recognition GUI, and

outputting the first updated speech recognition GUI for display comprises;

after outputting the at least one element for display at a first location of the first updated speech recognition GUI, outputting, by the computing device, for display at a second location of the first updated speech recognition GUI, the at least one element; and

after outputting the at least one element for display at the second location of the first updated speech recognition GUI, outputting, by the computing device, for display at the initial location, the at least one element, wherein the first location is above the initial location, wherein the second location is below the first location and the initial location.

3. The method of claim 1, wherein the animation of the change in the position of the at least on element to indicate that the voice-initiated action has been determined is a first animation, the method further comprising:

determining, by the computing device and based on the initial audio data, an absence of the voice-initiated action; and

responsive to determining the absence of the voice-initiated action;

refraining from outputting, by the computing device and for display, the first updated speech recognition GUI;

and

outputting, by the computing device and for display, a third updated speech recognition GUI including a second animation of the change in the position of the at least one element from the initial speech recognition GUI to indicate that the absence of the voice-initiated action has been determined, wherein the second animation is different from the first animation.

4. The method of claim 3, wherein:

outputting the initial speech recognition GUI for display comprises outputting, by the computing device and for display, the at least one element at an initial location of the initial speech recognition GUI, and

outputting the first updated speech recognition GUI for display comprises;

after outputting the at least one element for display at a first location of the first updated speech recognition GUI, outputting, by the computing device, for display at a second location of the first updated speech recognition GUI, the at least one element; and

after outputting the at least one element for display at the second location of the first updated speech recognition GUI, outputting, by the computing device, for display at the initial location, the at least one element, wherein the first location is positioned left or right of the initial location, wherein the second location is positioned opposite the first location and left or right of the initial location.

5. The method of claim 1, wherein determining the voice-initiated action further comprises:

identifying, by the computing device, at least one verb in the transcription; and

comparing, by the computing device, the at least one verb to one or more verbs from a set of verbs, each verb in the set of verbs corresponding to at least one action from a plurality of actions including the voice-initiated action.

6. The method of claim 1, wherein determining the voice-initiated action further comprises:

determining, by the computing device and based at least in part on data from the computing device, a context; and

determining, by the computing device and based at least in part on the context, the voice-initiated action.

7. The method of claim 1, wherein outputting, for display, the first updated speech recognition GUI comprises:

ceasing outputting, by the computing device and for display, the initial speech recognition GUI; and

outputting, by the computing device and for display, the first updated speech recognition GUI.

8. A computing device comprising:

at least one processor; and

at least one module operable by the at least one processor to;

while receiving initial audio data indicating an initial portion of voice input, output, for display, an initial speech recognition graphical user interface (GUI) including at least one element, wherein the at least one element is output for display in a first visual format to indicate the computing device is receiving the initial audio data;

generate, based at least in part on the initial audio data, a transcription of the initial audio data; and

while receiving additional audio data indicating a second portion of the voice input;

determine, based at least in part on a comparison of at least one a word from the transcription to a preconfigured set of actions, a voice-initiated action associated with the initial portion of the voice input; and

responsive to determining the voice-initiated action and prior to executing the voice-initiated action;

output, for display, a first updated speech recognition GUI including an animation of a change in a position of the at least one element, from the initial speech recognition GUI, wherein the animation of the change in the position indicates that the voice-initiated action has been determined based on the initial audio data;

after outputting the first update speech recognition GUI, output, for display, a second updated speech recognition GUI including the at least one element from the initial speech recognition GUI, the at least one element from the initial speech recognition GUI being displayed in a second visual format, different from the first visual format, to further indicate that the voice-initiated action has been determined from the initial audio data; and

executing the voice-initiated action based on the initial audio data and the additional audio data.

9. The computing device of claim 8, wherein the animation of the change in the position of the at least one element includes at least one of a bounce animation, a shake animation, a fold animation, a crinkle animation, a rotation animation, a zoom animation, or a morph-in-shape animation.

10. The computing device of claim 8, wherein the second portion of the voice input includes one or more parameters of the voice-initiated action.

11. A non-transitory computer-readable storage medium comprising instructions that, when executed, configure at least one processor to:

while receiving initial audio data indicating an initial portion of voice input, output, for display, an initial speech recognition graphical user interface (GUI) including at least one element, wherein the at least one element is output for display in a first visual format to indicate the computing device is receiving the initial audio data;

generate, based at least in part on the initial audio data, a transcription of the initial audio data; and

while receiving additional audio data indicating a second portion of the voice input;

determine, based at least in part on a comparison of at least one a word from the transcription to a preconfigured set of actions, a voice-initiated action associated with the initial portion of the voice input; and

responsive to determining the voice-initiated action and prior to executing the voice-initiated action;

output, for display, a first updated speech recognition GUI including an animation of a change in a position of the at least one element, from the initial speech recognition GUI, wherein the animation of the change in the position indicates that the voice-initiated action has been determined based on the initial audio data;

after outputting the first update speech recognition GUI, outputting, by the computing device and for display, a second updated speech recognition GUI including the at least one element from the initial speech recognition GUI, the at least one element from the initial speech recognition GUI being displayed in a second visual format, different from the first visual format, to further indicate that the voice-initiated action has been determined from the initial audio data; and

executing the voice-initiated action based on initial audio data and the additional audio data.

12. The non-transitory computer-readable storage medium of claim 11, wherein:

the first visual format of the at least one element from the initial speech recognition GUI comprises an image representative of a speech recognition mode of the computing device, and

the second visual format of the at least one element from the initial speech recognition GUI comprises an image representative of the voice-initiated action.

13. The non-transitory computer-readable storage medium of claim 11, wherein the animation of the change in the position of the at least one element comprises a morph animation of the image representative of the speech recognition mode changing into the image representative of the voice-initiated action.

14. The non-transitory computer-readable storage medium of claim 11, wherein the second portion of the voice input compliments the voice-initiated action and completes the voice-initiated action.