Visual indication of a recognized voice-initiated action

• US 9,990,177 B2
• Filed: 07/27/2016
• Issued: 06/05/2018
• Est. Priority Date: 03/17/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Faaborg, Alexander, Sonoda, Gustavo, Kaplan, Joshua Robin
• Primary Examiner(s): Yi, Rinna
• Application Number:
  US15/220,835
• Publication Number:
  US 20160335052A1
• Time in Patent Office: 678 Days
• Field of Search: 704231, 704233, 704235, 704246, 704260, 704270, 704275, 715706, 715709, 715727, 715728, 715763
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00

Abstract

A computing device is described that outputs, for display, an initial speech recognition graphical user interface (GUI) having at least one element. The computing device receives audio data and determines, based on the audio data, a voice-initiated action. Responsive to determining the voice-initiated action, the computing device outputs, for display, an updated speech recognition GUI having an animation of a change in a position of the at least one element to indicate that the voice-initiated action has been determined.

Claims

1. A method comprising:

while receiving an initial portion of a first voice input, outputting, by a computing device and for display, an initial speech recognition graphical user interface (GUI) including at least one element;

determining whether any voice-initiated action from a predetermined set of actions is associated with the initial portion of the first voice input;

while receiving a subsequent portion of the first voice input;

responsive to determining that no voice-initiated action from a predetermined set of actions is associated with the initial portion of the first voice input, outputting, by the computing device, for display, a first updated speech recognition GUI including a shake type animation in which the at least one element from the initial speech recognition GUI moves left and right one or more times in a horizontal direction to mimic a person shaking a head as an indication of a negative response that none of the voice-initiated actions from the predetermined set of actions has been determined based on the initial portion of the first voice input;

while receiving an initial portion of a second voice input outputting, by the computing device and for display, the initial speech recognition GUI including the at least one element; and

determining whether any voice-initiated action from the predetermined set of actions is associated with the initial portion of the second voice input; and

while receiving a subsequent portion of the second voice input;

responsive to determining that a particular voice-initiated action from the predetermined set of actions is associated with the initial portion of the second voice input;

outputting, by the computing device, for display, a second updated speech recognition GUI including a nod type animation in which the at least one element from the initial speech recognition GUI moves up and down one or more times in a vertical direction to mimic the person nodding the head as an indication of an affirmative response that the voice-initiated action has been determined based on the initial portion of the second voice input; and

after outputting the second updated speech recognition GUI, executing the voice-initiated action based on the initial portion of the second voice input and the subsequent portion of the second voice input.

2. The method of claim 1, wherein outputting the initial speech recognition GUI for display while receiving the initial portion of the second voice input includes outputting the at least one element for display in a first visual format indicating the computing device is receiving the second voice input.

3. The method of claim 2, wherein outputting the second updated speech recognition GUI for display includes outputting the at least one element for display in a second visual format, the second visual format being different than the first visual format and further indicating the affirmative response that the voice-initiated action has been determined based on the initial portion of the second voice input.

4. The method of claim 1, wherein:

outputting the initial speech recognition GUI for display comprises outputting the at least one element for display at an initial location of a display device,

outputting the first updated speech recognition GUI comprises outputting the at least one element for display at the initial location of the display device after outputting the at least one element for display at a second location of the display device, and

outputting the second updated speech recognition GUI comprises outputting the at least one element for display at the initial location of the display device after outputting the at least one element for display at the second location of the display device.

5. A computing device comprising:

at least one processor; and

at least one module operable by the at least one processor to;

while receiving an initial portion of a first voice input, output, for display, an initial speech recognition graphical user interface (GUI) including at least one element;

determine whether any voice-initiated action from a predetermined set of actions is associated with the initial portion of the first voice input;

while receiving a subsequent portion of the first voice input;

responsive to determining that no voice-initiated action from a predetermined set of actions is associated with the initial portion of the first voice input, output, for display, a first updated speech recognition GUI including a shake type animation in which the at least one element from the initial speech recognition GUI moves left and right one or more times in a horizontal direction to mimic a person shaking a head as an indication of a negative response that none of the voice-initiated actions from the predetermined set of actions has been determined based on the initial portion of the first voice input;

while receiving an initial portion of a second voice input output for display, the initial speech recognition GUI including the at least one element; and

determine whether any voice-initiated action from the predetermined set of actions is associated with the initial portion of the second voice input; and

while receiving a subsequent portion of the second voice input;

responsive to determining that a particular voice-initiated action from the predetermined set of actions is associated with the initial portion of the second voice input;

output, for display, a second updated speech recognition GUI including a nod type animation in which the at least one element from the initial speech recognition GUI moves up and down one or more times in a vertical direction to mimic the person nodding the head as an indication of an affirmative response that the voice-initiated action has been determined based on the initial portion of the second voice input; and

after outputting the second updated speech recognition GUI, execute the voice-initiated action based on the initial portion of the second voice input and the subsequent portion of the second voice input.

6. The computing device of claim 5, wherein the at least one module is further operable by the at least one processor to output the initial speech recognition GUI for display while receiving the initial portion of the second voice input by at least outputting the at least one element for display in a first visual format indicating the computing device is receiving the second voice input.

7. The computing device of claim 6, wherein the at least one module is further operable by the at least one processor to output the second updated speech recognition GUI for display by at least outputting the at least one element for display in a second visual format, the second visual format being different than the first visual format and further indicating the affirmative response that the voice-initiated action has been determined based on the initial portion of the voice second input.

8. The computing device of claim 5, wherein the at least one module is further operable by the at least one processor to:

output the initial speech recognition GUI for display by at least outputting the at least one element for display at an initial location of a display device;

output the first updated speech recognition GUI by at least outputting the at least one element for display at the initial location of the display device after outputting the at least one element for display at a second location of the display device; and

output the second updated speech recognition GUI by at least outputting the at least one element for display at the initial location of the display device after outputting the at least one element for display at the second location of the display device.

9. A non-transitory computer-readable storage medium comprising instructions that, when executed, cause at least one processor of a computing device to:

while receiving an initial portion of a first voice input, output, for display, an initial speech recognition graphical user interface (GUI) including at least one element;

determine whether any voice-initiated action from a predetermined set of actions is associated with the initial portion of the first voice input;

while receiving a subsequent portion of the first voice input;

responsive to determining that no voice-initiated action from a predetermined set of actions is associated with the initial portion of the first voice input, output, for display, a first updated speech recognition GUI including a shake type animation in which the at least one element from the initial speech recognition GUI moves left and right one or more times in a horizontal direction to mimic a person shaking a head as an indication of a negative response that none of the voice-initiated actions from the predetermined set of actions has been determined based on the initial portion of the first voice input;

while receiving an initial portion of a second voice input output for display, the initial speech recognition GUI including the at least one element; and

determine whether any voice-initiated action from the predetermined set of actions is associated with the initial portion of the second voice input; and

while receiving a subsequent portion of the second voice input;

responsive to determining that a particular voice-initiated action from the predetermined set of actions is associated with the initial portion of the second voice input;

output, for display, a second updated speech recognition GUI including a nod type animation in which the at least one element from the initial speech recognition GUI moves up and down one or more times in a vertical direction to mimic the person nodding the head as an indication of an affirmative response that the voice-initiated action has been determined based on the initial portion of the second voice input; and

after outputting the second updated speech recognition GUI, execute the voice-initiated action based on the initial portion of the second voice input and the subsequent portion of the second voice input.

10. The non-transitory computer-readable storage medium of claim 9, wherein the instructions, when executed, cause the at least one processor to output the initial speech recognition GUI for display while receiving the initial portion of the second voice input by at least outputting the at least one element for display in a first visual format indicating the computing device is receiving the second voice input.

11. The non-transitory computer-readable storage medium of claim 10, wherein the instructions, when executed, cause the at least one processor to output the second updated speech recognition GUI for display by at least outputting the at least one element for display in a second visual format, the second visual format being different than the first visual format and further indicating the affirmative response that the voice-initiated action has been determined based on the initial portion of the voice input.

12. The non-transitory computer-readable storage medium of claim 9, wherein the instructions, when executed, cause the at least one processor to:

output the initial speech recognition GUI for display by at least outputting the at least one element for display at an initial location of a display device;

output the first updated speech recognition GUI by at least outputting the at least one element for display at the initial location of the display device after outputting the at least one element for display at a second location of the display device; and

output the second updated speech recognition GUI by at least outputting the at least one element for display at the initial location of the display device after outputting the at least one element for display at the second location of the display device.