VISUAL FEEDBACK FOR SPEECH RECOGNITION SYSTEM

• US 20140207452A1
• Filed: 01/24/2013
• Published: 07/24/2014
• Est. Priority Date: 01/24/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Klein, Christian, Niman, Meg
• Granted Patent: US 9,721,587 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 21/10 Transforming into visible information

Abstract

Embodiments are disclosed that relate to providing visual feedback in a speech recognition system. For example, one disclosed embodiment provides a method including displaying a graphical feedback indicator having a variable appearance dependent upon a state of the speech recognition system. The method further comprises receiving a speech input, modifying an appearance of the graphical feedback indicator in a first manner if the speech input is heard and understood by the system, and modifying the appearance of the graphical feedback indicator in a different manner than the first manner if the speech input is heard and not understood.

Claims

1. On a computing device, a method of providing user feedback for a speech recognition system, the method comprising:

displaying a graphical feedback indicator having an appearance that varies based upon a state of the speech recognition system;

receiving a speech input; and

modifying an appearance of the graphical feedback indicator in a first manner if the speech input is heard and understood by the system, and modifying the appearance of the graphical feedback indicator in a different manner than the first manner if the speech input is heard and not understood.

2. The method of claim 1, further comprising determining a volume of the speech input and modifying the appearance of the graphical feedback indicator to indicate the volume.

3. The method of claim 2, wherein modifying the appearance of the graphical feedback indicator to indicate the volume comprises modifying a length of a volume indicator on the graphical feedback indicator in response to a change in volume.

4. The method of claim 1, further comprising determining a location of a user providing the speech input, and modifying the appearance of the graphical feedback indicator to indicate the location of the user.

5. The method of claim 3, wherein modifying the appearance of the graphical feedback indicator to indicate the location of the user comprises adjusting a location of a volume indicator in a direction corresponding to the location of the user.

6. The method of claim 1, further comprising modifying the appearance of the graphical feedback indicator during a continuous speech recognition mode to display one or more words of a speech input in real-time as each word is recognized.

7. The method of claim 1, further comprising displaying the graphical feedback indicator with a first appearance when the speech recognition system is operating in a local application scope, and with a second appearance, different from the first appearance, when the speech recognition system is operating in a global operating system scope.

8. The method of claim 1, further comprising identifying a user providing the speech input, and modifying the appearance of the graphical feedback indicator to indicate an identity of the user.

9. The method of claim 1, wherein modifying the appearance of the graphical feedback indicator in a different manner than the first manner comprises displaying a prompt for a user to provide additional user input.

10. A method of providing feedback for a speech recognition system of a computing device, the method comprising:

displaying a graphical feedback indicator with a first appearance indicating that the speech recognition system is in an active listening mode, the graphical feedback indicator having a variable appearance depending upon a state of the speech recognition system;

receiving a speech input;

if the speech input is heard, modifying the first appearance of the graphical feedback indicator to indicate a detected volume of the speech input; and

if the speech input is understood, modifying the first appearance of the graphical feedback indicator to indicate that the speech input is understood.

11. The method of claim 10, further comprising displaying the graphical feedback indicator with a second appearance when the state of the speech recognition system changes to a passive listening mode, the second appearance indicating that the speech recognition system is listening for one or more trigger phrases to change to an active listening mode.

12. The method of claim 10, wherein the first appearance indicates that the speech recognition system is in an active listening mode in a local context corresponding to a an application running on the computing device.

13. The method of claim 10, wherein the first appearance indicates that the speech recognition system is in an active listening mode in a global context.

14. The method of claim 10, further comprising modifying the appearance of the graphical feedback indicator during a continuous speech recognition mode to indicate one or more words of a speech input as the speech input is recognized.

15. The method of claim 10, further comprising modifying the appearance of the graphical feedback indicator to identify one or more words corresponding to a canonical form of a speech command corresponding to the speech input.

16. The method of claim 10, further comprising modifying the appearance of the graphical feedback indicator to indicate an uncertainty in a recognition of a speech input.

17. The method of claim 16, wherein indicating an uncertainty in recognition of a speech input includes displaying two or more possible speech recognition results for the speech input.

18. A computing system for performing speech recognition and providing feedback regarding the speech recognition, the computing system comprising:

a logic machine; and

a storage machine comprising instructions executable by the logic machine to;

output to a display device a graphical feedback indicator with a first appearance indicating that the speech recognition system is in an active listening mode, the graphical feedback indicator having a variable appearance depending upon a state of the speech recognition system;

receive, from one or more microphones, a speech input;

if the speech input is heard, modify the first appearance of the graphical feedback indicator to indicate a detected volume of the speech input in a location corresponding to a detected location of a source of the speech input; and

if the speech input is understood, modify the first appearance of the graphical feedback indicator to indicate that the speech input is understood.

19. The computing system of claim 18, the instructions being further executable to determining a location of a user providing the speech input, and modifying the appearance of the graphical feedback indicator to indicate the location of the user.

20. The computing system of claim 18, the instructions further executable to determine an identity of a user, and to modify the first appearance of the graphical feedback indicator to display the identity of the user.