Speech recognition status feedback user interface

• US 20030220798A1
• Filed: 05/24/2002
• Published: 11/27/2003
• Est. Priority Date: 05/24/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Iwema, Marieke, Chambers, Robert L., Schmid, Philipp H., Garside, Adrian
• Granted Patent: US 7,047,200 B2
• Time in Patent Office: Days
• US Class Current: 704/276
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems

Abstract

A user interface is described that informs the user as to the status of the operation of a voice recognition application. The user interface displays an indicator, such as a volume bar, each time that the voice recognition application records and identifies a volume event. The user interface also displays an indicator when the voice recognition application recognizes a volume event corresponding to a displayed volume event indicator. The interface thus confirms to a user that the voice recognition application is both recording and recognizing the words being spoken by the user. It also graphically informs the user of the delay the application is currently experiencing in recognizing the words that the user is speaking.

Claims

1. A method of displaying the status of a voice recognition process, comprising:

each time that a voice recognition process determines a volume event, displaying a volume event occurrence indicator; and

if the volume event occurrence indicator for a volume event is still displayed when the voice recognition process recognizes the volume event, displaying a volume event recognition indicator so as to associate the volume event recognition indicator with the volume event occurrence indicator.

2. The method recited in claim 1, further comprising:

displaying only a single volume event occurrence indicator at any time until the voice recognition process recognizes that a user has begun speaking a phrase.

3. The method recited in claim 2, further comprising:

after the voice recognition process recognizes that a user has begun speaking a phrase, displaying each new volume event occurrence indicator to the right of a previous volume event occurrence indicator.

4. The method recited in claim 1, wherein the volume event occurrence indicator is a bar.

5. The method recited in claim 4, wherein a height of the bar corresponds to a volume of the volume event.

6. The method recited in claim 4, wherein a color of the bar corresponds to a volume of the volume event.

7. The method recited in claim 1, wherein the volume event recognition indicator is a square.

8. The method recited in claim 1, wherein a color of the volume event recognition indicator corresponds to a confidence level in the recognition of the volume event.

9. The method recited in claim 1, further comprising:

displaying a phrase end indicator when the voice recognition process recognizes the end of a phrase being spoken by a user.

10. The method recited in claim 9, wherein the phrase end indicator indicates whether the phrase was accurately recognized by voice recognition application.

11. A method of displaying the status of a voice recognition process, comprising:

receiving a volume event determination notification from a voice recognition application, the volume event determination notification corresponding to a volume event;

in response to receiving the volume event determination notification, displaying a volume event occurrence indicator;

receiving a volume event recognition notification from the voice recognition application, the volume event recognition notification notifying that the volume event has been recognized, and

in response to receiving the volume event recognition notification, displaying a volume event recognition indicator so as to associate the volume event recognition indicator with the volume event occurrence indicator.

12. The method recited in claim 11, wherein displaying the volume event recognition indicator includes:

determining if the volume event occurrence indicator is still being displayed, and

if the volume event occurrence indicator is still being displayed, displaying the volume event recognition indicator adjacent to the volume event occurrence indicator; and

if the volume event occurrence indicator is not still being displayed, not displaying the volume event recognition indicator.

13. The method recited in claim 11, further comprising:

receiving a second volume event determination notification from a voice recognition application, the second volume event determination notification corresponding to a second volume event;

in response to receiving the second volume event determination notification, displaying a second volume event occurrence indicator;

receiving a second volume event recognition notification from the voice recognition application, the second volume event recognition notification notifying that the second volume event has been recognized, and

in response to receiving the second volume event recognition notification, displaying a second volume event recognition indicator so as to associate the second volume event recognition indicator with the second volume event occurrence indicator.

14. The method recited in claim 13, further comprising:

determining if the voice recognition application has provided a phrase start notification indicating that a user has started speaking a phrase, and

if the voice recognition application has provided a phrase start notification, displaying the second volume event occurrence indicator adjacent to the first volume event occurrence indicator, and

if the voice recognition application has not provided a phrase start notification, displaying the second volume event occurrence indicator in place of the first volume event occurrence indicator.

15. The method recited in claim 14, further comprising displaying the second volume event occurrence indicator to the right of the first volume event occurrence indicator.

16. The method recited in claim 13, further comprising:

receiving a third volume event determination notification from the voice recognition application before receiving the second volume event recognition notification, the third volume event determination notification corresponding to a third volume event;

in response to receiving the third volume event determination notification, displaying a third volume event occurrence indicator before displaying the second volume event recognition indicator.

17. The method recited in claim 11, wherein the volume event occurrence indicator is a bar.

18. The method recited in claim 17, wherein a height of the bar corresponds to a volume of the volume event.

19. The method recited in claim 17, wherein a color of the bar corresponds to a volume of the volume event.

20. The method recited in claim 11, wherein the volume event recognition indicator is a square.

21. The method recited in claim 11, wherein a color of the volume event recognition indicator corresponds to a confidence level in the recognition of the volume event.

22. The method recited in claim 11, further comprising:

receiving a phrase end notification from the voice recognition application notifying of the end of a phrase; and

in response to receiving the phrase end notification, displaying a phrase end indicator.

23. The method recited in claim 22, wherein

the phrase end indicator indicates whether the phrase was accurately recognized by the voice recognition application.

24. An apparatus for displaying the status of a voice recognition application, comprising:

a volume event notification module that receives a notification from the voice recognition application when a volume event is determined;

a volume event recognition module that receives a notification from the voice recognition application when a volume event is recognized; and

a user interface module that

displays a volume event occurrence indicator in response to the volume event notification module receiving a notification from the voice recognition application, and

in response to the volume event recognition module receiving the notification from the voice recognition application that a volume event has been recognized,

determines if the volume event occurrence indicator for the recognized volume event is being displayed, and

if a volume event occurrence indicator for the recognized volume event is being displayed, displays a volume event volume event recognition indicator so as to associate volume event volume event recognition indicator with the volume event occurrence indicator.

25. The apparatus recited in claim 24, further comprising:

a phrase start module that receives a notification from the voice recognition application when the voice recognition application recognizes the start of a phrase.

26. The apparatus recited in claim 25, wherein the user interface module displays only a single volume event occurrence indicator at any time until the phrase start module receives a notification from the voice recognition application that the voice recognition application has recognized the start of a phrase.

27. The apparatus recited in claim 24, further comprising:

a phrase end module that receives a notification from the voice recognition application when the voice recognition application recognizes the end of a phrase.

28. The apparatus recited in claim 27, wherein the user interface module displays a phrase end indicator in response to the phrase end module receiving a notification from the voice recognition application.

29. A voice recognition system, comprising:

a voice recognition application for determining and recognizing volume events; and

a user interface application that

displays a volume event occurrence indicator when the voice recognition application determines a volume event; and

if the volume event occurrence indicator is still displayed when the voice recognition application recognizes the volume event, displays a volume event recognition indicator so as to associate the volume event recognition indicator with the volume event occurrence indicator.

30. The recognition system recited in claim 29, wherein the voice recognition application includes

a volume event determination module for determining the volume event from one or more recorded sound samples, and

a voice recognition module for recognizing the volume event

31. The recognition system recited in claim 29, wherein the user interface application includes

a volume event notification module that receives a notification from the volume event determination module when the volume event determination module determines the volume event;

a volume event recognition module that receives a notification from the voice recognition application when the volume event is recognized; and

a user interface module that

displays a volume event occurrence indicator in response to the volume event notification module receiving a notification from the volume event determination module, and

in response to the volume event notification module receiving the notification from the voice recognition module,

determines if the volume event occurrence indicator for the recognized volume event is being displayed, and

if a volume event occurrence indicator for the recognized volume event is being displayed, displays a volume event recognition indicator so as to associate the volume event recognition indicator with the volume event occurrence indicator.

32. The apparatus recited in claim 31, wherein the user interface application further includes

a phrase start module that receives a notification from the voice recognition application when the voice recognition application recognizes the start of a phrase.

33. The apparatus recited in claim 32, wherein the user interface module displays only a single volume event occurrence indicator at any time until the phrase start module receives a notification from the voice recognition application that the voice recognition application has recognized the start of a phrase.

34. The apparatus recited in claim 31, wherein the user interface application further includes

a phrase end module that receives a notification from the voice recognition application when the voice recognition application recognizes the end of a phrase.

35. The apparatus recited in claim 34, wherein the user interface module displays a phrase end indicator in response to the phrase end module receiving a notification from the voice recognition application.