Method and system for displaying speech recognition status information in a visual notification area

• US 5,864,815 A
• Filed: 07/31/1995
• Issued: 01/26/1999
• Est. Priority Date: 07/31/1995
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Salin, Juha P., Spoltman, James H., Belgau, Ronald A., Rozak, Michael J.
• Primary Examiner(s): Dorvii, Richemond
• Assistant Examiner(s): Chawan, Vijay B.
• Application Number:
  US08/508,757
• Time in Patent Office: 1,275 Days
• Field of Search: 395/2.84, 395/2.52, 395/2.43, 395/2.79, 395/345-348, 395/340, 364/513, 340/721
• US Class Current: 704/275
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 2015/221 Announcement of recognition results

Abstract

A speech recognition system provides a user with graphical and textual feedback. The textual feedback is displayed in windows but occupies little of the available display space and are displayed only for a short period of time. The graphical feedback is displayed in a designated notification area and does not obscure any other displayed items. The feedback provided by the speech recognition system may indicate a current mode of operation of the speech recognition system as well as a state of processing of audio input by the speech recognition system.

Claims

1. In a computer system having a video display, an audio input device for receiving audio input from a user and a speech recognizer for recognizing components of speech in audio input, said speech recognizer operating in a number of different modes of operation, a method comprising the computer-implemented steps of:

providing a graphical user interface on the video display, said graphical user interface including a designated notification area for displaying graphical notifications;

determining a first current mode of operation of the speech recognizer;

displaying a first graphical notification of the first current mode of operation of the speech recognizer in the designated notification area;

determining a second current mode of operation of the speech recognizer; and

displaying a second graphical notification of the second current mode of the speech recognizer in the designated notification area, wherein the first notification differs from the second graphical notification.

2. The method of claim 1 wherein the first current mode of operation of the speech recognizer is determined to be a listening mode in which the speech recognizer is listening for and processing audio input received at the audio input device.

3. The method of claim 1 wherein the first current mode of operation of the speech recognizer is determined to be a non-listening mode in which the speech recognizer is not listening for or processing the audio input received at the audio input device.

4. The method of claim 1 wherein the first current mode of operation of the speech recognizer is determined to be a standing-by mode in which the speech recognizer is listening for and processing only selected audio input received at the audio input device.

5. The method of claim 1 wherein the step of displaying the graphical notification of the first current mode of operation of the speech recognizer in the designated notification area comprises displaying an icon that identifies the first current mode of operation of the speech recognizer in the designated notification area.

6. The method of claim 1 wherein the computer system runs an operating system and the operating system provides the graphical user interface.

7. The method of claim 1 further comprising the step of displaying text in a separate window that identifies the second current mode of operation.

8. The method of claim 7 further comprising the step of terminating the display of the text and the separate window after expiration of a predetermined time period.

9. The method of claim 1 wherein the computer system further comprises a mouse with at least one button, said mouse for manipulating a mouse cursor that points to locations on the video display and wherein the method further comprises the step of displaying a user interface element that enables the user to change the first current mode of operation of the speech recognizer in response to the mouse cursor pointing at the graphical notification and the user clicking the button on the mouse.

10. The method of claim 1 wherein the designated notification area does not obscure any other video output on the video display.

11. In a computer system having a video display, an audio input device for receiving audio input from a user and a speech recognizer for recognizing components of speech in audio input, a method comprising the computer-implemented steps of:

receiving audio input from the user through the audio input device;

processing the received audio input with the speech recognizer to attempt to recognize components of speech in the received audio input; and

displaying textual feedback regarding the processing of the received audio input by the speech recognizer in a tool tip window that is only displayed for a predetermined period of time.

12. The method of claim 11 wherein the displayed textual feedback indicates that the received audio input was recognized by the speech recognizer.

13. The method of claim 12 wherein the displayed textual feedback indicates what the received audio input was recognized as.

14. The method of claim 11 wherein the displayed textual feedback indicates that the received audio input was heard by the speech recognizer.

15. The method of claim 11 wherein the displayed textual feedback indicates that the received audio input was not recognized by the speech recognizer.

16. The method of claim 11 wherein the displayed textual feedback indicates that there are problems with the received audio input that make it difficult for the speech recognizer to process the received audio input.

17. The method of claim 11 further comprising the steps of:

providing a designated notification area on the video display for displaying graphical notifications by application programs;

displaying a graphical notification in the designated notification area to provide feedback to the user about processing of the received audio input by the speech recognizer.

18. The method of claim 17 wherein the displayed graphic notification indicates that the received audio input was recognized by the speech recognizer.

19. The method of claim 17 wherein the displayed graphical notification indicates that the received audio input was not recognized by the speech recognizer.

20. A computer system comprising:

a video display for displaying video images;

an audio input device for receiving audio input from a user;

a speech recognizer for recognizing components of speech in audio input received by the audio input device;

a component for providing a graphical notification area on the video display for displaying different types of graphical notifications to the user; and

a facility for displaying graphical notifications in the graphical notification area that identify current modes of operation of the speech recognizer.

21. The computer system of claim 20 further comprising a facility for displaying text in a window that identifies the current mode of operation of the speech recognizer.

22. The computer system of claim 21 wherein the facility for displaying text only displays in the window for a predetermined period of time.

23. A computer system comprising:

a video display for displaying video images;

an audio input device for receiving audio input from a user;

a speech recognizer for recognizing components of speech in audio input received by the audio input device; and

a first facility for displaying textual feedback to the user regarding processing of the received audio input by speech recognizer in a tool tip window that is only displayed for a predetermined period of time.

24. The computer system of claim 23 further comprising:

a second facility for providing a graphical notification area on the video display for displaying graphic notifications;

a third facility for displaying a graphical notification in the graphical notification area that provides feedback regarding processing of the received audio input by the speech recognizer.

25. In a computer system having a video display, an audio input device for receiving audio input from a user, a graphical user interface on the video display, said graphical user interface including a designated notification area for displaying graphical notifications and a speech recognizer for recognizing components of speech in audio input, said speech recognizer operating in a number of different modes of operation, a computer-readable medium holding computer-executable instructions for performing a method comprising the computer-implemented steps of:

determining a first current mode of operation of the speech recognizer;

displaying a first graphical notification of the first current mode of operation of the speech recognizer in the designated notification area;

determining a second current mode of operation of the speech recognizer; and

displaying a second graphical notification of the second current mode of the speech recognizer in the designated notification area, wherein the first notification differs from the second graphical notification.

26. The computer-readable medium of claim 25 wherein the first current mode of operation of the speech recognizer is determined to be a listening mode in which the speech recognizer is listening for and processing audio input received at the audio input device.

27. The computer-readable medium of claim 25 wherein the first current mode of operation of the speech recognizer is determined to be a non-listening mode in which the speech recognizer is not listening for or processing the audio input received at the audio input device.

28. The computer-readable medium of claim 25 wherein the first current mode of operation of the speech recognizer is determined to be a standing-by mode in which the speech recognizer is listening for and processing only selected audio input received at the audio input device.

29. The computer-readable medium of claim 25 wherein the step of displaying the graphical notification of the first current mode of operation of the speech recognizer in the designated notification area comprises displaying an icon that identifies the first current mode of operation of the speech recognizer in the designated notification area.

30. The computer-readable medium of claim 25 wherein the computer system runs an operating system and the operating system provides the graphical user interface.

31. The computer-readable medium of claim 25 wherein the method further comprises the step of displaying text in a separate window that identifies the second current mode of operation.

32. The computer-readable medium of claim 31 wherein the method further comprises the step of terminating the display of the text and the separate window after expiration of a predetermined time period.

33. The computer-readable medium of claim 25 wherein the designated notification area does not obscure any other video output on the video display.

34. In a computer system having a video display, an audio input device for receiving audio input from a user and a speech recognizer for recognizing components of speech in audio input, a computer-readable medium holding computer-executable instructions for performing a method comprising the computer-implemented steps of:

receiving audio input from the user through the audio input device;

processing the received audio input with the speech recognizer to attempt to recognize components of speech in the received audio input; and

displaying textual feedback regarding the processing of the received audio input by the speech recognizer in a tool tip window that is only displayed for a predetermined period of time.

35. The computer-readable medium of claim 34 wherein the displayed textual feedback indicates that the received audio input was recognized by the speech recognizer.

36. The computer-readable medium of claim 35 wherein the displayed textual feedback indicates what the received audio input was recognized as.

37. The computer-readable medium of claim 34 wherein the displayed textual feedback indicates that the received audio input was not recognized by the speech recognizer.

38. The computer-readable medium of claim 34 wherein the displayed textual feedback indicates that there are problems with the received audio input that make it difficult for the speech recognizer to process the received audio input.