Speech recognition user interface

• US 6,965,863 B1
• Filed: 11/12/1998
• Issued: 11/15/2005
• Est. Priority Date: 11/12/1998
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: DuVal, Cynthia, Zuberec, Sarah E., Rabelos, Benjamin N.
• Primary Examiner(s): ARMSTRONG, ANGELA A
• Application Number:
  US09/191,047
• Time in Patent Office: 2,560 Days
• Field of Search: 704/270, 704/275, 704/231, 704/246, 704/251, 704/276
• US Class Current: 704/270
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A speech recognition system having a user interface that provides both visual and auditory feedback to a user. The user interface includes an audio sound or speech generator that produces three distinct sounds: an “on” sound signifying that the speech recognition system is on and actively awaiting vocal input; an “off” sound indicating that the speech recognition system is off and in a sleep mode; and a “confirm” sound noting that an utterance has been recognized. The “on” sound is triggered by a key “wake up” command or by depression of button. Once awake, the speech recognition engine expects to receive an utterance within a predetermined response time. The “confirm” sound signals the start of the response time. If the response time lapses before a recognizable utterance is entered, the “off” sound is played. The user interface further includes a visual component in the form of a graphic that changes with the tolling of the response period. In one implementation, the count graphic is a progress bar that counts down or shortens in proportion to the passage of the response period. When the response time runs out, the progress bar disappears entirely. On the other hand, if the speech engine recognizes an utterance within the response period, the user interface plays the “confirm” sound and restarts the countdown graphic. The user interface may also change the color of the graphic elements briefly to reflect a correct voice entry.

Claims

1. A speech recognition system comprising:

a speech recognition engine to recognize an utterance, the speech recognition engine being configured to actively listen for the utterance for a predetermined response time, the speech recognition engine being configured to enter a dormant state if the utterance is not recognized within the predetermined amount of time, the speech recognition system remaining in the dormant state until recognition of a starter word that is independent of the utterance; and

a user interface to provide visual and auditory feedback indicating whether the speech recognition engine recognizes the utterance, the user interface being configured to;

(a) play an audible sound indicating recognition of the utterance;

(b) display a countdown graphic that changes with lapsing of the predetermined response time;

(c) restart the countdown graphic in the event the speech recognition engine recognizes the utterance.

2. A speech recognition system as recited in claim 1, wherein the response time is configurable.

3. A speech recognition system as recited in claim 1, wherein the user interface displays visual elements in a first color and briefly changes to a second color in the event the speech recognition engine recognizes the utterance.

4. A speech recognition system as recited in claim 1, wherein the countdown graphic comprises a progress bar that shortens as the response time diminishes.

5. A speech recognition system as recited in claim 1, wherein the user interface plays another audible sound when the response time has elapsed.

6. A speech recognition system as recited in claim 1, wherein the speech recognition engine is initially in a sleep mode and is awakened to an active mode upon detection of a starter utterance, the user interface plays another audible sound indicating that the speech recognition engine is in the active mode in the event the speech recognition engine recognizes the starter utterance.

7. A speech recognition system as recited in claim 1, wherein the speech recognition engine is initially in a sleep mode and is awakened to an active mode upon depression of a button, the user interface plays another audible sound indicating that the speech recognition engine is in the active mode in the event the speech recognition engine recognizes a starter utterance.

8. A speech recognition system comprising:

an application;

a vocabulary accessible by the application, the vocabulary holding a set of utterances applicable to the application;

a grammar that holds a subset of the utterances in the vocabulary;

a speech recognition engine to recognize the utterances in the grammar within a predetermined response time, the speech recognition engine being configured to enter a dormant state if the utterances are not recognized within the predetermined response of time; and

a user interface to display a countdown graphic that changes with lapsing of the response time, wherein the user interface restarts the countdown graphic in the event the speech recognition engine recognizes the one of the utterances.

9. A speech recognition system as recited in claim 8, wherein the user interface displays visual elements in a first color and briefly changes to a second color in the event the speech recognition engine recognizes one of the utterances.

10. A speech recognition system as recited in claim 8, wherein the countdown graphic comprises a progress bar that shortens as the response time diminishes.

11. A speech recognition system as recited in claim 8, wherein the user interface plays an audible sound when the speech recognition engine recognizes one of the utterances within the predetermined response time.

12. A speech recognition system as recited in claim 8, wherein the user interface plays an audible sound when the response time has elapsed.

13. A speech recognition system as recited in claim 8, wherein the speech recognition engine is initially in a sleep mode and is awakened to an active mode upon detection of a starter utterance, the user interface plays another audible sound indicating that the speech recognition engine is in the active mode in the event the speech recognition engine recognizes the starter utterance.

14. An entertainment system incorporating the speech recognition system as recited in claim 8.

15. A computing device incorporating the speech recognition system as recited in claim 8.

16. A user interface for an speech recognition system, the user interface comprising:

a display; and

a graphic progress bar shown on the display that indicates a response time in which the speech recognition system is awaiting a user to speak, the progress bar shortening with passage of the response time, wherein the graphic progress bar is lengthened to its initial position after each recognized user input, wherein the user interface plays an audible sound when the speech recognition engine recognizes one of the utterances within the predetermined response time, and wherein the user interface indicates that the speech recognition engine is in a dormant state when at least one of the utterances is not recognized within the predetermined response of time.

17. A user interface as recited in claim 16, wherein the graphic progress bar briefly changes color when a user input is recognized.

18. A speech recognition system incorporating the user interface as recited in claim 16.

19. A computing device incorporating the user interface as recited in claim 16.

20. A user interface for an speech recognition system, the user interface comprising:

a display;

an audio input to receive audible utterances;

a graphic shown on the display that indicates a fixed response time in which the speech recognition system is awaiting receipt of an utterance via the audio input, the graphic diminishing in size with the passage of time, the graphic returning to an original size after each recognized utterance; and

an audio generator to emit a first audible sound when the speech recognition system recognizes the utterance, the audio generator being further configured to emit a second audible sound when the fixed response time has expired before the utterance has been recognized, the second sound indicating that the speech recognition system has entered a dormant state.

21. A user interface as recited in claim 20, wherein the audio generator emits a second audible sound when the speech recognition system fails to recognize the utterance within the response time.

22. A speech recognition system incorporating the user interface as recited in claim 20.

23. A computing device incorporating the user interface as recited in claim 20.

24. A vehicle computer system comprising:

a computer;

an open platform operating system executing on the computer, the operating system being configured to support multiple applications; and

a speech recognition system to detect utterances used to control at least one of the applications running on the computer, the speech recognition system having a user interface to provide visual and auditory feedback indicating whether an utterance is recognized, the user interface being configured to play a first audible sound indicating recognition of the utterance and to display a graphic that diminishes in size from an original size with the passage of time, the graphic returning to the original size after each recognized utterance, the user interface being further configured to emit a second audible sound when a predetermined response time has expired before the utterance has been recognized, the second sound indicating that the speech recognition system has entered a dormant state.

25. A vehicle computer system as recited in claim 24, wherein the user interface restarts the graphic in the event the utterance is recognized.

26. A vehicle computer system as recited in claim 24, wherein the user interface displays visual elements in a first color and briefly changes to a second color in the event the utterance is recognized.

27. A vehicle computer system as recited in claim 24, wherein the graphic comprises a progress bar that shortens as the response time passes.

28. A vehicle computer system as recited in claim 24, wherein the user interface plays another audible sound when the response time has elapsed.

29. A vehicle computer system as recited in claim 24, wherein the speech recognition system is initially in a sleep mode and is awakened to an active mode upon detection of a starter utterance, the user interface plays another audible sound indicating that the speech recognition system is in the active mode in the event the starter utterance is recognized.

30. A collaboration system involving multiple interconnected devices comprising:

a voice input mechanism resident at each of the devices;

an audio output system resident at each of the devices; and

a user interface to provide visual and auditory feedback indicating when a party located at one of the devices can speak, the user interface being configured to play an audible sound when the party can begin speaking and to display a graphic that changes with lapsing of time to indicate a duration that the party can speak, the graphic diminishing in size from an original size with the passage of time, the graphic returning to the original size after each recognized utterance, wherein the user interface plays an audible sound upon recognizing an utterance within the duration that the party can speak, the user interface emitting a second audible sound when the duration has expired before the utterance has been recognized, the second sound indicating that the speech recognition system has entered a dormant state.

31. A method for operating a speech recognition system, comprising the following steps:

initiating a response time in which to receive an audible utterance;

displaying a graphic representing the response time;

playing a first sound when an audible utterance is recognized;

changing the graphic to indicate passage of the response time such that the graphic diminishes in size from an original size with the passage of time;

responsive to recognizing an utterance, presenting the graphic in the original size; and

responsive to expiration of the response time before the audible utterance has been recognized, emitting a second sound to indicate that the speech recognition system has entered a dormant state.

32. A method as recited in claim 31, wherein the displaying and changing steps comprise the steps of depicting a progress bar and shortening the progress bar as the response time passes.

33. A method as recited in claim 31, further comprising the step of changing a color of the graphic when an audible utterance is recognized.

34. A method as recited in claim 31, further comprising the step of playing a sound when no audible utterance is recognized within the response time.