Method and apparatus for reducing latency in speech-based applications

• US 20020099535A1
• Filed: 01/22/2001
• Published: 07/25/2002
• Est. Priority Date: 01/22/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Schmid, Philipp H., Garside, Adrian
• Granted Patent: US 6,961,694 B2
• Time in Patent Office: Days
• US Class Current: 704/4
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning

Abstract

A speech recognition interface and computer-readable medium build a grammar for speech recognition that reduces latency in speech-based applications. The interface and medium receive instructions to add a new phrase and semantic information to a grammar. The new phrase is combined with at least one other phrase in the grammar to form a composite grammar structure. The semantic information is then associated with a single word or transition in the grammar structure by selecting the first possible word or transition that semantically differentiates the new phrase from all other phrases in the grammar structure. By placing the semantic information in this position, the semantic information is placed as far forward in the grammar as possible without introducing semantic ambiguity into the grammar structure.

Claims

1. A computer-readable medium having computer-executable instructions for performing steps of:

receiving at least one instruction to add a new phrase and semantic information associated with the new phrase to a grammar;

combining the new phrase with at least one other phrase in the grammar to form a single grammar structure;

associating the semantic information with a single recognition value in the new phrase by selecting the first possible recognition value in the new phrase that can be associated with the semantic information without introducing semantic ambiguity into the grammar structure.

2. The computer-readable medium of claim 1 wherein the step of combining comprises adding a branch to the grammar structure to represent at least one recognition value of in the new phrase.

3. The computer-readable medium of claim 2 wherein associating the semantic information with a single recognition value in the new phrase comprises associating the semantic information with the first recognition value of the first branch added to the grammar structure for the new phrase.

4. The computer-readable medium of claim 3 wherein the computer-executable instructions are for performing further steps of:

identifying existing semantic information that was present in the grammar structure before the new phrase was added; and

shifting existing semantic information within the grammar structure so that the existing semantic information is not on a recognition value found in the new phrase.

5. The computer-readable medium of claim 4 wherein shifting existing semantic information comprises placing the semantic information on the first recognition value in each branch that is parallel to the branch added for the new phrase.

6. The computer-readable medium of claim 1 wherein the computer-executable instructions are for performing further steps of:

providing the grammar structure to a speech recognition engine;

receiving an indication of a hypothesis word recognized by the speech recognition engine based in part on the grammar structure; and

providing the semantic information associated with the hypothesis word to an application.

7. The computer-readable medium of claim 6 wherein the computer-executable instructions are for performing a further step of providing a confidence score to the application with the semantic information, the confidence score representing the probability that the hypothesis word has been properly recognized by the speech recognition engine.

8. The computer-readable medium of claim 7 wherein the computer-executable instruction include instructions defining the application and wherein the computer-executable instructions of the application are for performing a step of deciding whether to take an action before complete recognition of a phrase based on semantic information provided to the application with a hypothesis word.

9. The computer-readable medium of claim 8 wherein deciding whether to take an action before complete recognition comprises determining whether the risk of harm of taking the action is outweighed by the confidence score associated with the hypothesis word.

10. The computer-readable medium of claim 8 wherein deciding whether to take an action before complete recognition comprises determining whether the semantic information is sufficiently stable to take the action.

11. The computer-readable medium of claim 8 wherein the computer-executable instructions defining the application are for performing further steps of deciding not to take an action before complete recognition but deciding to provide user feedback indicating an action that could be taken based on the semantic information.

12. The computer-readable medium of claim 1 wherein the computer-executable instructions are for performing further steps comprising:

determining that the semantic information cannot be associated with a single recognition value without introducing semantic ambiguity into the grammar structure; and

returning an error message indicating that the semantic information would introduce semantic ambiguity into the grammar structure.

14. The speech recognition interface of claim 13 wherein the grammar structure building component adds transitions to the grammar structure by only adding a new transition if there are no existing transitions in the grammar structure that both extend from a same state as the new transition and have the same recognition value as the new transition.

15. The speech recognition interface of claim 14 wherein the semantic placement component additionally moves semantic information that is associated with at least one sequence of transitions other than the current sequence of transitions to avoid semantic ambiguity due to the addition of the current sequence of transitions.

16. The speech recognition interface of claim 15 wherein the semantic information moved by the placement component is found on a transition that forms part of the current sequence of transitions before it is moved.

17. The speech recognition interface of claim 16 wherein the semantic information moved by the placement component is found on a transition before the transition that differentiates the current sequence of transitions from all other transitions.

18. The speech recognition interface of claim 17 wherein the placement component moves the semantic information by associating the semantic information with each transition that extends from the same state as the transition that differentiates the current sequence of transitions from all other transitions.

19. The speech recognition interface of claim 13 further comprising:

a binary grammar building component that generates a compiled representation of the grammar structure; and

a context-free grammar engine that receives the compiled representation of the grammar structure and that provides the grammar structure to a speech recognition engine.

20. The speech recognition interface of claim 19 wherein the context-free grammar engine receives a hypothesis transition from the speech recognition engine before recognition of a complete utterance, identifies semantic information associated with the hypothesis transition, and provides the semantic information to an application.

13. A speech recognition interface for a speech recognition engine, the interface comprising:

a grammar structure building component that receives instructions to add transitions to a grammar structure and in response adds transitions to a grammar structure; and

a semantic placement component that places semantic information associated with a current sequence of transitions on a first transition in the grammar structure that differentiates the current sequence of transitions from all other sequences of transitions in the grammar structure.

21. A computer-readable medium having computer-executable instructions representing a speech-enabled application capable of performing steps of:

receiving semantic information associated with a hypothesis word identified by a speech recognition engine before an entire utterance has been recognized;

identifying a possible action to take based on the semantic information; and

determining whether to take the action before the entire utterance has been recognized.

22. The computer-readable medium of claim 21 wherein the computer-executable instructions are capable of performing a further step of receiving a confidence score for the hypothesis word and wherein determining whether to take the action comprises balancing the confidence score with a risk of harm associated with incorrectly taking the action.

23. The computer-readable medium of claim 21 wherein receiving semantic information for a hypothesis comprises receiving semantic information for each hypothesis in a sequence of hypotheses and wherein determining whether to take the action comprises determining the stability of the semantic information across the sequence of hypotheses.

24. The computer-readable medium of claim 21 wherein determining whether to take the action comprises determining not to take the action and instead providing feedback to the user identifying the action that was not taken.