Dynamic semantic control of a speech recognition system

• US 7,127,393 B2
• Filed: 02/10/2003
• Issued: 10/24/2006
• Est. Priority Date: 02/25/1999
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Speechworks International
• Inventors: Phillips, Michael S., Dahan, Jean-Guy, Barnard, Etienne, Metzger, Michael J.
• Primary Examiner(s): CHAWAN, VIJAY B
• Application Number:
  US10/364,298
• Publication Number:
  US 20040006465A1
• Time in Patent Office: 1,352 Days
• Field of Search: 704/240, 704/256, 704/9, 704/1, 704/232, 704/235, 704/260, 704/236, 704/242, 704/251, 704/252, 704/254, 704/255, 704/257, 704270-275, 707/4
• US Class Current: 704/240
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning

Abstract

A method and apparatus are provided for automatically recognizing words of spoken speech using a computer-based speech recognition system according to a dynamic semantic model. In an embodiment, the speech recognition system recognizes speech and generates one or more word strings, each of which is a hypothesis of the speech, and creates and stores a probability value or score for each of the word strings. The word strings are ordered by probability value. The speech recognition system also creates and stores, for each of the word strings, one or more keyword-value pairs that represent semantic elements and semantic values of the semantic elements for the speech that was spoken. One or more dynamic semantic rules are defined that specify how a probability value of a word string should be modified based on information about external conditions, facts, or the environment of the application in relation to the semantic values of that word string. A speech recognition application, upon receiving the keyword-value pairs, instructs the speech recognizer to modify one or more of the probability values, according to the dynamic semantic rules. The dynamic semantic rules are applied to the word strings and the keyword-value pairs. The speech recognizer modifies one or more of the probability values, re-orders the word strings, and returns control to the application. As a result, the speech recognizer may adjust dynamically to a changing likelihood that a speaker uttered a particular utterance, as the application executes, depending on the context of the application and the external factors.

Claims

1. A method for processing an utterance by a speech recognizer comprising:

processing a plurality of rules, each rule being associated with a condition external to the speech recognizer in which that rule is applicable and being based at least in part on an underlying meaning of one or more utterances;

receiving an utterance and producing a plurality of recognition hypotheses for the received utterance;

determining rules that are applicable to the utterance;

applying the determined rules to the plurality of recognition hypotheses for the received utterance; and

determining a recognition result for the received utterance based on results of applying the determined rules.

2. The method of claim 1 wherein each recognition hypothesis is associated with a numerical score, and applying the rules to the recognition hypotheses includes modifying the scores for the hypotheses.

3. The method of claim 2 wherein determining the recognition result includes selecting one of the recognition hypotheses according the modified scores for the hypotheses.

4. The method of claim 2 wherein determining the recognition result includes sorting the recognition hypotheses according to the modified scores for the hypotheses.

5. The method of claim 2 wherein the numerical score includes a probability.

6. The method of claim 1 wherein determining the recognition result includes selecting one of the recognition hypotheses based on the results of processing the rules.

7. The method of claim 1 wherein determining the recognition result includes determining an order for the recognition hypotheses based on the results of processing the rules.

8. The method of claim 1 wherein applying the rules includes rejecting one or more of the recognition hypotheses.

9. The method of claim 1 wherein each recognition hypothesis is associated with one or more semantic values, and applying the rules includes processing the semantic values.

10. The method of claim 9 wherein processing the semantic values in applying the rules includes evaluating expressions that depend on the semantic values.

11. The method of claim 1 further comprising storing the information defining the rules in association with the speech recognizer and wherein the processing of the rules is performed by the speech recognizer using the stored information.

12. The method of claim 11 wherein determining the recognition result is performed by the speech recognizer.

13. The method of claim 12 further comprising providing the recognition result to an application program.

14. The method of claim 13 further comprising accepting the information defining the rules from the application program.

15. The method of claim 1 further comprising passing data associated with each recognition hypothesis to a module external to the speech recognizer, and applying the applicable rules includes accepting information for use in determining the recognition result from the external module.

16. The method of claim 15, wherein each recognition hypothesis is associated with a numerical score, and applying the applicable rules includes modifying the scores for the hypotheses according to the accepted information.

17. The method of claim 16 wherein accepting the information includes accepting a plurality of numerical quantities for modifying the scores, each associated with a different one of the recognition hypotheses.

18. The method of claim 17 wherein modifying the scores includes combining each score and a corresponding one of the numerical quantities according to an arithmetic expression.

19. Software stored on a computer-readable medium for causing a computer to perform steps comprising:

processing a plurality of rules, each rule being associated with a condition external to a speech recognizer in which that rule is applicable and being based at least in part on an underlying meaning of one or more utterances;

receiving an utterance and producing a plurality of recognition hypotheses for the received utterance;

determining rules that are applicable to the received utterance;

applying the determined rules to the plurality of recognition hypotheses for the received utterance processed by the speech recognizer; and

determining a recognition result for the received utterance based on results of applying the determined rules.

20. The software of claim 19 wherein each recognition hypothesis is associated with a numerical score, and applying the rules to the recognition hypotheses includes modifying the scores for the hypotheses.

21. The software of claim 20 wherein determining the recognition result includes selecting one of the recognition hypotheses according the modified scores for the hypotheses.

22. The software of claim 20 wherein determining the recognition result includes sorting the recognition hypotheses according to the modified scores for the hypotheses.

23. The software of claim 22 wherein the numerical score includes a probability.

24. The software of claim 19 wherein determining the recognition result includes selecting one of the recognition hypotheses based on the results of processing the rules.

25. The software of claim 19 wherein determining the recognition result includes determining an order for the recognition hypotheses based on the results of processing the rules.

26. The software of claim 19 wherein applying the rules includes rejecting one or more of the recognition hypotheses.

27. The software of claim 19 wherein each recognition hypothesis is associated with one or more semantic values, and applying the rules includes processing the semantic values.

28. The software of claim 27 wherein processing the semantic values in applying the rules includes evaluating expressions that depend on the semantic values.

29. The software of claim 19 wherein the steps further comprise storing the information defining the rules in association with the speech recognizer and wherein the processing of the rules is performed by the speech recognizer using the stored information.

30. The software of claim 29 wherein determining the recognition result is performed by the speech recognizer.

31. The software of claim 30 wherein the steps further comprise providing the recognition result to an application program.

32. The software of claim 31 wherein the steps further comprise accepting the information defining the rules from the application program.

33. The software of claim 19 wherein the steps further comprise passing data associated with each recognition hypothesis to a module external to the speech recognizer, and applying the applicable rules includes accepting information for use in determining the recognition result from the external module.

34. The software of claim 33, wherein each recognition hypothesis is associated with a numerical score, and applying the applicable rules includes modifying the scores for the hypotheses according to the accepted information.

35. The software of claim 34 wherein accepting the information includes accepting a plurality of numerical quantities for modifying the scores, each associated with a different one of the recognition hypotheses.

36. The software of claim 35 wherein modifying the scores includes combining each score and a corresponding one of the numerical quantities according to an arithmetic expression.