Speech recognition language models

• US 6,167,377 A
• Filed: 03/28/1997
• Issued: 12/26/2000
• Est. Priority Date: 03/28/1997
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Dragon Systems, Inc. (Microsoft Corporation)
• Inventors: van Mulbregt, Paul A., Bibeault, Michael D., Roth, Robert, Gillick, Laurence S., Gould, Joel M.
• Primary Examiner(s): Smits, Talivaldis I.
• Assistant Examiner(s): Lerner, Martin
• Application Number:
  US08/825,535
• Time in Patent Office: 1,369 Days
• Field of Search: 704/3, 704/231, 704/243, 704/244, 704/252, 704/255, 704/257, 704/250, 704/240, 381/71.11, 708/322
• US Class Current: 704/240
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting

Abstract

Language model results are combined according to a combination expression to produce combined language model results for a set of candidates. A candidate is selected and the combination expression is adjusted using language model results associated with the selected candidate.

Claims

1. A method of speech recognition using multiple language models, the method comprising:

receiving language model results for speech recognition candidates for an utterance from multiple language models;

combining the results from the multiple language models according to a combination expression to produce combined language model results, the combination expression applying different combination weights to the results received from the respective language models;

selecting one of the candidates selecting based on the combined language model results;

adjusting at least one of the combination expression weights based on the selected candidate; and

repeating the combining, selecting, and adjusting for a subsequent utterance.

2. The method of claim 1, wherein:

the adjusting comprises adjusting the combination weights using language model results associated with the selected candidate.

3. The method of claim 2, wherein:

each language model result comprises a numerical score that indicates a likelihood associated with a candidate; and

the combination expression multiplies the numerical score for a language model by a combination weight associated with the language model.

4. The method of claim 1, wherein a language model result is associated with a topic language model generated from user-supplied text.

5. The method of claim 4, further comprising generating the topic language model by:

prompting a user to identify a source of text;

retrieving text from the source of text identified by the user; and

building the topic language model from the retrieved text.

6. The method of claim 5, wherein the source of text comprises a file including text.

7. The method of claim 1, further comprising selecting a candidate by evaluating the candidates using the combined language model results.

8. The method of claim 7, further comprising selecting a candidate by evaluating the candidates using the combined language model results and acoustic information associated with the candidates.

9. The method of claim 8, wherein the evaluating uses a vocabulary of words, and wherein building the topic language model further comprises:

prompting a user to identify a source of text;

retrieving text from the source of text identified by the user;

scanning the retrieved text to identify words from the text that are not included in the vocabulary of words;

displaying the identified words to the user;

prompting the user to select words to be included in the vocabulary; and

adding words selected by the user to the vocabulary.

10. The method of claim 9, further comprising building the topic language model from the retrieved text.

11. The method of claim 10, wherein the building the topic language model comprises building the topic language model for words included in the vocabulary of words.

12. The method of claim 1, wherein one of the language models comprises a language model formed from words in one or more user specified files.

13. The method of claim 1, wherein one of the language models comprises a language model formed from words in a document being dictated.

14. The method of claim 1, wherein one of the language models comprises a language model formed from a list of recently recognized words.

15. The method of claim 14, wherein the list of recently recognized words comprises a list of one thousand most recently recognized words.

16. The method of claim 1, wherein the combination expression is of the form:

space="preserve" listing-type="equation">P.sub.c (w.sub.a)=Σ

λ

.sub.x P.sub.x (w.sub.a)

where P_c (W_a) represents the combined language model score for word a, P_x (w_a) represents the score for word a for language model x, and λ

_x represents the combination expression weight for language model x.

17. The method of claim 16, wherein Σ

λ

_x =1.

18. The method of claim 1, wherein the adjusting comprises adjusting based on previous weights.

19. The method of claim 1, wherein the adjusting comprises adjusting using an adjustment factor that controls how rapidly the weights change.

20. A method of generating language model results for use in speech recognition, the method comprising:

prompting a user to identify a source of text;

retrieving text from the source of text identified by the user;

building a topic language model from the retrieved text;

producing combined language model results by combining language model results from the topic language model with language model results from one or more other language models according to a combination expression, the combination expression applying different combination weights to the results received from the respective language models; and

adjusting at least one of the combination weights based on performance of the language models.

21. The method of claim 20, further comprising:

scanning the retrieved text to identify words from the text that are not included in a vocabulary of words;

displaying the identified words to the user;

prompting the user to select words to be included in the vocabulary; and

adding words selected by the user to the vocabulary.

22. The method of claim 21, wherein the building the topic language model comprises building the topic language model for words included in the vocabulary of words.

23. The method of claim 20, wherein the source of text comprises a file including text.

24. A computer program, residing on a computer readable medium, for a speech recognition system comprising a processor and an input device, the computer program comprising instructions performing speech recognition by causing the processor to perform the following operations:

receive language model results for speech recognition candidates for an utterance from multiple language models;

combine the results from the multiple language models according to a combination expression to produce combined language model results, the combination expression applying different combination weights to the results received from the respective language models;

evaluate the candidates using the combined language model results to select one of the candidates;

adjust at least one of the combination expression weights based on the selected candidate; and

repeat the combining, selecting, and adjusting for a subsequent utterance.

25. The computer program of claim 24, further comprising instructions for causing a processor to:

prompt a user to identify a source of text;

retrieve text from the source of text identified by the user; and

build a topic language model from the retrieved text.

26. A computer program, residing on a computer readable medium, for a speech recognition system comprising a processor and an input device, the computer program comprising instructions for generating a topic language model for use in speech recognition by causing the processor to perform the following operations:

prompt a user to identify a source of text;

retrieve text from the source of text identified by the user;

build a topic language model from the retrieved text;

produce combined language model results by combining language model results from the topic language model with language model results from one or more other language models according to a combination expression, the combination expression applying different combination weights to the results received from the respective language models; and

adjust at least one of the combination weights based on performance of the language models.

27. The method of claim 26, wherein the source of text comprises a file including text.

28. A method of speech recognition using multiple language models, the method comprising:

receiving language model results for speech recognition candidates for an utterance from multiple language models, the language models including at least one language model formed from recently recognized words;

combining the results from the multiple language models according to a combination expression to produce combined language model results, the combination expression being of the form;

space="preserve" listing-type="equation">P.sub.c (w.sub.a)=Σ

λ

.sub.x P.sub.x (w.sub.a)

where P_c (w_a) represents the combined language model score for word a, P_x (w_a) represents the score for word a for language model x, and λ

_x represents a combination expression weight for language model x;

selecting one of the candidates based on the combined language model results;

adjusting at least one of the combination expression weights, λ

_x, based on the selected candidate; and

repeating the combining, selecting, and adjusting for a subsequent utterance.