Speech recognition system for providing voice recognition services using a conversational language model

• US 8,265,933 B2
• Filed: 12/22/2005
• Issued: 09/11/2012
• Est. Priority Date: 12/22/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Bates, Cary L., Wallenfelt, Brian P.
• Primary Examiner(s): He, Jialong
• Application Number:
  US11/316,263
• Publication Number:
  US 20070150278A1
• Time in Patent Office: 2,455 Days
• Field of Search: 704/256, 704/270, 704/275
• US Class Current: 704/270
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams

Abstract

Embodiments of the present invention provide a method, system and article of manufacture for adjusting a language model within a voice recognition system, based on text received from an external application. The external application may supply text representing the words of one participant to a text-based conversation. In such a case, changes may be made to a language model by analyzing the external text received from the external application.

Claims

1. A computer-implemented method for performing voice recognition, at least in part, using a language model incorporating a plurality of words and a plurality of n-grams, each of the plurality of n-grams formed by two or more of the plurality of words and defining a probability of occurrence of each word in the respective n-gram given the occurrence of one of the two or more of the plurality of words forming the respective n-gram, the method comprising:

receiving a first text comprising a sequence of one or more words from a first participant in a text-based conversation with a second participant via at least one first user device;

receiving a voice response provided by the second participant in the text-based conversation in response to the first text;

selecting at least one of the plurality of n-grams from the language model that includes at least one word in the first text; and

automatically recognizing at least a portion of the voice response to provide a second text, at least in part, by using the at least one selected n-gram;

maintaining a history of recognized words and received texts in the text-based conversation; and

maintaining an identifier representing a change in speaker from the first or the second participant to the other.

2. The method of claim 1, wherein the at least one first user device includes an instant messaging application, and wherein the first text is an instant message received by the second participant to the conversation via at least one second user device.

3. The method of claim 2, wherein the at least one first user device and/or the at least one second user device comprises a mobile telephone.

4. The method of claim 1, further comprising adjusting the probability for at least one word in the at least one selected n-gram based, at least in part, on the first text.

5. The method of claim 4, wherein the probability for the at least one candidate word of the at least one of the plurality of n-grams is increased.

6. The method of claim 4, wherein the probability for the at least one candidate word of the at least one of the plurality of n-grams is decreased.

7. At least one non-transitory computer-readable storage medium containing a program which, when executed on at least one computer, performs a method of performing voice recognition, at least in part, using a language model incorporating a plurality of words and a plurality of n-grams, each of the plurality of n-grams formed by two or more of the plurality of words and defining a probability of occurrence of each word in the respective n-gram given the occurrence of one of the two or more of the plurality of words forming the respective n-gram, the method comprising:

receiving a first text comprising a sequence of one or more words from a first participant in a text-based conversation with a second participant via at least one first user device;

receiving a voice response provided by the second participant in the text-based conversation in response to the first text;

selecting at least one of the plurality of n-grams from the language model that includes at least one word in the first text; and

automatically recognizing at least a portion of the voice response to provide a second text, at least in part, by using the at least one selected n-gram;

maintaining a history of recognized words and received texts in the text-based conversation; and

maintaining an identifier representing a change in speaker from the first or the second participant to the other.

8. The at least one non-transitory computer-readable storage medium of claim 7, wherein the at least one first user device includes an instant messaging application, and wherein the first text is an instant message received by the second participant to the conversation via at least one second user device.

9. The at least one non-transitory computer-readable storage medium of claim 8, wherein the at least one first user device and/or the at least one second user device comprises a mobile telephone.

10. The at least one non-transitory computer-readable storage medium of claim 7, further comprising adjusting the probability for at least one word in the at least one selected n-gram based, at least in part, on the first text.

11. The at least one non-transitory computer-readable storage medium of claim 10, wherein the probability for the at least one candidate word of the at least one of the plurality of n-grams is increased.

12. The at least one non-transitory computer-readable storage medium of claim 10, wherein the probability for the at least one candidate word of the at least one of the plurality of n-grams is decreased.

13. A computing device that performs voice recognition, at least in part, using a language model incorporating a plurality of words and a plurality of n-grams, each of the plurality of n-grams formed by two or more of the plurality of words and defining a probability of occurrence of each word in the respective n-gram given the occurrence of one of the two or more of the plurality of words forming the respective n-gram, the computing device comprising:

at least one input to receive a first text comprising a sequence of one or more words from a first participant in a text-based conversation with a second participant via at least one user device and receive a voice response provided by the second participant in the text-based conversation in response to the first text; and

at least one processor coupled to the at least one input, the at least one processor programmed to;

select at least one of the plurality of n-grams from the language model that includes at least one word in the first text; and

automatically recognize at least a portion of the voice response to provide a second text, at least in part, by using the at least one selected n-gram;

maintain a history of recognized words and received texts in the text-based conversation; and

maintain an identifier representing a change in speaker from the first or the second participant to the other.

14. The computing device of claim 13, wherein the at least one user device comprises an instant messaging application, and wherein the first text is an instant message received by the second participant to the conversation.

15. The computing device of claim 13, wherein the computing device and/or the at least one first user device comprises a mobile telephone.

16. The computing device of claim 13, wherein the at least one processor is further programmed to adjust the probability for at least one word in the at least one selected n-gram based, at least in part, on the first text.

17. The computing device of claim 16, wherein the probability for the at least one candidate word of the at least one of the plurality of n-grams is increased.

18. The computing device of claim 16, wherein the probability for the at least one candidate word of the at least one of the plurality of n-grams is decreased.

19. A computer-implemented method for performing voice recognition, at least in part, using a language model incorporating a plurality of words and a plurality of n-grams, each of the plurality of n-grams formed by two or more of the plurality of words and defining a probability of occurrence of each word in the respective n-gram given the occurrence of one of the two or more of the plurality of words forming the respective n-gram, the method comprising:

receiving a first text comprising a sequence of one or more words from a first participant in a text-based conversation with a second participant via at least one first user device;

receiving a voice response provided by the second participant in the text-based conversation in response to the first text;

selecting at least one of the plurality of n-grams from the language model that includes at least one word in the first text; and

automatically recognizing at least a portion of the voice response to provide a second text, at least in part, by using the at least one selected n-gram, wherein automatically recognizing comprises determining whether a number of messages from the first participant to the second participant exceeds a predetermined threshold.

20. The computer-implemented method of claim 19, further comprising adjusting the probability for at least one word in the at least one selected n-gram based, at least in part, on the first text.

21. The computer-implemented method of claim 19, wherein when the number of messages from the first participant to the second participant exceeds a predetermined threshold, automatically recognizing the at least a portion of the voice response includes using a history of recognized words and received texts between the first participant and the second participant.

22. At least one non-transitory computer-readable storage medium containing a program which, when executed on at least one computer, performs a method of performing voice recognition, at least in part, using a language model incorporating a plurality of words and a plurality of n-grams, each of the plurality of n-grams formed by two or more of the plurality of words and defining a probability of occurrence of each word in the respective n-gram given the occurrence of one of the two or more of the plurality of words forming the respective n-gram, the method comprising:

receiving a first text comprising a sequence of one or more words from a first participant in a text-based conversation with a second participant via at least one first user device;

receiving a voice response provided by the second participant in the text-based conversation in response to the first text;

selecting at least one of the plurality of n-grams from the language model that includes at least one word in the first text; and

automatically recognizing at least a portion of the voice response to provide a second text, at least in part, by using the at least one selected n-gram, wherein automatically recognizing comprises determining whether a number of messages from the first participant to the second participant exceeds a predetermined threshold.

23. The at least one non-transitory computer-readable storage medium of claim 22, further comprising adjusting the probability for at least one word in the at least one selected n-gram based, at least in part, on the first text.

24. The at least one non-transitory computer-readable storage medium of claim 22, wherein when the number of messages from the first participant to the second participant exceeds a predetermined threshold, automatically recognizing the at least a portion of the voice response includes using a history of recognized words and received texts between the first participant and the second participant.

25. A computing device that performs voice recognition, at least in part, using a language model incorporating a plurality of words and a plurality of n-grams, each of the plurality of n-grams formed by two or more of the plurality of words and defining a probability of occurrence of each word in the respective n-gram given the occurrence of one of the two or more of the plurality of words forming the respective n-gram, the computing device comprising:

at least one input to receive a first text comprising a sequence of one or more words from a first participant in a text-based conversation with a second participant via at least one user device and receive a voice response provided by the second participant in the text-based conversation in response to the first text; and

at least one processor coupled to the at least one input, the at least one processor programmed to;

select at least one of the plurality of n-grams from the language model that includes at least one word in the first text; and

automatically recognize at least a portion of the voice response to provide a second text, at least in part, by using the at least one selected n-gram,

wherein the at least one processor is programmed to automatically recognize at least a portion of the voice response at least in part by determining whether a number of messages from the first participant to the second participant exceeds a predetermined threshold.

26. The computing device of claim 25, wherein the at least one processor is configured to adjust the probability for at least one word in the at least one selected n-gram based, at least in part, on the first text.

27. The computing device of claim 25, wherein when the number of messages from the first participant to the second participant exceeds a predetermined threshold, the at least one processor is configured to automatically recognize the at least a portion of the voice response includes using a history of recognized words and received texts between the first participant and the second participant.