MULTIPLE PASS AUTOMATIC SPEECH RECOGNITION METHODS AND APPARATUS

• US 20150058018A1
• Filed: 08/23/2013
• Published: 02/26/2015
• Est. Priority Date: 08/23/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Kanthak, Stephan, Georges, Munir Nikolai Alexander
• Granted Patent: US 9,940,927 B2
• Time in Patent Office: Days
• US Class Current: 704/257
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

In some aspects, a method of recognizing speech that comprises natural language and at least one word specified in at least one domain-specific vocabulary is provided. The method comprises performing a first speech processing pass comprising identifying, in the speech, a first portion including the natural language and a second portion including the at least one word specified in the at least one domain-specific vocabulary, and recognizing the first portion including the natural language. The method further comprises performing a second speech processing pass comprising recognizing the second portion including the at least one word specified in the at least one domain-specific vocabulary.

Claims

1. A method of recognizing speech that comprises natural language and at least one word specified in at least one domain-specific vocabulary, the method comprising:

performing a first speech processing pass comprising;

identifying, in the speech, a first portion including the natural language and a second portion including the at least one word specified in the at least one domain-specific vocabulary; and

recognizing the first portion including the natural language; and

performing a second speech processing pass comprising;

recognizing the second portion including the at least one word specified in the at least one domain-specific vocabulary.

2. The method of claim 1, wherein performing the first speech processing pass comprises:

using at least one language model representing probabilities of transitioning from one or more words to one or more classes corresponding to the at least one domain-specific vocabulary and/or representing probabilities of transitioning from the one or more classes to the one or more words to recognize the first portion including the natural language; and

using at least one filler model to assist in identifying the second portion including the at least one word specified in the at least one domain-specific vocabulary.

3. The method of claim 2, wherein performing the first speech processing pass results in at least one recognized word from the first portion including the natural language and at least one tag associated with the at least one domain-specific vocabulary to identify the second portion, and wherein performing the second speech processing pass comprises recognizing the second portion identified by the at least one tag.

4. The method of claim 2, wherein the at least one filler model includes a first filler model and the at least one domain-specific vocabulary comprises a first domain-specific vocabulary, and wherein the first filler model is trained, at least in part, on data in the first domain-specific vocabulary, and wherein performing the second speech processing pass comprises recognizing a portion having words specified in the first domain-specific vocabulary using at least one grammar associated with the first domain-specific vocabulary.

5. The method of claim 2, wherein the at least one filler model includes at least one phoneme model to determine phoneme likelihoods in the second portion, the at least one phoneme model including a phoneme loop model and/or a phoneme N-gram model.

6. The method of claim 1, wherein the first speech processing pass is performed at a first location on a network and the second speech processing pass is performed at a second location on the network, wherein the first location is different from the second location.

7. The method of claim 2, wherein the second portion including the at least one word specified in the at least one domain-specific vocabulary includes a third portion including at least one word specified in a first domain-specific vocabulary and a fourth portion including at least one word specified in a second domain-specific vocabulary, wherein the at least one filler model includes a first filler model trained at least in part on data in the first domain-specific vocabulary, and a second filler model trained at least in part on data in the second domain-specific vocabulary, and wherein performing the second speech processing pass comprises:

using a first domain-specific speech recognizer adapted to recognize words in the first domain-specific vocabulary to recognize the third portion identified at least in part using the first filler model; and

using a second domain-specific recognizer adapted to recognize words in the second domain-specific vocabulary to recognize the fourth portion identified at least in part using the second filler model.

8. At least one computer readable medium storing instructions that, when executed by at least one processor, cause at least one computer to perform a method of recognizing speech that comprises natural language and at least one word specified in at least one domain-specific vocabulary, the method comprising:

performing a first speech processing pass comprising;

identifying, in the speech, a first portion including the natural language and a second portion including the at least one word specified in the at least one domain-specific vocabulary; and

recognizing the first portion including the natural language; and

performing a second speech processing pass comprising;

recognizing the second portion including the at least one word specified in the at least one domain-specific vocabulary.

9. The at least one computer readable medium of claim 8, wherein performing the first speech processing pass comprises:

using at least one language model representing probabilities of transitioning from one or more words to one or more classes corresponding to the at least one domain-specific vocabulary and/or representing probabilities of transitioning from the one or more classes to the one or more words to recognize the first portion including the natural language; and

using at least one filler model to assist in identifying the second portion including the at least one word specified in the at least one domain-specific vocabulary.

10. The at least one computer readable medium of claim 9, wherein performing the first speech processing pass results in at least one recognized word from the first portion including the natural language and at least one tag associated with the at least one domain-specific vocabulary to identify the second portion, and wherein performing the second speech processing pass comprises recognizing the second portion identified by the at least one tag.

11. The at least one computer readable medium of claim 9, wherein the at least one filler model includes a first filler model and the at least one domain-specific vocabulary comprises a first domain-specific vocabulary, and wherein the first filler model is trained, at least in part, on data in the first domain-specific vocabulary, and wherein performing the second speech processing pass comprises recognizing a portion having words specified in the first domain-specific vocabulary using at least one grammar associated with the first domain-specific vocabulary.

12. The at least one computer readable medium of claim 9, wherein the at least one filler model includes at least one phoneme model to determine phoneme likelihoods in the second portion, the at least one phoneme model including a phoneme loop model and/or a phoneme N-gram model.

13. The at least one computer readable medium of claim 8, wherein the first speech processing pass is performed at a first location on a network and the second speech processing pass is performed at a second location on the network, wherein the first location is different from the second location.

14. The at least one computer readable medium of claim 9, wherein the second portion including the at least one word specified in the at least one domain-specific vocabulary includes a third portion including at least one word specified in a first domain-specific vocabulary and a fourth portion including at least one word specified in a second domain-specific vocabulary, wherein the at least one filler model includes a first filler model trained at least in part on data in the first domain-specific vocabulary, and a second filler model trained at least in part on data in the second domain-specific vocabulary, and wherein performing the second speech processing pass comprises:

using a first domain-specific speech recognizer adapted to recognize words in the first domain-specific vocabulary to recognize the third portion identified at least in part using the first filler model; and

using a second domain-specific recognizer adapted to recognize words in the second domain-specific vocabulary to recognize the fourth portion identified at least in part using the second filler model.

15. A system comprising at least one computer for recognizing speech that comprises natural language and at least one word specified in at least one domain-specific vocabulary, the system comprising:

at least one computer readable medium storing instructions; and

at least one processor programmed by executing the instructions to perform;

a first speech processing pass comprising;

identifying, in the speech, a first portion including the natural language and a second portion including the at least one word specified in the at least one domain-specific vocabulary; and

recognizing the first portion including the natural language; and

a second speech processing pass comprising;

recognizing the second portion including the at least one word specified in the at least one domain-specific vocabulary.

16. The system of claim 15, wherein the at least one processor is programmed to perform the first speech processing pass further comprising:

using at least one language model representing probabilities of transitioning from one or more words to one or more classes corresponding to the at least one domain-specific vocabulary and/or representing probabilities of transitioning from the one or more classes to the one or more words to recognize the first portion including the natural language; and

using at least one filler model to assist in identifying the second portion including the at least one word specified in the at least one domain-specific vocabulary.

17. The system of claim 16, wherein performing the first speech processing pass results in at least one recognized word from the first portion including the natural language and at least one tag associated with the at least one domain-specific vocabulary to identify the second portion, and wherein performing the second speech processing pass comprises recognizing the second portion identified by the at least one tag.

18. The system of claim 16, wherein the at least one filler model includes a first filler model and the at least one domain-specific vocabulary comprises a first domain-specific vocabulary, and wherein the first filler model is trained, at least in part, on data in the first domain-specific vocabulary, and wherein the at least one processor is programmed to perform the second speech processing pass comprising recognizing a portion having words specified in the first domain-specific vocabulary using at least one grammar associated with the first domain-specific vocabulary.

19. The system of claim 16, wherein the at least one filler model includes at least one phoneme model to determine phoneme likelihoods in the second portion, the at least one phoneme model including a phoneme loop model and/or a phoneme N-gram model.

20. The system of claim 15, wherein the first speech processing pass is performed at a first location on a network and the second speech processing pass is performed at a second location on the network, wherein the first location is different from the second location.