Dynamically biasing language models

• US 9,502,032 B2
• Filed: 10/28/2014
• Issued: 11/22/2016
• Est. Priority Date: 10/08/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Aleksic, Petar, Moreno Mengibar, Pedro J.
• Primary Examiner(s): JACKSON, JAKIEDA R
• Application Number:
  US14/525,826
• Publication Number:
  US 20160104482A1
• Time in Patent Office: 756 Days
• Field of Search: 704/235
• US Class Current: 1/1
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 19/00 Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis in musical instruments G10H

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for speech recognition. In one aspect, a method comprises receiving audio data encoding one or more utterances; performing a first speech recognition on the audio data; identifying a context based on the first speech recognition; performing a second speech recognition on the audio data that is biased towards the context; and providing an output of the second speech recognition.

Claims

1. A method performed by one or more computers, the method comprising:

receiving audio data encoding one or more utterances;

generating a recognition lattice of the one or more utterances by performing speech recognition on the audio data using a first pass speech recognizer;

identifying a specific context for the one or more utterances that is referenced by the recognition lattice of the one or more utterances, generated by performing speech recognition on the audio data using the first pass speech recognizer, based on semantic analysis of the recognition lattice;

in response to identifying the specific context that is referenced by the recognition lattice, selecting a second pass speech recognizer that is biased towards the specific context that is referenced by the recognition lattice of the one or more utterances, generated by performing speech recognition on the audio data using the first pass speech recognizer, based on semantic analysis of the recognition lattice;

in parallel with generating a first transcription of the one or more utterances using the first pass speech recognizer, generating, by an automatic speech recognition engine, a second transcription of the one or more utterances by performing additional speech recognition on the audio data using the second pass speech recognizer that is biased towards the specific context that is referenced by the recognition lattice that was generated by performing speech recognition on the audio data using the first pass speech recognizer; and

providing an output transcription of one of the first transcription of the one or more utterances or the second transcription of the one or more utterances to initiate an operation based on the output transcription.

2. The method of claim 1, wherein generating the recognition lattice comprises generating one or more text phrases that acoustically match the one or more utterances according to a first language model.

3. The method of claim 2, wherein identifying the specific context that is referenced by the recognition lattice comprises determining that at least a first text phrase matches a first title of a first context language model of a plurality of context language models, wherein the first context language model comprises a biasing language model for the specific context.

4. The method of claim 2, wherein identifying the specific context that is referenced by the recognition lattice comprises supplying the recognition lattice to a classifier trained using recognition lattice data.

5. The method of claim 1, comprising determining whether a second pass speech recognizer that is biased towards the specific context that is referenced by the recognition lattice of the one or more utterances is likely to improve recognition results, based on the semantic analysis of the recognition lattice.

6. The method of claim 1, wherein selecting a second pass speech recognizer that is biased towards the specific context that is referenced by the recognition lattice of the one or more utterances generated by performing speech recognition on the audio data using the first pass speech recognizer based on semantic analysis of the recognition lattice comprises:

selecting a context language model for the specific context, the context language model specifying a plurality of potential output phrases associated with the specific context, wherein the number of potential output phrases is fewer than a number of potential output phrases of a general language model;

supplying the context language model, the general language model, and the audio data to an automatic speech recognition engine.

7. The method of claim 6, comprising supplying respective weights for the context language model and the general language model to the automatic speech recognition engine based on the recognition lattice.

8. The method of claim 6, wherein supplying the context language model and the general language model to the automatic speech recognition engine comprises combining the context language model and the general language model into a combined language model and supplying the combined language model to the automatic speech recognition engine.

9. The method of claim 1, wherein performing speech recognition on the audio data using a first pass speech recognizer comprises biasing the first pass speech recognizer.

10. The method of claim 1, comprising determining, based on the semantic analysis of the recognition lattice, that the recognition lattice defines an additional context for the one or more utterances, wherein the second pass speech recognizer is biased towards both the specific context and the additional context.

11. A system comprising:

one or more computers; and

one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving audio data encoding one or more utterances;

generating a recognition lattice of the one or more utterances by performing speech recognition on the audio data using a first pass speech recognizer;

identifying a specific context for the one or more utterances that is referenced by the recognition lattice of the one or more utterances, generated by performing speech recognition on the audio data using the first pass speech recognizer, based on semantic analysis of the recognition lattice;

in response to identifying the specific context that is referenced by the recognition lattice, selecting a second pass speech recognizer that is biased towards the specific context that is referenced by the recognition lattice of the one or more utterances, generated by performing speech recognition on the audio data using the first pass speech recognizer, based on semantic analysis of the recognition lattice;

in parallel with generating a first transcription of the one or more utterances using the first pass speech recognizer, generating, by an automatic speech recognition engine, a second transcription of the one or more utterances by performing additional speech recognition on the audio data using the second pass speech recognizer that is biased towards the specific context that is referenced by the recognition lattice that was generated by performing speech recognition on the audio data using the first pass speech recognizer; and

providing an output transcription of one of the first transcription of the one or more utterances or the second transcription of the one or more utterances to initiate an operation based on the output transcription.

12. The system of claim 11, wherein generating the recognition lattice comprises generating one or more text phrases that acoustically match the one or more utterances according to a first language model.

13. The system of claim 12, wherein identifying the specific context that is referenced by the recognition lattice comprises determining that at least a first text phrase matches a first title of a first context language model of a plurality of context language models, wherein the first context language model comprises a biasing language model for the specific context.

14. The system of claim 12, wherein identifying the specific context that is referenced by the recognition lattice comprises supplying the recognition lattice to a classifier trained using recognition lattice data.

15. The system of claim 11, comprising determining whether a second pass speech recognizer that is biased towards the specific context that is referenced by the recognition lattice of the one or more utterances is likely to improve recognition results, based on the semantic analysis of the recognition lattice.

16. The system of claim 11, wherein selecting a second pass speech recognizer that is biased towards the specific context that is referenced by the recognition lattice of the one or more utterances generated by performing speech recognition on the audio data using the first pass speech recognizer based on semantic analysis of the recognition lattice comprises:

selecting a context language model for the specific context, the context language model specifying a plurality of potential output phrases associated with the specific context, wherein the number of potential output phrases is fewer than a number of potential output phrases of a general language model;

supplying the context language model, the general language model, and the audio data to an automatic speech recognition engine.

17. The system of claim 16, the operations comprising supplying respective weights for the context language model and the general language model to the automatic speech recognition engine based on the recognition lattice.

18. The system of claim 16, wherein supplying the context language model and the general language model to the automatic speech recognition engine comprises combining the context language model and the general language model into a combined language model and supplying the combined language model to the automatic speech recognition engine.

19. The system of claim 16, wherein performing speech recognition on the audio data using a first pass speech recognizer comprises biasing the first pass speech recognizer.

20. The system of claim 11, the operations comprising determining, based on the semantic analysis of the recognition lattice, that the recognition lattice defines an additional context for the one or more utterances, wherein the second pass speech recognizer is biased towards both the specific context and the additional context.