Meta-data inputs to front end processing for automatic speech recognition

• US 9,953,638 B2
• Filed: 06/28/2012
• Issued: 04/24/2018
• Est. Priority Date: 06/28/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Willett, Daniel, Lf, Karl Jonas, Pan, Yue, Pinto, Joel, Gollan, Christian
• Primary Examiner(s): ALBERTALLI, BRIAN LOUIS
• Application Number:
  US14/411,236
• Publication Number:
  US 20150262575A1
• Time in Patent Office: 2,126 Days
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/14 using statistical models, e.g. Hidden Markov Models [HMMs] G10L15/18 takes precedence
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 25/30 using neural networks

Abstract

A computer-implemented method is described for front end speech processing for automatic speech recognition. A sequence of speech features which characterize an unknown speech input provided on an audio input channel and associated meta-data which characterize the audio input channel are received. The speech features are transformed with a computer process that uses a trained mapping function controlled by the meta-data, and automatic speech recognition is performed of the transformed speech features.

Claims

1. A method comprising:

receiving, by a computing device, a sequence of speech features that characterize an unknown speech input provided on an audio input channel controlled by an application executing on the computing device;

receiving meta-data that characterizes the audio input channel, an audio codec applied when generating the sequence of speech features, and a type of the application;

transforming the sequence of speech features using one or more trained mapping functions including a feature-space maximum mutual information (fMMI) mapping function, the one or more trained mapping functions controlled by the meta-data that characterizes the audio input channel, the audio codec applied when generating the sequence of speech features, and the type of the application, the fMMI mapping function using neural network based posterior estimates that use the meta-data as input; and

performing automatic speech recognition of the transformed speech features.

2. The method of claim 1, wherein the one or more trained mapping functions include at least one of a multilayered perceptron (MLP) mapping function or a deep belief network (DBN) mapping function.

3. The method of claim 2, wherein the MLP mapping function comprises a mapping function trained on segmentation for the sequence of speech features and the meta-data.

4. The method of claim 1, wherein the fMMI mapping function comprises a mapping function trained on MLP output posteriors of the MLP mapping function.

5. The method of claim 1, wherein the application executing on the computing device comprises a messaging application, and the meta-data characterizes the messaging application.

6. The method of claim 1, wherein the application executing on the computing device comprises a voice-search application, and the meta-data characterizes the voice-search application.

7. The method of claim 1, wherein the application executing on the computing device comprises a dictation application, and the meta-data characterizes the dictation application.

8. The method of claim 1, wherein transforming the sequence of speech features comprises normalizing the sequence of speech features to reduce channel impact of the audio input channel.

9. The method of claim 1, wherein transforming the sequence of speech features comprises normalizing the sequence of speech features to reduce speaker impact of a speaker of the unknown speech input.

10. The method of claim 1, wherein transforming the sequence of speech features comprises normalizing the sequence of speech features to reduce environmental impact of a source environment in which the unknown speech input is generated.

11. The method of claim 1, wherein the meta-data characterizes a source environment in which the unknown speech input is generated.

12. The method of claim 1, wherein the meta-data characterizes a microphone type of the audio input channel.

13. One or more non-transitory computer-readable media storing executable instructions that, when executed by a processor, cause a system to:

receive a sequence of speech features that characterize an unknown speech input provided on an audio input channel controlled by an application executing on the system;

receive meta-data that characterizes the audio input channel, an audio codec applied when generating the sequence of speech features, and a type of the application;

transform the sequence of speech features using one or more trained mapping functions including a feature-space maximum mutual information (fMMI) mapping function, the one or more trained mapping functions controlled by the meta-data that characterizes the audio input channel, the audio codec applied when generating the sequence of speech features, and the type of the application, the fMMI mapping function using neural network based posterior estimates that use the meta-data as input, wherein transforming the sequence of speech features comprises reducing a dimensionality of the sequence of speech features; and

perform automatic speech recognition of the transformed speech features.

14. The one or more non-transitory computer-readable media of claim 13, wherein the one or more trained mapping functions include a cepstral variance normalization (CVN) mapping function.

15. The one or more non-transitory computer-readable media of claim 13, wherein the one or more trained mapping functions include a linear discriminant analysis (LDA) mapping function.

16. The one or more non-transitory computer-readable media of claim 13, wherein the one or more trained mapping functions include a feature-space maximum likelihood linear regression (fMLLR) mapping function.

17. A system comprising:

at least one processor; and

one or more non-transitory computer-readable media storing executable instructions that, when executed by the at least one processor, cause the system to;

receive a sequence of speech features that characterize an unknown speech input provided on an audio input channel controlled by an application executing on the system;

receive meta-data that characterizes the audio input channel, an audio codec applied when generating the sequence of speech features, a microphone type of the audio input channel, and a type of the application;

transform the sequence of speech features using one or more trained mapping functions including a feature-space maximum mutual information (fMMI) mapping function, the one or more trained mapping functions controlled by the meta-data that characterizes the audio input channel, the audio codec applied when generating the sequence of speech features, the microphone type of the audio input channel, and the type of the application, the fMMI mapping function using neural network based posterior estimates that use the meta-data as input; and

perform automatic speech recognition of the transformed speech features.

18. The system of claim 17, wherein the microphone type is a built-in microphone, and the meta-data characterizes the audio input channel as the built-in microphone.

19. The system of claim 17, wherein the microphone type is an external microphone that is external to the system, and the meta-data characterizes the audio input channel as the external microphone.

20. The system of claim 17, wherein the meta-data further comprises a number representing a size of a data stream after audio encoding of the unknown speech input.