DIFFERENTIAL ACOUSTIC MODEL REPRESENTATION AND LINEAR TRANSFORM-BASED ADAPTATION FOR EFFICIENT USER PROFILE UPDATE TECHNIQUES IN AUTOMATIC SPEECH RECOGNITION

• US 20150149174A1
• Filed: 05/08/2012
• Published: 05/28/2015
• Est. Priority Date: 05/08/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Willett, Daniel, Gollan, Christian
• Granted Patent: US 9,406,299 B2
• Time in Patent Office: Days
• US Class Current: 704/246
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 15/285 Memory allocation or algorithm optimisation to reduce hardware requirements
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 17/06 Decision making techniques; Pattern matching strategies
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology

Abstract

A computer-implemented method is described for speaker adaptation in automatic speech recognition. Speech recognition data from a particular speaker is used for adaptation of an initial speech recognition acoustic model to produce a speaker adapted acoustic model. A speaker dependent differential acoustic model is determined that represents differences between the initial speech recognition acoustic model and the speaker adapted acoustic model. In addition, an approach is also disclosed to estimate speaker-specific feature or model transforms over multiple sessions. This is achieved by updating the previously estimated transform using only adaptation statistics of the current session.

Claims

1. A method comprising:

adapting, by a computing device, an initial speech recognition acoustic model to produce a speaker adapted acoustic model using speech recognition data from a particular speaker;

developing, by the computing device, a speaker differential acoustic model representing one or more differences between the initial speech recognition acoustic model and the speaker adapted acoustic model; and

storing the speaker differential acoustic model for subsequent speech recognition with the particular speaker.

2. The method of claim 1, further comprising:

retrieving the stored speaker differential acoustic model; and

processing the speaker differential acoustic model and the initial speech recognition acoustic model to derive the speaker adapted acoustic model.

3. The method of claim 2, further comprising:

using the derived speaker adapted acoustic model to perform speech recognition for the particular speaker.

4. The method of claim 1, wherein the storing the speaker differential acoustic model includes selecting a limited subset of parameter differences for storing to control size of the speaker differential acoustic model.

21. The method of claim 4, wherein the limited subset of parameter differences comprises most important parameter differences of the parameter differences.

22. The method of claim 1, further comprising:

ranking the one or more differences determined using a scoring function;

keeping a subset of the one or more differences, the subset selected based on the rank of the one or more differences;

optimizing different quantization ranges for parameter subsets of the speaker adapted acoustic model using the scoring function;

finding storage-efficient Gaussian component clustering of the speaker differential acoustic model using the scoring function; and

using the subset of the one or more differences, the different quantization ranges, and the storage-efficient Gaussian component clustering to minimize an acoustic model difference measure of the speaker adapted acoustic model against the initial speech recognition acoustic model.

23. The method of claim 3, wherein performing the speech recognition comprises:

producing a representative recognition output of the speech recognition data from the particular speaker, wherein the representative recognition output is constrained by a recognition language model and search algorithm; and

comparing the representative recognition output to the derived speaker adapted acoustic model.

24. The method of claim 1, further comprising:

adapting the speaker adapted acoustic model based on specific technical characteristics of a device used by the particular speaker.

5-8. -8. (canceled)

9. A method comprising:

loading a user profile for a particular speaker including an initial user feature transform representing speaker adapted speech recognition acoustic models;

performing speech recognition, by a computing device, for a session of speech utterances from the particular speaker using the initial user feature transform and a plurality of speaker independent speech recognition acoustic models;

determining a session update transform for a linear transform-based speaker adaptation of the initial user feature transform based on speech recognition data from the session;

producing an updated user feature transform by combining the initial user feature transform and the session update transform; and

storing in the user profile the updated user feature transform for subsequent speech recognition with the particular speaker.

10. The method of claim 9, wherein producing the updated user feature transform by combining the initial user feature transform and the session update transform is performed without a bias vector.

11. The method of claim 9, wherein producing the updated user feature transform by combining the initial user feature transform and the session update transform is performed with a bias vector.

12. The method of claim 11, wherein the bias vector is an acoustic observation vector for constrained maximum likelihood linear regression (CMLLR) adaptation.

13. The method of claim 11, wherein the bias vector is a component mean vector for maximum likelihood linear regression (MLLR) adaptation.

14. The method of claim 9, wherein the linear transform-based speaker adaptation uses relevance smoothing of speech recognition data from the session.

15. A non-transitory computer-readable medium storing computer-readable instructions that, when executed by a processor, cause a device to:

load a user profile for a particular speaker including an initial user feature transform representing speaker adapted speech recognition acoustic models;

perform speech recognition for a session of speech utterances from the particular speaker using the initial user feature transform and a plurality of speaker independent speech recognition acoustic models;

determine a session update transform for a linear transform-based speaker adaptation of the initial user feature transform based on speech recognition data from the session;

produce an updated user feature transform by combining the initial user feature transform and the session update transform; and

store in the user profile the updated user feature transform for subsequent speech recognition with the particular speaker.

16. The non-transitory computer-readable medium of claim 15, wherein producing the updated user feature transform by combining the initial user feature transform and the session update transform is performed without a bias vector.

17. The non-transitory computer-readable medium of claim 16, wherein producing the updated user feature transform by combining the initial user feature transform and the session update transform is performed with a bias vector.

18. The non-transitory computer-readable medium of claim 17, wherein the bias vector is an acoustic observation vector for constrained maximum likelihood linear regression (CMLLR) adaptation.

19. The non-transitory computer-readable medium of claim 17, wherein the bias vector is a component mean vector for maximum likelihood linear regression (MLLR) adaptation.

20. The non-transitory computer-readable medium of claim 15, wherein the linear transform-based speaker adaptation uses relevance smoothing of speech recognition data from the session.