Generating a task-adapted acoustic model from one or more supervised and/or unsupervised corpora

• US 20030182120A1
• Filed: 03/20/2002
• Published: 09/25/2003
• Est. Priority Date: 03/20/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hwang, Mei Yuh
• Granted Patent: US 7,031,918 B2
• Time in Patent Office: Days
• US Class Current: 704/251
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models

Abstract

Unsupervised speech data is provided to a speech recognizer that recognizes the speech data and outputs a recognition result along with a confidence measure for each recognized word. A task-related acoustic model is generated based on the recognition result, the speech data and the confidence measure. The speech data can be weighted by the confidence measure in generating the acoustic model so that only data that has been recognized with a high degree of confidence will weigh heavily in generation of the acoustic model.

Claims

1. An acoustic model (AM) for use in a speech recognition system, the AM comprising:

parameters indicative of unsupervised data weighted by a speech recognition confidence measure applied at a sub-utterance level.

2. The AM of claim 1 wherein the parameters are indicative of data weighted by the SR confidence measure at a word level.

3. The AM of claim 1 wherein the parameters are generated according to a Baum-Welch method having gamma counts weighted by the SR confidence measure.

4. The AM of claim 1 wherein the parameters are indicative of unsupervised task-dependent data, relevant to a desired task to be performed by the speech recognition system.

5. The AM of claim 4 wherein the parameters are further indicative of supervised, weighted relative to the unsupervised task-dependent data based on a volume of supervised data and unsupervised data used to generate the AM.

6. The AM of claim 5 wherein the parameters are further indicative of the supervised data weighted according to its relevance to the desired task.

7. The AM of claim 1 wherein the parameters are further indicative of the unsupervised data, weighted according to its relevance to a desired task to be performed by the speech recognition system.

10. The method of claim 1 and further comprising:

combining the confidence measure weighted AM with a supervised AM generated from supervised speech data to obtain a composite AM.

11. The method of claim 10 wherein combining comprises:

weighting a contribution of the confidence measure weighted AM and the supervised AM to the composite AM based on volume weights indicative of an amount of supervised and unsupervised speech data used to generate the AMs.

12. The method of claim 11 wherein the AMs each include Gaussian means and variances and wherein weighting comprises:

computing a volume weight for each Gaussian means based on a volume of data used to generate that Gaussian mean;

applying the volume weights computed to each Gaussian; and

combining the weighted Gaussian means for the confidence measure weighted AM and the supervised AM.

13. The method of claim 12 wherein the weighted Gaussian means comprises:

averaging the weighted Gaussian means.

14. The method of claim 12 wherein weighting comprises:

merging the Gaussian variances of the confidence measure weighed AM and the supervised AM at a count level.

15. The method of claim 10 wherein the unsupervised speech data comprises task-dependent data, relevant to a desired task to be performed by the speech recognition system.

16. The method of claim 15 wherein the supervised speech data comprises task-dependent supervised data, relevant to the desired task.

17. The method of claim 15 wherein the supervised speech data comprises task-independent data and wherein combining the confidence measure weighted AM with the supervised AM comprises:

generating the supervised AM based on a relevance of each word in the task-independent data to the desired task.

8. A method of generating an acoustic model (AM) for use in a speech recognition system, corresponding:

receiving unsupervised speech data as utterances formed of sub-utterances units, the utterances being represented by acoustic data;

generating a transcription for each utterance in the unsupervised speech data and a confidence measure for each sub-utterance unit, with a speech recognizer; and

generating a confidence measure weighted AM based on the acoustic data and the transcriptions weighted by the confidence measures on a sub-utterance level.

9. The method of claim 8 wherein generating the confidence measure comprises:

generating the confidence measure for each word in the utterance.

18. The method of claim 8 wherein the unsupervised speech data comprises task-independent speech data and further comprising:

generating a relevance measure for each word in the task-independent data, the relevance measure being indicative of a relevance of the word to a desired task to be performed by the speech recognition system.

19. The method of claim 18 wherein generating the confidence measure weighted AM comprises:

generating a composite AM based on the transcription weighted by the confidence measures and based on the relevance measure for each word in the unsupervised, task-independent speech data.

20. A method of generating an acoustic model (AM) for a speech recognition system, comprising:

receiving a task-dependent (TD) AM generated from task-dependent speech data, relevant to a desired task to be performed by the speech recognition system;

receiving a task-independent (TI) AM generated from task-independent speech data, the TI AM and the TD AM each including Gaussian means and variances; and

combining the Gaussian means and variances based on an amount of data used to generate each mean and each variance to obtain a composite AM.

21. The method of claim 20 wherein combining comprises:

applying data volume weights to each Guassian mean to obtain weighted Gaussian means, the data volume weight being indicative of the amount of data used to generate a corresponding Gaussian mean.

22. The method of claim 21 wherein combining further comprises:

averaging the weighted Gaussian means.

23. The method of claim 20 wherein combining comprises:

merging counts of the Gaussian variances for the TI AM and the TD AM.

24. An acoustic model (AM) generation system, comprising:

a speech recognizer receiving unsupervised speech data in the form of utterances with sub-utterance units and generating a transcription of the utterances and a confidence measure associated with each sub-utterance unit; and

an AM generator receiving the transcription and confidence measures and generating a confidence measure AM by weighting each word with its confidence measure.

26. The AM generation system of claim 24 and further comprising:

a task relevance AM generator receiving supervised task-independent (TI) speech data and generating a task relevance AM based on a relevance of each word in the TI speech data to a desired task for the AM.

27. The method of claim 26 and further comprising:

a composite AM generator generating a composite AM based on the task relevance AM and the confidence measure AM.

28. The method of claim 26 wherein the composite AM generator is configured to weight contributions of the relevance AM and the confidence measure AM based on amount of data used to generate the relevance AM and the confidence measure AM.

29. The method of claim 28 wherein the relevance AM and confidence measure AM each have Gaussian means and variances.

30. The method of claim 29 wherein the composite AM generator generates the composite AM by weighting each individual Gaussian means based on the amount of data used to generate the mean and combining the weighted means from the relevance AM and confidence measure AM.

31. The method of claim 30 wherein the composite AM generator generates the composite AM by combining counts associated with each variance in the relevance AM and the confidence measure AM.

32. The method of claim 24 wherein the unsupervised speech data comprises:

task-independent (TI) data, and further comprising;

a relevance generator generating a relevance measure for each word in the TI data, the relevance for each word in the TI data, the relevance measure being indicative of a relevance of the word to a desired task for the AM.

33. The method of claim 32 wherein the AM generator generates the confidence measure AM by weighting words in the TI data according to relevance measures associated with the words.

25. The AM generation system of cliam 24 wherein the AM generator comprises:

a weighting component weighting each word with its associated confidence measure; and

an AM training component training the confidence measure AM based on the weighted words.