Generating a task-adapted acoustic model from one or more supervised and/or unsupervised corpora

• US 7,031,918 B2
• Filed: 03/20/2002
• Issued: 04/18/2006
• Est. Priority Date: 03/20/2002
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hwang, Mei Yuh
• Primary Examiner(s): Storm, Donald L.
• Application Number:
  US10/103,184
• Publication Number:
  US 20030182120A1
• Time in Patent Office: 1,490 Days
• Field of Search: None
• US Class Current: 704/243
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models

Abstract

Unsupervised speech data is provided to a speech recognizer that recognizes the speech data and outputs a recognition result along with a confidence measure for each recognized word. A task-related acoustic model is generated based on the recognition result, the speech data and the confidence measure. Additional task independent model can be used. The speech data can be weighted by the confidence measure in generating the acoustic model so that only data that has been recognized with a high degree of confidence will weigh heavily in generation of the acoustic model. The acoustic model can be formed from a Gaussian mean and variance of the data.

Claims

1. A method of generating an acoustic model (AM) for use in a speech recognition system, comprising:

receiving unsupervised speech data as utterances formed of sub-utterances units, the utterances being represented by acoustic data and including words;

generatinq a transcription for each utterance in the unsupervised speech data and a confidence measure for each sub-utterance unit, with a speech recognizer;

generating a confidence measure weighted AM based on the acoustic data and the transcriptions weighted by the confidence measures on a sub-utterance level wherein generates the confidence measure for each word in the utterance; and

combining the confidence measure weight AM with a supervised AM generated from supervised speech data to obtain a composite AM.

2. The method of claim 1 wherein combining comprises:

weighting a contribution of the confidence measure weighted AM and the supervised AM to the composite AM based on volume weights indicative of an amount of supervised and unsupervised speech data used to generate the AMs.

3. The method of claim 2 wherein the AMs each include Gaussian means and variances and wherein weighting comprises:

computing a volume weight for each Gaussian means based on a volume of data used to generate that Gaussian mean;

applying the volume weights computed to each Gaussian mean; and

combining the weighted Gaussian means for the confidence measure weighted AM and the supervised AM.

4. The method of claim 3 wherein the weighted Gaussian means comprises:

averaging the weighted Gaussian means.

5. The method of claim 3 wherein weighting comprises:

merging the Gaussian variances of the confidence measure weighed AM and the supervised AM at a count level.

6. The method of claim 1 wherein the unsupervised speech data comprises task-dependent data, relevant to a desired task to be performed by the speech recognition system.

7. The method of claim 6 wherein the supervised speech data comprises task-dependent supervised data, relevant to the desired task.

8. The method of claim 6 wherein the supervised speech data comprises task-independent data including words and wherein combining the confidence measure weighted AM with the supervised AM comprises:

generating the supervised AM based on a relevance of each word in the task-independent data to the desired task.

9. A method of generating an acoustic model (AM) for use in a speech recognition system, comprising:

receiving unsupervised speech data as utterances formed of sub-utterances units, the utterances being represented by acoustic data;

generating a transcription for each utterance in the unsupervised speech data and a confidence measure for each sub-utterance unit, with a speech recognizer;

generating a confidence measure weighted AM based on the acoustic data and the transcriptions weighted by the confidence measure on a sub-utterance level; and

wherein the unsupervised speech data comprises task-independent speech data including words, and further comprising;

generating a relevance measure for each word in the task-independent data, the relevance measure being indicative of a relevance of the word to a desired task to be performed by the speech recognition system.

10. The method of claim 9 wherein generating the confidence measure weighted AM comprises:

generating a composite AM based on the transcription weighted by the confidence measures and based on the relevance measure for each word in the unsupervised, task-independent speech data.

11. A method of generating an acoustic model (AM) for a speech recognition system, comprising:

receiving a task-dependent (TD) AM generated from task-dependent speech data, relevant to a desired task to be performed by the speech recognition system;

receiving a task-independent (TI) AM generated from task-independent speech data, the TI AM and the TD AM each including Gaussian means and variances; and

combining the Gaussian means and variances based on an amount of data used to generate each mean and each variance to obtain a composite AM.

12. The method of claim 11 wherein combining comprises:

applying data volume weights to each Guassian mean to obtain weighted Gaussian means, the data volume weight being indicative of the amount of data used to generate a corresponding Gaussian mean.

13. The method of claim 12 wherein combining further comprises:

averaging the weighted Gaussian means.

14. The method of claim 11 wherein combining comprises:

merging counts of the Gaussian variances for the TI AM and the TD AM.

15. An acoustic model (AM) generation system, comprising:

a speech recognizer receiving unsupervised speech data in the form of utterances with sub-utterance units and words and generating a transcription of the utterances and a confidence measure associated with each sub-utterance unit;

an AM generator receiving the transcription and confidence measures and generating a confidence measure AM by weiqhting each word in the utterances with a confidence measure; and

a task relevance AM generator receiving supervised task-relevance (TI) speech data including words and generating a task relevance AM based on a relevance of each word in the TI speech data to a desired task for the task relevance AM.

16. The system of claim 15 and further comprising:

a composite AM generator generating a composite AM based on the task relevance AM and the confidence measure AM.

17. The system of claim 15 wherein the composite AM generator is configured to weight contributions of the relevance AM and the confidence measure AM based on amount of data used to generate the relevance AM and the confidence measure AM.

18. The system of claim 16 wherein the relevance AM and confidence measure AM each have Gaussian means and variances.

19. The system of claim 18 wherein the composite AM generator generates the composite AM by weighting each individual Gaussian means based on the amount of data used to generate the mean and combining the weighted means from the relevance AM and confidence measure AM.

20. The system of claim 19 wherein the composite AM generator generates the composite AM by combining counts associated with each variance in the relevance AM and the confidence measure AM.

21. An acoustic model (AM) generation system, comprising:

a speech recognizer receiving unsupervised speech data in the form of utterances with sub-utterance units and words generating a transcription of the utterances and a confidence measure associated with each sub-utterance unit;

an AM generator receiving the transcription and confidence measures and generating a confidence measure AM by weighting each word in the utterances with a confidence measure;

wherein the unsupervised speech data comprises a task-independent (TI) data including words

a relevance generator generating a relevance measure for each word in the TI data, the relevance for each word in the TI data, the relevance measure being indicative of a relevance of the word in the TI data to a desired task for the AM.

22. The system of claim 21 wherein the AM generator generates the confidence measure AM by weighting words in the TI data according to relevance measures associated with the words in the TI data.