Sequential, nonparametric speech recognition and speaker identification

• US 6,029,124 A
• Filed: 03/31/1998
• Issued: 02/22/2000
• Est. Priority Date: 02/21/1997
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Dragon Systems, Inc. (Microsoft Corporation)
• Inventors: Newman, Michael J., Corrada-Emmanuel, Andres, Peskin, Barbara R., Gillick, Laurence S.
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Wieland, Susan
• Application Number:
  US09/050,428
• Time in Patent Office: 693 Days
• Field of Search: 704/200, 704/240, 704/241, 704/245, 704/246, 704/238, 704/231, 704/250, 704/251
• US Class Current: 704/200
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/04 Training, enrolment or model building
• CPC Class Codes: G10L 17/14 Use of phonemic categorisation or speech recognition prior to speaker recognition or verification

Abstract

A speech sample is evaluated using a computer. Training data that include samples of speech are received and stored along with identification of speech elements to which portions of the training data are related. A speech sample is received and speech recognition is performed on the speech sample to produce recognition results. Finally, the recognition results are evaluated in view of the training data and the identification of the speech elements to which the portions of the training data are related. The technique may be used to perform tasks such as speech recognition, speaker identification, and language identification.

Claims

1. A method of evaluating a speech sample using a computer, the method comprising:

receiving training data comprising samples of speech;

storing the training data along with identification of speech elements to which portions of the training data are related;

receiving a speech sample;

performing speech recognition on the speech sample to produce recognition results; and

evaluating the recognition results in view of the training data and the identification of the speech elements to which the portions of the training data are related, wherein;

performing speech recognition on the speech sample comprises identifying portions of the speech sample to which different speech elements correspond; and

evaluating the recognition results comprises comparing a portion of the speech sample identified as corresponding to a particular speech element to one or more portions of the training data identified as corresponding to the particular speech element.

2. The method of claim 1, further comprising analyzing the training data to designate different portions of the training data as being related to different speech elements.

3. The method of claim 2, wherein analyzing the training data comprises performing large vocabulary continuous speech recognition on the training data to produce training data recognition results and a training data time alignment, the training data recognition results including a sequence of speech elements and the training data time alignment identifying portions of the training data to which each speech element corresponds.

4. The method of claim 1, wherein comparing comprises directly comparing the portion of the speech sample to a particular speech sample from the training data.

5. The method of claim 1, wherein performing speech recognition on the speech sample to produce recognition results comprises producing speech sample recognition results and a speech sample time alignment, the speech sample recognition results including a sequence of speech elements and the speech sample time alignment identifying portions of the speech sample to which each speech element corresponds.

6. The method of claim 1, wherein evaluating the recognition results comprises evaluating accuracy of the recognition results.

7. The method of claim 6, wherein the recognition results comprise an ordered list of hypotheses about contents of the speech sample, and wherein evaluating the recognition results comprises reordering the list as a result of the evaluated accuracy of each hypothesis.

8. The method of claim 1, wherein the training data are associated with a known speaker and evaluating the recognition results comprises determining a likelihood that the known speaker produced the speech sample.

9. The method of claim 1, wherein the training data are associated with different known speakers and evaluating the recognition results comprises determining which of the known speakers is more likely to have produced the speech sample.

10. The method of claim 1, wherein the training data are associated with a known language and evaluating the recognition results comprises determining a likelihood that the speech sample was spoken in the known language.

11. The method of claim 1, wherein the recognition results identify speech elements to which different portions of the speech sample correspond and evaluating the recognition results comprises comparing a portion of the speech sample identified as corresponding to a particular speech element to one or more portions of the training data identified as corresponding to the particular speech element.

12. The method of claim 11, wherein evaluating the recognition results comprises:

selecting a speech element from the recognition results;

comparing a portion of the speech sample corresponding to the selected speech element to a portion of the training data identified as corresponding to the selected speech element;

generating a speech element score as a result of the comparison;

repeating the selecting, comparing and generating steps for each speech element in the recognition results; and

producing an evaluation score for the recognition results based on the speech element scores for the speech elements in the recognition results.

13. The method of claim 12, wherein the compared portion of the speech sample and the compared portion of the training data are each represented by frames of data and wherein comparing the portions comprises aligning the frames of the portions using a dynamic programming technique that seeks to minimize an average distance between aligned frames.

14. The method of claim 13, wherein generating the speech element score comprises generating the score based on distances between aligned frames.

15. The method of claim 12, wherein producing the evaluation score for the recognition results comprises producing the evaluation score based on a subset of the speech element scores.

16. The method of claim 15, wherein producing the evaluation score for the recognition results comprises producing the evaluation score based on a best-scoring subset of the speech element scores.

17. The method of claim 12, further comprising normalizing the evaluation score.

18. The method of claim 17, wherein normalizing the evaluation score comprises:

receiving cohort data, the cohort data comprising samples of speech for a cohort of speakers;

storing the cohort data along with identification of speech elements to which portions of the cohort data are related;

evaluating the recognition results in view of the cohort data and the identification of the speech elements to which the portions of the cohort data are related to produce a cohort score; and

modifying the evaluation score based on the cohort score.

19. The method of claim 1, wherein performing speech recognition comprises performing large vocabulary continuous speech recognition.

20. The method of claim 1, wherein the training data comprises individual speech samples, each of which represents a single speech element uttered by a single speaker.

21. The method of claim 20, wherein evaluating recognition results comprises directly comparing the speech sample to individual speech samples from the training data corresponding to speech elements identified by the recognition results.

23. The computer program of claim 20, further comprising instructions for causing the processor to analyze the training data to designate different portions of the training data as being related to different speech elements.

22. A computer program, residing on a computer readable medium, for a system comprising a processor and an input device, the computer program comprising instructions for evaluating a speech sample by causing the processor to perform the following operations:

receive training data comprising samples of speech;

store the training data along with identification of speech elements to which portions of the training data are related;

receive a speech sample;

perform speech recognition on the speech sample to produce recognition results; and

evaluate the recognition results in view of the training data and the identification of the speech elements to which the portions of the training data are related,

wherein;

analyzing the training data comprises identifying portions of the training data to which different speech elements correspond;

performing speech recognition on the speech sample comprises identifying portions of the speech sample to which different speech elements correspond; and

evaluating the recognition results comprises comparing a portion of the speech sample identified as corresponding to a particular speech element to one or more portions of the training data identified as corresponding to the particular speech element.

24. The computer program of claim 22, wherein the training data are associated with a known speaker and evaluating the recognition results comprises determining a likelihood that the known speaker produced the speech sample.

25. The computer program of claim 22, wherein the training data are associated with different known speakers and evaluating the recognition results comprises determining which of the known speakers is more likely to have produced the speech sample.

26. The computer program of claim 22, wherein the training data are associated with a known language and evaluating the recognition results comprises determining a likelihood that the speech sample was spoken in the known language.

27. The computer program of claim 22, wherein evaluating the recognition results comprises evaluating accuracy of the recognition results.