Method for adapting a codebook for speech recognition

• US 8,346,551 B2
• Filed: 11/20/2009
• Issued: 01/01/2013
• Est. Priority Date: 11/21/2008
• Status: Active Grant

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Herbig, Tobias, Gerl, Franz
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US12/622,717
• Publication Number:
  US 20100138222A1
• Time in Patent Office: 1,138 Days
• Field of Search: 704/244
• US Class Current: 704/244
• CPC Class Codes: G10L 15/065 Adaptation

Abstract

A method for adapting a codebook for speech recognition, wherein the codebook is from a set of codebooks comprising a speaker-independent codebook and at least one speaker dependent codebook. A speech input is received and a feature vector based on the received speech input is determined. For each of the Gaussian densities, a first mean vector is estimated using an expectation process and taking into account the determined feature vector. For each of the Gaussian densities, a second mean vector using an Eigenvoice adaptation is determined taking into account the determined feature vector. For each of the Gaussian densities, the mean vector is set to a convex combination of the first and the second mean vector. Thus, this process allows for adaptation during operation and does not require a lengthy training phase.

Claims

1. A computer-implemented method for adapting a codebook for speech recognition, wherein the codebook is from a set of codebooks comprising a speaker-independent codebook and at least one speaker-dependent codebook, each codebook including a set of Gaussian densities, each Gaussian density being parameterized by a mean vector and a covariance matrix, the computer-implemented method comprising:

(a) receiving a speech input into a processor;

(b) determining a feature vector based on the received speech input;

(c) for each of the Gaussian densities, estimating a first mean vector using an expectation process and taking into account the determined feature vector;

(d) for each of the Gaussian densities, estimating a second mean vector using an Eigenvoice adaptation and taking into account the determined feature vector; and

(e) for each of the Gaussian densities, setting its mean vector to a convex combination of the first and the second mean vector, wherein the coefficient of the convex combination is determined individually for each of the Gaussian densities.

2. The method according to claim 1, wherein the expectation process is a maximum likelihood linear regression process or a maximum a posteriori process.

3. The method according, to claim 1, wherein the coefficient of the convex combination

is a function of the number of feature vectors assigned to the respective Gaussian density.

4. The method according to claim 1, further comprising:

performing speech recognition on the feature vector, wherein a confidence measure is determined with respect to the recognized feature vector, and wherein the coefficient of the convex combination is a function of the confidence measure.

5. A method according to claim 1, wherein steps (c) to (e) are preceded by the step of selecting the codebook from a set of codebooks according to a predetermined criterion.

6. A according to claim 5, wherein selecting further comprises:

identifying a speaker corresponding to the speech input and selecting the codebook corresponding to the identified speaker.

7. A method according to claim 5, wherein selecting comprises:

creating a new speaker-dependent codebook if the speaker corresponding to the speech input is not identified or if the set of codebooks does not contain a codebook corresponding to the identified speaker.

8. A method according to claim 5, wherein selecting comprises:

determining a score for assigning a sequence of feature vectors to each of the codebooks, and

selecting the codebook based on the determined scores.

9. A method according to claim 8, wherein determining a score comprises:

selecting the classes from the speaker-independent codebook to which a number of feature vectors above a predetermined threshold is assigned;

selecting the classes from the at least one speaker-dependent codebook that correspond to the selected classes from the speaker-independent codebook, and for each of the codebooks, determining a score based on the selected classes only.

10. A method according to claim 5, further comprising:

creating a new speaker-dependent codebook if the selected codebook based on the determined scores is the speaker-independent codebook.

11. A method according to claim 5, wherein selecting is performed using a Viterbi process.

12. A method according to claim 1, wherein (b) comprises processing the feature vector to reduce distortions to obtain a corrected feature vector.

13. A method according to claim 12, wherein processing is based on a conditional Gaussian Mixture Model obtained via a minimum mean square error estimate.

14. A computer program product including a non-transitory computer-readable medium having computer code thereon for adapting a codebook for speech recognition, wherein the

codebook is from a set of codebooks comprising a speaker-independent codebook and at least one speaker-dependent codebook, each codebook a set of Gaussian densities, the computer code comprising:

(a) computer code for receiving a speech input;

(b) computer code for determining a feature vector based on the received speech input;

(c) computer code, for each of the Gaussian densities, for estimating a first mean vector using an expectation process and taking into account the determined feature vector;

(d) computer code, for each of the Gaussian densities, for estimating a second mean vector using an Eigenvoice adaptation and taking into account the determined feature vector; and

(e) computer code, for each of the Gaussian densities, for setting its mean vector to a convex combination of the first and the second mean vector, wherein the coefficient of the convex combination is determined individually for each of the Gaussian densities.

15. The computer program product according to claim 14, wherein the computer code for the expectation process is a maximum likelihood linear regression or a maximum a posteriori process.

16. The computer program product according to claim 14, wherein the coefficient of the convex combination is a function of the number of feature vectors assigned to the respective Gaussian density.

17. The computer program product according to claim 14, further comprising:

computer code for performing speech recognition on the feature vector, wherein a confidence measure is determined with respect to the recognized feature vector, and wherein the coefficient of the convex combination is a function of the confidence measure.

18. A computer program product according to claim 14, wherein the computer code for (c) to (e) are preceded by computer code for selecting the codebook from a set of codebooks according to a predetermined criterion.

19. A computer program product according to claim 18, wherein the computer code for selecting comprises identifying a speaker corresponding to the speech input and selecting the codebook corresponding to the identified speaker.

20. A computer program product according to claim 18, wherein the computer code for selecting comprises creating a new speaker-dependent codebook if the speaker corresponding to the speech input is not identified or if the set of codebooks does not contain a codebook corresponding to the identified speaker.

21. A computer program product according to claim 18, wherein the computer code for selecting comprises:

computer code for determining a score for assigning a sequence of feature vectors to each of the codebooks, and

computer code for selecting the codebook based on the determined scores.

22. A computer program product according to claim 21, wherein the computer code for determining a score comprises:

computer code for selecting the classes from the speaker-independent codebook to which a number of feature vectors above a predetermined threshold is assigned;

computer code for selecting the classes from the at least one speaker-dependent codebook that correspond to the selected classes from the speaker-independent codebook, and

computer code, for each of the codebooks, for determining a score based on the selected classes only.

23. A computer program product according to claim 18, further comprising:

computer code for creating a new speaker-dependent codebook if the selected codebook based on the determined scores is the speaker-independent codebook.

24. A computer program product according to claim 18, wherein the computer code for selecting is performed using a Viterbi process.

25. A computer program product according to claim 14, wherein the computer code in (b) comprises computer code for processing the feature vector to reduce distortions to obtain a corrected feature vector.

26. A computer program product according to claim 25, wherein the computer code for processing is based on a conditional Gaussian Mixture Model obtained via a minimum mean square error estimate.