Adaptation of automatic speech recognition acoustic models

• US 8,155,961 B2
• Filed: 12/09/2008
• Issued: 04/10/2012
• Est. Priority Date: 12/09/2008
• Status: Expired due to Fees

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Tian, Jilei
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US12/330,921
• Publication Number:
  US 20100145699A1
• Time in Patent Office: 1,218 Days
• Field of Search: 704/257
• US Class Current: 704/257
• CPC Class Codes: G10L 15/07 to the speaker

Abstract

A user terminal may determine a phoneme distribution of a text corpus, determine an acoustic model gain distribution of phonemes of an acoustic model before and after adaptation of the acoustic model, determine a desired phoneme distribution based on the phoneme distribution and the acoustic model gain distribution, generate an adaptation sentence based on the desired phoneme distribution, and generate a prompt requesting a user speak the adaptation sentence.

Claims

1. A method comprising:

determining a phoneme distribution of a text corpus;

determining an acoustic model gain distribution of phonemes of an acoustic model before and after adaptation of the acoustic model;

determining, by a processor, a desired phoneme distribution based on the phoneme distribution and the acoustic model gain distribution;

generating an adaptation sentence based on the desired phoneme distribution; and

generating a prompt requesting a user speak the adaptation sentence.

2. The method of claim 1, further comprising adapting the acoustic model to generate an updated acoustic model based on updating statistical representations of the phonemes from speech input of the user speaking the adaptation sentence.

3. The method of claim 2, further comprising determining an updated acoustic model gain distribution based on the phonemes of the acoustic model and phonemes of the updated acoustic model.

4. The method of claim 3, further comprising determining that a similarity measure based on the updated acoustic model gain distribution satisfies a stopping criterion to end adaptation of the updated acoustic model.

5. The method of claim 1, wherein the acoustic model gain distribution is a similarity measure that measures similarity between the phonemes of the acoustic model before and after adaptation.

6. The method of claim 1, wherein the generation of the adaptation sentence comprises selecting a candidate adaptation sentence from a list of candidate adaptation sentences as the adaptation sentence.

7. The method of claim 1, wherein the generating of the adaptation sentence further comprises:

determining a plurality of candidate adaptation sentence phoneme distributions of a plurality of candidate adaptation sentences;

identifying a first candidate adaptation sentence of the plurality of candidate adaptation sentences that has a minimal cross entropy measure with the desired phoneme distribution; and

selecting the first candidate adaptation sentence as the adaptation sentence.

8. The method of claim 1, wherein the generation of the adaptation sentence further comprises identifying a word path that optimizes an accumulative score through a plurality of word list segments of a vocabulary.

9. The method of claim 1, wherein the generation of the adaptation sentence comprises modeling connections between words in successive word lists of a vocabulary as a bigram to determine a relationship between the words.

10. The method of claim 1, wherein the generation of the adaptation sentence comprises applying a finite state grammar to provide structure to the adaptation sentence.

11. The method of claim 1, wherein the phoneme distribution is calculated using a unigram language model.

12. An apparatus comprising:

a processor; and

a memory coupled to the processor and storing computer readable instructions, the memory and the instructions configured to, with the processor, cause the apparatus at least to;

determine a phoneme distribution of a text corpus;

determine an acoustic model gain distribution of phonemes of an acoustic model before and after adaptation of the acoustic model;

determine a desired phoneme distribution based on the phoneme distribution and the acoustic model gain distribution;

generate an adaptation sentence based on the desired phoneme distribution; and

generate a prompt requesting a user speak the adaptation sentence.

13. The apparatus of claim 12, wherein the computer readable instructions, when executed, cause the apparatus to adapt the acoustic model to generate an updated acoustic model based on updating statistical representations of the phonemes from speech input of the user speaking the adaptation sentence.

14. The apparatus of claim 13, wherein the computer readable instructions, when executed, cause the apparatus to determine an updated acoustic model gain distribution based on the phonemes of the acoustic model and phonemes of the updated acoustic model.

15. The apparatus of claim 14, wherein the computer readable instructions, when executed, cause the apparatus to determine that a similarity measure based on the updated acoustic model gain distribution satisfies a stopping criterion to end adaptation of the updated acoustic model.

16. The apparatus of claim 12, wherein the computer readable instructions, when executed, cause the apparatus to select a candidate adaptation sentence from a list of candidate adaptation sentences as the adaptation sentence.

17. The apparatus of claim 12, wherein the computer readable instructions for the generation of the adaptation sentence, when executed, cause the apparatus to:

determine a plurality of candidate adaptation sentence phoneme distributions of a plurality of candidate adaptation sentences;

identify a first candidate adaptation sentence of the plurality of candidate adaptation sentences that has a minimal cross entropy measure with the desired phoneme distribution; and

select the first candidate adaptation sentence as the adaptation sentence.

18. The apparatus of claim 12, wherein the computer readable instructions, when executed, cause the apparatus to identify a word path that optimizes an accumulative score through a plurality of word list segments of a vocabulary.

19. The apparatus of claim 12, wherein the computer readable instructions, when executed, cause the apparatus to model connections between words in successive word lists of a vocabulary as a bigram to determine a relationship between the words.

20. The apparatus of claim 12, wherein the computer readable instructions, when executed, cause the apparatus to apply a finite state grammar to provide structure to the adaptation sentence.

21. One or more non-transitory computer readable media storing computer-executable instructions which, when executed by a processor, cause an apparatus at least to:

determine a phoneme distribution of a text corpus;

determine an acoustic model gain distribution of phonemes of an acoustic model before and after adaptation of the acoustic model;

determine a desired phoneme distribution based on the phoneme distribution and the acoustic model gain distribution;

generate an adaptation sentence based on the desired phoneme distribution; and

generate a prompt requesting a user speak the adaptation sentence.

22. The one or more computer readable media of claim 21, storing further computer-executable instructions which, when executed, cause the apparatus to adapt the acoustic model to generate an updated acoustic model based on updating statistical representations of the phonemes from speech input of the user speaking the adaptation sentence.

23. The one or more computer readable media of claim 22, storing further computer-executable instructions which, when executed, cause the apparatus to determine an updated acoustic model gain distribution based on the phonemes of the acoustic model and phonemes of the updated acoustic model.

24. The one or more computer readable media of claim 23, storing further computer-executable instructions which, when executed, cause the apparatus to determine that a similarity measure based on the updated acoustic model gain distribution satisfies a stopping criterion to end adaptation of the updated acoustic model.

25. The one or more computer readable media of claim 21, storing further computer-executable instructions which, when executed, cause the apparatus to select a candidate adaptation sentence from a list of candidate adaptation sentences as the adaptation sentence.

26. The one or more computer readable media of claim 21, storing further computer-executable instructions, for the generation of the adaptation sentence, that, when executed, cause the apparatus to:

determine a plurality of candidate adaptation sentence phoneme distributions of a plurality of candidate adaptation sentences;

identify a first candidate adaptation sentence of the plurality of candidate adaptation sentences that has a minimal cross entropy measure with the desired phoneme distribution; and

select the first candidate adaptation sentence as the adaptation sentence.

27. The one or more computer readable media of claim 21, storing further computer-executable instructions which, when executed, cause the apparatus to identify a word path that optimizes an accumulative score through a plurality of word list segments of a vocabulary.

28. The one or more computer readable media of claim 21, storing further computer-executable instructions which, when executed, cause the apparatus to model connections between words in successive word lists of a vocabulary as a bigram to determine a relationship between the words.

29. The one or more computer readable media of claim 21, storing further computer-executable instructions which, when executed, cause the apparatus to apply a finite state grammar to provide structure to the adaptation sentence.

30. An apparatus comprising:

means for determining a phoneme distribution of a text corpus;

means for determining an acoustic model gain distribution of phonemes of an acoustic model before and after adaptation of the acoustic model;

a processor configured for determining a desired phoneme distribution based on the phoneme distribution and the acoustic model gain distribution;

means for generating an adaptation sentence based on the desired phoneme distribution; and

means for generating a prompt requesting a user speak the adaptation sentence.