Method for learning linguistically valid word pronunciations from acoustic data

• US 7,280,963 B1
• Filed: 09/12/2003
• Issued: 10/09/2007
• Est. Priority Date: 09/12/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Beaufays, Francoise, Sankar, Ananth, Weintraub, Mitchel, Williams, Shaun
• Primary Examiner(s): Lerner; Martin
• Application Number:
  US10/660,868
• Time in Patent Office: 1,488 Days
• Field of Search: 704/231, 704/236, 704/238, 704/239, 704/240, 704/243, 704/244, 704/251, 704/252, 704/254
• US Class Current: 704/236
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams

Abstract

A computerized method is provided for generating pronunciations for words and storing the pronunciations in a pronunciation dictionary. The method includes graphing sets of initial pronunciations; thereafter in an ASR subsystem determining a highest-scoring set of initial pronunciations; generating sets of alternate pronunciations, wherein each set of alternate pronunciations includes the highest-scoring set of initial pronunciations with a lowest-probability phone of the highest-scoring initial pronunciation substituted with a unique-substitute phone; graphing the sets of alternate pronunciations; determining in the ASR subsystem a highest-scoring set of alternate pronunciations; and adding to a pronunciation dictionary the highest-scoring set of alternate pronunciations.

Claims

1. In a computer system, a method of generating pronunciations for a word that is represented by a waveform and text, such that the pronunciations are spelled by phones in a phonetic alphabet for storage in a pronunciation dictionary, the method comprising:

a. graphing sets of initial pronunciations;

thereafter

b. in an ASR subsystem determining a highest-scoring set of initial pronunciations;

c. generating sets of alternate pronunciations, wherein each set of alternate pronunciations includes the highest-scoring set of initial pronunciations with a lowest-probability phone of the highest-scoring initial pronunciation substituted with a unique-substitute phone;

d. graphing the sets of alternate pronunciations;

e. determining in the ASR subsystem a highest-scoring set of alternate pronunciations; and

f. adding to a pronunciation dictionary the highest-scoring set of alternate pronunciations.

2. The method of claim 1, wherein step a. includes weighting the sets of initial pronunciations with linguistic probabilities.

3. The method of claim 2, wherein linguistic probabilities are calculated according to a transformation probability P(B_i|A), wherein B_i and A represent respective sequence of phones for respective sets of pronunciations.

4. The method of claim 1, wherein step d. includes weighting the sets of alternate pronunciations with linguistic probabilities.

5. The method of claim 4, wherein linguistic probabilities are calculated according to a transformation probability P(B_i|A), wherein B_i and A represent respective sequence of phones for respective sets of pronunciations.

6. The method of claim 1 further comprising in the ASR subsystem traversing branches of graphs of the sets of initial and alternate pronunciations to generate scores for the sets of initial and alternate pronunciations.

7. The method of claim 1 further comprising in the ASR subsystem generating transcriptions of acoustic data spoken by a plurality of speakers, wherein the transcriptions are included in the transcribed acoustic data.

8. The method of claim 7 further comprising in the ASR subsystem collecting feedback from the plurality of speakers to affirm or disaffirm correct generation of the transcriptions, wherein if the transcriptions are affirmed as correct then the transcriptions are entered in the transcribed acoustic data.

9. The method of claim 1 further comprising

g. generating a phone probability for each phone in the highest-scoring set of initial pronunciations, wherein the lowest-probability phone has a lowest-phone probability.

10. The method of claim 1, further comprising:

g. generating the sets of initial pronunciations from initial pronunciations generated by a letter-to-phone engine and/or extracted from the pronunciation dictionary.

11. The method of claim 1, wherein steps a., b., c., d., e., and f. are repeated for each waveform of a plurality of waveforms that represent the word.

12. The method of claim 1, wherein steps c., d., and e. are repeated using the highest-scoring set of alternate pronunciations as the highest-scoring set of initial pronunciations.

13. The method of claim 1, wherein the sets of alternate pronunciations include a set of alternate pronunciations that include the highest-scoring initial pronunciation with the lowest-probability phone removed.

14. The method of claim 1, wherein the sets of alternate pronunciations include additional sets of alternate pronunciations that include the highest-scoring initial pronunciation having a unique phone inserted adjacent to the lowest-probability phone.

15. The method of claim 1, wherein the sets of alternate pronunciations include additional sets of alternate pronunciations that include the highest-scoring initial pronunciation having a sequence of two phones substituted for the lowest-probability phone.

16. The method of claim 1, wherein the sets of alternate pronunciations include additional sets of alternate pronunciations that include the highest-scoring initial pronunciation having the lowest-probability phone and a right neighboring phone substituted with a unique phone.

17. The method of claim 1, wherein the sets of alternate pronunciations include additional sets of alternate pronunciations that include the highest-scoring initial pronunciation with the lowest-probability phone and a left neighboring phone substituted with a unique phone.

18. The method of claim 1, further comprising:

g. adding the highest-scoring set of initial pronunciations to the pronunciation dictionary.

19. In a computer system, a method of generating pronunciations for a word that is represented by a waveform and text, such that the pronunciations are spelled by phones in a phonetic alphabet for storage in a pronunciation dictionary, the method comprising:

a. graphing sets of initial pronunciations;

thereafter

b. in an ASR subsystem determining a highest-scoring set of initial pronunciations;

c. generating a set of alternate pronunciations that includes the highest-scoring set of initial pronunciations with a lowest-probability phone of the highest-scoring initial pronunciation substituted with a unique-substitute phone; and

d. adding to a pronunciation dictionary the set of alternate pronunciations and the highest-scoring set of initial pronunciations.

20. The method of claim 19, wherein step a. includes weighting the sets of initial pronunciations with linguistic probabilities.

21. The method of claim 20, wherein linguistic probabilities are calculated according to a transformation probability P(B_i|A), wherein B_i and A represent respective sequence of phones for respective sets of pronunciations.

22. The method of claim 19 further comprising in the ASR subsystem traversing branches of the graph to generate scores for the sets of initial pronunciations.

23. The method of claim 19 further comprising in the ASR subsystem generating transcriptions of acoustic data spoken by a plurality of speakers, wherein the transcriptions are included in the transcribed acoustic data.

24. The method of claim 23 further comprising in the ASR subsystem collecting feedback from the plurality of speakers to affirm or disaffirm correct generation of the transcriptions, wherein if the transcriptions are affirmed as correct then the transcriptions are entered in the transcribed acoustic data.

25. The method of claim 19 further comprising

e. generating a phone probability for each phone in the highest-scoring set of initial pronunciations, wherein the lowest-probability phone has a lowest-phone probability.

26. The method of claim 19 further comprising:

e. generating the sets of initial pronunciations from initial pronunciations generated by a letter-to-phone engine and/or extracted from the pronunciation dictionary.

27. The method of claim 19, wherein steps a., b., c., and d. are repeated for each waveform of a plurality of waveforms that represent the word.

28. The method of claim 19, wherein step c. is repeated using the set of alternate pronunciations as the highest-scoring set of initial pronunciations.