Method of recognizing phones in speech of any language

• US 7,406,408 B1
• Filed: 08/24/2004
• Issued: 07/29/2008
• Est. Priority Date: 08/24/2004
• Status: Active Grant

• Current Assignee: National Security Agency
• Original Assignee: The United States of America As Represented By The Director National Security Agency
• Inventors: Lackey, Bradley C., Walker, Brenton D., Schone, Patrick J.
• Primary Examiner(s): Lerner; Martin
• Application Number:
  US10/928,878
• Time in Patent Office: 1,435 Days
• Field of Search: 704/8, 704/10, 704/231, 704/240, 704/251, 704/257, 704/277
• US Class Current: 704/8
• CPC Class Codes: G10L 15/005 Language recognition

Abstract

Method of recognizing phones in speech of any language. Acquire phones for all languages and a set of languages. Acquire a pronunciation dictionary, a transcript of speech for the set of languages, and speech for the transcript. Receive speech containing unknown phones. If the speech'"'"'s language is unknown, compare it to the phones for all languages to determine the phones. If the language is known but no phones were acquired in that language, compare the speech to the phones for all languages to determine the phones. If phones were acquired in the speech'"'"'s language but no corresponding pronunciation dictionary was acquired, compare the speech to the phones for all languages to determine the phones. If a pronunciation dictionary was acquired for the phones in the speech'"'"'s language but no transcript was acquired then compare the speech to the phones for all languages to determine the phones.

Claims

1. A method of recognizing phones in speech of any language, comprising the steps of:

(a) acquiring phones for all languages, including information on the phones that indicate in which languages the phones are valid;

(b) acquiring phones for each of a user-definable number of languages;

(c) acquiring a pronunciation dictionary for said user-definable number of languages for which phones where acquired in step (b), and performing a statistical analysis of each pronunciation dictionary acquired to determine the probability of occurrence of each phone in the pronunciation dictionary;

(d) acquiring at least one transcript of speech for each of said user-definable number of languages, where each of the at least one transcript identifies the phones included therein, and performing a statistical analysis of each transcript acquired to determine the probability of occurrence of each phone in the transcript;

(e) acquiring speech for a user-definable number of the at least one transcript acquired in step (d) and performing a statistical analysis of each speech acquired to determine the probability of occurrence of each phone in the speech;

(f) receiving speech for which phones contained therein are not identified;

(g) if the language of the received speech is not known then comparing the received speech to the phones acquired in step (a) and declaring the received speech to include the phones acquired in step (a) to which the received speech most closely matches and stopping, otherwise proceeding to the next step;

(h) if the language of the received speech is known but no phones were acquired in step (b) in the language then comparing the received speech to the phones acquired in step (a) and declaring the received speech to include the phones acquired in step (a) to which the received speech most closely matches and stopping, otherwise proceeding to the next step;

(i) if phones were acquired in step (b) in the language of the received speech but no pronunciation dictionary was acquired in step (c) for the phones acquired in step (b) then comparing the received speech to the phones acquired in step (a) that are valid in the language of the received speech and declaring the received speech to include the phones acquired in step (a) that are valid in the language of the received speech to which the received speech most closely matches and stopping, otherwise proceeding to the next step; and

(j) if a pronunciation dictionary was acquired in step (c) for the phones acquired in step (b) in the language of the received speech but no transcript was acquired in step (d) in the language of the received speech then comparing the received speech to the phones acquired in step (a) that are valid in the language of the received speech and declaring the received speech to include the phones acquired in step (a) that are valid in the language of the received speech to which the received speech most likely matches considering the probability of occurrence of each phone in the pronunciation dictionary and stopping.

2. The method of claim 1, further including the step of if a transcript was acquired in step (d) but no speech was acquired in step (e) for the transcript then comparing the received speech to the phones acquired in step (a) that are valid in the language of the received speech and declaring the received speech to include the phones acquired in step (a) that are valid in the language of the received speech to which the received speech most likely matches considering the probability of occurrence of each phone in the transcript and stopping.

3. The method of claim 2, further including the step of if speech was acquired in step (e) for the transcript then replacing the phones acquired in step (a) that are valid in the language of the received speech with the phones of the acquired speech and comparing the received speech to the phones acquired in step (a) as replaced that are valid in the language of the received speech and declaring the received speech to include the phones acquired in step (a) as replaced that are valid in the language of the received speech to which the received speech most likely matches considering the probability of occurrence of each phone in the speech acquired in step (e) and stopping.

4. The method of claim 1, wherein said step of acquiring phones for all languages is comprised of the step of selecting a phone acquisition method from the list of phone acquisition methods consisting of acquiring an existing database of phones for all languages and creating a database of phones for all languages.

5. The method of claim 1, wherein said step of acquiring phones for each of said user-definable number of languages is comprised of the step of selecting a phone acquisition method from the list of phone acquisition methods consisting of acquiring an existing database of phones for each of said user-definable number of languages and creating a database of phones for each of said user-definable number of languages.

6. The method of claim 1, wherein said step of acquiring said pronunciation dictionary for said user-definable number of languages is comprised of the step of selecting a pronunciation dictionary acquisition method from the list of pronunciation dictionary acquisition methods consisting of acquiring an existing database of said pronunciation dictionary for said user-definable number of languages and creating said pronunciation dictionary for said user-definable number of languages.

7. The method of claim 1, wherein said step of acquiring at least one transcript of speech is comprised of the step of selecting a transcript acquisition method from the list of transcript acquisition methods consisting of acquiring an existing database of transcripts and creating transcripts.

8. The method of claim 1, wherein said step of acquiring speech for said user-definable number of the at least one transcript acquired in step (d) is comprised of the step of selecting a speech acquisition method from the list of speech acquisition methods consisting of acquiring an existing database of speech and creating a database of speech.

9. The method of claim 1, wherein said step of performing said statistical analysis of each pronunciation dictionary is comprised of the step of performing an n-gram analysis of each pronunciation dictionary.

10. The method of claim 1, wherein said step of performing a statistical analysis of each transcript is comprised of the step of performing an n-gram analysis of each transcript.

11. The method of claim 1, wherein said step of performing said statistical analysis of each speech acquired is comprised of the step of performing an n-gram analysis of each speech acquired.

12. The method of claim 1, wherein said method of acquiring phones for all languages is comprised of the steps of:

(a) receiving corpora;

(b) selecting a first corpus from the corpora;

(c) if the first corpus does not include a phonetic transcript of the first corpus then generating the phonetic transcript of the first corpus using a user-definable first pronunciation dictionary, where each word in the first corpus is associated with at least one sequence of at least one phone symbol;

(d) selecting a second corpus from the corpora;

(e) if the second corpus does not include a phonetic transcript of the second corpus then generating a phonetic transcript for the second corpus using a user-definable second pronunciation dictionary, where each word in the second corpus is associated with at least one sequence of at least one phone symbol;

(f) identifying at least one allophone for each phone in the second corpus;

(g) identifying for each phone in the second corpus one of the at least one allophone identified in step (f) that most closely matches the phone in the second corpus using a user-definable acoustic model for each of the at least one allophone;

(h) replacing each phone symbol in the phonetic transcript of the second corpus with a symbol for the corresponding allophone identified in step (g);

(i) combining the first corpus and the second corpus, including the phonetic transcript of the first corpus and the phonetic transcript of the second corpus, and designating the result as the first corpus; and

(j) returning to step (d) if there is another corpus in the corpora to be processed, otherwise stopping.

13. The method of claim 12, wherein the step of receiving corpora is comprised of receiving corpora that are not all in the same language.

14. The method of claim 13, wherein the step of selecting a first corpus from the corpora is comprised of the step of selecting a first corpus from the corpora for which most in known phonetically.

15. The method of claim 14, wherein the step of selecting a first corpus from the corpora is comprised of selecting a first corpus in a native language of a user of the method from the corpora.

16. The method of claim 15, wherein the step of generating the phonetic transcript of the first corpus is comprised of associating each word in the first corpus with at least one sequence of at least one phone symbol in the user-definable first pronunciation dictionary which most closely matches actual phones of the word.

17. The method of claim 16, wherein the step of generating the phonetic transcript of the second corpus is comprised of associating each word in the second corpus with at least one sequence of at least one phone symbol in the user-definable second pronunciation dictionary which most closely matches actual phones of the word.

18. The method of claim 17, further including the step of deleting a user-definable number of phonetic symbols from the transcript of the second corpus.

19. The method of claim 12, wherein the step of selecting a first corpus from the corpora is comprised of the step of selecting a first corpus from the corpora for which most in known phonetically.

20. The method of claim 12, wherein the step of selecting a first corpus from the corpora is comprised of selecting a first corpus in a native language of a user of the method from the corpora.

21. The method of claim 12, wherein the step of generating the phonetic transcript of the first corpus is comprised of associating each word in the first corpus with at least one sequence of at least one phone symbol in the user-definable first pronunciation dictionary which most closely matches actual phones of the word.

22. The method of claim 12, wherein the step of generating the phonetic transcript of the second corpus is comprised of associating each word in the second corpus with at least one sequence of at least one phone symbol in the user-definable second pronunciation dictionary which most closely matches actual phones of the word.

23. The method of claim 12, further including the step of deleting a user-definable number of phonetic symbols from the transcript of the second corpus.