Multiple language voice recognition

• US 8,788,256 B2
• Filed: 02/02/2010
• Issued: 07/22/2014
• Est. Priority Date: 02/17/2009
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Chen, Ruxin, Hernandez-Abrego, Gustavo, Omote, Masanori, Menendez-Pidal, Xavier
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US12/698,963
• Publication Number:
  US 20100211376A1
• Time in Patent Office: 1,631 Days
• Field of Search: 704/260, 704/2, 704/9, 704/270, 704/259, 704/257, 704/256, 704/255, 704/243, 704/235, 704/231, 704/209, 701/36, 434/156
• US Class Current: 704/2
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

Computer implemented speech processing generates one or more pronunciations of an input word in a first language by a non-native speaker of the first language who is a native speaker of a second language. The input word is converted into one or more pronunciations. Each pronunciation includes one or more phonemes selected from a set of phonemes associated with the second language. Each pronunciation is associated with the input word in an entry in a computer database. Each pronunciation in the database is associated with information identifying a pronunciation language and/or a phoneme language.

Claims

1. A computer speech processing apparatus, comprising:

a memory unit;

a processing unit coupled to the memory unit;

a pronunciation generator configured for execution by the processing unit, wherein the pronunciation generator is configured to generate pronunciations that include one or more pronunciations of an input orthographic word in a first language for a non-native speaker of the first language who is a native speaker of a second language, by;

a) comparing the input word to one or more words in a partial database of words and corresponding pronunciations to determine if the input word matches any words in the partial database, wherein each word in the partial database has an associated phoneme language and pronunciation language, entering a matching word and pronunciation into a Grammar and Dictionary (GnD) for the first language when the pronunciation language and phoneme language are both the first language, entering the entering a matching word and pronunciation into a Grammar and Dictionary (GnD) for the second language when the pronunciation language is the first language and phoneme language is the second language, and

b) generating a pronunciation for the input word when the input word does not match any word in the partial database and entering the input word and generated pronunciation into the GnD for the first language when the language of origin of the input word and the pronunciation language of the input word is the first language and entering the input word and generated pronunciation into the GnD for the second language when the language of origin of the input word is the first language and the pronunciation language of the input word is the second language, wherein generating the pronunciation includes converting each grapheme of the input word to a corresponding phoneme of the first language when the language of origin of the input word and the pronunciation language of the input word is the first language, and converting each grapheme of the input word to a corresponding phoneme of the first language and mapping each corresponding phoneme of the first language to one or more phonemes of the second language when the language of origin of the input word is the first language and the pronunciation language of the input word is the second language.

2. The apparatus of claim 1, further comprising:

a speech conversion unit configured to convert a sound signal produced by a speaker into a group of input phonemes, wherein the processing unit is configured to;

compare the group of input phonemes to one or more pronunciations in one or more entries in the database; and

trigger a change of state of the system corresponding to an entry that includes a pronunciation that matches the group of input phonemes.

3. The apparatus of claim 1, wherein converting each grapheme of the input word to a corresponding phoneme of the first language and mapping each corresponding phoneme of the first language to one or more phonemes of the second language includes mapping each corresponding phoneme of the first language phoneme into a corresponding intermediate phoneme and mapping each intermediate phoneme into a the one or more phonemes of the second language.

4. The apparatus of claim 1, wherein the pronunciation generator is further configured to determine a consensus from among a plurality of pronunciations for a given word in the database entry, and, based on the consensus, discard redundant, invalid or unlikely pronunciations from the entry.

5. In a computer speech processing apparatus having a processing unit and a memory unit, a computer implemented method for generating pronunciations that include one or more pronunciations of an input orthographic word in a first language for a non-native speaker of the first language who is a native speaker of a second language, the method comprising:

a) comparing the input word to one or more words in a partial database of words and corresponding pronunciations to determine if the input word matches any words in the partial database, wherein each word in the partial database has an associated phoneme language and pronunciation language, entering a matching word and pronunciation into a Grammar and Dictionary (GnD) for the first language when the pronunciation language and phoneme language are both the first language, entering the entering a matching word and pronunciation into a Grammar and Dictionary (GnD) for the second language when the pronunciation language is the first language and phoneme language is the second language, and

b) generating a pronunciation for the input word when the input word does not match any word in the partial database and entering the input word and generated pronunciation into the GnD for the first language when the language of origin of the input word and the pronunciation language of the input word is the first language and entering the input word and generated pronunciation into the GnD for the second language when the language of origin of the input word is the first language and the pronunciation language of the input word is the second language, wherein generating the pronunciation includes converting each grapheme of the input word to a corresponding phoneme of the first language when the language of origin of the input word and the pronunciation language of the input word is the first language, and converting each grapheme of the input word to a corresponding phoneme of the first language and mapping each corresponding phoneme of the first language to one or more phonemes of the second language when the language of origin of the input word is the first language and the pronunciation language of the input word is the second language.

6. The method of claim 5, further comprising:

c) converting a sound signal produced by a speaker into a group of input phonemes;

d) comparing the group of phonemes to one or more entries in the database; and

e) executing a command corresponding to an entry that matches the group of input phonemes.

7. The method of claim 5, converting each grapheme of the input word to a corresponding phoneme of the first language and mapping each corresponding phoneme of the first language to one or more phonemes of the second language includes mapping each corresponding phoneme of the first language phoneme into a corresponding intermediate phoneme and mapping each intermediate phoneme into the one or more phonemes of the second language.

8. The method of claim 5, further comprising determining a consensus from a plurality of pronunciations for a given word in the database entry, and, based on the consensus, discarding redundant, invalid or unlikely pronunciations.

9. A non-transitory computer readable storage medium, having embodied therein computer readable instructions for implementing computer speech processing in a computer speech processing apparatus having a processing unit and a memory unit, the instructions comprising:

a) one or more instructions that, when executed by the processor, compare the input word to one or more words in a partial database of words and corresponding pronunciations to determine if the input word matches any words in the partial database, wherein each word in the partial database has an associated phoneme language and pronunciation language, enter a matching word and pronunciation into a Grammar and Dictionary (GnD) for the first language when the pronunciation language and phoneme language are both the first language, enter the entering a matching word and pronunciation into a Grammar and Dictionary (GnD) for the second language when the pronunciation language is the first language and phoneme language is the second language; and

b) generate a pronunciation for the input word when the input word does not match any word in the partial database and entering the input word and generated pronunciation into the GnD for the first language when the language of origin of the input word and the pronunciation language of the input word is the first language and entering the input word and generated pronunciation into the GnD for the second language when the language of origin of the input word is the first language and the pronunciation language of the input word is the second language, wherein generating the pronunciation includes converting each grapheme of the input word to a corresponding phoneme of the first language when the language of origin of the input word and the pronunciation language of the input word is the first language, and convert each grapheme of the input word to a corresponding phoneme of the first language and mapping each corresponding phoneme of the first language to one or more phonemes of the second language when the language of origin of the input word is the first language and the pronunciation language of the input word is the second language.