Method and system for assessing pronunciation difficulties of non-native speakers by entropy calculation

• US 7,778,834 B2
• Filed: 08/11/2008
• Issued: 08/17/2010
• Est. Priority Date: 01/11/2005
• Status: Active Grant

• Current Assignee: Educational Testing Service
• Original Assignee: Educational Testing Service
• Inventors: Lawless, Rene, Futagi, Yoko, Higgins, Derrick, Zechner, Klaus
• Primary Examiner(s): Hudspeth; David R
• Assistant Examiner(s): SHAH, PARAS D
• Application Number:
  US12/189,506
• Publication Number:
  US 20080294440A1
• Time in Patent Office: 736 Days
• Field of Search: 704/270, 704/271, 704/272, 704/251, 704/246, 704/258, 704/260, 704/9, 704/257, 434/327, 434/167, 434/185
• US Class Current: 704/270
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules

Abstract

The present disclosure presents a useful metric for assessing the relative difficulty which non-native speakers face in pronouncing a given utterance and a method and systems for using such a metric in the evaluation and assessment of the utterances of non-native speakers. In an embodiment, the metric may be based on both known sources of difficulty for language learners and a corpus-based measure of cross-language sound differences. The method may be applied to speakers who primarily speak a first language speaking utterances in any non-native second language.

Claims

1. A computer-implemented method of assessing pronunciation difficulties of a non-native speaker, the method comprising:

determining one or more sources of pronunciation difficulties between a language of a non-native speaker and a second language;

assigning a weight to each source;

calculating using a processor a phonetic difficulty score based on the one or more sources and the weight assigned to each source;

calculating using the processor a language model score based on a sound comparison between the language of the non-native speaker and the second language, including calculating a language model for the language of the non-native speaker and the second language and calculating a cross-entropy of an utterance with respect to the language model of the language of the non-native speaker inversely weighted by the cross-entropy of the utterance with respect to the language model of the second language, wherein calculating the cross-entropy of an utterance with respect to the language model of the language of the non-native speaker comprises assigning a lower score to utterances of the second language that are similar to sounds of the language of the non-native speaker, and assigning a higher score to utterances of the second language that are not similar to sounds of the language of the non-native speaker;

normalizing using the processor the phonetic difficulty score and the language model score; and

calculating a metric from the normalized phonetic difficulty score and the normalized language model score.

2. The method of claim 1, wherein calculating the language model for the language of the non-native speaker and the second language comprises phonetically transcribing the language of the non-native speaker and the second language by applying a phonetic dictionary to text representing recorded utterances of each language.

3. The method of claim 1, wherein calculating the metric comprises calculating the sum of the normalized phonetic difficulty score and the normalized language model score.

4. The method of claim 1, wherein said determining comprises determining a plurality of different types of phonetic sources of pronunciation difficulties between the language of the non-native speaker and the second language, said determining including determining multiple types of sounds in the second language that cause pronunciation difficulties.

5. The method of claim 4, wherein the multiple types of sounds in the second language that cause pronunciation difficulties include:

a first sound that does not contrast with a second sound in the language of the non-native speaker; and

a third sound that does not exist in language of the non-native speaker.

6. The method of claim 5, wherein the language of the non-native speaker is Japanese, the second language is English, the first sound is /r/, the second sound is /l/, and the third sound is /θ

/.

7. A system for assessing the pronunciation difficulties of a non-native speaker comprising:

a processor; and

a processor-readable storage medium in communication with the processor, wherein the processor-readable storage medium contains one or more programming instructions for causing the processor to perform a method of assessing the pronunciation difficulties of a non-native speaker, the method comprising;

determining one or more sources of pronunciation difficulties between a language of a non-native speaker and a second language,

assigning a weight to each source,

calculating a phonetic difficulty score based on the one or more sources and the weight assigned to each source,

calculating a language model score based on a sound comparison between the language of the non-native speaker and the second language, by calculating a language model for the language of the non-native speaker and the second language and calculating a cross-entropy of an utterance with respect to the language model of the language of the non-native speaker inversely weighted by the cross-entropy of the utterance with respect to the language model of the second language, wherein calculating the cross-entropy of an utterance with respect to the language model of the language of the non-native speaker comprises assigning a lower score to utterances of the second language that are similar to sounds of the language of the non-native speaker, and assigning a higher score to utterances of the second language that are not similar to sounds of the language of the non-native speaker,

normalizing the phonetic difficulty score and the language model score, and

calculating a metric from the normalized phonetic difficulty score and the normalized language model score.

8. The system of claim 7, wherein calculating the language model for the language of the non-native speaker and the second language comprises phonetically transcribing the language of the non-native speaker and the second language by applying a phonetic dictionary to text representing recorded utterances of each language.

9. The system of claim 7, wherein calculating the metric comprises calculating the sum of the normalized phonetic difficulty score and the normalized language model score.

10. The system of claim 7, wherein said determining comprises determining a plurality of different types of phonetic sources of pronunciation difficulties between the language of the non-native speaker and the second language, said determining including determining multiple types of sounds in the second language that cause pronunciation difficulties.

11. The system of claim 10, wherein the multiple types of sounds in the second language that cause pronunciation difficulties include:

a first sound that does not contrast with a second sound in the language of the non-native speaker; and

a third sound that does not exist in language of the non-native speaker.

12. The system of claim 11, wherein the language of the non-native speaker is Japanese, the second language is English, the first sound is /r/, the second sound is /l/, and the third sound is /θ

/.

13. A non-transitory processor-readable storage medium for assessing pronunciation difficulties of a non-native speaker, comprising computer-executable instructions, which when executed cause a processing system to execute steps comprising:

determining one or more sources of pronunciation difficulties between a language of a non-native speaker and a second language,

assigning a weight to each source,

calculating a phonetic difficulty score based on the one or more sources and the weight assigned to each source,

calculating a language model score based on a sound comparison between the language of the non-native speaker and the second language, by calculating a language model for the language of the non-native speaker and the second language and calculating a cross-entropy of an utterance with respect to the language model of the language of the non-native speaker inversely weighted by the cross-entropy of the utterance with respect to the language model of the second language, wherein calculating the cross-entropy of an utterance with respect to the language model of the language of the non-native speaker comprises assigning a lower score to utterances of the second language that are similar to sounds of the language of the non-native speaker, and assigning a higher score to utterances of the second language that are not similar to sounds of the language of the non-native speaker,

normalizing the phonetic difficulty score and the language model score, and

calculating a metric from the normalized phonetic difficulty score and the normalized language model score.

14. The non-transitory processor-readable storage medium of claim 13, wherein calculating a language model for the language of the non-native speaker and the second language comprises phonetically transcribing the first language of the non-native speaker and the second language by applying a phonetic dictionary to text representing recorded utterances of each language.

15. The non-transitory processor-readable storage medium of claim 13, wherein calculating the metric comprises calculating the sum of the normalized phonetic difficulty score and the normalized language model score.

16. The non-transitory processor-readable storage medium of claim 13, wherein said determining comprises determining a plurality of different types of phonetic sources of pronunciation difficulties between the language of the non-native speaker and the second language, said determining including determining multiple types of sounds in the second language that cause pronunciation difficulties.

17. The non-transitory processor-readable storage medium of claim 13, wherein the multiple types of sounds in the second language that cause pronunciation difficulties include:

a first sound that does not contrast with a second sound in the language of the non-native speaker; and

a third sound that does not exist in language of the non-native speaker.

18. The non-transitory processor-readable storage medium of claim 17, wherein the language of the non-native speaker is Japanese, the second language is English, the first sound is /r/, the second sound is /l/, and the third sound is /θ

/.