Utterance Verification and Pronunciation Scoring by Lattice Transduction

• US 20120078630A1
• Filed: 09/27/2010
• Published: 03/29/2012
• Est. Priority Date: 09/27/2010
• Status: Active Grant

• Current Assignee: Rosetta Stone Limited (Rosetta Stone Incorporated)
• Original Assignee: Rosetta Stone Limited (Rosetta Stone Incorporated)
• Inventors: Pellom, Bryan, Hagen, Andreas
• Granted Patent: US 8,880,399 B2
• Time in Patent Office: Days
• US Class Current: 704/254
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

In the field of language learning systems, proper pronunciation of words and phrases is an integral aspect of language learning, determining the proximity of the language learner'"'"'s pronunciation to a standardized, i.e. ‘perfect’, pronunciation is utilized to guide the learner from imperfect toward perfect pronunciation. In this regard, a phoneme lattice scoring system is utilized, whereby an input from a user is transduced into the perfect pronunciation example in a phoneme lattice. The cost of this transduction may be determined based on a summation of substitutions, deletions and insertions of phonemes needed to transducer from the input to the perfect pronunciation of the utterance.

Claims

1. A method for verifying the accuracy of an utterance comprising the steps of:

a. creating and storing in a computer a phoneme lattice for an ideal utterance;

b. determining an ideal path through the phoneme lattice for the ideal utterance;

c. receiving an input utterance from a user;

d. transducing the input utterance into the ideal utterance utilizing the phoneme lattice;

e. computing a transduction cost; and

f. determining the accuracy of the input utterance utilizing the transduction cost.

2. The method of claim 1, further comprising the steps of:

a. tagging each phoneme instance of the ideal utterance with a begin time, an end time and an ideal duration;

b. measuring the actual duration of each phoneme of the input utterance; and

c. scoring the ideal versus the actual duration and utilizing the determined score in the step of determining the accuracy of the input utterance.

3. The method of claim 1 wherein determining the accuracy of the input utterance utilizes a phoneme confusion matrix.

4. The method of claim 3 further wherein the phoneme confusion matrix provides modified substitution costs depending on a likelihood that two phonemes are confused with one another.

5. The method of claim 1, further comprising the steps of:

a. transforming the phoneme lattice into a directed acyclic graph.

6. A method comprising accepting an utterance as input, transducing the utterance into an expected phoneme sequence, calculating a transduction cost of said transducing, and signaling a user that said utterance is a substantially correct pronunciation of said expected phoneme sequence if said transduction cost is below a predetermined threshold.

7. The method of claim 6 further comprising signaling said user that said utterance is either an incorrect pronunciation of said expected phoneme sequence, or that said utterance is out of grammar, depending upon said transduction cost.