Method and apparatus for providing an interactive language tutor

• US 20040006461A1
• Filed: 02/10/2003
• Published: 01/08/2004
• Est. Priority Date: 07/03/2002
• Status: Active Grant

• Current Assignee: WSOU Investments, LLC (WSOU Holdings, LLC)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: Sayeed, Zulfiquar, Gupta, Sunil K., Raghavan, Prabhu, Lu, ZiYi, Vinchhi, Chetan, Sethuraman, Aravind
• Granted Patent: US 7,299,188 B2
• Time in Patent Office: Days
• US Class Current: 704/200
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit

Abstract

A method and apparatus for generating a pronunciation score by receiving a user phrase intended to conform to a reference phrase and processing the user phrase in accordance with at least one of an articulation-scoring engine, a duration scoring engine and an intonation-scoring engine to derive thereby the pronunciation score. The scores provided by the various scoring engines are adapted to provide a visual and/or numerical feedback that provides information pertaining to correctness or incorrectness in one or more speech-features such as intonation, articulation, voicing, phoneme error and relative word duration. Such useful interactive feedback will allow a user to quickly identify the problem area and take remedial action in reciting “tutor” sentences or phrases.

Claims

1. A method for providing speech-feature feedback, said method comprising the steps of:

presenting a reference utterance;

displaying graphically a speech-feature associated with speech of said reference utterance;

receiving a user utterance that represents speech of said reference utterance by a user; and

displaying graphically a speech-feature associated with speech of said user utterance.

2. The method of claim 1, wherein said speech-feature represents intonation.

3. The method of claim 1, wherein said speech-feature represents acoustic quality.

4. The method of claim 1, wherein said speech-feature represents word duration.

5. The method of claim 1, further comprising the step of:

displaying at least one score associated with said speech-feature.

6. The method of claim 5, wherein said at least one score comprises an intonation score.

7. The method of claim 5, wherein said at least one score comprises an acoustic quality score.

8. The method of claim 5, wherein said at least one score comprises a word duration score.

9. The method of claim 1, further comprising the step of:

playing said reference utterance.

10. The method of claim 2, wherein said speech-feature associated with speech of said reference utterance and said speech-feature associated with speech of said user utterance are graphically displayed as intonation curves.

11. The method of claim 4, wherein said speech-feature associated with speech of said reference utterance is presented graphically as a plurality of intonation curves corresponding to a plurality of words within said reference utterance.

12. The method of claim 3, wherein said speech-feature associated with speech of said user utterance is presented graphically as a plurality of international phonetic symbols corresponding to a selected word from within said reference utterance.

13. The method of claim 12, wherein one or more words within said reference utterance are highlighted to indicate one or more words of said user utterance that are deemed mispronounced.

14. The method of claim 12, wherein a subset of said plurality of interniational phonetic symbols are highlighted to indicate one or more phonemes of said user utterance that are deemed mispronounced.

15. The method of claim 1, wherein said method is implemented by a handheld apparatus.

16. The method of claim 1, further comprising the step of:

downloading a set of reference utterances from a remote source.

17. The method of claim 5, wherein said one or more numeral scores associated with said speech-feature are graphically displayed for illustrating historical performance.

18. A computer readable medium for storing instructions that, when executed by a processor, perform a method for providing speech-feature feedback, said method comprising the steps of:

presenting a reference utterance;

displaying graphically a speech-feature associated with speech of said reference utterance;

receiving a user utterance that represents speech of said reference utterance by a user; and

displaying graphically a speech-feature associated with speech of said user utterance.

19. Apparatus for providing speech-feature feedback, said apparatus comprising:

means for presenting a reference utterance;

means for displaying graphically a speech-feature associated with speech of said reference utterance;

means for receiving a user utterance that represents speech of said reference utterance by a user; and

means for displaying graphically a speech-feature associated with speech of said user utterance.

20. A handheld device, comprising:

a voice input port for receiving a voiced utterance that represents speech of a reference utterance;

a display device for displaying at least pronunciation scoring information;

a memory for storing software instructions representing an articulation scoring engine, a duration scoring engine, an intonation scoring engine, and a plurality of reference utterances; and

a processor for executing said software instructions representing said scoring engines to derive thereby a pronunciation score for said received utterance.