METHOD AND APPARATUS FOR INTERACTIVE LANGUAGE INSTRUCTION

• US 20030028378A1
• Filed: 09/09/1999
• Published: 02/06/2003
• Est. Priority Date: 09/09/1999
• Status: Active Grant

• Current Assignee: WSOU Investments, LLC (WSOU Holdings, LLC)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: ZHOU, QIRU, ZHONG, JIALIN, MCNERNEY, MICHELLE, AUGUST, KATHERINE GRACE, SHIH, CHI-LIN, SURENDRAN, ARUN CHANDRASEKARAN, BLACKWOOD, NADINE, LI, QI P.
• Granted Patent: US 7,149,690 B2
• Time in Patent Office: Days
• US Class Current: 704/260
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G09B 5/04 with audible presentation of the material to be studied sound recording or reproducing G11B
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence

Abstract

A method and apparatus for interactive language instruction is provided that displays text files for processing, provide key features and functions for interactive learning, displays facial animation, and provides a workspace for language building functions. The system includes a stored set of language rules as part of the text-to-speech sub-system, as well as another stored set of rules as applied to the process of learning a language. The method implemented by the system includes digitally converting text to audible speech, providing the audible speech to a user or student (with the aid of an animated image in selected circumstances), prompting the student to replicate the audible speech, comparing the student'"'"'s replication with the audible speech provided by the system, and providing feedback and reinforcement to the student by, for example, selectively recording or playing back the audible speech and the student'"'"'s replication.

Claims

1. A system for interactive language instruction for a user comprising:

a first module configured to convert input text to audible speech in a selected language, the audible speech being patterned after a model;

a user interface configured to receive utterances spoken by a user in response to a prompt to replicate the audible speech; and

,

a second module configured to recognize the utterances and provide feedback to the user as to a precision at which the user replicates the audible speech in the selected language based on a comparison of the utterances to one of the audible speech and the model.

2. The system as set forth in claim 1 further comprising a third module synchronized to the first module, a third module producing an animated image of a human face and head pronouncing the audible speech.

3. The system as set forth in claim 2 wherein the animated image of the human face and head portrays a transparent face and head.

4. The system as set forth in claim 2 wherein the first and third modules further include controls to control one of the volume, speed, and vocal characteristics of the video image and the audible speech.

5. The system as set forth in claim 1 wherein the model is one of a predictive model, phoneme model, a diphone model, and a dynamically generated model.

6. The system as set forth in claim 1 wherein the first module includes files storing model pronunciations for words comprising the input text.

7. The system as set forth in claim 1 further comprising lesson files wherein the input text is based on data stored in the lesson files.

8. The system as set forth in claim 1 wherein the input text is based on data received from a source outside of the system.

9. The system as set forth in claim 1 wherein the system further includes dictionary files.

10. The system as set forth in claim 1 wherein the system further comprises a record and playback module.

11. The system as set forth in claim 1 wherein the system further includes a table storing mapping information between word subgroups and vocabulary words.

12. The system as set forth in claim 1 wherein the system further includes a table storing mapping information between words and vocabulary words.

13. The system as set forth in claim 1 wherein the system further includes a table storing mapping information between words and examples of parts of speech.

14. The system as set forth in claim 1 wherein the system further includes tables of punctuation.

15. The system as set forth in claim 1 wherein the system includes specific pronunciation files.

16. A system comprising:

a first module configured to convert input text to audible speech in a selected language, the audible speech indicative of a model;

a second module synchronized to the first module, the second module producing an animated image of a human face and head pronouncing the audible speech;

a user interface positioned to receive utterances spoken by a user in response to a prompt to replicate the audible speech; and

,

a third module configured to recognize the utterances and provide feedback to the user as to a precision at which the user replicates the speech in the selected language based on a comparison of the utterances to one of the audible speech and the model.

17. A method for voice interactive language instruction comprising:

converting input text data to audible speech data;

generating audible speech comprising phonemes based on the audible speech data;

outputting the audible speech through an audio output device;

generating an animated image of a face and head pronouncing the audible speech;

synchronizing the audible speech and the video image;

prompting the user to replicate the audible speech;

recognizing utterances generated by the user in response to the prompting;

comparing the audible speech to the utterances; and

,

providing feedback to the user based on the comparison.

18. The method as set forth in claim 17 further comprising receiving the input text from one of a network, a stored lesson file, a scanner, and the internet.

19. The method as set forth in claim 17 wherein the feedback comprises providing a playback of selected portions of the audible speech and utterances.