Interactive language learning system

• US 5,010,495 A
• Filed: 02/02/1989
• Issued: 04/23/1991
• Est. Priority Date: 02/02/1989
• Status: Expired due to Term

• Current Assignee: American Language Academy
• Original Assignee: American Language Academy
• Inventors: Willetts, John A.
• Primary Examiner(s): KEMENY, EMANUEL
• Application Number:
  US07/305,223
• Time in Patent Office: 810 Days
• Field of Search: 364/513.5, 364/419, 364/900 MS File
• US Class Current: 704/235
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G09B 5/04 with audible presentation of the material to be studied sound recording or reproducing G11B
• CPC Class Codes: G09B 7/04 characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question, supplying a further explanation

Abstract

An interactive computer assisted language learning system which allows a student to select a model phrase from text displayed on an electronic display; record (in digitized form) his own pronunciation of that phrase; and instantly listen to the digitized vocal version of the selected phrase and his own recorded pronunciation for comparison purposes. An audio CLIP mode permits the student to select any (random) portion of displayed text (e.g., a phrase, a small part of a phrase, a single word, a syllable, or a phoneme) using cursor control or the like and to control the system to play the voice corresponding to that selected portion. A SoundSort text reconstruction exercise based on aural clues automatically randomizes the order of plural phrases, provides digitized utterances of the phrases in the randomized order, and requires the student to reconstruct the original order using a visual display interface. Integration of digitized sound in a high-level authoring system (as distinct from an authoring language) is provided. An easy-to-use "WYSIWYG" ("What you see is what you get") user interface reduces or eliminates user mistakes and associated frustration and does not require the user to have any programming ability. An extremely flexible authoring system allows a teacher to link recorded digitized sound with customized on-screen text (which may but need not match the digitized sound). This allows a wide variety of free-form exercises to be created.

Claims

1. An interactive language learning system comprising:

storing means for storing a digitized speech version of a passage of language and for also storing a digital data textual version of said same passage;

a display for displaying text corresponding to said passage;

selecting means operatively connected to said display and to said storing means and operable by a user for selecting a portion of said passage, and for causing the portion of said stored digital data textual version corresponding to said selected passage portion to be displayed as text on said display; and

speech processing means for;

selecting the portion of said stored digitized speech version corresponding to said selected portion of said passage,

converting said selected digitized speech version portion to audio signals for use in generating speech sounds,

converting audio signals representing user speech into digitized speech signals representing said user speech, and

subsequently reconverting said digitized speech signals representing said user speech to audio signals for use in generating further speech sounds so as to permit said user to listen to and compare his own speech with a spoken version of said passage corresponding to said stored digitized speech version.

2. A system as in claim 1 wherein:

said system further includes a transducer which converts user speech to audio signals;

said speech processing means includes means connected to said transducer for converting said audio signals to digitized speech signals and for temporarily storing said digitized speech signals;

said display also displays a symbol; and

said system further includes user input means for permitting said user to (i) select said portion of said passage by manipulating the position of said symbol displayed by said display with respect to said displayed text, and (ii) control said speech processing means to rapidly alternate (a) converting said temporarily stored digitized speech signals representing his own speech to audio signals, and (b) converting said digitized speech signals corresponding to said selected portion of said passage to audio signals so as to alternately generate sounds corresponding to said user'"'"'s speech and sounds corresponding to said stored digitized speech version.

3. A system as in claim 1 wherein:

said speech processing means generates processor interrupts; and

said system further includes interrupt means for reading said digitized speech version from said storing means for conversion to audio signals by said speech processing means in response to said generated processor interrupts.

4. A system as in claim 1 wherein:

said selecting means includes means for selecting the position and length of a portion of said passage;

said speech processing means includes further selecting means for selecting only those portions of said stored digitized speech version corresponding to said selected passage portion; and

said speech processing means also includes means for converting only said selected stored digitized speech version portions to audio signals.

5. A system as in claim 1 wherein said first-mentioned selecting means comprises cursor control means manipulable by said user for selecting portions of said text displayed by said display and for thereby selecting corresponding portions of said stored digitized speech version for conversion to audio signals.

6. A system as in claim 5 further including means connected to said cursor control means and to said display for causing said selected text portions displayed by said display to have a different displayed appearance than the non-selected displayed text portions.

7. A system as in claim 5 wherein said system further includes text display selection means, manipulable by said user and operatively connected to said display, for alternately selecting:

(a) display of only said selected text portions, and (b) display of the entire textual version of said passage including said selected text portion.

8. A system as in claim 1 wherein said speech processing means includes means for converting between audio signals and adaptive differential pulse code modulation encoded digitized speech signals representing said audio signals.

9. An interactive language learning system comprising:

means for providing digitized speech signals representing a sequence of spoken phrases having an initial order;

re-ordering means connected to said providing means for re-ordering said plural phrases into a sequence having an order different from said initial order;

symbol display means connected to said re-ordering means for associating a symbol with each of said plural phrases and for presenting a display of said symbols in said re-ordered sequence;

input means operatively connected to said re-ordering means and operable by a user for permitting said user to further re-order said plural phrases into a still further order and to thus change the order in which said symbols are presented by said symbol display means;

speech processing means connected to said providing means and to said re-ordering means for converting said digitized speech signals to audio speech signals in said re-ordered sequence so as to generate spoken versions of said phrases in said re-ordered sequence; and

testing means connected to said input means for comparing the user-selected re-ordered sequence with said initial order.

10. A system as in claim 9 wherein:

said input means includes means for selecting any one of said symbols; and

said speech processing means is also connected to said user input means and to said symbol display means and includes means for converting the stored digitized speech associated with said selected symbol to audio signals.

11. A system as in claim 9 wherein:

said symbol display means includes;

a left-hand display column which displays said symbols in said first-mentioned re-ordered sequence, and

a right-hand display column which displays said symbols in said user-specified further order; and

said input means includes means for moving said symbols from said left-hand display column to said right-hand display column in response to user commands.

12. A system as in claim 11 wherein said speech processing means is also connected to said user input means and includes means for generating sounds corresponding to said phrases in said initial order.

13. A system as in claim 9 wherein said speech processing means is also connected to said user input means and includes means for selecting a starting point within said re-ordered sequence, said selected starting point being different from the beginning of said sequence, and for converting said corresponding digitized speech signals to audio signals in said initial order of said phrases beginning from said starting point so as to provide audible speech corresponding to less than said entire sequence of phrases.

14. An interactive language lesson composing system comprising:

user input means for inputting characters in response to user manipulation thereof;

display means for displaying said inputted characters;

storing means for storing digital data;

word processing means connected to said user input means, said display means and said storing means, for permitting said user to input and edit a main text display format, for causing said storing means to store said main text display format, for permitting said user to input and edit a secondary text display format, and for causing said storing means to store said secondary text display format;

selecting means connected to said user input means, said display means and said storing means for permitting said user to select plural portions of said main text display format; and

speech processing means connected to receive an audio signal for converting said audio signal to digitized speech and for causing said storing means to store said digitized speech in discrete segments, each of said discrete segments being associated with a different one of said plural selected main text display format portions.

15. An interactive language learning system comprising:

user input means for inputting characters in response to user manipulation thereof;

display means for displaying said inputted characters;

storing means for storing digitized speech corresponding to a text passage and for also storing a textual version of said text passage;

speech processing means connected to said storing means for converting said digitized speech to audio signals; and

comparing means connected to said input means for comparing said user inputted characters to said stored textual version of said text passage.

16. A system as in claim 15 further including means responsive to said stored text passage and to said input means and also connected to said display means for:

controlling said display means to display only those user inputted characters which said comparing means reveals correspond to the characters of said stored text passage and to also display punctuation and spaces found in said stored text passage so as to relieve said user from having to input said punctuation and spaces.

17. An interactive language learning system comprising:

means for providing digitized speech signals representing a sequence of spoken phrases having an initial order;

re-ordering means connected to said providing means for re-ordering said plural phrases into a sequence having an order different from said initial order;

user input means operatively connected to said re-ordering means and operable by a user for permitting said user to further re-order said plural phrases into a user-specified order; and

speech processing means, connected to said providing means and to said re-ordering means and responsive to said digitized speech signals, for generating audible versions of said phrases so as to provide audible cues to said user.

18. An interactive language learning system as in claim 17 further including a display for displaying symbols representing said plural phrases in at least said user-specified order.

19. An interactive language learning system comprising:

random access storing means for storing digitized speech signals representing a sequence of plural speech phonemes;

display means for displaying the text corresponding to said plural phonemes;

user input means, operatively connected to said storing means and to said display means and operable by a user, for permitting said user to specify a sub-sequence of said speech phoneme sequence by delimiting corresponding text; and

speech processing means, connected to said storing means and to said user input means and responsive to said stored digitized speech signals, for generating audible versions of phonemes in said user-specified sub-sequence.

20. An interactive language learning system as in claim 19 further including:

means for converting audio signals representing user speech into digitized speech signals representing said user speech and for subsequently reconverting said speech signals representing said user speech to audio signals so as to permit said user to listen to and compare his own speech with said audible versions of phonemes in said user-specified sub-sequence.

21. An interactive language learning system as in claim 19 wherein said user input means includes pointing symbol means operatively coupled to said display means for causing at least one pointing symbol to be displayed on said display at a position relative to said displayed test that is specified by said user, said pointing symbol means indicating said specified sub-sequence.

22. An interactive language learning system comprising:

a storage device which stores digitized speech signals representing a sequence of speech phrases;

a display capable of displaying symbols corresponding to said phrases;

a user input device operatively connected to said storing means and to said display means and operable by a user;

a speech processing circuit, connected to said storage device and to said user input device and responsive to said stored digitized speech signals, said speech processing circuit generating audible versions of said phrases in response to said digitized speech signals, said speech processing circuit also converting user speech into further digitized speech signals and generating audible versions of said user speech in response to said further digitized speech signals; and

a processor operatively coupled to said display, said user input device and said speech processing circuit, said processor being preprogrammed so as to selectively perform at least the following functions;

(a) a lab function in which said processor controls said speech processing circuit to alternately generate audible versions of said stored digitized speech signals and said further user-speech digitized speech signals so as to permit the user to compare his own pronunciation with the pronunciation specified by said stored digitized speech signals,

(b) a clip function in which said processor permits said user to specify subsequences of said speech phrases sequence and controls said speech processing circuit to generate an audible version of said subsequences, and

(c) a sound sort function in which said processor controls said display to present said symbols in an order different from the order specified by said sequence, permits said user to re-order said presented symbols, and controls said speech processing circuit to generate audible versions of said phrases.

23. An interactive language learning system as in claim 22 wherein said processor in said clip function controls said speech processing circuit to alternately generate audible versions of said stored digitized speech signals and said further user-speech digitized speech signals so as to permit said user to listen to and compare his own speech with said audible versions of said user-specified sub-sequence.

24. An interactive language learning system as in claim 22 wherein said processor in said sound sort function permits said user to compare said re-ordered sequence with said initial order specified by said sequence.

25. An interactive language learning system comprising:

a storage device which stores digitized speech signals representing a sequence of speech phrases;

a display capable of displaying symbols corresponding to said phrases;

a user input device operatively connected to said storing means and to said display means and operable by a user;

a speech processing circuit, connected to said storage device and to said user input device and responsive to said stored digitized speech signals, said speech processing circuit generating audible versions of said phrases in response to said digitized speech signals, said speech processing circuit also converting user speech into further digitized speech signals and generating audible versions of said user speech in response to said further digitized speech signals; and

a data processor including the following;

means for controlling said speech processing circuit to alternately generate audible versions of said stored digitized speech signals and said further user-speech digitized speech signals so as to permit the user to compare his own pronunciation with the pronunciation specified by said stored digitized speech signals,

means for permitting said user to specify subsequences of said speech phrases sequence and for controlling said speech processing circuit to generate an audible version of said subsequences, and

means for controlling said display to present said symbols in an order different from the order specified by an initial sequence, for permitting said user to re-order said presented symbols, for controlling said speech processing circuit to generate audible versions of said phrases, and for permitting said user to audibly and/or visually compare said re-ordered sequence with said initial sequence.