Speech recognition apparatus and method for learning

• US 5,717,828 A
• Filed: 03/15/1995
• Issued: 02/10/1998
• Est. Priority Date: 03/15/1995
• Status: Expired due to Fees

• Current Assignee: Activision Entertainment Holdings, Inc. (Vivendi SE)
• Original Assignee: Syracuse Language Systems
• Inventors: Rothenberg, Martin
• Primary Examiner(s): Zele, Krista
• Assistant Examiner(s): Wolinsky, Scott
• Application Number:
  US08/405,049
• Time in Patent Office: 1,063 Days
• Field of Search: 395/2.79, 395/2.85, 395/2.1, 395/2, 395/2.8, 395/2.81, 395/2.86, 395/2.6, 395/2.61, 434/185, 434/156, 434/159, 434/167
• US Class Current: 704/251
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems

Abstract

The invention extends the capability of conventional computer speech recognition programs to reliably recognize and understand large word and phrase vocabularies for teaching language comprehension and oral production skills. At each step of a teaching program, information is supplied to the user such that some responses in the language being taught are correct (or appropriate) and some are incorrect (or inappropriate), with these respective sets of responses judiciously selected to teach some language aspect. A subset of allowable correct and incorrect responses is selected such that a speech recognition subprogram readily discerns certain allowable responses from other allowable responses, including each incorrect response being discriminable from each correct response. The meanings of at least the correct allowable responses are made clear by aural or visual information, such as graphic images, printed text, or translations into the user'"'"'s native language. A model of the correct pronunciation of each allowable response may be available, with these models presented either exhaustively or selectively. The invention includes a mechanism for differentiating between improperly formed and deficient productions and providing corrective feedback upon deficient production. The mechanism for differentiating may include dummy, improperly formed speech models, and augmentative voice analyzers may be used. Augmentative monitors capturing non-acoustic measurements may be included.

Claims

1. A method for using a computer to teach language skills comprising:

presenting auditory or visual information to a user that defines for the user at least one correct response and at least one incorrect response, each response corresponding to a stored internal speech pattern, wherein said speech patterns are readily differentiable from each other by a comparison subprogram, which is part of a computer program running on said computer;

selecting, from a set of stored internal speech patterns, using said computer program, a subset containing at least one internal speech pattern corresponding to a correct response and at least one internal speech pattern corresponding to an incorrect response, each internal speech pattern corresponding to a correct response being readily differentiable from each internal speech pattern corresponding to an incorrect response by said comparison subprogram;

presenting or making available to a user auditory or visual information that presents the correct pronunciation of each of the responses corresponding to an internal speech pattern in the subset and also the meaning of at least the correct responses corresponding to the internal speech patterns in the subset;

obtaining a speech response segment from the user;

comparing, using the comparison subprogram, the speech response segment to the subset of internal speech patterns and determining the degree of matching to each internal speech pattern in the subset of internal speech patterns; and

presenting to the user an auditory or visual response to the speech segment based on the results of the step of comparing.

2. The method of claim 1 wherein the step of selecting further comprises adding to the subset an internal speech pattern corresponding to an improperly formed or pronounced response which is a variation of a properly formed or pronounced response in the subset of responses.

3. The method of claim 2 wherein the step of comparing further comprises directing the user toward a more properly formed or pronounced speech response if the speech response segment most closely matches the improperly formed or pronounced response.

4. The method of claim 1 wherein the step of selecting further comprises selecting and adding to the subset of internal speech patterns an improperly pronounced variation of a response in the subset of internal speech patterns such that the improperly pronounced response represents a speech response segment indicative of the user'"'"'s native speech pattern.

5. The method of claim 4 wherein the step of comparing further comprises directing the user toward a more properly pronounced speech response if the speech response segment most closely matches an improperly pronounced response.

6. The method of claim 1 wherein the step of presenting or making available further comprises automatically representing aurally to the user each response in the subset of internal speech patterns.

7. The method of claim 1 wherein the step of presenting or making available further comprises automatically representing visually to the user each response in the subset of internal speech patterns.

8. The method of claim 1 wherein the step of presenting or making available further comprises representing aurally to the user a response in the subset of internal speech patterns when a signal is generated by the user.

9. The method of claim 1 wherein the step of presenting auditory or visual information further comprises encouraging the user to use perceptual skills, cognitive skills, memory skills, or motor skills to identify a correct response.

10. The method of claim 1 wherein the step of obtaining further comprises obtaining a measurement of the nasality of the speech response segment.

11. The method of claim 1 wherein the step of obtaining further comprises obtaining the spectral qualities of the speech response segment.

12. The method of claim 1 wherein the step of obtaining further comprises obtaining a measurement of the airflow from a mouth of the user during the release phase of a stop or plosive consonant in the speech response segment.

13. The method of claim 1 wherein the step of obtaining further comprises obtaining a measurement of the intonation pattern of the speech response segment.

14. A language learning apparatus comprising:

at least one transducer;

a digital computer having both visual and auditory output capabilities and auditory input capabilities; and

a program running on the digital computer which analyzes input speech patterns derived from said at least one tranducer, the program including a comparison program which can compare the input speech patterns to each member of a subset of a total set of speech patterns stored in a memory located in the digital computer, with a result of each such comparison being output to a decision mechanism program, and the comparison program receiving an identification code for specifying which of a total set of internally stored speech patterns are to be included in the subset for each such comparison.

15. The apparatus of claim 14 further comprising an augmentative speech analyzer with input from at least one transducer and output connected to a decision mechanism.

16. The apparatus of claim 15 wherein the augmentative speech analyzer is a voice intonation analyzer.

17. The apparatus of claim 15 wherein the augmentative speech analyzer is a voice spectral quality analyzer.

18. The apparatus of claim 15 wherein the augmentative speech analyzer is a nasal vibration monitor.

19. The apparatus of claim 15 wherein the augmentative speech analyzer is an electroglottograph.

20. The apparatus of claim 15 wherein the augmentative speech analyzer is a throat microphone.

21. The apparatus of claim 15 wherein the augmentative speech analyzer is a nasal airflow monitor.

22. The apparatus of claim 15 wherein the augmentative speech analyzer is an oral airflow monitor.

23. A method for using a computer to teach language skills comprising:

presenting auditory or visual information to a user that defines for the user a plurality of appropriate oral responses, at least two of said appropriate oral responses being in a set of at least two internal speech patterns and being readily differentiable from each other by a comparison subprogram, which is part of a computer program running on said computer;

selecting, from the set of at least two internal speech patterns, using said computer program, a subset of responses that are properly pronounced, and that contains at least two appropriate responses, each appropriate response being readily differentiable from each other appropriate response by said comparison subprogram;

presenting or making available to a user auditory or visual information that communicates the correct pronunciation of cach of the responses in the subset;

obtaining a speech response segment from the user;

comparing, using the comparison subprogram, the speech response segment to the subset of responses and determining the degree of matching to each response in the subset of responses; and

presenting to the user an auditory or visual response to the speech response segment based on the results of the step of comparing.