Speech training method with alternative proper pronunciation database

• US 6,963,841 B2
• Filed: 01/09/2003
• Issued: 11/08/2005
• Est. Priority Date: 04/21/2000
• Status: Active Grant

• Current Assignee: Lessac Technologies Incorporated
• Original Assignee: Lessac Technologies Incorporated
• Inventors: Wilson, H. Donald, Marple, Gary, Handal, Anthony H., Lessac, Michael
• Primary Examiner(s): Azad, Abul K.
• Application Number:
  US10/339,370
• Publication Number:
  US 20030182111A1
• Time in Patent Office: 1,034 Days
• Field of Search: 704/235, 704/251, 704/257, 704/260, 704/243, 704/270, 704/276
• US Class Current: 704/270
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G09B 5/04 with audible presentation of the material to be studied sound recording or reproducing G11B

Abstract

In accordance with a present invention speech training system is disclosed. It uses a microphone to receive audible sounds input by a user into a first computing device having a program with a database consisting of (i) digital representations of known audible sounds and associated alphanumeric representations of the known audible sounds, and (ii) digital representations of known audible sounds corresponding to mispronunciations resulting from known classes of mispronounced words and phrases. The method is performed by receiving the audible sounds in the form of the electrical output of the microphone. A particular audible sound to be recognized is converted into a digital representation of the audible sound. The digital representation of the particular audible sound is then compared to the digital representations of the known audible sounds to determine which of those known audible sounds is most likely to be the particular audible sound being compared to the sounds in the database. In response to a determination of error corresponding to a known type or instance of mispronunciation, the system presents an interactive training program from the computer to the user to enable the user to correct such mispronunciation.

Claims

1. A computer-implemented speech training method utilizing a computerized system, the computerized system being capable of inputting speech from a user, of recognizing the input speech and of outputting to the user information as to the speech recognized and audible words to be heard by the user wherein the computerized system has a data processing capability, optionally has a display screen and has access to a data storage facility and wherein a first proper pronunciation database of properly and individually pronounced words is stored in the data storage facility, the speech training method comprising:

(a) the user inputting to the computerized system desired words as speech;

(b) the computerized system outputting to the user information as to the speech recognized including information regarding at least one improperly pronounced word and information as to proper pronunciation of the at least one improperly pronounced word to help the user correct the mispronunciation;

(c) employing, stored in the data storage facility, a second proper pronunciation database comprising alternative phonetic sequences for the pronunciation of selected words in the first proper pronunciation database, the selected words being words each having an alternative proper pronunciations in a sequence of words; and

(d) the computerized system referencing said first and second proper pronunciation databases for the speech input by the user to determine the propriety of pronunciation of each desired word according to the context of the input word when each input word is recognized;

wherein the information regarding the at least one improperly pronounced word comprises information as to mispronunciation of the word as an individual word or as a word in one of the sequences of words in the second proper pronunciation database whereby the system can help the user speak the at least one improperly pronounced word correctly according to whether the word is to be pronounced as an individual word or as a word in one of the sequences of words.

2. A speech training method according to claim 1 wherein the second proper pronunciation database includes an associated grapheme and pronunciation rule for each selected word and for each alternative phonetic sequence for the selected word.

3. A speech training method according to claim 2 wherein the alternative phonetic sequences comprise a phonetic sequence for each selected word together with a phonetic sequence for the next preceding word in context or for the next succeeding word in context or for both said next preceding and succeeding words.

4. A speech training method according to claim 3 wherein the phonetic sequences for each selected word are stored in the database as a single pronunciation record for the selected word.

5. A speech training method according to claim 2 wherein the alternative phonemes and corresponding alternative graphemes for a selected word map to a single correct text form of the selected word.

6. A speech training method according to claim 1 wherein the context of the selected word comprises the position of the selected word in a sequence of words uttered.

7. A speech training method according to claim 1 wherein the first and second proper pronunciation databases are available for audio playback to the user through the audio output device.

8. A speech training method according to claim 7 further comprising:

(e) the user again speaking the at least one improperly pronounced word or words into the computerized system with attempted improved pronunciation;

(f) the system indicating to the user whether the at least one word or words is properly pronounced and playing back to the user through the audio output device a properly pronounced word or words intended to train the user in the proper pronunciation of the at least one improperly pronounced word.

9. A speech training method according to claim 8 comprising generating the second proper pronunciation database by recording statistically representative samples of the phonetic sequences of combinations of words in a language reference dictionary pronounced in accordance with pronunciation rules wherein the pronunciation rules are dependent upon the selected word, and the relationship of the selected word to the beginning or following word, or both the beginning and following words, in a phrase to be spoken.

10. A speech training method according to claim 9 wherein the statistically representative sample comprises sentence structures and pronunciation sequences and by the method comprises employing a computational linguistic analysis of beginnings and endings of words and by identifying combinations that are cognitively meaningful to draw the statistically representative sample.

11. A speech training method according to claim 9 wherein each word in the sample is pronounced by certified speech practitioners of a common method of voice use and training, optionally the Lessac system, the method of voice use and training having its own phonetics and pronunciation rules.

12. A speech training method according to claim 11 wherein the spoken words in the sample are representative of words spoken by high pitched, normally pitched, low pitched, rapidy spoken, normally spoken and slowly spoken speakers, by male, female, young and old speakers and by speakers of normal pitch.

13. A speech training method according to claim 11 comprising including in the spoken words in the recorded sample a full range of audible frequencies including harmonics created by the voice, the audible frequencies optionally being in the range of from about 500 to about 20,000 Hz.

14. A speech training method according to claim 11 comprising employing a certified speech practitioner speaker to listen to high-fidelity reproductions of the pronunciation of words for the second proper pronunciation database made by other certified speech practitioners.

15. A speech training method according to claim 14 wherein the listening practitioner notes and removes words having unintended errors of pronunciation and removes preceding and following words having proper alternative pronunciations.

16. A speech training method according to claim 8 wherein the first and second proper pronunciation databases comprise standard native language pronunciations, optionally standard American English pronunciations, of the phonetic sequences for the words in the databases.

17. A speech training method according to claim 16 comprising further employing, stored in the data storage facility, a proper pronunciation dialect database comprising acoustic profiles, graphemes, and text for identifying correct alternative words and pronunciations of words according to a known dialect of the native language.

18. A speech training method according to claim 16 wherein the proper pronunciation dialect database comprises a database of rules-based dialectic pronunciations according to the Lessac system of pronunciation and communication.

19. A speech training method according to claim 16 comprising further employing, stored in the data storage facility, an alternative proper pronunciation dialect database comprising alternative phonetic sequences for a dialect where the pronunciation of a word is modified according to the word'"'"'s position in a sequence of words.

20. A speech training method according to claim 16 comprising further employing, stored in the data storage facility, a pronunciation error database of phonetic sequences, acoustic profiles, graphemes and text for correctly identifying alternative pronunciations of words according to commonly occurring errors of articulation by native speakers of the language.

21. A speech training method according to claim 16 comprising further employing, stored in the data storage facility, a Lessac pronunciation error database of common mispronunciations according to the Lessac system of pronunciation and communication.

22. A speech training method according to claim 16 comprising employing, stored in the data storage facility, an individual word mispronunciation database.

23. A speech training method according to claim 16 comprising employing, stored in the data storage facility, a database of common word mispronunciations occurring when speaking a sequence of words.

24. A speech training method according to claim 16 comprising further employing, stored in the data storage facility, two or more databases selected from the group consisting of:

a proper pronunciation dialect database comprising acoustic profiles, graphemes, and text for identifying correct alternative words and pronunciations of words according to a known dialect of the native language;

a database of rules-based dialectic pronunciations according to the Lessac system of pronunciation and communication an alternative proper pronunciation dialect database comprising alternative phonetic sequences for a dialect where the pronunciation of a word is modified because of the word'"'"'s position in a sequence of words;

a pronunciation error database of phonetic sequences, acoustic profiles, graphemes and text for correctly identifying alternative pronunciations of words according to commonly occurring errors of articulation by native speakers of the language;

a Lessac pronunciation error database of common mispronunciations according to the Lessac system of pronunciation and communication;

an individual word mispronunciation database; and

a database of common word mispronunciations when speaking a sequence of words.

25. A speech training method according to claim 24 wherein the computerized system comprises a standard language model, a common phoneme error model, an exercise word model, an exercise phrase model, an exercise word error model and an exercise phrase error model.

26. A speech training method according to claim 8 wherein the computerized system comprises the display screen and a training text displayable on the display screen wherein the training text includes both exercise word models and exercise phrase models wherein the exercise word models and the exercise phrase models are organized in groups having common mispronunciation characteristics.

27. A speech training method according to claim 8 wherein the computerized system comprises at least one model selected from the group consisting of a standard language model, a common phoneme error model, an exercise word model, an exercise phrase model, an exercise word error model and an exercise phrase error model.

28. A speech training method according to claim 24 wherein the computerized system comprises a speech recognition program capable of recognizing words, mispronunciations and phoneme errors, and comprising audio recordings of proper pronunciations, the speech recognition program being operative during speech training sessions employing with the inventive interactive program wherein if no pronunciation error has occurred twice consecutively, the computerized system leaves the speech training routine and functions as a voice recognition system.

29. A speech training method according to claim 8 wherein the computerized system comprises a speech recognition program capable of recognizing words, mispronunciations and phoneme errors, and comprising audio recordings of the user'"'"'s pronunciations, plus speech sounds for proper pronunciations, the speech recognition program being operative during the speech training.

30. A speech training method according to claim 29 wherein the computerized system is loaded with a speech recognition engine, the speech recognition engine having text editing and speech training utilities and with databases of word models, phrase models, vocal recordings and error models, the computerized system optionally being a personal computer available to the user.

31. A speech training method according to claim 8 wherein the computerized system comprises a speech recognition engine and wherein speech training and speech recognition are integrated to provide smooth switching between training and recognition modes to enable speech recognition to proceed and be intermittently supplemented with speech training as improperly pronounced words are encountered.

32. A speech training method as claimed in claim 31 wherein said computerized system provides the user speech training using Lessac System techniques.

33. A speech training method as claimed in claim 32, wherein the computerized system instructs the user in the correct pronunciation of a phoneme in terms of the sound of a musical instrument.

34. A method of speech training as claimed in claim 32, wherein the user is given the option of receiving speech training or training the program to recognize the user'"'"'s speech pattern.

35. Programs and databases recorded on a CD-ROM or on permanent electronic storage media wherein the programs and databases capable of implementing a speech training method according to claim 1.

36. A computerized speech training system comprising a data processing unit, random access volatile memory available to the data processing unit, an audio device to audibilize output from the computerized system to be heard by the user and, optionally, a display screen, the computerized system having access to a data storage facility and further comprising stored in the data storage facility, a software program and at least one database to implement a speech training method wherein the software program comprises procedures to effect a speech training method according to claim 1.

37. A computerized speech training system according to claim 36 comprising, stored in the data storage facility, two or more databases selected from the group consisting of:

a proper pronunciation dialect database comprising acoustic profiles, graphemes, and text for identifying correct alternative words and pronunciations of words according to a known dialect of the native language;

a database of rules-based dialectic pronunciations according to the Lessac system of pronunciation and communication;

an alternative proper pronunciation dialect database comprising alternative phonetic sequences for a dialect where the pronunciation of a word is modified because of the word'"'"'s position in a sequence of words;

a pronunciation error database of phonetic sequences, acoustic profiles, graphemes and text for correctly identifying alternative pronunciations of words according to commonly occurring errors of articulation by native speakers of the language;

a Lessac pronunciation error database of common mispronunciations according to the Lessac system of pronunciation and communication;

an individual word mispronunciation database; and

a database of common word mispronunciations when speaking a sequence of words.

38. A computer-implemented speech training method utilizing a computerized system, the computerized system being capable of inputting speech from a user, of recognizing the input speech and of outputting to the user information as to the speech recognized and audible words to be heard by the user wherein the computerized system has a data processing capability, optionally has a display screen, and has access to a data storage facility and wherein a first proper pronunciation database of properly and individually pronounced words is stored in the data storage facility the speech training method comprising:

(a) the user inputting to the computerized system desired words as speech;

(b) the computerized system outputting to the user information as to the speech recognized including information regarding at least one improperly pronounced word and information as to proper pronunciation of the at least one improperly pronounced word to help the user correct the mispronunciation;

(c) employing stored in the data storage facility, a second proper pronunciation database comprising alternative phonetic sequences for the pronunciation of selected words in the first proper pronunciation database, the selected words being words each having an alternative proper pronunciations in a sequence of words; and

(d) the computerized system referencing said first and second proper pronunciation databases for the speech input by the user to determine the propriety of pronunciation of each desired word according to the context of the input word when each input word is recognized wherein the information regarding the at least one improperly pronounced word comprises information as to mispronunciation of the word as an individual word or as a word in one of the sequences of words in the second proper pronunciation database whereby the system can help the user speak the at least one improperly pronounced word correctly according to whether the word is to be pronounced as an individual word or as a word in one of the sequences of words;

(e) generating the proper pronunciation database by recording specimen samples representative of the phonetic sequences of combinations of words in a language reference dictionary; and

(f) each word in the sample being pronounced by persons who are certified speech practitioners of a common method of voice use and training, optionally the Lessac system, the method of voice use and training having its own phonetics and pronunciation rules.

39. A speech training method according to claim 38 wherein the spoken words in the sample are representative of words spoken by high pitched, normally pitched, low pitched, rapidly spoken, normally spoken and slowly spoken speakers, by male, female, young and old speakers and by speakers of normal pitch.

40. A speech training method according to claim 39 comprising employing a certified speech practitioner speaker to listen to high-fidelity reproductions of the pronunciation of words for the second proper pronunciation database, their recordings being made by other certified speech practitioners.

41. A speech training method according to claim 40 wherein the listening practitioner notes and removes words having unintended errors of pronunciation and removes preceding and following words having proper alternative pronunciations.

42. A speech training method according to claim 38 comprising including in the spoken words in the recorded sample a full range of audible frequencies in the range of from about 500to about 20,000 Hz.