Feedback modification for accent reduction

• US 5,995,932 A
• Filed: 12/31/1997
• Issued: 11/30/1999
• Est. Priority Date: 12/31/1997
• Status: Expired due to Fees

• Current Assignee: Scientific Learning Corp. (Carnegie Learning, Inc.)
• Original Assignee: Scientific Learning Corp. (Carnegie Learning, Inc.)
• Inventors: Houde, John F.
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Chawan, Vijay B
• Application Number:
  US09/001,508
• Time in Patent Office: 699 Days
• Field of Search: 704/255, 704/254, 704/200, 704/275, 704/270, 704/261, 704/271, 434/169, 434/167, 434/157
• US Class Current: 704/261
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G09B 5/04 with audible presentation of the material to be studied sound recording or reproducing G11B
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

A training system used while a person is speaking uses a feedback modification technique to reduce accents. As the speaker is speaking, the system feeds back to the speaker the speaker'"'"'s speech in "real-time" so that the speaker, in effect, hears what he or she is saying while saying it. The system includes a detector configured to monitor a speaker'"'"'s speech to detect a preselected target vowel sound that the speaker wishes to produce accurately. In response to the detector detecting a "target" vowel sound, a cue generator generates a sensory cue (e.g., an amplification of the "target" vowel sound) that is perceived by the speaker. As the speaker is speaking, the generator feeds back to the speaker the sensory cue along with the speech so that the cue is coincident with the "target" vowel sound.

Claims

1. A method for training a speaker to reduce the speaker'"'"'s accent in speaking a language, the speaker'"'"'s speech comprising a series of sounds, the method comprising:

preselecting a sound in the language;

monitoring the speaker'"'"'s speech for the preselected sound using an acoustic input device;

generating a cue in response to detecting the preselected sound using a signal processing device; and

feeding back the cue to the speaker using an output device, the cue being in a form that is detectable to the speaker, wherein the cue is indicative of the speaker forming the preselected sound correctly.

2. The method of claim 1 wherein the cue is fed back to the speaker with a delay of less than or equal to thirty milliseconds.

3. The method of claim 1 wherein the cue is an amplified in volume version of the detected preselected sound.

4. The method of claim 3 wherein the amplification is configurable.

5. The method of claim 1 wherein the preselected sound in a vowel sound.

6. The method of claim 5 wherein the preselected sound is a vowel sound missing from a language natively spoken by the speaker.

7. The method of claim 6 wherein monitoring the speaker'"'"'s speech comprises monitoring the speaker'"'"'s speech for F1 and F2 formants that substantially match F1 and F2 formants of the preselected sound.

8. An apparatus for training a speaker to reduce the speaker'"'"'s accent in speaking a language, the speaker'"'"'s speech comprising a series of sounds, the apparatus comprising:

means for monitoring the speaker'"'"'s speech for a preselected sound;

means for generating a cue in response to detecting the preselected sound; and

means for feeding back the cue to the speaker in a form that is detectable to the speaker the, cue indicating that the speaker has formed the preselected sound correctly.

9. The apparatus of claim 8 wherein the means for feeding back is configured to feed back the cue to the speaker with a delay of less than or equal to thirty milliseconds.

10. The apparatus of claim 8 wherein the cue is an amplification in volume of the detected preselected sound.

11. The apparatus of claim 10 wherein the amplification is configurable.

12. The apparatus of claim 8 wherein the preselected sound in a vowel sound.

13. The apparatus of claim 12 wherein the preselected sound is a vowel sound different from a vowel set of a language natively spoken by the speaker.

14. The apparatus of claim 13 wherein monitoring the speaker'"'"'s speech comprises monitoring the speaker'"'"'s speech for F1 and F2 formants that substantially match F1 and F2 formants of the preselected sound.

15. A system for training a speaker to modify the speaker'"'"'s accent in speaking a language, the speaker'"'"'s speech comprising a series of sounds, the system comprising:

an acoustic input device, wherein the acoustic input device is configured to receive the speaker'"'"'s speech;

an acoustic output device, wherein the acoustic output device is configured to generate sound that can be sensed by the speaker; and

an accent modifier coupled to the acoustic input device and the acoustic output device, wherein the accent modifier is configured to;

receive a signal representing the speaker'"'"'s speech from the acoustic input device,

detect whether a sound in the speaker'"'"'s speech substantially matches a preselected sound,

generate a cue in response to detecting the match, and

provide the cue to the acoustic output device.

16. The system of claim 15 wherein the accent modifier is further configured to provide to the acoustic output device the signal representing each unmatching sound at a reference amplification level.

17. The system of claim 16 wherein the cue is an electromagnetic signal representing the detected preselected sound with an amplification greater than the reference amplification level.

18. The system of claim 17 wherein the accent modifier provides the cue and the electromagnetic signals representing the unmatched sounds to the acoustic output device with a delay of less than or equal to thirty milliseconds.

19. The system of claim 15 wherein the preselected sound is a vowel sound missing from a language natively spoken by the speaker.

20. The system of claim 15 wherein the accent modifier is configured to monitor the speaker'"'"'s speech for F1 and F2 formants that substantially match F1 and F2 formants of the preselected sound.

21. An accent modifier for use in combination with an acoustic input device and an acoustic output device in training a speaker to modify the speaker'"'"'s accent in speaking a language, the acoustic input device configured to receive the speaker'"'"'s speech and generate a signal as a function of the speech, the acoustic output device configured to generate sound that can be sensed by the speaker, the speaker'"'"'s speech comprising a series of sounds, the accent modifier having an input interface configured to be coupled to the acoustic input device and an output interface configured to be coupled to the acoustic output device, wherein the accent modifier is configured to:

receive a signal representing the speaker'"'"'s speech at the input interface,

detect whether a sound in the speaker'"'"'s speech substantially matches a preselected sound,

generate a cue in response to detecting the match, and

provide the cue to the output interface.

22. The accent modifier of claim 21 wherein the accent modifier is further configured to provide to the output device a signal representing each unmatching sound at a reference amplification level.

23. The accent modifier of claim 22 wherein the cue is a signal representing the detected preselected sound with an amplification greater than the reference amplification level.

24. The accent modifier of claim 23 wherein the accent modifier provides the cue and the signals representing the unmatched sounds to the acoustic output device with a delay less than or equal to thirty milliseconds.

25. The accent modifier of claim 21 wherein the preselected sound is a vowel sound missing from a language natively spoken by the speaker.

26. The accent modifier of claim 21 wherein the accent modifier is configured to monitor the speaker'"'"'s speech for F1 and F2 formants that substantially match F1 and F2 formants of the preselected sound.

27. An accent modifier for use in combination with an acoustic input device and an acoustic output device in training a speaker to modify the speaker'"'"'s accent in speaking a language, the acoustic input device configured to receive the speaker'"'"'s speech and generate a signal as a function of the speech, the acoustic output device configured to generate sound that can be sensed by the speaker, the speaker'"'"'s speech comprising a series of sounds, wherein the accent modifier comprises:

an input interface configured to be coupled to the acoustic input device;

an output interface configured to be coupled to the acoustic output device;

a formant estimator coupled to the input interface, the formant estimator being configured to extract formant information from the speaker'"'"'s speech;

a formant pattern recognizer coupled to the formant estimator, the sound recognizer being configured to generate a match score indicative of how well formants in sounds of the speaker'"'"'s speech that match formants of a preselected sound; and

a cue generator coupled to the formant pattern recognizer and the output interface, the cue generator being configured to generate a cue as a function of the match score.

28. The accent modifier of claim 27 wherein the cue generator is further configured to provide to the output interface a signal representing each unmatching sound at a reference amplification level.

29. The accent modifier of claim 28 wherein the cue is a signal representing a detected sound having formants substantially matching formants of the preselected sound, with an amplification greater than the reference amplification level.