Language independent suprasegmental pronunciation tutoring system and methods

• US 6,397,185 B1
• Filed: 03/29/1999
• Issued: 05/28/2002
• Est. Priority Date: 03/29/1999
• Status: Expired due to Term

• Current Assignee: Lexia Learning Systems LLC (Rosetta Stone Incorporated), Rosetta Stone Limited (Rosetta Stone Incorporated)
• Original Assignee: Betteraccent
• Inventors: Komissarchik, Edward, Komissarchik, Julia
• Primary Examiner(s): Tsang, Fan
• Assistant Examiner(s): Opsasnick, Michael N.
• Application Number:
  US09/282,050
• Time in Patent Office: 1,156 Days
• Field of Search: 704/270,276 434/156,167,169
• US Class Current: 704/270
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence

Abstract

A pronunciation training system and methods are provided as a series of programmed routines stored on an item of removable storage media, and select information generated by a speech analysis engine to compute and display graphical representations of metrics useful to a student. The student selects from among a plurality of pre-recorded utterances spoken by a native speaker, and the student then records his or her pronunciation of the utterance. The software computes and displays graphical metrics for the native speaker'"'"'s utterance and the student'"'"'s utterance, in any of a variety of formats, on a side-by-side basis. The system also permits the student to repeat selected phrases and to monitor improvement by similarity between the graphical metrics.

Claims

1. A speech analysis system for voice and pronunciation training, the system stored as a series of programmed routines on a removable item of storage media, the system comprising:

a data file that stores a pre-recorded utterance by a native speaker comprising a sequence of syllables and a first example metric corresponding to a pitch value of a vocal part of each one of the sequence of syllables;

a speech capture routine that captures a sample utterance spoken by a user;

a segmentation routine that segments the sample utterance into a sequence of syllables;

a computation routine that computes a first sample metric corresponding to a pitch value of a vocal part of each one of the sequence of syllables for the sample utterance; and

a display routine for graphically displaying the first example metric and the first sample metric to the user on a side-by-side basis.

2. The speech analysis system of claim 1 wherein the segmentation routine divides the sample utterance into vowel segments and non-vowel segments.

3. The speech analysis system of claim 2 wherein the segmentation routine regroups the vowel segments and non-vowel segments into syllables depending upon relative energy levels of the vowel segments.

4. The speech analysis system of claim 1 further comprising a routine that deletes parasitic vowels at the beginning or end of the sample utterance.

5. The speech analysis system of claim 1 wherein the display routine graphically identifies boundaries between syllables in the sequence of syllables of the sample utterance.

6. The speech analysis system of claim 1 further comprising a series of navigation screens that enable a user to select from amongst a curriculum comprising a plurality of pre-recorded utterances.

7. The speech analysis system of claim 1 wherein the segmentation routine and the computation routine are capable of processing pre-recorded utterances independently of the language of the native speaker.

8. The speech analysis system of claim 1 wherein the data file also stores a second example metric corresponding to a duration, and an energy of the vocal part of, each one of the sequence of syllables of the pre-recorded utterance, the computation routine further computing a second sample metric corresponding to a duration and an energy of the vocal part of each one of the sequence of syllables of the sample utterance, and the display routine further selectably displays either the first example metric and the first sample metric to the user on a side-by-side basis or the second example metric and the second sample metric to the user on a side-by-side basis.

9. The speech analysis system of claim 8 wherein the display routine graphically represents the sequence of syllables as a series of steps, wherein for each syllable in the sequence, a length of each step represents the duration of the corresponding syllable and a height of each step represents the energy of the corresponding syllable.

10. The speech analysis system of claim 1 wherein the display routine further selectably displays either the first example metric and the first sample metric to the user on a side-by-side basis or a waveform for the pre-recorded utterance and a waveform for the sample utterance to the user on a side-by-side basis.

11. A speech analysis system for voice and pronunciation training, the system stored as a series of programmed routines on a removable item of storage media, the system comprising:

a data file that stores a pre-recorded utterance by a native speaker comprising a sequence of syllables and a first example metric corresponding to a duration, and an energy of a vocal part of, each one of the sequence of syllables;

a speech capture routine that captures a sample utterance spoken by a user;

a segmentation routine that segments the sample utterance into a sequence of syllables;

a computation routine that computes a first sample metric corresponding to a duration, and an energy of a vocal part of, each one of the sequence of syllables for the sample utterance; and

a display routine for graphically displaying the first example metric and the first sample metric to the user on a side-by-side basis.

12. The speech analysis system of claim 11 wherein the display routine graphically represents the sequence of syllables as a series of steps, wherein for each syllable in the sequence, a length of each step represents the duration of the corresponding syllable and a height of each step represents the energy of the corresponding syllable.

13. The speech analysis system of claim 11 wherein the segmentation routine divides the sample utterance into vowel segments and non-vowel segments.

14. The speech analysis system of claim 13 wherein the segmentation routine regroups the vowel segments and non-vowel segments into syllables depending upon relative energy levels of the vowel segments.

15. The speech analysis system of claim 11 further comprising a routine that deletes parasitic vowels at the beginning or end of the sample utterance.

16. The speech analysis system of claim 11 wherein the display routine graphically identifies boundaries between syllables in the sequence of syllables of the sample utterance.

17. The speech analysis system of claim 11 further comprising a series of navigation screens that enable a user to select from amongst a curriculum comprising a plurality of pre-recorded utterances.

18. The speech analysis system of claim 11 wherein the segmentation routine and the computation routine are capable of processing pre-recorded utterances independently of the language of the native speaker.

19. The speech analysis system of claim 11 wherein the data file also stores a second example metric corresponding to a pitch value of a vocal part of each one of the sequence of syllables of the pre-recorded utterance, the computation routine further computing a second example metric corresponding to a pitch value of a vocal part of each one of the sequence of syllables of the sample utterance, and the display routine further selectably displays either the first example metric and the first sample metric to the user on a side-by-side basis or the second example metric and the second sample metric to the user on a side-by-side basis.

20. The speech analysis system of claim 11 wherein the display routine further selectably displays either the first example metric and the first sample metric to the user on a side-by-side basis or a waveform for the pre-recorded utterance and a waveform for the sample utterance to the user on a side-by-side basis.

21. A method of analyzing speech for voice and pronunciation training, the method comprising:

storing a pre-recorded utterance by a native speaker comprising a sequence of syllables and a first example metric corresponding to a pitch value of a vocal part of each one of the sequence of syllables;

capturing a sample utterance spoken by a user;

segmenting the sample utterance into a sequence of syllables;

computing a first sample metric corresponding to a pitch value of a vocal part of each one of the sequence of syllables for the sample utterance; and

graphically displaying the first example metric and the first sample metric to the user on a side-by-side basis.

22. The method of claim 21 wherein segmenting the sample utterance further comprises dividing the sample utterance into vowel segments and non-vowel segments.

23. The method of claim 22 wherein segmenting the sample utterance further comprises regrouping the vowel segments and non-vowel segments into syllables depending upon relative energy levels of the vowel segments.

24. The method of claim 21 further comprising deleting parasitic vowels at the beginning or end of the sample utterance.

25. The method of claim 21 further comprising graphically identifying boundaries between syllables in the sequence of syllables of the sample utterance.

26. The method of claim 21 further comprising navigating through a plurality of navigation screens to select from amongst a curriculum comprising a plurality of pre-recorded utterances.

27. The method of claim 21 wherein segmenting the sample utterance and computing a first sample metric are performed independently of the language of the native speaker of the pre-recorded utterance.

28. The method of claim 21 further comprising:

storing a second example metric corresponding to a duration, and an energy of the vocal part of, each one of the sequence of syllables of the pre-recorded utterance;

computing a second sample metric corresponding to a duration and an energy of the vocal part of each one of the sequence of syllables of the sample utterance; and

selectably graphically displaying either the first example metric and the first sample metric to the user on a side-by-side basis or the second example metric and the second sample metric to the user on a side-by-side basis.

29. The method of claim 28 wherein graphically displaying the second sample metric comprises graphically representing the sequence of syllables as a series of steps, wherein for each syllable in the sequence, a length of each step represents the duration of the corresponding syllable and a height of each step represents the energy of the corresponding syllable.

30. The method of claim 21 further comprising selectably graphically displaying either the first example metric and the first sample metric to the user on a side-by-side basis or a waveform for the pre-recorded utterance and a waveform for the sample utterance to the user on a side-by-side basis.

31. A method of analyzing speech for voice and pronunciation training, the method comprising:

storing a pre-recorded utterance by a native speaker comprising a sequence of syllables and a first example metric corresponding to a duration, and an energy of a vocal part of, each one of the sequence of syllables;

capturing a sample utterance spoken by a user;

segmenting the sample utterance into a sequence of syllables;

computing a first sample metric corresponding to a duration, and an energy of a vocal part of, each one of the sequence of syllables for the sample utterance; and

graphically displaying the first example metric and the first sample metric to the user on a side-by-side basis.

32. The method of claim 31 wherein graphically displaying the second sample metric comprises graphically representing the sequence of syllables as a series of steps, wherein for each syllable in the sequence, a length of each step represents the duration of the corresponding syllable and a height of each step represents the energy of the corresponding syllable.

33. The method of claim 31 wherein segmenting the sample utterance further comprises dividing the sample utterance into vowel segments and non-vowel segments.

34. The method of claim 33 wherein segmenting the sample utterance further comprises regrouping the vowel segments and non-vowel segments into syllables depending upon relative energy levels of the vowel segments.

35. The method of claim 31 further comprising deleting parasitic vowels at the beginning or end of the sample utterance.

36. The method of claim 31 further comprising graphically identifying boundaries between syllables in the sequence of syllables of the sample utterance.

37. The method of claim 31 further comprising navigating through a plurality of navigation screens to select from amongst a curriculum comprising a plurality of pre-recorded utterances.

38. The method of claim 31 wherein segmenting the sample utterance and computing a first sample metric are performed independently of the language of the native speaker of the pre-recorded utterance.

39. The method of claim 31 further comprising:

storing a second example metric corresponding to a pitch value of a vocal part of each one of the sequence of syllables of the pre-recorded utterance;

computing a second sample metric corresponding to a pitch value of a vocal part of each one of the sequence of syllables of the sample utterance; and

selectably graphically displaying either the first example metric and the first sample metric to the user on a side-by-side basis or the second example metric and the second sample metric to the user on a side-by-side basis.

40. The method of claim 31 further comprising selectably graphically displaying either the first example metric and the first sample metric to the user on a side-by-side basis or a waveform for the pre-recorded utterance and a waveform for the sample utterance to the user on a side-by-side basis.