Systems and Methods for Assessment of Non-Native Speech Using Vowel Space Characteristics

• US 20120016672A1
• Filed: 07/14/2011
• Published: 01/19/2012
• Est. Priority Date: 07/14/2010
• Status: Active Grant

• Current Assignee: Educational Testing Service
• Original Assignee: Educational Testing Service
• Inventors: Chen, Lei, Evanini, Keelan, Sun, Xie
• Granted Patent: US 9,262,941 B2
• Time in Patent Office: Days
• US Class Current: 704/236
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 25/15 the extracted parameters being formant information
• CPC Class Codes: G10L 25/27 characterised by the analysis technique
• CPC Class Codes: G10L 25/60 for measuring the quality of voice signals

Abstract

Computer-implemented systems and methods are provided for assessing non-native speech proficiency. A non-native speech sample is processed to identify a plurality of vowel sound boundaries in the non-native speech sample. Portions of the non-native speech sample are analyzed within the vowel sound boundaries to extract vowel characteristics. The vowel characteristics are used to identify a plurality of vowel space metrics for the non-native speech sample, and the vowel space metrics are used to determine a non-native speech proficiency score for the non-native speech sample.

Claims

1. A computer-implemented method of assessing non-native speech proficiency, comprising:

processing, using a processing system, a non-native speech sample to identify a plurality of vowel sound boundaries in the non-native speech sample;

analyzing, using the processing system, portions of the non-native speech sample within the vowel sound boundaries to extract vowel characteristics;

processing, using the processing system, the vowel characteristics to identify a plurality of vowel space metrics for the non-native speech sample; and

processing system, using the processing system, the vowel space metrics to determine a non-native speech proficiency score for the non-native speech sample.

2. The method of claim 1, wherein the vowel characteristics comprise vowel formant measurements.

3. The method of claim 2, wherein a vowel formant measurement comprises a measurement of an amplitude peak in a vowel spectrum that indicates a resonant frequency of a vowel.

4. The method of claim 1, wherein the vowel space metric comprises a vowel space range, a vowel space area, a vowel space dispersion, a within category vowel dispersion, or a F2-F1 distance.

5. The method of claim 4, wherein the vowel space metric comprises a vowel space range, wherein the vowel space range, F1 Range, is calculated according to

F1Range=Max_F1(AA)−

Min_F1(IY)

where Max_F1(AA) is the maximum value of the first vowel formant measurements for all AA sounds produced by a speaker, and Min_F1(IY) is the minimum value of the first vowel formant measurements for all IY sounds produced by a speaker.

6. The method of claim 5, wherein a second vowel space metric comprises a second vowel space range, wherein the second vowel space range, F2Range, is calculated according to

F2Range=Max_F2(IY)−

Min_F2(OW)

where Max_F2(IY) is the maximum value of the second vowel formant measurements for all IY sounds produced by a speaker, and Min_F2(OW) is the minimum value of the second vowel formant measurements for all OW sounds produced by the speaker,

wherein the non-native speech proficiency score is determined based on the F1Range and F2Range vowel space metrics.

7. The method of claim 1, wherein the vowel space metric is calculated based on a first vowel characteristic, formant F1, and a second vowel characteristic, formant F2;

wherein the vowel space metric comprises an area of a vowel triangle defined by a mean F1 and F2 values of three vowel sounds.

8. The method of claim 7, wherein the area is calculated according to:

area=√

{square root over (s(s−

D_{IY, AA} )(s−

D_{AA, OW} )(s−

D_{OW, IY}))}

where s=0.5*(D_{IY, AA}+D_{AA, OW}+D_{OW, IY}), V represents mean F1 and F2 values for vowel V, and D_x,y represents a Euclidean distance between two values in an F1-F2 plane;

D_x,y=√

{square root over ((F1_x−

F1_y)²+(F2_x−

F2_y)² )}{square root over ((F1_x−

F1_y)²+(F2_x−

F2_y)² )}

9. The method of claim 1, wherein the vowel space metric is calculated based on a first vowel space characteristic, formant F1, and a second vowel space characteristic, formant F2;

wherein the vowel space metric comprises a vowel space dispersion based on an average distance from individual peripheral vowel tokens to a center of a vowel space.

10. The method of claim 9, wherein the vowel space center, {square root over (V)}, is a mean of the F1 and F2 vowel space characteristics, wherein the vowel space dispersion is calculated according to:

11. The method of claim 1, wherein the vowel space metric is calculated based on a first vowel space characteristic, formant F1, and a second vowel space characteristic, formant F2;

wherein the vowel space metric is a within category vowel space dispersion.

12. The method of claim 11, wherein the within category vowel space dispersion is calculated according to:

13. The method of claim 1, wherein the vowel space metric is calculated based on a first vowel space characteristic, Fl, and a second vowel space characteristic, F2;

wherein the vowel space metric comprises an F2-F1 distance.

14. The method of claim 1, wherein the non-native speech sample is scored by inputting the vowel space metric into a model, wherein the model is trained based on a correlation between the vowel space metric and human generated non-native speech proficiency scores.

15. The method of claim 1, wherein the non-native speech proficiency score is automatically provided via a graphical user interface to provide feedback on non-native speech proficiency.

16. The method of claim 15, wherein the feedback offers vowel pronunciation suggestions for improving communicative competence through better vowel pronunciation.

17. The method of claim 1, further comprising performing automatic speech recognition on the non-native speech sample to generate a transcript, wherein the transcript is used to identify the vowel sound boundaries.

18. The method of claim 1, wherein a forced alignment procedure is used to identify the plurality of vowel sound boundaries.

19. The method of claim 1, wherein the non-native speech proficiency score is determined based on one or more non-vowel-space-based metrics.

20. The method of claim 19, wherein the non-native speech proficiency score is further based on a stress metric, an intonation metric, a vocabulary metric, or a grammar metric.

21. A computer-implemented system for assessing non-native speech proficiency, comprising:

one or more data processors;

one or more computer-readable storage mediums encoded with instructions for commanding the one or more data processors to execute steps that include;

processing a non-native speech sample to identify a plurality of vowel sound boundaries in the non-native speech sample;

analyzing portions of the non-native speech sample within the vowel sound boundaries to extract vowel characteristics;

using the vowel characteristics to identify a plurality of vowel space metrics for the non-native speech sample; and

using the vowel space metrics to determine a non-native speech proficiency score for the non-native speech sample.

22. A non-transitory computer-readable storage medium comprising instructions for which when executed cause a processing system to execute steps comprising:

processing a non-native speech sample to identify a plurality of vowel sound boundaries in the non-native speech sample;

analyzing portions of the non-native speech sample within the vowel sound boundaries to extract vowel characteristics;

using the vowel characteristics to identify a plurality of vowel space metrics for the non-native speech sample; and

using the vowel space metrics to determine a non-native speech proficiency score for the non-native speech sample.