Method and system for the automatic generation of speech features for scoring high entropy speech

• US 20060074655A1
• Filed: 09/20/2004
• Published: 04/06/2006
• Est. Priority Date: 09/20/2004
• Status: Active Grant

• Current Assignee: Educational Testing Service
• Original Assignee: Educational Testing Service
• Inventors: Zechner, Klaus, Bejar, Isaac
• Granted Patent: US 7,392,187 B2
• Time in Patent Office: Days
• US Class Current: 704/243
• CPC Class Codes: G10L 15/063 Training

Abstract

A method and system for automatically generating a scoring model for scoring a speech sample are disclosed. One or more training speech samples are received in response to a prompt. One or more speech features are determined for each of the training speech samples. A scoring model is then generated based on the speech features. At least one of the training speech samples may be a high entropy speech sample. An evaluation speech sample is received and a score is assigned to the evaluation speech sample using the scoring model. The evaluation speech sample may be a high entropy speech sample.

Claims

1. A method of automatically generating a scoring model for scoring a speech sample, the method comprising:

receiving one or more training speech samples in response to a prompt;

determining one or more speech features for each of the training speech samples; and

generating a scoring model based on the speech features, wherein the scoring model is effective for scoring high entropy evaluation speech responses.

2. The method of claim 1 wherein at least one of the training speech samples comprises a high entropy speech sample.

3. The method of claim 1 wherein the one or more speech features comprise one or more of the following for each training speech sample:

a previously assigned score;

a lexical count;

a fluency measure;

a rate of speech measure;

a lexical similarity measure; and

a speech sample utterance duration.

4. The method of claim 3 wherein the previously assigned score comprises a score assigned by a human rater characterizing a speaking proficiency of the training speech sample.

5. The method of claim 3 wherein the lexical count comprises one or more of the following:

a number of utterances in response to a prompt for the training speech sample;

a number of words used in the training speech sample;

a number of disfluencies used in the training speech sample;

a number of tokens used in the training speech sample, wherein the number of tokens comprises a sum of the number of words and the number of disfluencies;

a number of unique word forms used in the training speech sample; and

a ratio equal to the number of unique word forms used in the training speech sample divided by the number of tokens used in the training speech sample.

6. The method of claim 3 wherein the fluency measure comprises one or more of the following:

a number of silent periods during the training speech sample, wherein each silent period comprises a duration;

a ratio equal to the number of silent periods during the training speech sample divided by a number of words used in the training speech sample;

a mean of the durations for all silent periods during the training speech sample; and

a standard deviation of the durations of all silent periods during the training speech sample.

7. The method of claim 3 wherein the rate of speech measure comprises one or more of the following:

a number of words per time unit in the training speech sample;

a number of disfluencies per time unit in the training speech sample;

a number of unique word forms per time unit in the training speech sample; and

a number of silent periods per time unit in the training speech sample.

8. The method of claim 3 wherein the lexical similarity measure comprises one or more of the following:

an inner product of word frequencies for the training speech sample and a content vector, wherein the content vector comprises raw frequencies of word forms based on a corpus related to the prompt; and

a ratio equal to the inner product divided by a number of words in the training speech sample.

9. The method of claim 1 wherein generating a scoring model comprises:

generating a classification tree based on a regression, for each training speech sample, between a score assigned to the training speech sample and the one or more speech features determined for the training speech sample.

10. The method of claim 1 wherein generating a scoring model comprises:

generating a scoring model based on weighting factors for the one or more speech features assigned by one or more expert judges.

11. The method of claim 1, further comprising:

receiving an evaluation speech response; and

assigning a score to the evaluation speech response using the scoring model.

12. The method of claim 11 wherein the evaluation speech response comprises a high entropy speech sample.

13. A method for scoring a high entropy speech sample, the method comprising:

retrieving a scoring model based on one or more speech features; and

assigning a score to a high entropy speech sample using the scoring model.

14. The method of claim 13 wherein the one or more speech features comprise one or more of the following for each high entropy speech sample:

a previously assigned score;

a lexical count;

a fluency measure;

a rate of speech measure;

a lexical similarity measure; and

a speech sample utterance duration.

15. The method of claim 14 wherein the previously assigned score comprises a score assigned by a human rater characterizing a speaking proficiency of the high entropy speech sample.

16. The method of claim 14 wherein the lexical count comprises one or more of the following:

a number of utterances in response to a prompt for the high entropy speech sample;

a number of words used in the high entropy speech sample;

a number of disfluencies used in the high entropy speech sample;

a number of tokens used in the high entropy speech sample, wherein the number of tokens comprises a sum of the number of words and the number of disfluencies;

a number of words used in the high entropy speech sample; and

a ratio equal to the number of unique word forms used in the high entropy speech sample divided by the number of tokens used in the high entropy speech sample.

17. The method of claim 14 wherein the fluency measure comprises one or more of the following:

a number of silent periods during the high entropy speech sample, wherein each silent period comprises a duration;

a ratio equal to the number of silent periods during the high entropy speech sample divided by a number of words used in the high entropy speech sample;

a mean of the durations for all silent periods during the high entropy speech sample; and

a standard deviation of the durations of all silent periods during the high entropy speech sample.

18. The method of claim 14 wherein the rate of speech measure comprises one or more of the following:

a number of words per time unit in the high entropy speech sample;

a number of disfluencies per time unit in the high entropy speech sample;

a number of unique word forms per time unit in the high entropy speech sample; and

a number of silent periods per time unit in the high entropy speech sample.

19. The method of claim 14 wherein the lexical similarity measure comprises one or more of the following:

an inner product of word frequencies for the high entropy speech sample and a content vector, wherein the content vector comprises raw frequencies of word forms based on a corpus related to the prompt; and

a ratio equal to the inner product divided by a number of words in the high entropy speech sample.

20. A system for automatically generating a scoring model for scoring a high entropy speech sample, the system comprising:

a processor; and

a processor-readable storage medium,

wherein the processor-readable storage medium contains one or more programming instructions for performing a method for automatically generating a scoring model for scoring a high entropy speech sample, the method comprising;

receiving one or more high entropy training speech samples in response to a prompt,

determining one or more speech features for each of the high entropy training speech samples, and

generating a scoring model based on the speech features.

21. The system of claim 20 wherein the processor-readable storage medium further contains one or more programming instructions for performing the following:

receiving a high entropy evaluation speech sample; and

assigning a score to the high entropy evaluation speech sample using the scoring model.

22. A system for scoring a high entropy speech sample, the system comprising:

a processor; and

a processor-readable storage medium,

wherein the processor-readable storage medium contains one or more programming instructions for performing a method for scoring a high entropy speech sample, the method comprising;

retrieving a scoring model based on one or more speech features, and

assigning a score to a high entropy speech sample using the scoring model.