Automatic measurement of speech fluency

• US 9,230,539 B2
• Filed: 07/17/2013
• Issued: 01/05/2016
• Est. Priority Date: 01/06/2009
• Status: Active Grant

• Current Assignee: Regents of The University of Minnesota (University of Minnesota)
• Original Assignee: Regents of The University of Minnesota (University of Minnesota)
• Inventors: Pakhomov, Serguei V. S.
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US13/944,635
• Publication Number:
  US 20130304472A1
• Time in Patent Office: 902 Days
• Field of Search: 704/270
• US Class Current: 1/1
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

Techniques are described for automatically measuring fluency of a patient'"'"'s speech based on prosodic characteristics thereof. The prosodic characteristics may include statistics regarding silent pauses, filled pauses, repetitions, or fundamental frequency of the patient'"'"'s speech. The statistics may include a count, average number of occurrences, duration, average duration, frequency of occurrence, standard deviation, or other statistics. In one embodiment, a method includes receiving an audio sample that includes speech of a patient, analyzing the audio sample to identify prosodic characteristics of the speech of the patient, and automatically measuring fluency of the speech of the patient based on the prosodic characteristics. These techniques may present several advantages, such as objectively measuring fluency of a patient'"'"'s speech without requiring a manual transcription or other manual intervention in the analysis process.

Claims

1. A system comprising:

a computer-readable medium having stored thereon model data that includes at least one model of a characteristic of a language spoken by a patient; and

a speech analyzer configured to analyze an audio sample including speech of the patient, identify phonemes from the speech of the patient, analyze the identified phonemes to identify prosodic characteristics of the speech of the patient using the model, and automatically measure fluency of the speech of the patient based on the prosodic characteristics.

2. The system of claim 1, wherein the speech analyzer comprises an analysis module configured to analyze speech of the patient of the audio sample to determine, from the speech of the patient of the audio sample, at least one of a count of total number of pauses, a count of a number of silent pauses, a count of a number of filled pauses, an average of the total number of pauses, a standard deviation in the total number of pauses, a count of a number of repetitions, and fluctuations in a fundamental frequency.

3. The system of claim 1, wherein the model data includes an acoustic model that models phoneme boundaries, and

wherein the speech analyzer references the acoustic model to identify locations of phonemes, silences, and white noise.

4. The system of claim 3, wherein the model data includes a phoneme model that models phonemes, and

wherein the speech analyzer references the phoneme model to identify particular phonemes.

5. The system of claim 1, wherein the model data includes a phoneme model that models phonemes, and

wherein the speech analyzer references the phoneme model to identify particular phonemes.

6. The system of claim 1, wherein the speech analyzer comprises a recognition engine to identify pauses of the speech of the patient of the audio sample.

7. The system of claim 6, wherein the recognition engine is configured to produce a list of tagged symbols to identify phonemes, silent pauses, filled pauses, and a duration of each pause.

8. The system of claim 1, wherein the speech analyzer comprises a repetition detector configured to identify repetitions of the speech of the patient of the audio sample.

9. The system of claim 1, further comprising a spectrogram calculator to calculate a spectrogram from a portion of the audio sample.

10. A device comprising:

a computer-readable medium having stored thereon model data that includes at least one model of a characteristic of a language spoken by a patient; and

one or more processors configured to analyze an audio sample including speech of the patient, identify phonemes from the speech of the patient, analyze the identified phonemes to identify prosodic characteristics of the speech of the patient using the model, and automatically measure fluency of the speech of the patient based on the prosodic characteristics.

11. The device of claim 10, wherein the one or more processors are configured to analyze speech of the patient of the audio sample to determine, from the speech of the patient of the audio sample, at least one of a count of total number of pauses, a count of a number of silent pauses, a count of a number of filled pauses, an average of the total number of pauses, a standard deviation in the total number of pauses, a count of a number of repetitions, and fluctuations in a fundamental frequency.

12. The device of claim 10, wherein the one or more processors are further configured to calculate a spectrogram from a portion of the audio sample.

13. The device of claim 10, wherein the model data includes an acoustic model that models phoneme boundaries, and

wherein the one or more processors reference the acoustic model to identify locations of phonemes, silences, and white noise.

14. The device of claim 13, wherein the model data includes a phoneme model that models phonemes, and

wherein the speech analyzer references the phoneme model to identify particular phonemes.

15. The device of claim 10, wherein the model data includes a phoneme model that models phonemes, and

wherein the speech analyzer references the phoneme model to identify particular phonemes.

16. The device of claim 10, wherein the one or more processors are further configured to identify pauses of the speech of the patient of the audio sample.

17. The device of claim 16, wherein the one or more processors are configured to produce a list of tagged symbols to identify phonemes, silent pauses, filled pauses, and a duration of each pause.

18. The device of claim 10, wherein the one or more processors are configured to identify repetitions of the speech of the patient of the audio sample.