AUTOMATIC MEASUREMENT OF SPEECH FLUENCY

• US 20100174533A1
• Filed: 01/05/2010
• Published: 07/08/2010
• Est. Priority Date: 01/06/2009
• Status: Active Grant

• Current Assignee: Regents of The University of Minnesota (University of Minnesota)
• Original Assignee: Regents of The University of Minnesota (University of Minnesota)
• Inventors: Pakhomov, Serguei V.S.
• Granted Patent: US 8,494,857 B2
• Time in Patent Office: Days
• US Class Current: 704/205
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

Techniques are described for automatically measuring fluency of a patient'"'"'s speech based on prosodic characteristics thereof. The prosodic characteristics may include statistics regarding silent pauses, filled pauses, repetitions, or fundamental frequency of the patient'"'"'s speech. The statistics may include a count, average number of occurrences, duration, average duration, frequency of occurrence, standard deviation, or other statistics. In one embodiment, a method includes receiving an audio sample that includes speech of a patient, analyzing the audio sample to identify prosodic characteristics of the speech of the patient, and automatically measuring fluency of the speech of the patient based on the prosodic characteristics. These techniques may present several advantages, such as objectively measuring fluency of a patient'"'"'s speech without requiring a manual transcription or other manual intervention in the analysis process.

Claims

1. A method comprising:

receiving, with a speech analysis device, an audio sample that includes speech of a patient;

analyzing, with the speech analysis device, the audio sample to identify prosodic characteristics of the speech of the patient; and

automatically measuring, with the speech analysis device, fluency of the speech of the patient based on the prosodic characteristics.

2. The method of claim 1, wherein analyzing the audio sample to identify prosodic characteristics comprises identifying pauses between phonemes of the speech of the patient from the audio sample.

3. The method of claim 2, wherein identifying pauses comprises identifying silent pauses between phonemes.

4. The method of claim 2, wherein identifying pauses comprises identifying filled pauses that include at least one filled pause phoneme.

5. The method of claim 2, wherein automatically measuring fluency comprises counting a number of pauses that occur in the speech of the patient of the audio sample.

6. The method of claim 5, wherein counting a number of pauses comprises counting a number of silent pauses and counting a number of filled pauses.

7. The method of claim 2, wherein automatically measuring fluency comprises identifying a frequency of pauses of the speech of the patient of the audio sample.

8. The method of claim 2, wherein automatically measuring fluency comprises identifying an average duration of pauses of the speech of the patient of the audio sample.

9. The method of claim 1, wherein analyzing the audio sample to identify prosodic characteristics comprises identifying repetitions of the speech of the patient of the audio sample.

10. The method of claim 9, wherein automatically measuring fluency comprises counting a number of repetitions that occur in the speech of the patient of the audio sample.

11. The method of claim 9, wherein identifying repetitions comprises:

calculating a first spectrogram for a first portion of the audio sample;

calculating a second spectrogram for a second portion of the audio sample;

comparing the first spectrogram to the second spectrogram to determine similarity between the first spectrogram and the second spectrogram; and

determining that the second portion is a repetition of the first portion when the similarity between the first spectrogram and the second spectrogram exceeds a threshold.

12. The method of claim 11, wherein comparing the first spectrogram to the second spectrogram comprises:

computing a cosine angle between each of a plurality of vectors of the first spectrogram and a corresponding one of a plurality of vectors of the second spectrogram;

determining a mean of the computed cosine angles; and

determining that the similarity between the first spectrogram and the second spectrogram is the computed mean of the determined cosine angles.

13. The method of claim 1, wherein analyzing the audio sample to identify prosodic characteristics comprises identifying a fundamental frequency of the speech of the patient of the audio sample.

14. The method of claim 1, further comprising diagnosing a disorder of the patient according to the fluency of the speech of the patient.

15. The method of claim 14, wherein the disorder comprises at least one of fronto-temporal dementia, Alzheimer'"'"'s disease, schizophrenia, epilepsy, an autism spectrum disorder, and a side effect caused by medication.

16. The method of claim 1, further comprising monitoring changes in the fluency of the speech of the patient over time.

17. The method of claim 16, further comprising modifying a therapy regimen based on changes in the fluency of the speech of the patient.

18. The method of claim 1, further comprising determining efficacy of a drug based on the fluency of the speech of the patient.

19. A system comprising:

a speech analyzer to analyze an audio sample including speech of a patient to automatically measure fluency of the speech of the patient; and

model data that includes at least one model of a characteristic of a language spoken by the patient.

20. The system of claim 19, wherein the speech analyzer comprises a recognition engine to identify pauses of the speech of the patient of the audio sample.

21. The system of claim 20, wherein the recognition engine produces a list of tagged symbols to identify phonemes, silent pauses, filled pauses, and a duration of each pause.

22. The system of claim 19, wherein the speech analyzer comprises a repetition detector to identify repetitions of the speech of the patient of the audio sample.

23. The system of claim 19, further comprising a spectrogram calculator to calculate a spectrogram from a portion of the audio sample.

24. The system of claim 19, wherein the speech analyzer comprises an analysis module to analyze speech of the patient of the audio sample to determine, from the speech of the patient of the audio sample, at least one of a count of total number of pauses, a count of a number of silent pauses, a count of a number of filled pauses, an average of the total number of pauses, a standard deviation in the total number of pauses, a count of a number of repetitions, and fluctuations in a fundamental frequency.

25. The system of claim 19, wherein the model data includes an acoustic model that models phoneme boundaries and a phoneme model that models phonemes,

wherein the speech analyzer refers to the acoustic model to identify locations of phonemes, silences, and white noise, and wherein the speech analyzer refers to the phoneme model to identify particular phonemes.

26. A computer-readable storage medium comprising instructions for causing a programmable processor to:

receive an audio sample that includes speech of a patient;

analyze the audio sample to identify prosodic characteristics of the speech of the patient; and

automatically measure fluency of the speech of the patient based on the prosodic characteristics.

27. A method comprising:

providing an audio sample that includes speech of a patient to a speech analysis device, wherein the speech analysis device analyzes the audio sample to identify prosodic characteristics of the speech of the patient and automatically measures fluency of the speech of the patient based on the prosodic characteristics; and

receiving an indication of the fluency of the speech of the patient from the speech analysis device.

28. The method of claim 27, further comprising diagnosing a disorder of the patient according to the fluency of the speech of the patient.

29. The method of claim 28, wherein the disorder comprises at least one of fronto-temporal dementia, Alzheimer'"'"'s disease, schizophrenia, epilepsy, an autism spectrum disorder, and a side effect caused by medication.

30. The method of claim 27, further comprising monitoring changes in the fluency of the speech of the patient over time.

31. The method of claim 30, further comprising modifying a therapy regimen based on changes in the fluency of the speech of the patient.

32. The method of claim 27, further comprising determining efficacy of a drug based on the fluency of the speech of the patient.