Automated verbal fluency assessment

• US 9,576,593 B2
• Filed: 03/14/2013
• Issued: 02/21/2017
• Est. Priority Date: 03/15/2012
• Status: Active Grant

• Current Assignee: Regents of The University of Minnesota (University of Minnesota)
• Original Assignee: Regents of The University of Minnesota (University of Minnesota)
• Inventors: Pakhomov, Serguei V. S., Hemmy, Laura Sue, Lim, Kelvin O.
• Primary Examiner(s): Shah, Paras D
• Application Number:
  US14/379,654
• Publication Number:
  US 20150058013A1
• Time in Patent Office: 1,440 Days
• Field of Search: 704/210, 704/211, 704/231, 704/251, 704/270, 704/271, 704/272, 704/275, 434/156, 434/167, 434/178, 434/185
• US Class Current: 1/1
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 2025/783 based on threshold decision
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: G10L 25/66 for extracting parameters related to health condition detecting or measuring for diagnostic purposes A61B5/00
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal

Abstract

Techniques are described for calculating one or more verbal fluency scores for a person. An example method includes classifying, by a computing device, samples of audio data of speech of a person, based on amplitudes of the samples, into a first class of samples including speech or sound and a second class of samples including silence. The method further includes analyzing the first class of samples to determine a number of words spoken by the person, and calculating a verbal fluency score for the person based at least in part on the determined number of words spoken by the person.

Claims

1. A method comprising:

obtaining, by a computing device via a speech analyzer, a waveform representing a digital recording of audio speech of a person, the speech analyzer comprising at least one of a microphone, an interface to a sound recorder, an interface to a database, or an interface to a data storage system;

measuring, by the computing device, amplitudes of waves within the waveform, the waves corresponding to samples of the digital recording of the audio speech of the person;

classifying, by a silence detector of the computing device, the samples of the digital recording of the audio data of the speech of the person, based on the measured amplitudes of the samples and on a silence threshold, into a first class of samples including speech or sound and a second class of samples including silence, wherein classifying the samples comprises;

sorting, by the silence detector, the samples of the audio data in an order defined by the amplitudes of the samples of the audio data;

determining, by the silence detector, the silence threshold based on the amplitudes of the samples of the audio data, wherein determining the silence threshold comprises;

calculating, by the silence detector, linear regressions of the sorted samples in the sorted order; and

determining, by the silence detector, the silence threshold as the amplitude of one of the samples for which a slope of the calculated linear regression exceeds a predetermined value;

classifying, by the silence detector, samples having amplitudes above the silence threshold as belonging to the first class; and

classifying, by the silence detector, samples having amplitudes below the silence threshold as belonging to the second class;

analyzing, by the computing device, the first class of samples to determine a number of words spoken by the person;

calculating, by the computing device, a verbal fluency score for the person based at least in part on the determined number of words spoken by the person, and

outputting, by the computing device, the verbal fluency score.

2. The method of claim 1, wherein the predetermined value comprises −

0.2.

3. The method of claim 1, wherein analyzing the first class of samples comprises determining a first subset of samples of the first class including speech sound and a second subset of samples of the first class including non-speech sound.

4. The method of claim 3, further comprising determining the number of words as a number of contiguous samples in the audio data belonging to the first subset that start with a sample above the silence threshold and end with a sample below the silence threshold.

5. The method of claim 3, wherein determining the first subset and the second subset comprises:

classifying contiguous samples in the first class of the audio data for which a fundamental frequency can be calculated as belonging to the first subset; and

classifying contiguous samples in the first class of the audio data for which a fundamental frequency cannot be calculated as belonging to the second subset.

6. The method of claim 1, further comprising:

determining a number of pauses as a number of contiguous samples in the second class that start with a sample below the silence threshold and end with a sample below the silence threshold.

7. The method of claim 6, further comprising:

measuring a duration associated with each pause of the pauses;

calculating an average duration comprising a mean value of the measured durations; and

calculating a standard deviation of the measured durations from the average duration.

8. The method of claim 1, further comprising:

classifying, by the computing device, second samples of second audio data of speech of the person, based on second amplitudes of the second samples and on a second silence threshold, into the first class and the second class;

calculating a second verbal fluency score based at least in part on the number of words spoken by the person; and

calculating a learning score based at least in part on a change from the verbal fluency score to the second verbal fluency score.

9. The method of claim 8, wherein the silence threshold and the second silence threshold comprise equal values.

10. The method of claim 8, wherein calculating the learning score further comprises:

plotting at least the verbal fluency score and the second verbal fluency score on a graph; and

calculating a slope associated with the graph.

11. The method of claim 1, further comprising:

receiving the samples as at least a portion of a verbal fluency test of the person.

12. The method of claim 11, further comprising:

outputting the verbal fluency score.

13. The method of claim 1, wherein analyzing the first class of samples comprises excluding non-speech sounds in the first class of samples from the number of words spoken by the person, comprising:

calculating an average duration of the samples in the first class of samples;

calculating a standard deviation of durations of the samples in the first class of samples; and

classifying samples having durations that deviate from the average duration by at least one standard deviation as non-speech sounds.

14. The device of claim 13, wherein to analyze the first class of samples, the one or more processors are configured to exclude non-speech sounds in the first class of samples from the number of words spoken by the person, and wherein to exclude the non-speech sounds, the one or more processors are configured to:

calculate an average duration of the samples in the first class of samples;

calculate a standard deviation of durations of the samples in the first class of samples; and

classify samples having durations that deviate from the average duration by at least one standard deviation as non-speech sounds.

15. A device comprising:

a memory storing instructions defining at least a silence detector;

a speech analyzer comprising at least one of a microphone, an interface to a sound recorder, an interface to a database, or an interface to a data storage system, wherein the speech analyzer is configured to obtain a waveform representing a digital recording of audio speech of a person;

one or more processors configured to execute the instructions, wherein execution of the instructions causes the one or more processors to;

measure amplitudes of waves within the waveform, the waves corresponding to samples of the digital recording of the audio speech of the person;

execute the silence detector to classify the samples of the digital recording of the audio data of the speech of the person, based on the measured amplitudes of the samples and on a silence threshold, into a first class of samples including speech or sound and a second class of samples including silence, wherein to classify the samples, the silence detector is configured to;

sort the samples of the audio data in an order defined by the amplitudes of the samples of the audio data;

determine the silence threshold based on the amplitudes of the samples of the audio data, wherein to determine the silence threshold, the silence detector is configured to;

calculate linear regressions of the sorted samples in the sorted order; and

determine the silence threshold as the amplitude of one of the samples for which a slope of the calculated linear regression exceeds a predetermined value;

classify samples having amplitudes above the silence threshold as belonging to the first class; and

classify samples having amplitudes below the silence threshold as belonging to the second class;

analyze the first class of samples to determine a number of words spoken by the person;

calculate a verbal fluency score for the person based at least in part on the determined number of words spoken by the person; and

output the verbal fluency score.

16. The device of claim 15, wherein to analyze the first class of samples, the one or more processors are configured to:

determine a first subset of samples of the first class including speech sound and a second subset of samples of the first class including non-speech sound.

17. The device of claim 16, wherein to determine the first subset and the second subset, the one or more processors are configured to:

classify contiguous samples in the first class of the audio data for which a fundamental frequency can be calculated as belonging to the first subset; and

classify contiguous samples in the first class of the audio data for which a fundamental frequency cannot be calculated as belonging to the second subset.

18. The device of claim 15, wherein the one or more processors are further configured to:

classify second samples of second audio data of speech of the person, based on second amplitudes of the second samples and on a second silence threshold, into the first class and the second class;

calculate a second verbal fluency score based at least in part on the number of words spoken by the person; and

calculate a learning score based at least in part on a change from the verbal fluency score to the second verbal fluency score.

19. A non-transitory computer-readable medium comprising instructions that, when executed, cause a processor of a computing device to:

obtain, via a speech analyzer of the computing device, a waveform representing a digital recording of audio speech of a person, the speech analyzer comprising at least one of a microphone, an interface to a sound recorder, an interface to a database, or an interface to a data storage system;

measure amplitudes of waves within the waveform, the waves corresponding to samples of the digital recording of the audio speech of the person;

execute a silence detector of the computing device to classify the samples of the digital recording of the audio data of the speech of the person, based on the measured amplitudes of the samples and on a silence threshold, into a first class of samples including speech or sound and a second class of samples including silence, wherein to classify the samples, the instructions for the silence detector cause the processor to;

sort the samples of the audio data in an order defined by the amplitudes of the samples of the audio data;

determine the silence threshold based on the amplitudes of the samples of the audio data, wherein to determine the silence threshold, the silence detector is configured to;

calculate linear regressions of the sorted samples in the sorted order; and

determine the silence threshold as the amplitude of one of the samples for which a slope of the calculated linear regression exceeds a predetermined value;

classify samples having amplitudes above the silence threshold as belonging to the first class; and

classify samples having amplitudes below the silence threshold as belonging to the second class;

analyze the first class of samples to determine a number of words spoken by the person;

calculate a verbal fluency score for the person based at least in part on the determined number of words spoken by the person; and

output the verbal fluency score.

20. The non-transitory computer-readable storage medium of claim 19, wherein the instructions that cause the processor to analyze the first class of samples comprise instructions that cause the processor to exclude non-speech sounds in the first class of samples from the number of words spoken by the person, and wherein the instructions that cause the processor to exclude the non-speech sounds comprise instructions that cause the processor to:

calculate an average duration of the samples in the first class of samples;

calculate a standard deviation of durations of the samples in the first class of samples; and

classify samples having durations that deviate from the average duration by at least one standard deviation as non-speech sounds.