System and method for expressive language, developmental disorder, and emotion assessment

• US 9,240,188 B2
• Filed: 01/23/2009
• Issued: 01/19/2016
• Est. Priority Date: 09/16/2004
• Status: Active Grant

• Current Assignee: LENA Foundation
• Original Assignee: LENA Foundation
• Inventors: Paul, Terrance D., Xu, Dongxin D., Gray, Sharmistha S., Yapanel, Umit, Gilkerson, Jill S., Richards, Jeffrey A.
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): KOVACEK, DAVID M
• Application Number:
  US12/359,124
• Publication Number:
  US 20090191521A1
• Time in Patent Office: 2,552 Days
• Field of Search: 704 1- 10, 704231-269, 704E17001-E17016, 704E15001-E1505, 704E11001-E11007
• US Class Current: 1/1
• CPC Class Codes: A61B 2503/06 Children, e.g. for attention deficit diagnosis
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/4803 Speech analysis specially adapted for diagnostic purposes
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/075 supervised, i.e. under machine guidance
• CPC Class Codes: G10L 15/14 using statistical models, e.g. Hidden Markov Models [HMMs] G10L15/18 takes precedence
• CPC Class Codes: G10L 17/16 Hidden Markov models [HMM]
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: G10L 2015/022 Demisyllables, biphones or triphones being the recognition units
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G10L 25/66 for extracting parameters related to health condition detecting or measuring for diagnostic purposes A61B5/00

Abstract

In one embodiment, the system and method for expressive language development; a method for detecting autism in a natural language environment using a microphone, sound recorder, and a computer programmed with software for the specialized purpose of processing recordings captured by the microphone and sound recorder combination; and the computer programmed to execute a method that includes segmenting an audio signal captured by the microphone and sound recorder combination using the computer programmed for the specialized purpose into a plurality recording segments. The method further includes determining which of the plurality of recording segments correspond to a key child. The method also includes extracting acoustic parameters of the key child recordings and comparing the acoustic parameters of the key child recordings to known acoustic parameters for children. The method returns a determination of a likelihood of autism.

Claims

1. A method for detecting autism in a natural language environment using a microphone, a sound recorder, and a computer programmed with software for a specialized purpose of processing recordings captured by a combination of the microphone and the sound recorder, the computer programmed to execute the method comprising:

segmenting an audio signal captured by the combination using the computer programmed for the specialized purpose into a plurality of recording segments;

determining which of the plurality of recording segments correspond to a key child;

determining which of the plurality of recording segments that correspond to the key child are classified as key child recordings;

extracting acoustic parameters of the key child recordings;

comparing the acoustic parameters of the key child recordings to known acoustic parameters for children, wherein the known acoustic parameters are developed through clustering the known acoustic parameters from recordings of known children using K-means clustering, and the known acoustic parameters of known children are tuned to autism; and

determining a likelihood of autism for the key child.

2. The method of claim 1, wherein the acoustic parameters of the key child recordings include canonical syllables.

3. The method of claim 2, wherein the acoustic parameters of the key child recordings are divided into 4 categories, including a rhythm/syllabicity category, a low spectral tilt and high pitch control category, a wide formant bandwidth and low pitch control category, and a duration of islands within utterances category.

4. The method of claim 3, wherein the rhythm/syllabicity category includes a voiced parameter, a canonical syllable transitions category, and a spectral entropy category.

5. The method of claim 3, wherein the low spectral tilt and high pitch control category includes a mean pitch high parameter, a low tilt of spectrum parameter, and a high frequency energy concentration parameter.

6. The method of claim 3, wherein the wide formant bandwidth and low pitch control category includes a Mean pitch low parameter and Wide Bandwidth of first two formants parameter.

7. The method of claim 3, wherein the duration of islands within utterances category includes a short parameter, a medium parameter, a long parameter, and an extra long parameter, wherein the extra long parameter is longer than the short parameter, the medium parameter, and the long parameter, wherein the long parameter is longer than the short parameter and the medium parameter, and wherein the medium parameter is longer than the short parameter.

8. The method of claim 1, wherein the comparing the acoustic parameters of the key child recordings to the known acoustic parameters for the children includes Logical Regression Analysis.

9. The method of claim 1, wherein:

the key child recordings are broken down into cry, vegetative-sound, and vocalization categories; and

the extracting the acoustic parameters of the key child recordings is applied only to the vocalization category of the key child recordings.

10. The method of claim 1, further comprising:

performing acoustic analysis on the key child recordings.

11. The method of claim 10, wherein the performing the acoustic analysis on the key child recordings includes transforming the recordings to reveal formant contours, formant bandwidths, pitch contours, spectral analysis, and duration analysis.

12. The method of claim 11, wherein the acoustic analysis is used to perform the extracting the acoustic parameters of the key child recordings.

13. The method of claim 1, further comprising:

sending instructions to transform a display of a user to display the likelihood of autism.

14. The method of claim 1, further comprising:

transforming an information storage device to store the likelihood of autism.

15. A method for detecting autism, comprising:

transforming an audio recording to output an indication of autism for a key child on an output mechanism selected from the group consisting of a display, a printing device, an electronic storage device, and an audio output device;

the transforming of the audio recording performed by comparing the audio recording to a model developed by analyzing transparent parameters of a plurality of sound recordings captured in a natural language environment, wherein the transparent parameters are developed through clustering the transparent parameters from recordings of known children, the clustering the transparent parameters is performed according to an unsupervised clustering method, the unsupervised clustering method comprising K-means clustering, the transparent parameters of known children are tuned to autism, and the analyzing the transparent parameters for each sound recording of the plurality of sound recordings comprises;

segmenting the sound recording into a plurality of recording segments, wherein the sound recording is captured by a microphone and sound recorder combination;

determining which of the plurality of recording segments correspond to the key child;

determining which of the plurality of recording segments that correspond to the key child are classified as key child recordings; and

extracting acoustic parameters of the key child recordings.

16. The method of claim 15, wherein the comparing the audio recording to the model includes Logical Regression Analysis.

17. The method of claim 15, wherein:

the key child recordings are broken down into cry, vegetative-sound, and vocalization categories; and

the extracting the acoustic parameters of the key child recordings is applied only to the vocalization category of the key child recordings.

18. The method of claim 15, further comprising:

performing acoustic analysis on the key child recordings.

19. The method of claim 18 wherein the performing the acoustic analysis on the key child recordings includes transforming the audio recording to reveal formant contours, formant bandwidths, pitch contours, spectral analysis, and duration analysis.

20. The method of claim 19, wherein the acoustic analysis is used to perform the extracting of the acoustic parameters of the key child recordings.

21. The method of claim 15, wherein the acoustic parameters include canonical syllables.

22. A method for detecting a disorder in a natural language environment using a microphone, a sound recorder, and a computer programmed with software for a specialized purpose of processing recordings captured by a combination of the microphone and the sound recorder, the computer programmed to execute the method comprising:

segmenting an audio signal captured by the combination using the computer programmed for the specialized purpose into a plurality of recording segments;

determining which of the plurality of recording segments correspond to a key subject;

determining which of the plurality of recording segments that correspond to the key subject are classified as key subject recordings;

extracting acoustic parameters of the key subject recordings;

comparing the acoustic parameters of the key subject recordings to known acoustic parameters for subjects, wherein the known acoustic parameters are developed through clustering the known acoustic parameters from recordings of known children and the clustering is performed according to an unsupervised clustering method, the unsupervised clustering method comprising K-means clustering, and the known acoustic parameters of known children are tuned to autism; and

determining a likelihood of the disorder for the key subject.

23. The method of claim 22, further comprising:

transforming an electronic storage device to record the likelihood of the disorder.