System and method for assessing expressive language development of a key child

• US 10,573,336 B2
• Filed: 02/17/2018
• Issued: 02/25/2020
• Est. Priority Date: 09/16/2004
• Status: Active Grant

• Current Assignee: LENA Foundation
• Original Assignee: LENA Foundation
• Inventors: Paul, Terrance D., Xu, Dongxin D., Gray, Sharmistha Sarkar, Yapanel, Umit, Gilkerson, Jill S., Richards, Jeffrey A.
• Primary Examiner(s): He, Jialong
• Application Number:
  US15/898,512
• Publication Number:
  US 20180174601A1
• Time in Patent Office: 738 Days
• CPC Class Codes: A61B 2503/06 Children, e.g. for attention deficit diagnosis
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/4803 Speech analysis specially adapted for diagnostic purposes
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/075 supervised, i.e. under machine guidance
• CPC Class Codes: G10L 15/14 using statistical models, e.g. Hidden Markov Models [HMMs] G10L15/18 takes precedence
• CPC Class Codes: G10L 17/16 Hidden Markov models [HMM]
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: G10L 2015/022 Demisyllables, biphones or triphones being the recognition units
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G10L 25/66 for extracting parameters related to health condition detecting or measuring for diagnostic purposes A61B5/00

Abstract

A method of assessing expressive language development of a key child. The method can include processing an audio recording taken in a language environment of the key child to identify segments of the audio recording that correspond to vocalizations of the key child. The method also can include applying an adult automatic speech recognition phone decoder to the segments of the audio recordings to identify each occurrence of a plurality of phone categories and to determine a duration for each of the plurality of phone categories. The method additionally can include determining a duration distribution for the plurality of phone categories based on the durations for the plurality of phone categories. The method further can include using the duration distribution for the plurality of phone categories in an age-based model to assess the expressive language development of the key child. The age-based model is selected based on a chronological age of the key child and the age-based model includes a plurality of different weights associated with the plurality of phone categories. Other embodiments are described.

Claims

1. A system for assessing expressive language development of a key child, comprising:

one or more processors;

one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform;

processing an audio recording taken in a language environment of the key child to identify segments of the audio recording that correspond to vocalizations of the key child;

applying an adult automatic speech recognition phone decoder to the segments of the audio recording that correspond to the vocalizations of the key child to identify each occurrence of a plurality of phone categories in the segments and to determine a respective duration for each of the plurality of phone categories in the segments, wherein each of the plurality of phone categories corresponds to a respective pre-defined speech sound that is a smallest phonetic unit capable of conveying a distinction in meaning in a predetermined language, and wherein the plurality of phone categories are a first subset of the vocalizations of the key child; and

determining a duration distribution for the plurality of phone categories based on the respective durations for the plurality of phone categories; and

using the duration distribution for the plurality of phone categories in an age-based model to assess the expressive language development of the key child, wherein the age-based model is selected based on a chronological age of the key child, and the age-based model comprises a plurality of different weights associated with the plurality of phone categories.

2. The system of claim 1, wherein:

applying the adult automatic speech recognition phone decoder comprises identifying occurrences of a plurality of non-phone categories;

each of the plurality of non-phone categories corresponds to a respective pre-defined non-speech sound; and

the plurality of non-phone categories are a second subset of the vocalizations of the key child.

3. The system of claim 1, wherein there is a correlation between the respective duration for each of the plurality of phone categories to chronological age.

4. The system of claim 1, wherein the age-based model is based on an age in months.

5. The system of claim 1, wherein the age-based model is an adjustment of the chronological age of the key child.

6. The system of claim 1, wherein using the duration distribution for the plurality of phone categories in the age-based model to assess the expressive language development of the key child comprises computing an estimated developmental age for the key child based at least in part on the plurality of different weights associated with the plurality of phone categories.

7. The system of claim 6, wherein the computing instructions are further configured to perform using the estimated developmental age for the key child to determine an estimated mean length of utterance for the key child.

8. The system of claim 6, wherein the computing instructions are further configured to perform:

processing at least one additional audio recording taken in the language environment of the key child to identify additional segments of the audio recording that correspond to the vocalizations of the key child;

applying the adult automatic speech recognition phone decoder to the additional segments of the audio recording to identify each occurrence of the plurality of phone categories in the additional segments and to determine an additional respective duration for each of the plurality of phone categories in the additional segments;

determining an additional duration distribution for the plurality of phone categories based on the additional respective durations for the plurality of phone categories;

using the additional duration distribution for the plurality of phone categories in the age-based model to assess the expressive language development of the key child and to generate an additional estimated developmental age for the key child based at least in part on the plurality of different weights associated with the plurality of phone categories; and

averaging the estimated developmental age and the additional estimated developmental age.

9. The system of claim 1, wherein:

applying the adult automatic speech recognition phone decoder comprises identifying occurrences of a plurality of non-phone categories, wherein each of the plurality of non-phone categories corresponds to a respective pre-defined non-speech sound, and wherein the plurality of non-phone categories are a second subset of the vocalizations of the key child;

the age-based model is an adjustment of the chronological age of the key child; and

the age-based model is based on an age in months.

10. The system of claim 1, wherein the computing instructions are further configured to perform:

computing a standard score and a developmental age for the key child; and

generating an output comprising at least one of;

the standard score or the developmental age.

11. A method of assessing expressive language development of a key child, comprising:

processing an audio recording taken in a language environment of the key child to identify segments of the audio recording that correspond to vocalizations of the key child;

applying an adult automatic speech recognition phone decoder to the segments of the audio recording to identify each occurrence of a plurality of phone categories in the segments and to determine a respective duration for each of the plurality of phone categories in the segments, wherein each of the plurality of phone categories corresponds to a respective pre-defined phoneme speech sound that is a smallest phonetic unit capable of conveying a distinction in meaning in a predetermined language, and wherein the plurality of phone categories are a first subset of the vocalizations of the key child;

determining a duration distribution for the plurality of phone categories based on the respective durations for the plurality of phone categories; and

using the duration distribution for the plurality of phone categories in an age-based model to assess the expressive language development of the key child, wherein the age-based model is selected based on a chronological age of the key child, and the age-based model comprises a plurality of different weights associated with the plurality of phone categories.

12. The method of claim 11, wherein:

applying the adult automatic speech recognition phone decoder comprises identifying occurrences of a plurality of non-phone categories;

each of the plurality of non-phone categories corresponds to a respective pre-defined non-speech sound; and

the plurality of non-phone categories are a second subset of the vocalizations of the key child.

13. The method of claim 11, wherein there is a correlation between the respective duration for each of the plurality of phone categories to chronological age.

14. The method of claim 11, wherein the age-based model is based on an age in months.

15. The method of claim 11, wherein the age-based model is an adjustment of the chronological age of the key child.

16. The method of claim 11, wherein using the duration distribution for the plurality of phone categories in the age-based model to assess the expressive language development of the key child comprises computing an estimated developmental age for the key child based at least in part on the plurality of different weights associated with the plurality of phone categories.

17. The method of claim 16 further comprising using the estimated developmental age for the key child to determine an estimated mean length of utterance for the key child.

18. The method of claim 16 further comprising:

processing, using an audio engine, at least one additional audio recording taken in the language environment of the key child to identify additional segments of the audio recording that correspond to the vocalizations of the key child;

applying the adult automatic speech recognition phone decoder to the additional segments of the audio recording to identify each occurrence of the plurality of phone categories in the additional segments and to determine an additional respective duration for each of the plurality of phone categories in the additional segments;

determining an additional duration distribution for the plurality of phone categories based on the additional respective durations for the plurality of phone categories;

using the additional duration distribution for the plurality of phone categories in the age-based model to assess the expressive language development of the key child and to generate an additional estimated developmental age for the key child based at least in part on the plurality of different weights associated with the plurality of phone categories; and

averaging the estimated developmental age and the additional estimated developmental age.

19. The method of claim 11, wherein:

applying the adult automatic speech recognition phone decoder comprises identifying occurrences of a plurality of non-phone categories, wherein each of the plurality of non-phone categories corresponds to a respective pre-defined non-speech sound, and wherein the plurality of non-phone categories are a second subset of the vocalizations of the key child;

the age-based model is an adjustment of the chronological age of the key child; and

the age-based model is based on an age in months.

20. The method of claim 11 further comprising:

computing a standard score and a developmental age for the key child; and

generating an output comprising at least one of;

the standard score or the developmental age.