Systems and methods for automatic determination of infant cry and discrimination of cry from fussiness

• US 10,529,357 B2
• Filed: 12/07/2018
• Issued: 01/07/2020
• Est. Priority Date: 12/07/2017
• Status: Active Grant

• Current Assignee: LENA Foundation
• Original Assignee: LENA Foundation
• Inventors: Richards, Jeffrey A., Hannon, Stephen M.
• Primary Examiner(s): Paul, Disler
• Application Number:
  US16/213,468
• Publication Number:
  US 20190180772A1
• Time in Patent Office: 396 Days
• Field of Search: 381 56- 57, 704246, 704270
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band
• CPC Class Codes: G10L 25/27 characterised by the analysis technique
• CPC Class Codes: G10L 25/30 using neural networks
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: H04R 29/00 Monitoring arrangements; Testing arrangements for hearing aids H04R25/30; detection of loudspeaker connection H04R5/04; sound-field adaptation dependent on speaker detection H04S7/308

Abstract

A method including receiving one or more datasets of audio data of a key child captured in a natural sound environment of the key child. The method also includes segmenting each of the one or more datasets of audio data to create audio segments. The audio segments include cry-related segments and non-cry segments. The method additionally includes determining periods of the cry-related segments that satisfy one or more threshold non-sparsity criteria. The method further includes performing a classification on the periods to classify each of the periods as either a cry period or a fussiness period. Other embodiments are described.

Claims

1. A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform;

receiving one or more datasets of audio data of a key child captured in a natural sound environment of the key child;

segmenting each of the one or more datasets of audio data to create audio segments, the audio segments comprising cry-related segments and non-cry segments;

determining periods of the cry-related segments that satisfy one or more threshold non-sparsity criteria; and

performing a classification on the periods to classify each of the periods as either a cry period or a fussiness period,

wherein;

performing the classification on the periods comprises;

applying a binary logistic regression model on features of the periods; and

the features of the periods used in the binary logistic regression model comprise;

a slope of the period;

a median decibel sound pressure level of the period;

a total number of the periods of the cry-related segments, including (a) the periods of the cry-related segments that satisfy the one or more threshold non-sparsity criteria and (b) the periods of the cry-related segments that do not satisfy the one or more threshold non-sparsity criteria;

an age of the key child; and

a gender of the key child.

2. The system of claim 1, wherein the computing instructions are further configured to perform:

outputting metrics based on the classification of the periods.

3. The system of claim 1, wherein the periods are separated by gaps of at least a first time-duration threshold without any cry-related segments.

4. The system of claim 3, wherein the first time-duration threshold is approximately 15 seconds to approximately 5 minutes.

5. The system of claim 1, wherein the one or more threshold non-sparsity criteria comprise a first non-sparsity criterion in which each of the periods that satisfies the first non-sparsity criterion has a period duration longer than a second time-duration threshold.

6. The system of claim 5, wherein the second time-duration threshold is approximately 10 seconds to approximately 5 minutes.

7. The system of claim 1, wherein:

the one or more threshold non-sparsity criteria comprise a second non-sparsity criterion in which each period of the periods that satisfies the second non-sparsity criterion has an aggregated cry duration that exceeds a third time-duration threshold; and

the aggregated cry duration for the each period is an aggregate of durations of the cry-related segments within the each period.

8. The system of claim 7, wherein the third time-duration threshold is approximately 5 seconds to approximately 30 seconds.

9. The system of claim 1, wherein the one or more datasets of audio data of the key child comprise datasets of audio data in which the key child is less than 4 months old.

10. The system of claim 1, wherein:

the periods are separated by gaps of at least a first time-duration threshold without any cry-related segments;

the first time-duration threshold is approximately 15 seconds to approximately 5 minutes;

the one or more threshold non-sparsity criteria comprise a first non-sparsity criterion in which each of the periods that satisfies the first non-sparsity criterion has a period duration longer than a second time-duration threshold; and

the second time-duration threshold is approximately 10 seconds to approximately 5 minutes.

11. The system of claim 10, wherein:

the one or more threshold non-sparsity criteria comprise a second non-sparsity criterion in which each period of the periods that satisfies the second non-sparsity criterion has an aggregated cry duration that exceeds a third time-duration threshold;

the aggregated cry duration for the each period is an aggregate of durations of the cry-related segments within the each period; and

the third time-duration threshold is approximately 5 seconds to approximately 30 seconds.

12. A method being implemented via execution of computing instructions configured to run on one or more processors and stored on one or more non-transitory computer-readable media, the method comprising:

receiving one or more datasets of audio data of a key child captured in a natural sound environment of the key child;

segmenting each of the one or more datasets of audio data to create audio segments, the audio segments comprising cry-related segments and non-cry segments;

determining periods of the cry-related segments that satisfy one or more threshold non-sparsity criteria; and

performing a classification on the periods to classify each of the periods as either a cry period or a fussiness period,

wherein;

performing the classification on the periods comprises;

applying a binary logistic regression model on features of the periods; and

the features of the periods used in the binary logistic regression model comprise;

a slope of the period;

a median decibel sound pressure level of the period;

a total number of the periods of the cry-related segments, including (a) the periods of the cry-related segments that satisfy the one or more threshold non-sparsity criteria and (b) the periods of the cry-related segments that do not satisfy the one or more threshold non-sparsity criteria;

an age of the key child; and

a gender of the key child.

13. The method of claim 12 further comprising:

outputting metrics based on the classification of the periods.

14. The method of claim 12, wherein the periods are separated by gaps of at least a first time-duration threshold without any cry-related segments.

15. The method of claim 14, wherein the first time-duration threshold is approximately 15 seconds to approximately 5 minutes.

16. The method of claim 12, wherein the one or more threshold non-sparsity criteria comprise a first non-sparsity criterion in which each of the periods that satisfies the first non-sparsity criterion has a period duration longer than a second time-duration threshold.

17. The method of claim 16, wherein the second time-duration threshold is approximately 10 seconds to approximately 5 minutes.

18. The method of claim 12, wherein:

the one or more threshold non-sparsity criteria comprise a second non-sparsity criterion in which each period of the periods that satisfies the second non-sparsity criterion has an aggregated cry duration that exceeds a third time-duration threshold; and

the aggregated cry duration for the each period is an aggregate of durations of the cry-related segments within the each period.

19. The method of claim 18, wherein the third time-duration threshold is approximately 5 seconds to approximately 30 seconds.

20. The method of claim 12, wherein the one or more datasets of audio data of the key child comprise datasets of audio data in which the key child is less than 4 months old.

21. The method of claim 12, wherein:

the periods are separated by gaps of at least a first time-duration threshold without any cry-related segments;

the first time-duration threshold is approximately 15 seconds to approximately 5 minutes;

the one or more threshold non-sparsity criteria comprise a first non-sparsity criterion in which each of the periods that satisfies the first non-sparsity criterion has a period duration longer than a second time-duration threshold; and

the second time-duration threshold is approximately 10 seconds to approximately 5 minutes.

22. The method of claim 21, wherein:

the one or more threshold non-sparsity criteria comprise a second non-sparsity criterion in which each period of the periods that satisfies the second non-sparsity criterion has an aggregated cry duration that exceeds a third time-duration threshold;

the aggregated cry duration for the each period is an aggregate of durations of the cry-related segments within the each period; and

the third time-duration threshold is approximately 5 seconds to approximately 30 seconds.