Systems and methods of screening for medical states using speech and other vocal behaviors

• US 8,784,311 B2
• Filed: 10/03/2011
• Issued: 07/22/2014
• Est. Priority Date: 10/05/2010
• Status: Active Grant

• Current Assignee: University of Florida Research Foundation, Incorporated (State University System of Florida)
• Original Assignee: University of Florida Research Foundation, Incorporated (State University System of Florida)
• Inventors: Shrivastav, Rahul, Rosenbek, John Clyde, Harnsberger, James D., Anand, Supraja
• Primary Examiner(s): Thomson, William
• Assistant Examiner(s): SORIANO, BOBBY GILES
• Application Number:
  US13/251,755
• Publication Number:
  US 20120265024A1
• Time in Patent Office: 1,023 Days
• Field of Search: 600300-301, 705 2- 4
• US Class Current: 600/301
• CPC Class Codes: A61B 5/0004 characterised by the type of physiological signal transmitted
• CPC Class Codes: A61B 5/16 Devices for psychotechnics using teaching or educational appliances G09B1/00 - G09B7/00; Testing reaction times ; Devices for evaluating the psychological state
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/40 Detecting, measuring or recording for evaluating the nervous system A61B5/4806, A61B5/4821, A61B5/4824 take precedence
• CPC Class Codes: A61B 5/4803 Speech analysis specially adapted for diagnostic purposes
• CPC Class Codes: G10L 25/66 for extracting parameters related to health condition detecting or measuring for diagnostic purposes A61B5/00
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals
• CPC Class Codes: G16H 20/70 relating to mental therapies, e.g. psychological therapy or autogenous training
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: G16H 50/30 for calculating health indices; for individual health risk assessment

Abstract

Systems and methods of screening for neurological and other diseases utilizing a subject'"'"'s speech behavior are provided. According to one embodiment, a system is provided that includes an identification device used to determine a health state of a subject by receiving, as input to an interface of the device, one or more speech samples from the subject. The speech samples can be provided to the device by an intentional action of a user or passively due to the device being in the signal path of the subject'"'"'s speech. The samples are communicated to a processor that identifies the acoustic measures of the samples and compares the acoustic measures of the samples with baseline acoustic measures stored in a memory of the device. The results of this determination can be communicated back to the subject or provided to a third party.

Claims

1. A method for screening for neurological or other diseases, the method comprising:

receiving, at an identification device, a speech sample from a subject;

performing a signal analysis, via a processor of the identification device, to identify acoustic measures or language markers from the speech sample, wherein performing the signal analysis to identify acoustic measures or language markers from the speech sample comprises identifying the language markers by;

performing an automatic speech recognition process to recognize words, phrases, or sentences;

classifying the recognized words, phrases, or sentences into syntactical categories; and

determining phrase and/or sentence complexity by evaluating a number and order of syntactical categories found in the speech sample;

determining, via the processor, a likelihood and type of disease state based upon the identified acoustic measures or language markers; and

outputting the likelihood and type of disease state.

2. The method according to claim 1, further comprising:

outputting a treatment program corresponding to the health state of the subject.

3. The method according to claim 1, wherein outputting the likelihood and type of disease state provides information regarding the subject'"'"'s adherence to a treatment program and/or information regarding a change or progression in disease state from a previous determination of the likelihood and type of disease state.

4. The method according to claim 1, wherein receiving the speech sample from the subject comprises:

receiving, through a telephony, voice over internet protocol (VoIP) or cellular service provider, an outgoing call from the subject.

5. The method according to claim 1, wherein receiving the speech sample from the subject comprises:

receiving a recorded speech sample uploaded to a network via a website interface.

6. The method according to claim 1, wherein receiving the speech sample from the subject comprises:

recording the speech sample through a microphone.

7. The method according to claim 1, wherein the acoustic measures comprise acoustic measures of speech, wherein performing the signal analysis to identify the acoustic measures or language markers from the speech sample comprises identifying the acoustic measures by:

performing an acoustic analysis to quantify at least one measure from the group consisting of fundamental frequency characteristics, articulatory characteristics, speech/voice quality, prosodic characteristics, and speaking rate.

8. The method according to claim 1, wherein the acoustic measures comprise acoustic measures of cough, wherein performing the signal analysis to identify acoustic measures or language markers from the speech sample comprises identifying the acoustic measures by:

determining intensity and frequency of cough from the speech sample.

9. The method according to claim 1, wherein determining the likelihood and type of disease state based upon the identified acoustic measures or language markers comprises:

comparing the identified acoustic measures with normative baseline acoustic measures.

10. The method according to claim 1, wherein determining the likelihood and type of disease state based upon the identified acoustic measures or language markers comprises:

comparing the identified acoustic measures or language markers with previously obtained measures or markers from the same subject.

11. The method according to claim 10, wherein determining the likelihood and type of disease state based upon the identified acoustic measures or language markers further comprises:

estimating trajectories or rate-of-change of the acoustic measures or the language markers of the subject to reach a diagnostic decision.

12. The method according to claim 1, wherein determining the likelihood and type of disease state based upon the identified acoustic measures or language markers comprises:

comparing the identified acoustic measures with corresponding measures obtained from a set of other speakers.

13. The method according to claim 1, wherein determining the likelihood and type of disease state based upon the identified acoustic measures or language markers comprises:

performing a criterion-based test comparing the identified acoustic measures or language markers to a set of rules to reach a diagnostic decision.

14. The method according to claim 1, wherein determining the likelihood and type of disease state based upon the identified acoustic measures or language markers comprises:

using statistics, pattern recognition, and/or machine learning algorithms to combine information from the identified acoustic measures or identified language markers.

15. The method according to claim 14, wherein performing the signal analysis to identify the acoustic measures or language markers from the speech sample comprises identifying both the acoustic measures and the language markers from the speech sample, wherein the using statistical approaches, pattern recognition, and/or machine learning algorithms combines information from both the identified acoustic measures and the identified language markers.

16. The method according to claim 1, wherein outputting the diagnosis of the health state of the subject comprises transmitting results and/or instructions to the subject via phone, email, website interface, text messaging, mail, attached printer, or networked printer.

17. The method according to claim 1, wherein outputting the diagnosis of the health state of the subject comprises outputting a request to the subject for further testing.

18. The method according to claim 1, further comprising:

transmitting a notification to the subject to provide the speech sample at regularly scheduled intervals.

19. The method according to claim 1, further comprising associating with the likelihood and type of disease state information related to geographical location of the subject from which the speech sample is received.

20. The method according to claim 1, further comprising:

performing a pre-processing of the speech sample to clean the speech sample or select segments of the speech sample before performing the signal analysis to identify the acoustic measures or language markers.

21. The method according to claim 1, further comprising performing a non-linear transformation of the speech sample before performing the signal analysis to identify the acoustic measures or language markers.

22. A system for screening for neurological or other diseases, the system comprising:

an application service provider for receiving a speech sample from a subject; and

one or more storage media having stored thereon computer-executable instructions comprising;

a pre-processing module for receiving the speech sample from the application service provider and cleaning the speech sample or selecting segments of the speech sample for further processing;

a speech metric module for receiving the speech sample from the pre-processing module and identifying acoustic measures of the speech sample;

a language marker module for receiving the speech sample from the pre-processing module, performing automatic speech recognition;

classifying recognized words, phrases, or sentences into syntactical categories; and

determining phrase and/or sentence complexity;

a coding module for combining acoustic measures and/or language markers obtained by the speech metric module and the language marker module; and

a comparator for comparing the output of the coding module with normative data, criteria, or previous output of the coding module stored in a memory of the system and outputting a decision indicating a likelihood and type of disease state.

23. A system for screening for diseases, the system comprising:

an application service provider for receiving a speech sample from a subject; and

one or more storage media having stored thereon computer-executable instructions comprising;

a pre-processing module for receiving the speech sample from the application service provider, cleaning the speech sample, and selecting segments of the speech sample for further processing;

a speech metric module for receiving the segments of speech sample from the pre-processing module and identifying acoustic measures of the speech sample; and

a comparator for comparing the output of the speech metric module with normative data, criteria, or previous output of a coding module stored in a memory of the system and outputting a decision indicating a likelihood and type of disease state,

wherein the identified acoustic measures are used as a sole biomarker of the likelihood and type of disease state for the decision indicating the likelihood and type of disease state.

24. The system according to claim 23, wherein identifying the acoustic measures of the speech sample comprises:

computing specific acoustic measures from each segment;

comparing the specific acoustic measures across successive segments;

performing a machine learning process to identify acoustic patterns of a cough from the specific acoustic measures;

identifying instances of cough in the speech sample according to the acoustic patterns of the cough; and

determining characteristics of the cough including frequency, intensity, and voice quality according to the instances of cough.

25. A non-transitory computer-readable medium comprising instructions stored thereon for performing a method for screening for neurological or other diseases, the instructions comprising steps for:

receiving a speech sample,

segmenting the speech sample into windows and performing pattern recognition to locate a particular pattern in each window;

identifying acoustic measures of the speech sample including characteristics of the particular pattern; and

comparing the acoustic measures of the speech sample including the characteristics of the particular pattern with normative data, criteria, or previous output stored in a memory to provide an indication of disease diagnosis,

wherein the identified acoustic measures are used as a sole biomarker of the indication of disease diagnosis.

26. The non-transitory computer-readable medium according to claim 25, wherein the instructions further comprise steps for:

associating with the indication of disease diagnosis information related to geographical location of an individual from which the speech sample is obtained and/or the individual'"'"'s age and gender.

27. A method for screening for neurological or other diseases, the method comprising:

receiving, at an identification device, a speech sample from a subject;

performing a signal analysis, via a processor of the identification device, to identify acoustic measures or language markers from the speech sample;

determining, via the processor, a likelihood and type of disease state based upon the identified acoustic measures or language markers; and

outputting the likelihood and type of disease state,

wherein performing the signal analysis to identify acoustic measures or language markers from the speech sample comprises analyzing conversational speech in the speech sample.

28. A method for screening for neurological or other diseases, the method comprising:

receiving, at an identification device, a speech sample from a subject;

performing a signal analysis, via a processor of the identification device, to identify acoustic measures or language markers from the speech sample;

determining, via the processor, a likelihood and type of disease state based upon the identified acoustic measures or language markers; and

outputting the likelihood and type of disease state,

wherein performing the signal analysis to identify acoustic measures or language markers from the speech sample comprises identifying both the acoustic measures and the language markers from the speech sample.

29. A method for screening for neurological or other diseases, the method comprising:

receiving, at an identification device, a speech sample from a subject;

performing a signal analysis, via a processor of the identification device, to identify acoustic measures or language markers from the speech sample;

determining, via the processor, a likelihood and type of disease state based upon the identified acoustic measures or language markers; and

outputting the likelihood and type of disease state, wherein the identified acoustic measures, language markers, or a combination of the identified acoustic measures and language markers are used as a sole biomarker of the likelihood and type of disease state for determining the likelihood and type of disease state.