Sentiment analysis of mental health disorder symptoms

• US 10,276,190 B2
• Filed: 06/19/2017
• Issued: 04/30/2019
• Est. Priority Date: 06/19/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ashoori, Maryam, Briggs, Benjamin D., Clevenger, Lawrence A., Clevenger, Leigh Anne H.
• Primary Examiner(s): Sirjani, Fariba
• Application Number:
  US15/626,782
• Publication Number:
  US 20180366143A1
• Time in Patent Office: 680 Days
• Field of Search: None
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G10L 25/66 for extracting parameters related to health condition detecting or measuring for diagnostic purposes A61B5/00
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

Monitoring and analysis of a user'"'"'s speech to detect symptoms of a mental health disorder by continuously monitoring a user'"'"'s speech in real-time to generate audio data based, transcribing the audio data to text and analyzing the text of the audio data to determine a sentiment of the audio data is disclosed. A trained machine learning model may be applied to correlate the text and the determined sentiment to clinical information associated with symptoms of a mental health disorder to determine whether the symptoms are a symptom event. The initial determination may be transmitted to a second device to determine (and/or verify) whether or not the symptom event was falsely recognized. The trained machine learning model may be updated based on a response from the second device.

Claims

1. A computer system comprising:

an audio input device associated with a first party and receiving a speech from the first party; and

at least one hardware processor of said computer system in communication with the audio input device and configured to;

cause the audio input device to continuously and in real-time monitor the speech of said first party;

cause the audio input device to generate audio data based on the monitored speech;

transcribe the audio data to text;

analyze the text of the audio data to determine a sentiment;

train a model, using machine learning, to correlate the text and the determined sentiment to clinical information associated with one or more symptoms of a health disorder;

store the audio data, the text, the determined sentiment and an output of the trained machine learning model in a database as historical data, said machine learning model being trained to correlate based at least in part on the historical data that is stored in the database;

the at least one hardware processor is further configured to;

develop, over time, a behavioral baseline condition for the first party based on a history of audio data and corresponding text sentiment analysis results;

compare a result of analyzing the text of the current audio data against a baseline condition of said first party, and

determine, based on said comparison, whether the first party is exhibiting a new or different symptom; and

schedule, via an interface device, a checkup or appointment with a health care practitioner regarding said new or different symptom, wherein the at least one hardware processor is further configured to;

analyze the sentiment of the speech and a duration of said sentiment to identify a mood swing event including a time of occurrence, how quickly the mood swing event occurs, and for how long a mood swing event occurs;

determine, over time, a pattern and frequency of each identified mood swing event;

compare a determined pattern and frequency of the mood swing events against a database of known mood swing patterns;

predict, based on said comparing determined frequency and pattern of mood swing events, a mood swing occurrence exhibited by said first party in the future, and

generate an output message via an interface, said message indicating said predicted potential mood swing of said first party.

2. The system of claim 1, the at least one hardware processor further configured to apply the trained machine learning model to correlate at least one speech attribute of the audio data to the clinical information associated with one or more symptoms of a health disorder.

3. The system of claim 2, wherein the at least one speech attribute is selected from a group consisting of:

a volume of the audio data and a rate of speech of the audio data.

4. The system of claim 1, the at least one hardware processor further configured to:

determine whether or not symptoms are a symptom event, and upon determining

that the symptoms are a symptom event;

determine that the symptom event was falsely recognized; and

update the trained machine learning model based on the audio data, the text and the determined sentiment, with a confirmed non-symptom event, in response to determining that the symptom event was falsely recognized.

5. The system of claim 1, the at least one hardware processor further configured to:

determine whether or not symptoms are a symptom event, and upon determining that the symptoms are a symptom event;

determine the symptom event was correctly recognized; and

update the trained machine learning model based on the audio data, the text and the determined sentiment with a confirmed symptom event, in response to determining the symptom event was correctly recognized.

6. The system of claim 1, the at least one hardware processor further configured to:

transmit a message including an indication of whether or not the symptoms are a symptom event to a computing device associated with a second user;

receive from the computing device associated with the second user a message indicating whether or not the symptom event was falsely recognized; and

update the trained machine learning model based on the received message.

7. The computer system of claim 1, wherein the hardware processor is further configured to:

determine, based on said comparing a result of analyzing the text of the current audio data against the baseline condition, whether a medication or treatment administered to said first party is effective in reducing the first party'"'"'s symptoms.

8. The computer system of claim, 1 wherein the hardware processor is further configured to:

train said machine learning model based at least in part on training inputs comprising a testing feature vector, said testing feature vector having data representing one or more of;

a first party'"'"'s rate of speech, a volume level of speech, a repeating speech pattern, a determined sentiment, a determined mental health diagnosis, an indication of a physical manifestation of said one or more symptoms, medication, and treatments applied to said first party.

9. The computer system of claim 1, wherein the hardware processor is further configured to:

transmit a message including an indication of a detected mood swing event associated with the first party to a second party;

receive, from a device associated with the second party, a message indicating whether or not the mood swing event was correctly recognized or falsely recognized; and

update the trained machine learning model based on the received message.

12. The computer program product of claim 6, wherein the at least one speech attribute is selected from a group consisting of:

a volume level and a rate of speech.

14. The computer program product of claim 12, wherein the instructions when executed by at least one processor cause the at least one processor to:

train said machine learning model based at least in part on training inputs comprising a testing feature vector, said testing feature vector having data representing one or more of;

said first party'"'"'s rate of speech, said volume level of speech, a repeating speech pattern, a determined sentiment, a determined mental health diagnosis, an indication of a physical manifestation of said one or more symptoms, medication, and treatments applied to said first party.

10. A computer program product, said computer program product comprising a non-transitory computer readable storage medium having instructions stored thereon that, when executed by at least one processor:

cause an audio input device associated with a first party and receiving a speech from the first party, to continuously and in real-time monitor the speech of said first party;

cause the audio input device to generate audio data based on the monitored speech;

transcribe the audio data to text;

analyze the text of the audio data to determine a sentiment;

train a model, using machine learning, to correlate the text and the determined sentiment to clinical information associated with one or more symptoms of a health disorder;

store the audio data, the text, the determined sentiment and an output of the trained machine learning model in a database as historical data, said machine learning model being trained to correlate based at least in part on the historical data that is stored in the database;

develop, over time, a behavioral baseline condition for the first party based on a history of audio data and corresponding text sentiment analysis results;

compare a result of analyzing the text of the current audio data against a baseline condition of said first party, and

determine, based on said comparison, whether the first party is exhibiting a new or different symptom; and

schedule, via an interface device, a checkup or appointment with a health care practitioner regarding said new or different symptom, wherein the instructions when executed by at least one processor further cause the at least one processor to;

analyze the sentiment of the speech and a duration of said sentiment to identify a mood swing event including a time of occurrence, how quickly the mood swing event occurs, and for how long a mood swing event occurs;

determine, over time, a pattern and frequency of each identified mood swing event;

compare a determined pattern and frequency of the mood swing events against a database of known mood swing patterns;

predict, based on said comparing determined frequency and pattern of mood swing events, a mood swing occurrence exhibited by said first party in the future, and generate an output message via an interface, said message indicating said predicted potential mood swing of said first party.

11. The computer program product of claim 10, the computer readable storage medium having further instructions that, when executed by the at least one processor:

apply the trained machine learning model to correlate at least one speech attribute of the audio data to the clinical information associated with one or more symptoms of a health disorder.

13. The computer program product of claim 10, wherein the instructions when executed by at least one processor cause the at least one processor to:

determine, based on said comparing a result of analyzing the text of the current audio data against the baseline condition, whether a medication or treatment administered to said first party is effective in reducing the first party'"'"'s symptoms.

15. The computer program product of claim 10, wherein the instructions when executed by at least one processor cause the at least one processor to:

transmit a message including an indication of a detected mood swing event associated with the first party to a second party;

receive, from a device associated with the second party, a message indicating whether or not the mood swing event was correctly recognized or falsely recognized; and

update the trained machine learning model based on the received message.