Sentiment analysis of mental health disorder symptoms

• US 10,580,435 B2
• Filed: 11/27/2017
• Issued: 03/03/2020
• Est. Priority Date: 06/19/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ashoori, Maryam, Briggs, Benjamin D., Clevenger, Lawrence A., Clevenger, Leigh Anne H.
• Primary Examiner(s): Sirjani, Fariba
• Application Number:
  US15/822,854
• Publication Number:
  US 20180366144A1
• Time in Patent Office: 827 Days
• Field of Search: None
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G10L 25/66 for extracting parameters related to health condition detecting or measuring for diagnostic purposes A61B5/00
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

Monitoring and analysis of a user'"'"'s speech to detect symptoms of a mental health disorder by continuously monitoring a user'"'"'s speech in real-time to generate audio data based, transcribing the audio data to text and analyzing the text of the audio data to determine a sentiment of the audio data is disclosed. A trained machine learning model may be applied to correlate the text and the determined sentiment to clinical information associated with symptoms of a mental health disorder to determine whether the symptoms are a symptom event. The initial determination may be transmitted to a second device to determine (and/or verify) whether or not the symptom event was falsely recognized. The trained machine learning model may be updated based on a response from the second device.

Claims

1. A computer-implemented method, comprising:

monitoring continuously and in real-time, by an audio input device associated with a first party, speech from the first party;

generating, by the audio input device, audio data based on monitored speech;

transcribing, using a hardware processor of a computing device, the audio data to text;

analyzing, by the computing device, the text of the audio data to determine a sentiment;

training a model, using machine learning, to correlate the text and the determined sentiment to clinical information associated with one or more symptoms of a health disorder;

storing the audio data, the text, the determined sentiment and an output of the trained machine learning model in a database as historical data, wherein the trained machine learning model is trained based at least in part on the historical data that is stored in the database; and

developing, over time, a behavioral baseline condition for the first party based on a history of audio data and corresponding text sentiment analysis results;

comparing a result of analyzing the text of the current audio data against a baseline condition of said first party, and

determining, based on said comparison, whether the first party is exhibiting a new or different symptom; and

scheduling, via an interface device, a checkup or appointment with a health care practitioner regarding said new or different symptom,

wherein a symptom includes a mood swing event, said method further comprising;

analyzing, by the hardware processor, the sentiment of the speech and a duration of said sentiment to identify a mood swing event including a time of occurrence, how quickly the mood swing event occurs, and for how long a mood swing event occurs;

determining, by the hardware processor, over time, a pattern and frequency of each identified mood swing event;

comparing a determined pattern and frequency of the mood swing events against a database of known mood swing patterns;

predicting, based on said comparing determined frequency and pattern of mood swing events, a mood swing occurrence exhibited by said first party in the future, and

generating an output message via an interface, said message indicating said predicted potential mood swing of said first party.

2. The method of claim 1, further comprising applying the trained machine learning model to correlate the text, the determined sentiment, and at least one speech attribute of the audio data to the clinical information associated with one or more symptoms of a health disorder.

3. The method of claim 2, wherein the at least one speech attribute is selected from a group consisting of:

a volume level and a rate of speech.

4. The method of claim 1, further comprising:

determining, by the hardware processor, whether or not symptoms are a symptom event, and upon determining that the symptoms are a symptom event,

determining that the symptom event was falsely recognized; and

updating the trained machine learning model based on the audio data, the text and the determined sentiment, with a confirmed non-symptom event, in response to determining that the symptom event was falsely recognized.

5. The method of claim 1, further comprising:

determining whether or not symptoms are a symptom event, and upon determining that the symptoms are a symptom event,

determining the symptom event was correctly recognized; and

updating the trained machine learning model based on the audio data, the text and the determined sentiment with a confirmed symptom event, in response to determining the symptom event was correctly recognized.

6. The method of claim 1, further comprising:

transmitting a message including an indication of whether or not the symptoms are a symptom event to a computing device associated with a second user;

receiving from the computing device associated with the second user a message indicating whether or not the symptom event was falsely recognized; and

updating the trained machine learning model based on the received message.

7. The method of claim 1, further comprising:

determine, using the hardware processor, based on said comparing a result of analyzing the text of the current audio data against the baseline condition, whether a medication or treatment administered to said first party is effective in reducing the first party'"'"'s symptoms.

8. The method of claim 1, further comprising:

training, using said hardware processor, said machine learning model based at least in part on training inputs comprising a testing feature vector, said testing feature vector having data representing one or more of;

a first party'"'"'s rate of speech, a volume level of speech, a repeating speech pattern, a determined sentiment, a determined mental health diagnosis, an indication of a physical manifestation of said one or more symptoms, medication, and treatments applied to said first party.

9. The method of claim 1, further comprising:

transmitting, using said hardware processor, a message including an indication of a detected mood swing event associated with the first party to a second party;

receiving, from a device associated with the second party, a message indicating whether or not the mood swing event was correctly recognized or falsely recognized; and

updating the trained machine learning model based on the received message.