Systems and methods for mental health assessment

• US 10,748,644 B2
• Filed: 09/04/2019
• Issued: 08/18/2020
• Est. Priority Date: 06/19/2018
• Status: Active Grant

• Current Assignee: Ellipsis Health, Inc.
• Original Assignee: Ellipsis Health, Inc.
• Inventors: Shriberg, Elizabeth E., Aratow, Michael, Islam, Mainul, Harati Nejad Torbati, Amir Hossein, Rutowski, Tomasz, Lin, David, Lu, Yang, Haque, Farshid, Rogers, Robert D.
• Primary Examiner(s): Saint-Vil, Eddy
• Application Number:
  US16/560,720
• Publication Number:
  US 20200118458A1
• Time in Patent Office: 349 Days
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 5/14551 for measuring blood gases
• CPC Class Codes: A61B 5/164 Lie detection
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/4088 Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia
• CPC Class Codes: A61B 5/4803 Speech analysis specially adapted for diagnostic purposes
• CPC Class Codes: A61B 5/7275 Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 25/66 for extracting parameters related to health condition detecting or measuring for diagnostic purposes A61B5/00
• CPC Class Codes: G16H 10/20 for electronic clinical trials or questionnaires
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 15/00 ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/30 : for calculating health indices; for individual health risk assessment

Abstract

The present disclosure provides systems and methods for assessing a mental state of a subject in a single session or over multiple different sessions, using for example an automated module to present and/or formulate at least one query based in part on one or more target mental states to be assessed. The query may be configured to elicit at least one response from the subject. The query may be transmitted in an audio, visual, and/or textual format to the subject to elicit the response. Data comprising the response from the subject can be received. The data can be processed using one or more individual, joint, or fused models. One or more assessments of the mental state associated with the subject can be generated for the single session, for each of the multiple different sessions, or upon completion of one or more sessions of the multiple different sessions.

Claims

1. A method for identifying whether a subject is at risk of having a mental or physiological condition, comprising:

(a) obtaining data from said subject, said data comprising speech data and optionally associated visual data;

(b) processing said data using a plurality of machine learning models comprising a natural language processing (NLP) model and an acoustic model to generate an NLP output and an acoustic output, wherein said plurality of machine learning models comprises a neural network trained on labeled speech data collected from one or more other subjects, wherein said labeled speech data for each of said one or more other subjects is labeled as (i) having, to some level, said mental or physiological condition or (ii) not having said mental or physiological condition;

(c) fusing said NLP output and said acoustic output by (1) applying weights to said NLP output and said acoustic output to generate weighted outputs and (2) generating a composite output from said weighted outputs, wherein said NLP output and said acoustic output each comprise a plurality of outputs corresponding to a plurality of time segments of said speech data, and wherein said weights in (1) are temporally-based; and

(d) outputting an electronic report identifying whether said subject is at risk of having said mental or physiological condition, based at least on said composite output, which risk is quantified in a form of a score having a confidence level provided in said report.

2. The method of claim 1, wherein said speech data in (a) is obtained by:

(i) transmitting at least one query in an audio or textual format to said subject, wherein said at least one query is configured to elicit at least one verbal response from said subject; and

(ii) receiving said speech data comprising said at least one verbal response from said subject in response to transmitting said at least one query.

3. The method of claim 2, wherein said at least one query comprises a plurality of queries and said at least one response comprises a plurality of responses, wherein said plurality of queries is transmitted in a sequential manner to said subject and configured to systematically elicit said plurality of responses from said subject.

4. The method of claim 3, wherein said plurality of queries is structured in a hierarchical manner such that each subsequent query of said plurality of queries is structured as a logical follow on to said subject'"'"'s response to a preceding query, and is configured to assess or draw inferences on a plurality of aspects of said mental or physiological condition of said subject.

5. The method of claim 4, further comprising:

updating said score and said confidence level based at least in part on a follow-on response from said subject to said subsequent query.

6. The method of claim 5, further comprising:

updating or assigning a clinical value to said updated score having said confidence level.

7. The method of claim 5, further comprising:

determining whether said confidence level meets a predetermined criterion; and

generating one or more additional queries to assess or draw inferences on said plurality of aspects of said mental or physiological condition of said subject.

8. The method of claim 7, further comprising:

transmitting said one or more additional queries to said subject to elicit one or more additional verbal responses; and

determining whether said confidence level meets said predetermined criterion, based at least in part on said speech data comprising said one or more additional verbal responses.

9. The method of claim 8, further comprising:

continuing to transmit said one or more additional queries to said subject to elicit said one or more additional verbal responses, until said confidence level is determined to meet said predetermined criterion.

10. The method of claim 2, wherein said confidence level is based at least in part on a length and/or duration measure of said at least one verbal response.

11. The method of claim 2, wherein said confidence level is based at least in part on an evaluated truthfulness of said at least one verbal response.

12. The method of claim 1, wherein said confidence level is based at least in part on a quality measure of said speech data associated with metadata of said speech data, said subject, or a context of said speech data.

13. The method of claim 1, wherein said confidence level is based at least in part on an acoustic, NLP, or speech-recognition confidence measure of said data.

14. The method of claim 3, further comprising:

assigning a plurality of weights to said plurality of responses.

15. The method of claim 14, wherein said plurality of weights are assigned based at least in part on a type of query configured to elicit each of said responses.

16. The method of claim 1, wherein said plurality of machine learning models are provided as (i) one or more individual models, (ii) jointly as two or more separate models, and/or (iii) a fused model comprising a composite model that is an aggregate of two or more different models.

17. The method of claim 1, wherein said processed data comprises one or more model outputs generated from said one or more models, and wherein said one or more model outputs comprise one or more of the following:

an NLP output or an acoustic output.

18. The method of claim 1, wherein said weights are based at least in part on confidence measures associated with each of said NLP output and said acoustic output.

19. The method of claim 17, wherein said score is generated at least in part by fusing two or more of said model outputs.

20. The method of claim 19, wherein said two or more of said model outputs comprise at least:

(1) a first model output having a first confidence measure, and (2) a second model output having a second confidence measure, and wherein said confidence level of said score is generated at least in part by fusing said first confidence measure and said second confidence measure.

21. The method of claim 20, wherein said first model output corresponds to said NLP output, and said second model output corresponds to said acoustic output.

22. The method of claim 20, wherein said two or more of said model outputs further comprise at least (3) a third model output having a third confidence measure, and wherein said confidence level of said score is generated by fusing said first confidence measure, said second confidence measure, and said third confidence measure.

23. The method of claim 22, wherein said first model output corresponds to said NLP output and said second model output corresponds to said acoustic output.

24. The method of claim 1, wherein said electronic report is usable by a user to identify whether said subject is at risk of having said mental or physiological condition.

25. The method of claim 24, wherein said score has a clinical value, wherein said user is a healthcare provider or entity, and wherein said electronic report comprising said score having said clinical value is usable by said healthcare provider or said entity to evaluate or provide care for said subject, when said subject is identified to be at risk of having said mental or physiological condition.

26. The method of claim 1, wherein at least one of said plurality of machine learning models is a deep learning model.

27. The method of claim 1, wherein applying weights to said NLP output and said acoustic output does not change a magnitude of said NLP output or said acoustic output.