Medical assessment based on voice

• US 10,311,980 B2
• Filed: 05/07/2018
• Issued: 06/04/2019
• Est. Priority Date: 05/05/2017
• Status: Active Grant

• Current Assignee: Canary Speech, LLC
• Original Assignee: Canary Speech, LLC
• Inventors: Kim, Jangwon, Kwon, Namhee, O'Connell, Henry, Walstad, Phillip, Yang, Kevin Shengbin
• Primary Examiner(s): Le, Linh Giang
• Application Number:
  US15/973,504
• Publication Number:
  US 20180322961A1
• Time in Patent Office: 393 Days
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/4088 Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia
• CPC Class Codes: A61B 5/4803 Speech analysis specially adapted for diagnostic purposes
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G06F 2111/10 Numerical modelling
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 25/66 for extracting parameters related to health condition detecting or measuring for diagnostic purposes A61B5/00
• CPC Class Codes: G16H 10/20 for electronic clinical trials or questionnaires
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/50 for simulation or modelling of medical disorders
• CPC Class Codes: G16H 80/00 ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring

Abstract

Apparatuses, systems, methods, and computer program products are disclosed for medical assessment based on voice. A query module is configured to audibly question a user from a speaker of a mobile computing device. A response module is configured to receive a verbal response of a user from a microphone of a mobile computing device. A detection module is configured to provide an assessment for a user of a medical condition based on an analysis of a received verbal response of the user.

Claims

1. An apparatus comprising:

a query module installed on a mobile computing device and configured to audibly question a user from a speaker of the mobile computing device with one or more open ended questions, wherein the one or more open ended questions were previously selected by;

computing a medical diagnosis score for each speech data item of a training corpus by processing the speech data items with a machine learning model,

computing a prompt selection score for each prompt of a plurality of prompts using the medical diagnosis scores, and

selecting a subset of prompts from the plurality of prompts using the prompt selection scores, the subset of prompts comprising the one or more open ended questions;

a response module installed on the mobile computing device and configured to receive a conversational verbal response of the user from a microphone of the mobile computing device in response to the one or more open ended questions; and

a detection module installed on the mobile computing device and configured to display on a screen of the mobile computing device a machine learning assessment for the user of a medical condition based on a machine learning analysis of the received conversational verbal response of the user.

2. The apparatus of claim 1, wherein the detection module is configured to determine the assessment based on one or more acoustic features of the received verbal response without regard to language features of the received verbal response such that the assessment is independent of one or more of a language and a dialect of the received verbal response.

3. The apparatus of claim 1, wherein the user comprises a medical trial participant and the assessment comprises an assessment of an efficacy of a medical treatment for the medical condition.

4. The apparatus of claim 3, wherein the detection module comprises one of a plurality of distributed detection modules disposed on mobile computing devices for a plurality of medical trial participants comprising at least a placebo group not receiving the medical treatment and a group receiving the medical treatment, the plurality of distributed detection modules configured to provide blind assessments of the medical condition for both the placebo group and the group receiving the medical treatment.

5. The apparatus of claim 3, wherein the assessment of the efficacy of the medical treatment is based at least partially on one or more biomarkers of the received verbal response indicating a quality of life for the user.

6. The apparatus of claim 5, wherein the one or more biomarkers indicate one or more of physical fatigue, tiredness, mental fatigue, stress, anxiety, and depression.

7. The apparatus of claim 1, wherein the user comprises a prospective medical trial participant and the assessment comprises a suitability of the user for a medical trial for the medical condition.

8. The apparatus of claim 1, wherein the assessment comprises a first score determined for the user on the mobile computing device.

9. The apparatus of claim 8, wherein the assessment further comprises a second score determined for the user on a backend server in communication with the mobile computing device over a network.

10. The apparatus of claim 1, wherein the medical condition comprises a concussion.

11. The apparatus of claim 1, wherein the medical condition comprises one or more of depression, a stroke, Alzheimer'"'"'s disease, and Parkinson'"'"'s disease.

12. The apparatus of claim 1, further comprising an interface module configured to play a recording of the received verbal response of the user and recordings of received verbal responses of a plurality of other users to a different user based on hierarchical access control permissions for the different user.

13. The apparatus of claim 1, wherein the response module is further configured to receive data from one or more sensors of the mobile computing device and the detection module is further configured to base the analysis at least partially on the received data.

14. The apparatus of claim 13, wherein the one or more sensors comprise an image sensor and the received data comprises one or more images of the user.

15. The apparatus of claim 13, wherein the one or more sensors comprise a touchscreen and the received data comprises touch inputs received from the user during an interactive video game configured to test the user for one or more symptoms of the medical condition.

16. The apparatus of claim 13, wherein the one or more sensors comprise one or more of an accelerometer and a gyroscope and the received data comprises movement information for the mobile computing device by the user.

17. A system comprising:

a plurality of distributed voice modules installed on computing devices for a plurality of users, the plurality of distributed voice modules configured to audibly question the plurality of users from speakers of the computing devices with one or more open ended questions and to record verbal responses from the plurality of users on the computing devices; and

a backend computer server device configured to store at least baseline recorded verbal responses from the plurality of users, test case recorded verbal responses from the plurality of users, and machine learning assessments of a medical condition for at least the test case recorded verbal responses and to audibly play the stored baseline recorded verbal responses and the test case recorded verbal responses, and to display the assessments to at least a subset of the plurality of users on the computing devices through the plurality of distributed voice modules installed on the computing devices, the backend computer server device further configured to select the one or more open ended questions by;

computing a medical diagnosis score for each speech data item of a training corpus by processing the speech data items with a machine learning model,

computing a prompt selection score for each prompt of a plurality of prompts using the medical diagnosis scores, and

selecting a subset of prompts from the plurality of prompts using the prompt selection scores, the subset of prompts comprising the one or more open ended questions.

18. The system of claim 17, wherein the plurality of users comprises participants in a medical trial for the medical condition and the subset of the plurality of users comprises one or more administrators of the medical trial with hierarchical access control permissions to access the stored baseline recorded verbal responses, test case recorded verbal responses, and assessments.

19. The system of claim 17, wherein the plurality of distributed voice modules are configured to determine the assessments on the computing devices based on the baseline recorded verbal responses and the test case recorded verbal responses.

20. The system of claim 17, wherein the backend computer server device is configured to determine the assessments based on the baseline recorded verbal responses and the test case recorded verbal responses.

21. A system comprising:

means for computing a medical diagnosis score for each speech data item of a training corpus by processing the speech data items with a machine learning model;

means for computing a prompt selection score for each prompt of a plurality of prompts using the medical diagnosis scores;

means for selecting a subset of prompts from the plurality of prompts using the prompt selection scores, the subset of prompts comprising one or more open ended questions;

means for audibly questioning a user from an application installed on a mobile computing device with the one or more open ended questions;

means for receiving a verbal response of the user on the mobile computing device from a microphone of the mobile computing device in response to the one or more open ended questions;

means for assessing the user for a medical condition based on the received verbal response of the user; and

means for displaying the assessment to the user on a screen of the mobile computing device.

22. The system of claim 21, further comprising:

means for authenticating different users in a hierarchy of users; and

means for providing access to different recordings and different assessments to the different users based on hierarchical access control permissions for the hierarchy of users.

23. A method comprising:

computing a medical diagnosis score for each speech data item of a training corpus by processing the speech data items with a machine learning model;

computing a prompt selection score for each prompt of a plurality of prompts using the medical diagnosis scores;

selecting a subset of prompts from the plurality of prompts using the prompt selection scores, the subset of prompts comprising one or more open ended questions;

audibly querying a user with the one or more open ended questions using a user interface of a computing device;

recording, on a computing device, one or more baseline verbal responses of the user to the one or more open ended questions;

audibly re-querying the user, in response to a potential concussion event, with the one or more open ended questions using a user interface of a computing device;

recording, on a computing device, one or more test case verbal responses of the user to the re-queried one or more open ended questions;

assessing, on a computing device, a likelihood that the user has a concussion based on a voice analysis of the recorded one or more baseline verbal responses and the recorded one or more test case verbal responses; and

visually displaying, on a screen of a computing device, the likelihood that the user has a concussion.

24. The method of claim 23, wherein the voice analysis is based on one or more acoustic features without regard to language features such that the assessing is independent of one or more of a language and a dialect of the recorded one or more baseline verbal responses and the recorded one or more test case verbal responses.

25. The method of claim 23, further comprising receiving data associated with the user from one or more of an image sensor, a touchscreen, an accelerometer, and a gyroscope, wherein the assessing is further based at least partially on the received data.