Systems and methods to determine user state

• US 10,265,008 B2
• Filed: 03/13/2014
• Issued: 04/23/2019
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Aptima Inc.
• Original Assignee: Aptima Inc.
• Inventors: Durkee, Kevin, Pappada, Scott, Ortiz, Andres, DePriest, William, Feeney, John, Geyer, Alexandra, Sullivan, Seamus, Wiggins, Sterling
• Primary Examiner(s): Natnithithadha, Navin
• Application Number:
  US14/772,827
• Publication Number:
  US 20160007899A1
• Time in Patent Office: 1,867 Days
• Field of Search: 600544, 600545
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/316 Modalities, i.e. specific diagnostic methods
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/7203 for noise prevention, reduction or removal
• CPC Class Codes: A61B 5/7246 using correlation, e.g. template matching or determination of similarity
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: A61B 5/7475 User input or interface means, e.g. keyboard, pointing device, joystick
• CPC Class Codes: G09B 23/28 for medicine
• CPC Class Codes: G16H 50/30 for calculating health indices; for individual health risk assessment

Abstract

Computer based systems and methods for estimating a user state are disclosed. In some embodiments, the methods comprise inputting a first input at an intermittent interval and a second input at a frequent interval into a user state estimation model to estimate the user state. In some embodiments, the first inputs are enhanced by injecting a noise input to create a plurality of enhanced first inputs whereby the plurality of enhance first inputs correspond to the plurality of second inputs at the frequent interval. In some embodiments, the first input comprises a self-reported input and the second inputs comprise a physiological input, a performance input or a situational input. In some embodiments, a machine learning algorithm creates the state estimation model. In some embodiments, the state estimation model estimates a future user state. In some embodiments, a computer based system for estimating a user state is provided.

Claims

1. A computer based method to estimate a user state, the method comprising:

receiving a plurality of first inputs at an intermittent interval;

receiving a plurality of second inputs at a frequent interval;

injecting a plurality of third inputs as a noise input into the plurality of first inputs at a non-random interval to create a plurality of enhanced first inputs whereby the plurality of enhanced first inputs correspond to the plurality of second inputs at the frequent interval;

inputting the enhanced first inputs and the plurality of second inputs into a state estimation model whereby a user state can be estimated based on the plurality of second inputs; and

estimating the user state based on the plurality of second inputs.

2. The computer based method of claim 1 wherein the noise input comprises a physiological correlate for the user state.

3. The computer based method of claim 2 wherein the noise input is injected into the plurality of first inputs according to an empirical relationship of the known physiological correlate to the first inputs.

4. The computer based method of claim 2 wherein the noise input is injected into the plurality of first inputs according to an empirical relationship of the physiological correlate to the first inputs.

5. The computer based method of claim 2 wherein the physiological correlate for the user state comprises a ratio of electroencephalogram (EEG) measures comprising a Frontal Midline Theta data over a Parietal Midline Alpha for the user state of a user workload.

6. The computer based method of claim 1 wherein the state estimation model comprises a relational database containing a weighted relationship of the plurality of second inputs to the plurality of first inputs.

7. The computer based method of claim 6 wherein the plurality of second inputs comprise a plurality of physiological inputs of a user and the plurality of first inputs comprise a plurality of state inputs of the user.

8. The computer based method of claim 1 further comprising inputting the enhanced first inputs and the plurality of second inputs into a machine learning model to create the state estimation model.

9. The computer based method of claim 8 wherein the machine learning model comprises an artificial neural network.

10. The computer based method of claim 9 wherein the artificial neural network is configured to find a mathematical relationship between a physiological feature and a state measurement.

11. The computer based method of claim 1 wherein the plurality of first inputs comprises a self-reported data of the user state.

12. The computer based method of claim 1 wherein the plurality of second inputs comprises at least one selected from a group comprising:

a performance measure; and

a situational measure.

13. The computer based method of claim 1 further comprising:

receiving a plurality of enhanced actual first inputs and a plurality of actual second inputs;

retraining the state estimation model with the plurality of enhance actual first inputs and the plurality of actual second inputs to create a retrained state estimation model; and

estimating the user state based on the plurality of actual second inputs.

14. The computer based method of claim 13 wherein the retraining the state estimation model comprises inputting a plurality of enhanced actual first inputs and a plurality of actual second inputs into a machine learning model to create the retrained state estimation model.

15. The computer based method of claim 1 wherein:

the estimate of the user state comprises an objective measure of a cognitive state of a trainee in a computer based training simulator; and

the cognitive state comprises one selected from a group comprising;

a real-time engagement of the trainee in a training situation, and

a real-time workload of the trainee in the training situation.

16. The computer based method of claim 1 wherein the estimate of the user state comprises an objective measure of a cognitive state of a trainee in a computer based training simulator and the method further comprises:

presenting a training situation to the trainee through a user interface; and

adapting a interface based on objective measure of the cognitive state of the trainee.

17. A computer based method to estimate a state of a user, the method comprising:

receiving a plurality of first inputs at an intermittent interval;

receiving a plurality of second inputs at a frequent interval;

injecting a noise input into the plurality of first inputs at the frequent interval to create a plurality of enhanced first inputs whereby the plurality of enhanced first inputs correspond to the plurality of second inputs at the frequent interval;

inputting the enhanced first inputs and the plurality of second inputs into a state estimation model whereby a state of a user can be estimated based on the plurality of second inputs; and

estimating a future state of the user at a future time utilizing the state estimation model.

18. The computer based method of claim 17 further comprising:

modifying a user interface based on the future state of the user.

19. The method of claim 18 wherein the step of modifying the user interface based on the future state of the user comprises:

eliminating an unnecessary information set from a user display; and

displaying a necessary information set based on the future state of the user.

20. A computer based system for estimating a user state, the computer based system comprising:

a processor;

a non-transitory computer readable medium having a computer readable program code embodied therein, said computer readable program code configured to be executed to implement a method comprising;

receiving a plurality of first inputs at an intermittent interval;

receiving a plurality of second inputs at a frequent interval;

injecting a plurality of third inputs as a noise input into the plurality of first inputs at a non-random interval to create a plurality of enhanced first inputs whereby the plurality of enhanced first inputs correspond to the plurality of second inputs at the frequent interval;

inputting the enhanced first inputs and the plurality of second inputs into a state estimation model whereby a user state can be estimated based on the plurality of second inputs; and

estimating the user state based on the plurality of second inputs.