Systems, computer medium and methods for management training systems

• US 10,475,351 B2
• Filed: 12/04/2015
• Issued: 11/12/2019
• Est. Priority Date: 12/04/2015
• Status: Active Grant

• Current Assignee: Saudi Arabian Oil Company (Government of Saudi Arabia)
• Original Assignee: Saudi Arabian Oil Company (Government of Saudi Arabia)
• Inventors: Horseman, Samantha J., Mattson, Brent
• Primary Examiner(s): Hong, Thomas J
• Application Number:
  US14/959,244
• Publication Number:
  US 20170162072A1
• Time in Patent Office: 1,439 Days
• Field of Search: 434236
• CPC Class Codes: A61B 2503/12 Healthy persons not otherwise provided for, e.g. subjects of a marketing survey
• CPC Class Codes: A61B 2505/09 Rehabilitation or training
• CPC Class Codes: A61B 5/002 Monitoring the patient using a local or closed circuit, e.g. in a room or building A61B5/0017 takes precedence
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/02 Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: A61B 5/021 Measuring pressure in heart or blood vessels A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/0531 Measuring skin impedance
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: A61B 5/14532 for measuring glucose, e.g. by tissue impedance measurement
• CPC Class Codes: A61B 5/14551 for measuring blood gases
• CPC Class Codes: A61B 5/162 Testing reaction times
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/318 Heart-related electrical modalities, e.g. electrocardiography [ECG]
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/389 Electromyography [EMG]
• CPC Class Codes: A61B 5/486 Bio-feedback A61B5/375 takes precedence
• CPC Class Codes: A61B 5/4884 inducing physiological or psychological stress, e.g. applications for stress testing
• CPC Class Codes: A61B 5/6803 : Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/681 : Wristwatch-type devices
• CPC Class Codes: A61B 5/7267 : involving training the classification device
• CPC Class Codes: A61B 5/7278 : Artificial waveform generation or derivation, e.g. synthesising signals from measured signals A61B5/327 takes precedence
• CPC Class Codes: A61B 5/7425 : Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image geometric transforms for image registration G06T3/14; enhancement using image fusion G06T5/50; algorithms for registration of images G06T7/30
• CPC Class Codes: A61B 5/746 : Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: G06Q 10/0639 : Performance analysis of employees; Performance analysis of enterprise or organisation operations
• CPC Class Codes: G09B 19/00 : Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 7/02 : of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
• CPC Class Codes: G09B 9/00 : Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients

Abstract

A training system, including a plurality of sensors to obtain a plurality of biometrics from a first user. A stress level, a level of interest, a level of engagement, a level of alertness, and a level of excitement are determined responsive to analysis of ones of the plurality of biometrics. An indication is displayed of the obtained biometrics, the determined stress level, and the determined levels of interest, engagement, alertness, and excitement.

Claims

1. A training system comprising:

one or more processors;

one or more input and output units in communication with the one or more processors;

one or more heart rate sensors in communication with the one or more input and output units;

one or more respiratory rate sensors in communication with the one or more input and output units;

one or more skin conductance sensors in communication with the one or more input and output units;

one or more blood glucose sensors in communication with the one or more input and output units;

one or more blood pressure sensors in communication with the one or more input and output units;

one or more neural sensors in communication with the one or more input and output units;

one or more facial recognition sensors in communication with the one or more input and output units and positioned to capture images of physical facial features;

one or more displays in communication with the one or more processors; and

one or more non-transitory processor-readable media in communication with the one or more processors, the one or more non-transitory processor-readable media having processor-readable instructions stored therein that when executed cause the training system to perform the steps of;

providing a virtual reality training session;

obtaining biometric data from a first user during the virtual reality training session, the obtaining comprising the steps of;

converting measurements from the one or more heart rate sensors into electronic heart rate data,

converting respiratory rate measurements from the one or more respiratory rate sensors into electronic respiratory rate data,

converting skin conductance measurements from the one or more skin conductance sensors into electronic skin conductance data,

converting blood glucose measurements from the one or more blood glucose sensors into electronic blood glucose data,

converting blood pressure measurements from the one or more blood pressure sensors into electronic blood pressure data,

converting neural signals measured by the one or more neural sensors into electronic neural data,

converting physical facial features captured by the one or more facial recognition sensors into electronic facial data indicative of one or more of;

gender, age, and emotion of the first user,

determining a stress level of the first user responsive to analysis of at least the electronic heart rate data, the electronic respiratory rate data, the electronic skin conductance data, the electronic blood glucose data, and the electronic blood pressure data, and

determining a level of interest, a level of engagement, a level of alertness, and a level of excitement responsive to analysis of at least the electronic neural data and the electronic facial data;

displaying, in real time on the one or more displays, a first indication of one or more of the electronic heart rate data, the electronic respiratory data, the electronic skin conductance data, the electronic blood glucose data, the electronic blood pressure data, the electronic neural data, the electronic facial data, the determined stress level, and the determined levels of interest, engagement, alertness, and excitement;

determining, based on the biometric data obtained, avatar images indicative biometric states of the first user at different points in time during the virtual reality training session; and

displaying a post-training review comprising display of an animation of the avatar images that provides a visual representation of development of the biometric state of the first user during the virtual reality training session.

2. The system of claim 1, wherein the displaying step comprises display of the first indication within a virtual reality interface associated with the virtual reality training session.

3. The system of claim 2, wherein the virtual reality interface is configured to include an avatar representing the first user, and wherein display of the first indication comprises determination of one or more graphical operation based upon at least a portion of the obtained biometric data and application of the one or more graphical operation to the avatar representing the first user.

4. The system of claim 3, wherein the non-transitory processor-readable media has processor-readable instructions stored therein that when executed cause the training system to monitor one or more of the one or more heart rate sensors, the one or more respiratory rate sensors, the one or more skin conductance sensors, the one or more blood glucose sensors, the one or more blood pressure sensors, the one or more neural sensors and the one or more facial recognition sensors for changes in the obtained biometric data;

and

determine one or more further graphical operation responsive to determination of a change in the obtained biometric data and apply the one or more further graphical operation to the displayed avatar.

5. The system of claim 1, wherein the non-transitory processor-readable media has processor-readable instructions stored therein that when executed cause the training system to provide a second indication of one or more of the electronic heart rate data, the electronic respiratory data, the electronic skin conductance data, the electronic blood glucose data, the electronic blood pressure data, the electronic neural data, the electronic facial data, the determined stress level, and the determined levels of interest, engagement, alertness, and excitement to a second user.

6. The system of claim 5, wherein providing the second indication to a second user comprises provision of at least the second indication to the second user in real-time during the virtual reality training session.

7. The system of claim 5, wherein the non-transitory processor-readable media has processor-readable instructions stored therein that when executed cause the training system to store at least a portion of the obtained biometric data;

and

wherein providing the second indication to the second user comprises transmission of the stored at least a portion of the obtained biometric data to the second user for review.

8. The system of claim 1, wherein the non-transitory processor-readable media has processor-readable instructions stored therein that when executed cause the training system to generate one or more alerts responsive to obtaining the biometric data.

9. The system of claim 8, wherein providing an indication of obtained biometric data to a user comprises provision of the one or more alerts to the user.

10. The system of claim 1, wherein the non-transitory processor-readable medium has processor-readable instructions stored therein that when executed cause the system to monitor the obtained biometric data in real-time to determine whether one or more biometric boundary conditions are exceeded.

11. The system of claim 10, wherein the non-transitory processor-readable media has processor-readable instructions stored therein that when executed cause the training system to generate one or more alerts responsive to obtaining the biometric data and wherein generating one or more alerts is responsive to determination that one or more biometric boundary conditions are exceeded.

12. The system of claim 1, wherein providing the virtual reality training session comprises:

receiving a selection of a training module from one of a plurality of training modules; and

determining biometric data required by the selected training module;

wherein the step of obtaining biometric data is responsive to the determination of the biometric data required by the selected training module.

13. The system of claim 1, wherein a virtual reality simulation of the virtual reality training session comprises a plurality of paths, and the method further comprises selecting one or more of the plurality of paths responsive to the obtained biometric data.

14. A method of providing training in a training system, the method comprising:

obtaining biometric data from a first user during a virtual reality training session, the obtaining comprising;

converting measurements from one or more heart rate sensors into electronic heart rate data,

converting respiratory rate measurements from one or more respiratory rate sensors into electronic respiratory rate data,

converting skin conductance measurements from one or more skin conductance sensors into electronic skin conductance data,

converting blood glucose measurements from one or more blood glucose sensors into electronic blood glucose data,

converting blood pressure measurements from one or more blood pressure sensors into electronic blood pressure data,

converting neural signals measured by one or more neural sensors into electronic neural data,

converting physical facial features captured by one or more facial recognition sensors into electronic facial data indicative of one or more of;

gender, age, and emotion of the first user,

determining a stress level of the first user responsive to analysis of at least the electronic heart rate data, the electronic respiratory rate data, the electronic skin conductance data, the electronic blood glucose data, and the electronic blood pressure data, and

determining a level of interest, a level of engagement, a level of alertness, and a level of excitement responsive to analysis of at least the electronic neural data and the electronic facial data; and

displaying, in real time on the one or more displays, a first indication of one or more of the electronic heart rate data, the electronic respiratory data, the electronic skin conductance data, the electronic blood glucose data, the electronic blood pressure data, the electronic neural data, the electronic facial data, the determined stress level, and the determined levels of interest, engagement, alertness, and excitement;

determining, based on the biometric data obtained, avatar images indicative biometric states of the first user at different points in time during the virtual reality training session; and

displaying a post-training review comprising display of an animation of the avatar images that provides a visual representation of development of the biometric state of the first user during the virtual reality training session.

15. The method of claim 14, further comprising the step of displaying an avatar representing the first user within a virtual reality interface associated with the virtual reality training session, determination of one or more graphical operation based upon at least a portion of the obtained biometric data and application of the one or more graphical operation to the displayed avatar.

16. The method of claim 15, further comprising the steps of:

monitoring one or more of the one or more heart rate sensors, the one or more respiratory rate sensors, the one or more skin conductance sensors, the one or more blood glucose sensors, the one or more blood pressure sensors, the one or more neural sensors and the one or more facial recognition sensors for a change in the obtained biometric data; and

determination of one or more further graphical operation responsive to determination of a change in the obtained biometric data and application of the one or more further graphical operation to the displayed avatar.

17. The method of claim 14, further comprising the step of providing a second indication of one or more of the electronic heart rate data, the electronic respiratory data, the electronic skin conductance data, the electronic blood glucose data, the electronic blood pressure data, the electronic neural data, the electronic facial data, the determined stress level, and the determined levels of interest, engagement, alertness, and excitement to a second user.

18. The method of claim 14, further comprising a real-time determination of whether the obtained biometric data indicates that one or more biometric boundary conditions are exceeded;

and

generation of one or more alerts responsive to a determination that one or more biometric boundary conditions are exceeded.

19. The method of claim 18, wherein the step of providing an indication of obtained biometric data to a user comprises providing one or more of the one or more alerts to the user.

20. A non-transitory computer readable medium comprising program instructions stored thereon that are executable by one or more processors to cause the following operations for providing training in a training system:

obtaining biometric data from a first user during a virtual reality training session, the obtaining comprising;

converting measurements from one or more heart rate sensors into electronic heart rate data,

converting respiratory rate measurements from one or more respiratory rate sensors into electronic respiratory rate data,

converting skin conductance measurements from one or more skin conductance sensors into electronic skin conductance data,

converting blood glucose measurements from one or more blood glucose sensors into electronic blood glucose data,

converting blood pressure measurements from one or more blood pressure sensors into electronic blood pressure data,

converting neural signals measured by one or more neural sensors into electronic neural data,

converting physical facial features captured by one or more facial recognition sensors into electronic facial data indicative of one or more of;

gender, age, and emotion of the first user,

determining a stress level of the first user responsive to analysis of at least the electronic heart rate data, the electronic respiratory rate data, the electronic skin conductance data, the electronic blood glucose data, and the electronic blood pressure data, and

determining a level of interest, a level of engagement, a level of alertness, and a level of excitement responsive to analysis of at least the electronic neural data and the electronic facial data;

displaying, in real time on the one or more displays, a first indication of one or more of the electronic heart rate data, the electronic respiratory data, the electronic skin conductance data, the electronic blood glucose data, the electronic blood pressure data, the electronic neural data, the electronic facial data, the determined stress level, and the determined levels of interest, engagement, alertness, and excitement,

determining, based on the biometric data obtained, avatar images indicative biometric states of the first user at different points in time during the virtual reality training session; and

displaying a post-training review comprising display of an animation of the avatar images that provides a visual representation of development of the biometric state of the first user during the virtual reality training session.