Simulation based training system for measurement of team cognitive load to automatically customize simulation content

• US 10,783,801 B1
• Filed: 12/21/2017
• Issued: 09/22/2020
• Est. Priority Date: 12/21/2016
• Status: Active Grant

• Current Assignee: Aptima Inc.
• Original Assignee: Aptima Inc.
• Inventors: Beaubien, Jeffrey, Feeney, John J., DePriest, William N., Pappada, Scott M.
• Primary Examiner(s): McClellan, James S.
• Application Number:
  US15/850,998
• Time in Patent Office: 1,006 Days
• CPC Class Codes: A61B 5/0245 by using sensing means generating electric signals, i.e. ECG signals
• CPC Class Codes: A61B 5/16 Devices for psychotechnics using teaching or educational appliances G09B1/00 - G09B7/00; Testing reaction times ; Devices for evaluating the psychological state
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 23/28 for medicine
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G09B 7/10 wherein a set of answers is common to a plurality of questions
• CPC Class Codes: G09B 9/00 Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131

Abstract

In one example embodiment of the invention, a simulation based training system is provided having a sensor that unobtrusively collects objective data for individuals and teams experiencing training content to determine the cognitive states of individuals and teams; time-synchronizes the various data streams; automatically determines granular and objective measures for individual cognitive load (CL) of individuals and teams; and automatically determines a cognitive load balance (CLB) and a relative cognitive load (RCL) measure in real or near-real time. Data is unobtrusively gathered through physiological or other activity sensors such as electroencephalogram (EEG) and electrocardiogram (ECG) sensors. Some embodiments are further configured to also include sociometric data in the determining cognitive load. Sociometric data may be obtained through the use of sociometric badges. Some embodiments further automatically customize the simulation content by automatically selecting content based on the CL of the individuals and teams.

Claims

1. A simulation based training system for real-time measurement of a team cognitive load to automatically customize a simulation content for a team, the simulation based training system comprising:

a processor based simulator comprising;

a user interface,

a cognitive load measuring subsystem, and

a simulation content customization subsystem;

the user interface configured to present a first simulation content concurrently to a team comprising a first user and a second user;

a first sensor comprising a first physiological sensor configured to capture a first raw input from the first user and communicate the first raw input to the cognitive load measuring subsystem;

a second sensor comprising a second physiological sensor configured to capture a second raw input from the second user and communicate the second raw input to the cognitive load measuring subsystem;

the cognitive load measuring subsystem comprising;

a processor,

an input translator configured to translate the first raw input to a first compliant input value and translate the second raw input to a second compliant input value,

a predefined individual cognitive load model configured to determine a first cognitive load value from the first compliant input value and determine a second cognitive load value from the second compliant input value,

a team cognitive load model configured to determine a team cognitive load value from the first cognitive load value and the second cognitive load value, and

the cognitive load measuring subsystem configured to communicate the team cognitive load value to the simulation content customization subsystem;

the simulation content customization subsystem configured to determine a second simulation content based on the team cognitive load value and communicate the second simulation content as a media file to the user interface;

the processor based simulator configured to communicate the second simulation content to the user interface as a customized simulation content;

the cognitive load measuring subsystem further comprises a predefined range of values for a Zone of Proximal Development (ZPD) for the team; and

the simulation content customization subsystem determines the second simulation content based on a comparison of the team cognitive load value to the ZPD for the team whereby a second team cognitive load value for the second simulation content is more likely to be within the ZPD.

2. The simulation based training system of claim 1 wherein the second simulation content comprises a media file.

3. The simulation based training system of claim 1 wherein the individual cognitive load model comprises a time-delay neural network (TDNN).

4. The simulation based training system of claim 1 wherein the first sensor comprises an ECG sensor and the first raw input is an ECG data.

5. The simulation based training system of claim 1 wherein the first sensor comprises an EEG sensor and the first raw input is an EEG data.

6. The simulation based training system of claim 1 wherein the first sensor comprises a wireless neurophysiological monitor.

7. The simulation based training system of claim 1 wherein the first sensor is a sociometric sensor.

8. The simulation based training system of claim 1 wherein the first sensor is an audio sensor.

9. The simulation based training system of claim 1 further comprising:

a relative cognitive load model configured to determine a relative cognitive load value from the team cognitive load value and one of the first cognitive load value or the second cognitive load value; and

the second simulation content is also determined based on the relative cognitive load value.

10. The simulation based training system of claim 9 wherein the second simulation content to the user interface is further based on a first relative cognitive load value of the first user and a second relative cognitive load value of the second user whereby a difference between the first relative cognitive load value and the second relative cognitive load value of the second user is smaller.

11. A simulation based training system for real-time measurement of a team cognitive load to automatically customize a simulation content for a team, the simulation based training system comprising:

a processor based simulator comprising;

a user interface,

a cognitive load measuring subsystem, and

a simulation content customization subsystem;

the user interface configured to present a first simulation content concurrently to a team comprising a first user and a second user;

a first sensor comprising a first physiological sensor configured to capture a first raw input from the first user and communicate the first raw input to the cognitive load measuring subsystem;

a second sensor comprising a second physiological sensor configured to capture a second raw input from the second user and communicate the second raw input to the cognitive load measuring subsystem;

the cognitive load measuring subsystem comprising;

a processor,

an input translator configured to translate the first raw input to a first compliant input value and translate the second raw input to a second compliant input value,

a predefined individual cognitive load model configured to determine a first cognitive load value from the first compliant input value and determine a second cognitive load value from the second compliant input value,

a team cognitive load model configured to determine a team cognitive load value from the first cognitive load value and the second cognitive load value,

the cognitive load measuring subsystem configured to communicate the team cognitive load value to the simulation content customization subsystem, and

a relative cognitive load model configured to determine a relative cognitive load value from the team cognitive load value and one of the first cognitive load value or the second cognitive load value;

the simulation content customization subsystem configured to determine a second simulation content based on the relative cognitive load value and communicate the second simulation content as a media file to the user interface; and

the processor based simulator configured to communicate the second simulation content to the user interface as a customized simulation content.

12. The simulation based training system of claim 11 wherein:

the cognitive load measuring subsystem further comprises a predefined range of values for a Zone of Proximal Development (ZPD) for the team; and

the simulation content customization subsystem determines the second simulation content based on a comparison of the team cognitive load value to the ZPD for the team whereby a second cognitive load value is more likely to be within the ZPD.

13. The simulation based training system of claim 11 wherein the individual cognitive load model comprises a time-delay neural network (TDNN).

14. The simulation based training system of claim 11 wherein the first sensor comprises an ECG sensor and the first raw input is an ECG data.

15. The simulation based training system of claim 11 wherein the first sensor comprises an EEG sensor and the first raw input is an EEG data.

16. The simulation based training system of claim 11 wherein the first sensor comprises a wireless neurophysiological monitor.

17. The simulation based training system of claim 11 wherein the first sensor is a sociometric sensor.

18. The simulation based training system of claim 11 wherein the first sensor is an audio sensor.

19. The simulation based training system of claim 11 wherein the cognitive load measuring subsystem comprising further comprises:

a cognitive load balance model configured to determine a cognitive load balance value from a first relative cognitive load value of the first user and a second relative cognitive load value of the second user.

20. The simulation based training system of claim 19 wherein the second simulation content to the user interface is further based on the first relative cognitive load value of the first user and the second relative cognitive load value of the second user whereby a difference between the first relative cognitive load value and the second relative cognitive load value of the second user is smaller.