AUTOMATED EXECUTION AND EVALUATION OF NETWORK-BASED TRAINING EXERCISES

• US 20090208910A1
• Filed: 02/18/2009
• Published: 08/20/2009
• Est. Priority Date: 02/19/2008
• Status: Active Grant

• Current Assignee: Architecture Technology Corporation
• Original Assignee: Architecture Technology Corporation
• Inventors: Donovan, Matthew, Brueckner, Stephen, Adelstein, Frank N., Bar, Haim
• Granted Patent: US 9,076,342 B2
• Time in Patent Office: Days
• US Class Current: 434/11
• CPC Class Codes: G09B 19/003 Repetitive work cycles; Sequence of movements
• CPC Class Codes: G09B 19/0053 Computers, e.g. programming
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 9/00 Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131
• CPC Class Codes: H04L 63/029 Firewall traversal, e.g. tunnelling or, creating pinholes
• CPC Class Codes: H04L 63/1441 Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
• CPC Class Codes: H04L 63/1458 Denial of Service
• CPC Class Codes: H04L 63/1475 Passive attacks, e.g. eavesdropping or listening without modification of the traffic monitored

Abstract

This disclosure generally relates to automated execution and evaluation of computer network training exercises, such as in a virtual machine environment. An example environment includes a control and monitoring system, an attack system, and a target system. The control and monitoring system initiates a training scenario to cause the attack system to engage in an attack against the target system. The target system then performs an action in response to the attack. Monitor information associated with the attack against the target system is collected by continuously monitoring the training scenario. The attack system is then capable of sending dynamic response data to the target system, wherein the dynamic response data is generated according to the collected monitor information to adapt the training scenario to the action performed by the target system. The control and monitoring system then generates an automated evaluation based upon the collected monitor information.

Claims

1. A method comprising:

providing a training environment that includes a control and monitoring system, an attack system, and a target system that are each executable by one or more processors;

initiating, by the control and monitoring system, a training scenario to cause the attack system to engage in an attack against the target system;

performing an action by the target system in response to the attack;

collecting monitor information associated with the attack against the target system by continuously monitoring the training scenario;

sending dynamic response data from the attack system to the target system based upon the collected monitor information to adapt the training scenario to the action performed by the target system; and

generating, by the control and monitoring system, an automated evaluation based upon the collected monitor information.

2. The method of claim 1, wherein the attack system and the target system are included within one attack/target system, and wherein the attack against the target system comprises an insider attack.

3. The method of claim 1, wherein collecting the monitor information associated with the training scenario comprises collecting information associated with the action performed by the target system.

4. The method of claim 1, wherein collecting the monitor information associated with the training scenario comprises collecting active feedback data and passive feedback data, wherein the active feedback data comprises one or more of action data, state data, and metric data, and wherein the passive feedback data comprises one or more of notebook data, instant message data, and state knowledge data.

5. The method of claim 1, wherein one or more of the control and monitoring system, the attack system, and the target system include one or more virtual machines for performing training activities.

6. The method of claim 1, wherein sending dynamic response data from the attack system to the target system comprises:

maintaining a state on the attack system that is based upon the collected monitor information;

receiving a notification of the action performed by the target system;

updating the state based upon the action performed by the target system; and

generating the dynamic response data according to the updated state.

7. The method of claim 1, wherein performing the action by the target system comprises receiving user input from a user of the target system specifying the action to be performed by the target system, wherein collecting the monitor information comprises receiving additional user input from the user indicating a reason for performing the action, and wherein generating the automated evaluation comprises analyzing the additional user input to determine if the indicated reason for performing the action is correct according to the training scenario.

8. The method of claim 7, wherein receiving the additional user input comprises receiving the additional user input either by way of an electronic notebook that records information entered by the user or by way of an instant message exchange.

9. The method of claim 1, further comprising:

sending scenario traffic for the training scenario on a first communication channel;

sending out-of-band data for the training scenario on a second communication channel that is distinct from the first communication channel; and

monitoring and controlling the training scenario within the training environment using the out-of-band data, wherein the monitor information comprises the out-of-band data, wherein the out-of-band data does not interfere with the scenario traffic sent on the first communication channel.

10. A computer-readable medium comprising instructions that, when executed, cause one or more processors to:

provide a training environment that includes a control and monitoring system, an attack system, and a target system;

initiate, by the control and monitoring system, a training scenario to cause the attack system to engage in an attack against the target system;

perform an action by the target system in response to the attack;

collect monitor information associated with the attack against the target system by continuously monitoring the training scenario;

send dynamic response data from the attack system to the target system based upon the collected monitor information to adapt the training scenario to the action performed by the target system; and

generate, by the control and monitoring system, an automated evaluation based upon the collected monitor information.

11. A system comprising:

one or more processors;

an attack system executable by the one or more processors;

a target system executable by the one or more processors; and

a control and monitoring system executable by the one or more processors and configured to initiate a training scenario that causes the attack system to engage in an attack against the target system, and further configured to collect monitor information associated with the attack by continuously monitoring the training scenario,

wherein the target system is configured to perform an action in response to the attack,

wherein the attack system is configured to send dynamic response data to the target system based upon the collected monitor information to adapt the training scenario to the action performed by the target system, and

wherein the control and monitoring system is configured to generate an automated evaluation based upon the collected monitor information.

12. The system of claim 11, wherein the attack system and the target system are included within one attack/target system, and wherein the attack against the target system comprises an insider attack.

13. The system of claim 11, wherein the monitor information associated with the training scenario comprises information associated with the action performed by the target system.

14. The system of claim 11, wherein the monitor information associated with the training scenario comprises active feedback data and passive feedback data, wherein the active feedback data comprises one or more of action data, state data, and metric data, and wherein the passive feedback data comprises one or more of notebook data, instant message data, and state knowledge data.

15. The system of claim 11, wherein one or more of the control and monitoring system, the attack system, and the target system include one or more virtual machines for performing training activities.

16. The system of claim 11, wherein the attack system is configured to send dynamic response data to the target system by maintaining a state that is based upon the collected monitor information, receiving a notification of the action performed by the target system, updating the state based upon the action performed by the target system, and generating the dynamic response data according to the updated state.

17. The system of claim 11, wherein the target system is configured to perform the action by receiving user input from a user specifying the action to be performed by the target system, wherein the control and monitoring system is configured collect the monitor information by collecting, from the target system, additional user input from the user indicating a reason for performing the action, and wherein the control and monitoring system is configured to generate the automated evaluation by analyzing the additional user input to determine if the indicated reason for performing the action is correct according to the training scenario.

18. The system of claim 17, wherein the target system is configured to receive the additional user input by receiving the additional user input either by way of an electronic notebook that records information entered by the user or by way of an instant message exchange.

19. The system of claim 11, wherein the attack system and the target system are each configured to send scenario traffic for the training scenario on a first communication channel and to send out-of-band data for the training scenario on a second communication channel that is distinct from the first communication channel, wherein the control and monitoring system is further configured to monitor the training scenario within the training environment by using the out-of-band data, wherein the monitor information comprises the out-of-band data, and wherein the out-of-band data does not interfere with the scenario traffic sent on the first communication channel.

20. A method comprising:

providing a training environment that includes a control and monitoring system, an attack system, and a target system each executable by one or more processors;

initiating, by the control and monitoring system, a training scenario to cause the attack system to engage in an attack against the target system;

sending scenario traffic for the training scenario on a first communication channel;

sending out-of-band data for the training scenario on a second communication channel that is distinct from the first communication channel, wherein the out-of-band data is not visible to a trainee and does not interfere with the scenario traffic sent on the first communication channel; and

monitoring the training scenario by the control and monitoring system using the out-of-band data.

21. The method of claim 20, wherein one or more of the control and monitoring system, the attack system, and the target system include one or more virtual machines for performing training activities.

22. The method of claim 20, wherein:

sending the out-of-band data comprises sending a portion of the out-of-band data to the control and monitoring system from one or more of the attack system and the target system; and

monitoring the training scenario comprises using the portion of the out-of-band data received by the control and monitoring system to monitor the training scenario.

23. The method of claim 20, wherein sending the out-of-band data comprises sending a portion of the out-of-band data from the control and monitoring system to control behavior of one or more of the attack system and the target system during the training scenario.