Automated execution and evaluation of network-based training exercises

• US 9,384,677 B2
• Filed: 04/10/2015
• Issued: 07/05/2016
• Est. Priority Date: 02/19/2008
• Status: Active Grant

• Current Assignee: Architecture Technology Corporation
• Original Assignee: Architecture Technology Corporation
• Inventors: Brueckner, Stephen K., Adelstein, Frank N., Bar, Haim Yehuda, Donovan, Matthew P.
• Primary Examiner(s): Utama, Robert J
• Application Number:
  US14/683,923
• Publication Number:
  US 20150213730A1
• Time in Patent Office: 452 Days
• US Class Current: 1/1
• CPC Class Codes: G09B 19/003 Repetitive work cycles; Sequence of movements
• CPC Class Codes: G09B 19/0053 Computers, e.g. programming
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 9/00 Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131
• CPC Class Codes: H04L 63/029 Firewall traversal, e.g. tunnelling or, creating pinholes
• CPC Class Codes: H04L 63/1441 Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
• CPC Class Codes: H04L 63/1458 Denial of Service
• CPC Class Codes: H04L 63/1475 Passive attacks, e.g. eavesdropping or listening without modification of the traffic monitored

Abstract

This disclosure generally relates to automated execution and evaluation of computer network training exercises, such as in a virtual machine environment. An example environment includes a control and monitoring system, an attack system, and a target system. The control and monitoring system initiates a training scenario to cause the attack system to engage in an attack against the target system. The target system then performs an action in response to the attack. Monitor information associated with the attack against the target system is collected by continuously monitoring the training scenario. The attack system is then capable of sending dynamic response data to the target system, wherein the dynamic response data is generated according to the collected monitor information to adapt the training scenario to the action performed by the target system. The control and monitoring system then generates an automated evaluation based upon the collected monitor information.

Claims

1. A method comprising:

during a computer-based training exercise, initiating, by an attack system, a simulated attack against a target system, wherein the target system is configured to respond to actions specified by a human trainee, wherein the target system performs a corrective or preventive action that is specified by the human trainee in response to the simulated attack, and wherein the attack system is configured to initiate a change in the simulated attack by dynamically responding to the corrective or preventive action performed by the target system and specified by the human trainee;

collecting information associated with the corrective or preventive action performed by the target system in response to the simulated attack; and

based on the corrective or preventive action performed by the target system;

updating a state of the attack system;

automatically generating, by the attack system and based on the updated state of the attack system, dynamic response data;

sending the dynamic response data from the attack system to the target system, wherein sending the dynamic response data initiates the change in the simulated attack against the target system; and

upon completion of the training exercise, generating an automated evaluation of a performance of the human trainee.

2. The method of claim 1, wherein the attack system and the target system comprise one or more virtual machines.

3. The method of claim 1, further comprising:

processing scenario traffic for the training exercise on a first communication channel;

processing out-of-band data for the training exercise on a second communication channel that is distinct from the first communication channel, such that the out-of-band data does not interfere with the scenario traffic; and

controlling the training exercise using the out-of-band data.

4. The method of claim 1, wherein collecting the information associated with the corrective or preventive action performed by the target system comprises receiving user input from the user indicating a reason for performing the corrective or preventive action.

5. The method of claim 4, wherein generating the automated evaluation comprises analyzing the user input to determine if the indicated reason for performing the corrective or preventive action is correct according to the training exercise.

6. The method of claim 4, wherein receiving the user input comprises receiving the user input either by way of an electronic notebook that records information entered by the human trainee or by way of an instant message exchange with the human trainee.

7. The method of claim 1,

wherein collecting the information associated with the corrective or preventive action performed by the target system comprises collecting active feedback data and passive feedback data,

wherein the active feedback data comprises one or more of action data, state data, or metric data, and

wherein the passive feedback data comprises one or more of notebook data, instant message data, or state knowledge data.

8. A non-transitory computer-readable storage medium comprising instructions that, when executed, cause one or more processors to:

during a training exercise, initiate, by an attack system, a simulated attack against a target system, wherein the target system is configured to respond to actions specified by a human trainee, wherein the target system performs a corrective or preventive action that is specified by the human trainee in response to the simulated attack, and wherein the attack system is configured to initiate a change in the simulated attack by dynamically responding to the corrective or preventive action performed by the target system and specified by the human trainee;

collect information associated with the corrective or preventive action performed by the target system in response to the simulated attack; and

based on the corrective or preventive action performed by the target system;

update a state of the attack system;

automatically generate, by the attack system and based on the updated state of the attack system, dynamic response data;

send the dynamic response data from the attack system to the target system, wherein sending the dynamic response data initiates the change in the simulated attack against the target system; and

upon completion of the training exercise, generate an automated evaluation of a performance of the human trainee.

9. The non-transitory computer-readable storage medium of claim 8, wherein the attack system and the target system comprise one or more virtual machines.

10. The non-transitory computer-readable storage medium of claim 8, further comprising instructions that, when executed, cause the one or more processors to:

process scenario traffic for the training exercise on a first communication channel;

process out-of-band data for the training exercise on a second communication channel that is distinct from the first communication channel, such that the out-of-band data does not interfere with the scenario traffic; and

control the training exercise using the out-of-band data.

11. The non-transitory computer-readable storage medium of claim 8, wherein the instructions that cause the one or more processors to collect the information associated with the corrective or preventive action performed by the target system comprise instructions that cause the one or more processors to receive user input from the user indicating a reason for performing the corrective or preventive action.

12. The non-transitory computer-readable storage medium of claim 11, wherein the instructions that cause the one or more processors to generate the automated evaluation comprise instructions that cause the one or more processors to analyze the user input to determine if the indicated reason for performing the corrective or preventive action is correct according to the training exercise.

13. The non-transitory computer-readable storage medium of claim 11, wherein the instructions that cause the one or more processors to receive the user input comprise instructions that cause the one or more processors to receive the user input either by way of an electronic notebook that records information entered by the human trainee or by way of an instant message exchange with the human trainee.

14. The non-transitory computer-readable storage medium of claim 8,

wherein the instructions that cause the one or more processors to collect the information associated with the corrective or preventive action performed by the target system comprise instructions that cause the one or more processors to collect active feedback data and passive feedback data,

wherein the active feedback data comprises one or more of action data, state data, or metric data, and

wherein the passive feedback data comprises one or more of notebook data, instant message data, or state knowledge data.

15. A system comprising:

one or more processors,

wherein the one or more processors are configured to;

during a training exercise, initiate, by an attack system, a simulated attack against a target system, wherein the target system is configured to respond to actions specified by a human trainee, wherein the target system performs a corrective or preventive action that is specified by the human trainee in response to the simulated attack, and wherein the attack system is configured to initiate a change in the simulated attack by dynamically responding to the corrective or preventive action performed by the target system and specified by the human trainee;

collect information associated with the corrective or preventive action performed by the target system in response to the simulated attack; and

based on the corrective or preventive action performed by the target system;

update a state of the attack system;

automatically generate, by the attack system and based on the updated state of the attack system, dynamic response data;

send the dynamic response data from the attack system to the target system, wherein sending the dynamic response data initiates the change in the simulated attack against the target system; and

upon completion of the training exercise, generate an automated evaluation of a performance of the human trainee.

16. The system of claim 15, wherein the attack system and the target system comprise one or more virtual machines.

17. The system of claim 15, wherein the one or more processors are further configured to:

process scenario traffic for the training exercise on a first communication channel;

process out-of-band data for the training exercise on a second communication channel that is distinct from the first communication channel, such that the out-of-band data does not interfere with the scenario traffic; and

control the training exercise using the out-of-band data.

18. The system of claim 15, wherein the one or more processors configured to collect the information associated with the corrective or preventive action performed by the target system are configured to receive user input from the user indicating a reason for performing the corrective or preventive action.

19. The system of claim 18, wherein the one or more processors configured to generate the automated evaluation are configured to analyze the user input to determine if the indicated reason for performing the corrective or preventive action is correct according to the training exercise.

20. The system of claim 18, wherein the one or more processors configured to receive the user input are configured to receive the user input either by way of an electronic notebook that records information entered by the human trainee or by way of an instant message exchange with the human trainee.

21. The system of claim 15,

wherein the one or more processors configured to collect the information associated with the corrective or preventive action performed by the target system are configured to collect active feedback data and passive feedback data,

wherein the active feedback data comprises one or more of action data, state data, or metric data, and

wherein the passive feedback data comprises one or more of notebook data, instant message data, or state knowledge data.

22. The system of claim 15, wherein the attack system and the target system are included within one attack/target system, and wherein the attack against the target system comprises an insider attack.