Techniques for customer-derived training of intrusion management systems

• US 10,686,811 B1
• Filed: 11/21/2014
• Issued: 06/16/2020
• Est. Priority Date: 11/21/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Ehle, Braxton Chase
• Primary Examiner(s): Zoubair, Noura
• Application Number:
  US14/550,313
• Time in Patent Office: 2,034 Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

Techniques described and suggested herein include various systems and methods for using customer-initiated security tests to generate training data for use in improving detection and mitigation capabilities related network intrusion and data security attacks. Such techniques may include implementing machine learning techniques to refine security models used therewith. For example, customers of a computing resource provider may notify the computing resource provider that a security test is scheduled to occur. In response, in some embodiments, information related to the security test may be used to improve an implemented security model.

Claims

1. A system, comprising:

one or

more processors; and

memory including instructions that, as a result of execution by the one or more processors, cause the system to;

generate a security model that characterizes network data by applying attack classifications;

determine that a security test is to be performed to a target endpoint by a client device associated with a customer of a computing resource service provider, the target endpoint being a virtual computer system hosted by the computing resource service provider;

obtain, from the customer via the client device, a plurality of attributes of the security test, wherein the plurality of attributes include a first identifier of the client device, second identifier of the target endpoint, and a length of time specified for performing the security test;

modify, based at least in part on the plurality of attributes of the security test, the security model to produce a modified security model to be usable in determining traffic profiles from ingestion of network traffic between the client device and the target endpoint;

determine, based at least in part on the plurality of attributes of the security test and the security model, a subset of the network traffic between the client device and the target endpoint in connection with the security test, the subset of the network traffic conforming to the plurality of attributes of the security test;

process, by at least utilizing the modified security model, the subset of the network traffic to determine a traffic profile;

process the traffic profile using the attack classifications to generate training data; and

update, based at least in part on the training data, the modified security model by at least generating a new attack classification that matches the traffic profile with a higher confidence level than the attack classifications used to generate the training data.

2. The system of claim 1, wherein, the instructions, if executed by the one or more processors, cause the system to further update the security model, based at least in part on the training data, with a confidence level associated with the attack classifications used to generate the training data.

3. The system of claim 1, wherein if the training data indicates that the traffic profile fails to match a subset of the attack classifications, the instructions, if executed by the one or more processors, cause the system to at least:

determine whether the security test is performed; and

update the security model so as to improve a detection baseline of the security model.

4. The system of claim 1, wherein the security test simulates a distributed denial of service attack upon the target endpoint.

5. A computer-implemented method, comprising:

generating a security model that characterizes network data by applying attack classifications;

obtaining, from a customer of a computing resource service provider via a client device, information relating to a security test between the client device and a virtual computer system, the information including a source identifier that identifies the client device, a target identifier that identifies the virtual computer system, and a duration of the security test, the security test being directed to the virtual computer system from the client device;

configuring, based at least in part on the information relating to the security test, the security model to produce a configured security model to be usable in determining traffic profiles from ingestion of network traffic between the client and the virtual computer system;

monitoring the network traffic between the client device and the virtual computer system in connection with the security test;

using the configured security model to determine a traffic profile for the security test;

comparing the traffic profile to the attack classifications to generate training data; and

updating the configured security model using the training data by at least generating a new attack classification that matches the traffic profile with a higher confidence level than the attack classifications used to generate the training data.

6. The computer-implemented method of claim 5, wherein one or more computer systems monitor, during the security test, the network traffic between the client and the virtual computer system for a subset of the network traffic that conforms to one or more attributes.

7. The computer-implemented method of claim 5, further comprising updating the security model, by updating, based at least in part on the training data, a confidence level associated with the attack classifications used to generate the training data.

8. The computer-implemented method of claim 5, further comprising:

detecting whether the security test is occurring; and

updating the security model by improving a detection baseline of the security model.

9. The computer-implemented method of claim 8, wherein the detection baseline is improved by using, at least in part, the training data to refine at least a subset of the attack classifications.

10. The computer-implemented method of claim 5, wherein the security test simulates a distributed denial of service attack upon the virtual computer system.

11. The computer-implemented method of claim 5, wherein machine learning techniques are used in conjunction with the training data to update the security model.

12. The computer-implemented method of claim 5, further comprising:

determining an expected type of traffic between the client device and the virtual computer system based at least in part on other information relating to the client device or the virtual computer system.

13. A non-transitory computer-readable storage medium storing thereon executable instructions that, as a result of execution by one or more processors of a computer system, cause the computer system to at least:

generate a security model that characterizes network data by applying a plurality of attack classifications;

obtain, from a customer of a computing resource service provider via a source client, information relating to a security test being performed to a destination client by the source client, the destination client being a virtual computer system of the computing resource service provider, the information including a first identifier corresponding to the source client, a second identifier corresponding to the destination client, and an amount of time for the security test;

modify, based at least in part on the information, the security model to produce a modified security model to configure the modified security model to be usable in determining traffic profiles from ingestion of network traffic between the source client and the destination client;

monitor the network traffic between the source client and the destination client in connection with the security test;

use the modified security model to determine a traffic profile for a portion of time during which the security test between the source client and the destination client is monitored;

generate training data by assessing the traffic profile against the plurality of attack classifications associated with the modified security model; and

update the modified security model using the training data by at least generating a new attack classification that matches the traffic profile with a higher confidence level than the plurality of attack classifications used to generate the training data.

14. The non-transitory computer-readable storage medium of claim 13, wherein the information is received from an entity operatively connected to the computer system and includes at least a test type and a duration for the security test.

15. The non-transitory computer-readable storage medium of claim 13, wherein the executable instructions that cause the computer system to monitor the network traffic further cause the computer system to monitor the security test by sampling the network traffic between the source client and the destination client associated with the security test.

16. The non-transitory computer-readable storage medium of claim 13, wherein the source client is operatively connected to the computer system.

17. The non-transitory computer-readable storage medium of claim 15, wherein the source client is a virtual computer system of the computing resource service provider.

18. The non-transitory computer-readable storage medium of claim 13, wherein:

the executable instructions that cause the computer system to modify the security model using the training data include instructions that cause the computer system to modify the security model by using binary classification techniques; and

the training data is used input for the binary classification techniques.

19. The non-transitory computer-readable storage medium of claim 13, wherein the security model is a supervised machine learning model.

20. The non-transitory computer-readable storage medium of claim 13, wherein the security test is a penetration test.