Attack traffic signature generation using statistical pattern recognition

• US 8,997,227 B1
• Filed: 02/27/2012
• Issued: 03/31/2015
• Est. Priority Date: 02/27/2012
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Mhatre, Amit J., Kiggins, Andrew John, Diggins, Michael F.
• Primary Examiner(s): NGUYEN, TU T
• Application Number:
  US13/406,274
• Time in Patent Office: 1,128 Days
• Field of Search: 726/23
• US Class Current: 726/23
• CPC Class Codes: G06F 21/566 Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1458 Denial of Service

Abstract

A pattern recognition security system (“PRSS”) generates a packet signature from network traffic, including attack packets. The PRSS can utilize a statistical pattern recognition based approach to generate attack traffic signatures, such as for DDoS or DoS attacks. In some embodiments, the PRSS dynamically creates training sets from actual captured data, allowing the PRSS to adapt to changes in network attacks. For example, more sophisticated DDoS attacks commonly rotate through different attacking computers to vary the packet attributes of attack packets sent to a target system. However, as the PRSS can determine packet signatures based on the actual captured data packets, the PRSS can adapt to the changes in the attack. In some embodiments, the PRSS may determine packet signatures in real-time or near real time during an attack, allowing the PRSS to quickly react to changes in attack traffic.

Claims

1. A method for generating attack packet signatures, the method comprising:

by a signature generating system comprising computing hardware;

determining baseline values for attributes of a second set of data packets captured during normal operations;

obtaining data packets corresponding to a network attack on a target system;

determining packet probabilities for the obtained data packets, wherein each packet probability indicates a probability that an individual data packet of the obtained data packets corresponds to a network attack, and wherein each packet probability is based at least partly on the determined baseline values for the attributes of the second set of data packets;

designating at least a portion of the data packets as attack packets based at least partly on the determined packet probabilities that individual data packets of the obtained data packets correspond to a network attack;

generating a training data set including the attack packets designated based at least partly on the determined probabilities; and

generating a packet signature for attack packets based at least partly on the training data set including the attack packets designated based at least partly on the determined packet probabilities.

2. The method of claim 1, wherein the signature generating system uses a decision tree algorithm to generate the packet signature.

3. The method of claim 1, further comprising providing the packet signature to the target system.

4. The method of claim 1, further comprising filtering out likely attack packets based on the packet signature.

5. The method of claim 1, further comprising generating a packet filter rule based at least partly on the packet signature.

6. The method of claim 1, wherein the training data set includes non-attack packets.

7. The method of claim 1, wherein designating at least a portion of the data packets as attack packets comprises:

clustering the data packets into normal and attack clusters based at least partly on the determined packet probabilities; and

designating packets of the attack cluster as attack packets.

8. The method of claim 7, wherein low probability packets are designated as attack packets.

9. The method of claim 1, wherein the network attack comprises a denial of service (DoS) attack.

10. The method of claim 1, wherein designating at least a portion of the data packets as attack packets comprises determining that the data packets of the designated portion have an above average probability to correspond to a network attack, wherein the average probability is determined by averaging packet probabilities of a majority of the data packets.

11. The method of claim 10 further comprising designating a second portion of the data packets as non-attack packets based at least partly on determining that the data packets of the designated second portion have below the average probability to correspond to a network attack.

12. A system for generating packet signatures, the system comprising:

a packet capture module configured to capture data packets during a network attack on a target computing system;

a training data generator configured to generate a training data set from the captured data packets, wherein the training data set includes attack packets selected from the captured data packets based at least in part on a set of determined packet probabilities, each packet probability indicating a probability that an individual data packet of the captured data packets corresponds to a network attack; and

a signature generator configured to generate a packet signature for attack packets based at least partly on the training data including the attack packets selected based at least partly on the determined packet probabilities that individual packets correspond to a network attack.

13. The system of claim 12, wherein the attack packets are identified based at least partly on calculated probabilities of individual data packets having particular packet attributes.

14. The system of claim 12, wherein the attack packets are identified based at least partly on number of packets transmitted by a packet source.

15. The system of claim 12, wherein the attack packets are identified based at least partly on clustering algorithms for grouping the captured data packets.

16. The system of claim 12, wherein the signature generator uses a decision tree algorithm to generate the packet signature.

17. The system of claim 12, further comprising the target computing system, wherein the target computing system receives the packet signature and subsequently filters packets based on the packet signature.

18. The system of claim 17, wherein the target computing system includes a protected computing host for hosting customer applications.

19. The system of claim 17, wherein a packet filter rule is generated from the packet signature and packets received on the target computing system are filtered using the packet filter rule.

20. The system of claim 12, further comprising a network device, the network device configured to filter packets directed to the target computing system based at least partly on the packet signature.

21. The system of claim 12, wherein the signature generator is configured to generate the packet signature substantially in real-time in the course of the network attack.

22. The system of claim 12, further comprising a network attack detector in communication with the signature generator, wherein the signature generator is further configured to generate a packet signature in response to the network attack detector detecting a network attack.

23. Non-transitory computer storage having stored thereon instructions that, when executed by a computer system, cause the computer system to:

compare a first set of data packets captured during a network attack on a target system with a second set of data packets captured during a different time period than the network attack to determine a set of packet probabilities, each packet probability indicating a probability that an individual data packet of the first set of data packets corresponds to a network attack;

designate at least a portion of the data packets as attack packets based at least partly on the determined packet probabilities that individual packets of the first set of data packets correspond to a network attack; and

generate a packet signature for attack packets based at least partly on the identified attack packets designated based at least partly on the determined packet probabilities.

24. The non-transitory computer storage of claim 23, wherein the packet signature is generated using a decision tree algorithm.

25. The non-transitory computer storage of claim 23, further having instructions that cause the computer system to determine packet attributes that indicate an attack packet based at least partly on the packet attributes of the identified attack packets.

26. The non-transitory computer storage of claim 25, wherein the packet signature is generated based at least partly on the determined packet attributes that indicate an attack packet.

27. The non-transitory computer storage of claim 25, wherein particular values of the packet attributes are determined to indicate an attack packet.

28. The non-transitory computer storage of claim 23, wherein the packet signature specifies characterizing attributes of attack packets.