Adaptive Behavioral HTTP Flood Protection

• US 20080086434A1
• Filed: 10/09/2007
• Published: 04/10/2008
• Est. Priority Date: 10/09/2006
• Status: Active Grant

• Current Assignee: Radware Limited
• Original Assignee: Radware Limited
• Inventors: Chesla, Avi
• Granted Patent: US 7,617,170 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06N 5/048 Fuzzy inferencing
• CPC Class Codes: H04L 2463/141 Denial of service attacks against endpoints in a network
• CPC Class Codes: H04L 63/1458 Denial of Service

Abstract

A system and method to detect and mitigate denial of service and distributed denial of service HTTP “page” flood attacks. Detection of attack/anomaly is made according to multiple traffic parameters including rate-based and rate-invariant parameters in both traffic directions. Prevention is done according to HTTP traffic parameters that are analyzed once a traffic anomaly is detected. This protection includes a differential adaptive mechanism that tunes the sensitivity of the anomaly detection engine. The decision engine is based on a combination between fuzzy logic inference systems and statistical thresholds. A “trap buffer” characterizes the attack to allow an accurate mitigation according to the source IP(s) and the HTTP request URL'"'"'s that are used as part of the attack. Mitigation is controlled through a feedback mechanism that tunes the level of rate limit factors that are needed in order to mitigate the attack effectively while letting legitimate traffic to pass.

Claims

1. An anomaly detection engine comprising:

an interface to receive at least the following;

a plurality of real-time statistical parameters or a plurality of normal base line values, said plurality of real-time statistical parameters comprising at least one rate-based parameter and at least one rate-invariant parameter;

embedded correlation rules;

a degree of anomaly generator generating a degree of anomaly (DoA) based on said received plurality of real-time statistical parameters and said plurality of normal base line values; and

when said generated degree of anomaly indicates a network attack, said decision engine communicates with at least one “

trap”

buffer to characterize the anomaly.

2. The anomaly detection engine of claim 1, wherein said normal base line values are generated by a learning mechanism comprising any of the following:

day-time differential averaging or continuous IIR filtering/continuous averaging, said normal base line values dynamically updated over a predetermined time period so that said anomaly detection engine tunes its detection sensitivity based on said updated normal base line values, said learning mechanism dynamically picked from said day-time differential averaging or said continuous IIR filtering/continuous averaging based on a behavior of a network protected environment.

3. The anomaly detection engine of claim 1, wherein said anomaly detection engine is implemented in server accessible over a network.

4. The anomaly detection engine of claim 3, wherein said network is any of, or a combination of, the following:

a local area network, a wide area network, the Internet, or a cellular network.

5. The anomaly detection engine of claim 3, wherein said network is the Internet and said server is a web server.

6. The anomaly detection engine of claim 1, wherein said network attack is a HTTP flood attack.

7. The anomaly detection engine of claim 6, wherein at least one of said real-time statistical parameters is HTTP request URL distribution.

8. The anomaly detection engine of claim 7, wherein said remainder of said real-time statistical parameters additionally comprises any of, or a combination of, the following:

number of HTTP requests per second, HTTP outbound bandwidth per second, ratio between HTTP request to outbound HTTP bandwidth per second, number of HTTP request per TCP connection, and number of HTTP requests per second per source IP.

9. The anomaly detection engine of claim 5, wherein said at least one “

trap”

buffer further comprises;

at least one source IP trap buffer, said at least one source IP trap buffer detects abnormal repetitions of HTTP request URL'"'"'s per source IP address and protected web server;

at least one HTTP request size trap buffer, said at least one HTTP request trap buffer detecting abnormal repetitions of HTTP request URL'"'"'s per protected web server only.

10. The anomaly detection engine of claim 9, wherein said IP trap buffer and HTTP request size trap buffer is created according to two lists:

a source IP list and a size list, said source IP list including all suspicious source IP addresses and said size list including all suspicious HTTP request URL sizes.

11. The anomaly detection engine of claim 10, wherein said inclusions in each list is determined according to at least one pre-determined threshold.

12. The anomaly detection engine of claim 1, wherein said at least one rate-based parameter is any of the following:

a get request per second counter—

Get_C, which is incremented by one upon classification of any HTTP Get or Post request toward a protected server;

an other request per second counter—

Other_C, which is incremented by one upon classification of any HTTP request type that is not Get or Post toward a protected server;

a post request per second counter—

Post_C, which is incremented by one upon classification of any HTTP Post request toward a protected server;

an outbound bandwidth per second counter—

OutB_C, which measures the outbound HTTP traffic that is generated by a protected server;

a source HTTP request per second counter—

S_R_C, which is attached to each source IP address that “

owns”

a session toward a protected server and wherein the S_R_C counter is incremented by one upon classification of any HTTP Get or Post request toward a protected server; and

a HTTP request per connection counter—

Con_R_C, which is maintained per TCP connection and wherein the Con_R_C counter is incremented upon classification of any HTTP Get or Post request toward a protected server.

13. The anomaly detection engine of claim 12, wherein said at least one rate-invariant parameter is any of the following:

a ratio between outbound bandwidth to inbound HTTP requests—

Out/R_P, which measures an average outbound bandwidth per HTTP Get or Post requests; and

a HTTP request short-term size distribution table—

S_D_T, which measures the current size distribution of HTTP request URL'"'"'s, and upon classification of a HTTP Get or Post requests, the HTTP Get or Post request URL being indexed.

14. The anomaly detection engine of claim 13, wherein said Out/R_P ratio parameter is calculated as follows:

15. The anomaly detection engine of claim 13, wherein HTTP request size index in said S_D_T is calculated as follows:

Request Size Index=INT(Request Size/10),

where said Request Size measures the length in bytes of a request URL path.

16. The anomaly detection engine of claim 15, wherein a measure of deviation from an expected Request URL Size is set over a predetermined time period.

17. The anomaly detection engine of claim 2, wherein said learning mechanism maintains a long-term size distribution table—

L_D_T, wherein a measure of deviation from an expected request size occurrences is set over a predetermined time period.

18. The anomaly detection engine of claim 17, wherein a set of expected values E(i) in calculated from said L_D_T as follows:

E(i)=P(i)×

R_C_n, where R_C_n is the total number of HTTP request (Post and Get) in one second and P(i) is the probability of each size grade.

19. The anomaly detection engine of claim 18, wherein, the normal, suspect and attack edges matrix are calculated as follows:

Normal edge;

N(i)=E(i),

Attack edge;

A(i)=√

{square root over (l×

[E(i)×

(E(i))}{square root over (l×

[E(i)×

(E(i))}+√

{square root over (E(i)×

P))}],

Suspect edge;

S(i)=√

{square root over (A×

N)},

where P is a confidence level, l is a configurable detection sensitivity level, and N is the number of entries in said L_D_T.

20. The anomaly detection engine of claim 1, wherein, after said characterization of an anomaly, any of the following mitigation mechanisms are implemented:

rate limit traffic flows which are defined by both Source IP(s) and associated HTTP request URL checksum towards a protected server;

rate limit traffic flows which are defined only by HTTP request URL checksum(s) toward a protected server; and

blocking source IP(s) toward a protected server,

wherein a rate limit factor is set according to adapted HTTP request URL size distribution (E(i)).

21. The anomaly detection engine of claim 20, wherein said mitigation mechanisms is activated according to any of the following types of feedback:

a DoA that is generated according to all inbound traffic parameter toward the attacked server prior to mitigation; and

a DoA that is generated according to remaining traffic after mitigation.

22. An article of manufacture comprising a computer user medium having computer readable program code embodied therein which implements a method to detect an anomaly, said medium comprising:

computer readable program code aiding in receiving at least the following;

a plurality of real-time statistical parameters or a plurality of normal base line values, said plurality of real-time statistical parameters comprising at least one rate-based parameter or at least one rate-invariant and rate-based parameter;

computer readable program code maintaining a plurality of embedded correlation rules;

computer readable program code generating a degree of anomaly (DoA) based on said received plurality of real-time statistical parameters and said plurality of normal base line values; and

when said generated degree of anomaly indicates a network attack, computer readable program code aiding in communicating with at least one buffer to characterize the anomaly.

23. The article of manufacture of claim 22, wherein said real-time statistical parameters is HTTP request-URL size distribution along with are any of, or a combination of, the following:

number of HTTP requests per second, HTTP outbound bandwidth per second, ratio between HTTP request to outbound HTTP bandwidth per second, number of HTTP request per TCP connection, and number of HTTP requests per second per source IP.