Stateful attack protection

• US 7,607,170 B2
• Filed: 12/22/2004
• Issued: 10/20/2009
• Est. Priority Date: 12/22/2004
• Status: Active Grant

• Current Assignee: Radware Limited
• Original Assignee: Radware Limited
• Inventors: Chesla, Avi
• Primary Examiner(s): Simitoski; Michael J
• Application Number:
  US11/018,255
• Publication Number:
  US 20060137009A1
• Time in Patent Office: 1,763 Days
• Field of Search: 726/22, 726/24, 726/25, 726/23, 709/224
• US Class Current: 726/22
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00

Abstract

A method for detecting an attack in a computer network includes monitoring communication traffic transmitted over connections on the network that are associated with a stateful application protocol so as to detect respective states of the connections, and analyzing a distribution of the states so as to detect the attack.

Claims

1. A computer-implemented method for protecting a computer network, comprising:

monitoring communication traffic transmitted between clients outside the protected network and an application server inside the protected network over connections on the network that are associated with a stateful application protocol implemented by the application server;

implementing a state machine that tracks the connections between the clients and the application server, and makes transitions between state machine states based on application commands and replies generated by the clients and application server, in accordance with rules of the stateful application protocol, so as to detect respective application states of the connections;

analyzing a distribution of the application states so as to detect an attack on the application server; and

filtering traffic entering the network in order to block traffic participating in the attack, wherein analyzing the distribution comprises interpreting as indicative of the attack a number of connections in one of the application states that is beyond a certain number of standard deviations from an average number of connections in the other application states.

2. The method according to claim 1, wherein monitoring comprises detecting respective application handshake states of the connections, and wherein analyzing comprises analyzing the distribution of the application handshake states so as to detect the attack.

3. The method according to claim 1, wherein the stateful application protocol includes Simple Mail Transfer Protocol (SMTP).

4. The method according to claim 1, wherein the stateful application protocol is selected from a list consisting of:

Post Office Protocol (POP), Internet Message Access Protocol (IMAP), and File Transfer Protocol (FTP).

5. The method according to claim 1, wherein analyzing the distribution comprises interpreting a number of connections from a single source address that is greater than a threshold value as indicative of the attack.

6. The method according to claim 1, wherein analyzing the distribution comprises interpreting a deviation of the distribution from a baseline distribution of the application states as indicative of the attack.

7. The method according to claim 6, wherein analyzing the distribution comprises interpreting a disproportionately large number of connections in one of the application states compared to the baseline distribution as indicative of the attack.

8. The method according to claim 6, wherein the baseline distribution is a generally uniform distribution of the application states, and wherein analyzing the distribution comprises interpreting the deviation of the distribution from the generally uniform baseline distribution as indicative of the attack.

9. The method according to claim 8, wherein analyzing the distribution comprises interpreting a disproportionately large number of connections in one of the application states as indicative of the attack.

10. The method according to claim 1, wherein analyzing the distribution comprises calculating an average number of connections for each of the application states.

11. The method according to claim 10, wherein calculating the average number of connections comprises calculating the average number of connections for each of the application states on an aggregate basis for all source addresses of the traffic.

12. The method according to claim 10, wherein calculating the average number of connections comprises repeatedly calculating the average number for recent traffic.

13. The method according to claim 12, wherein calculating the average number of connections comprises applying Infinite Impulse Response (IIR) filtering.

14. The method according to claim 12, wherein calculating the average number of connections comprises calculating the average number using a moving window.

15. The method according to claim 1, wherein the connections are additionally associated with a transport layer protocol, and wherein analyzing the distribution comprises:

analyzing a pattern of the connections at the transport layer so as to detect a potential attack; and

analyzing the distribution of the application states so as to make a determination that the potential attack is the attack.

16. The method according to claim 1, wherein analyzing the distribution comprises:

measuring a time-related property of traffic entering the network;

transforming the time-related property of the traffic into a frequency domain; and

analyzing the distribution of the application states and the property in the frequency domain so as to detect the attack.

17. The method according to claim 16, wherein measuring the time-related property comprises measuring arrival times of packets, and wherein transforming the time-related property comprises determining a spectrum of packet arrival frequency.

18. Apparatus for protecting an application server inside a computer network in communication with clients outside the network, the apparatus comprising:

an interface; and

a network security processor, which is adapted to monitor, via the interface. communication traffic transmitted between the clients and the application server over connections on the network that are associated with a stateful application protocol implemented by the application server;

to implement a state machine that tracks the connections between the clients and the application server, and makes transitions between state machine states based on application commands and replies generated by the clients and application server, in accordance with rules of the stateful application protocol, so as to detect respective application states of the connection;

to analyze a distribution of the application states so as to detect an attack on the application server; and

to filter traffic entering the network in order to block traffic participating in the attack, wherein the network security processor is adapted to interpret as indicative of the attack a number of connections in one of the application states that is beyond a certain number of standard deviations from an average number of connections in the other application states.

19. The apparatus according to claim 18, wherein the network security processor is adapted to detect respective application handshake states of the connections, and to analyze the distribution of the application handshake states so as to detect the attack.

20. The apparatus according to claim 18, wherein the stateful application protocol includes Simple Mail Transfer Protocol (SMTP).

21. The apparatus according to claim 18, wherein the stateful application protocol is selected from a list consisting of:

Post Office Protocol (POP), Internet Message Access Protocol (IMAP), and File Transfer Protocol (FTP).

22. The apparatus according to claim 18, wherein the network security processor is adapted to interpret a number of connections from a single source address that is greater than a threshold value as indicative of the attack.

23. The apparatus according to claim 18, wherein the network security processor is adapted to interpret a deviation of the distribution from a baseline distribution of the application states as indicative of the attack.

24. The apparatus according to claim 23, wherein the network security processor is adapted to interpret a disproportionately large number of connections in one of the application states compared to the baseline distribution as indicative of the attack.

25. The apparatus according to claim 23, wherein the baseline distribution is a generally uniform distribution of the application states, and wherein the network security processor is adapted to interpret the deviation of the distribution from the generally uniform baseline distribution as indicative of the attack.

26. The apparatus according to claim 25, wherein the network security processor is adapted to interpret a disproportionately large number of connections in one of the application states as indicative of the attack.

27. The apparatus according to claim 18, wherein the network security processor is adapted to analyze the distribution by calculating an average number of connections for each of the application states.

28. The apparatus according to claim 27, wherein the network security processor is adapted to calculate the average number of connections for each of the application states on an aggregate basis for all source addresses of the traffic.

29. The apparatus according to claim 27, wherein the network security processor is adapted to repeatedly calculate the average number for recent traffic.

30. The apparatus according to claim 29, wherein the network security processor is adapted to calculate the average number of connections by applying Infinite Impulse Response (IIR) filtering.

31. The apparatus according to claim 29, wherein the network security processor is adapted to calculate the average number of connections using a moving window.

32. The apparatus according to claim 18, wherein the connections are additionally associated with a transport layer protocol, and wherein the network security processor is adapted to analyze the distribution by analyzing a pattern of the connections at the transport layer so as to detect a potential attack, and analyzing the distribution of the application states so as to make a determination that the potential attack is the attack.

33. The apparatus according to claim 18, wherein the network security processor is adapted to analyze the distribution by:

measuring a time-related property of traffic entering the network,

transforming the time-related property of the traffic into a frequency domain, and

analyzing the distribution of the application states and the property in the frequency domain so as to detect the attack.

34. The apparatus according to claim 33, wherein the network security processor is adapted to measure the time-related property by measuring arrival times of packets, and to transform the time-related property by determining a spectrum of packet arrival frequency.

35. A computer software product for protecting a computer network, the product comprising a tangible computer-readable medium in which program instructions are stored, which instructions, when read by a computer, cause the computer to monitor communication traffic transmitted between clients outside the protected network and an application server inside the protected network over connections on the network that are associated with a stateful application protocol implemented by the application server;

to implement a state machine that tracks the connections between the clients and the application server, and makes transitions between state machine states based on application commands and replies generated by the clients and application server, in accordance with rules of the stateful application protocol, so as to detect respective application states of the connections;

to analyze a distribution of the application states so as to detect an attack on the application server; and

to filter traffic entering the network in order to block traffic participating in the attack, wherein the instructions cause the computer to interpret as indicative of the attack a number of connections in one of the application states that is beyond a certain number of standard deviations from an average number of connections in the other application states.

36. The product according to claim 35, wherein the instructions cause the computer to detect respective application handshake states of the connections, and to analyze the distribution of the application handshake states so as to detect the attack.

37. The product according to claim 35, wherein the stateful application protocol includes Simple Mail Transfer Protocol (SMTP).

38. The product according to claim 35, wherein the stateful application protocol is selected from a list consisting of:

Post Office Protocol (POP), Internet Message Access Protocol (IMAP), and File Transfer Protocol (FTP).

39. The product according to claim 35, wherein the instructions cause the computer to interpret a number of connections from a single source address that is greater than a threshold value as indicative of the attack.

40. The product according to claim 35, wherein the instructions cause the computer to interpret a deviation of the distribution from a baseline distribution of the application states as indicative of the attack.

41. The product according to claim 40, wherein the instructions cause the computer to interpret a disproportionately large number of connections in one of the application states compared to the baseline distribution as indicative of the attack.

42. The product according to claim 40, wherein the baseline distribution is a generally uniform distribution of the application states, and wherein the instructions cause the computer to interpret the deviation of the distribution from the generally uniform baseline distribution as indicative of the attack.

43. The product according to claim 42, wherein the instructions cause the computer to interpret a disproportionately large number of connections in one of the application states as indicative of the attack.

44. The product according to claim 35, wherein the instructions cause the computer to analyze the distribution by calculating an average number of connections for each of the application states.

45. The product according to claim 44, wherein the instructions cause the computer to calculate the average number of connections for each of the application states on an aggregate basis for all source addresses of the traffic.

46. The product according to claim 44, wherein the instructions cause the computer to repeatedly calculate the average number for recent traffic.

47. The product according to claim 46, wherein the instructions cause the computer to calculate the average number of connections by applying Infinite Impulse Response (IIR) filtering.

48. The product according to claim 46, wherein the instructions cause the computer to calculate the average number of connections using a moving window.

49. The product according to claim 35, wherein the connections are additionally associated with a transport layer protocol, and wherein the instructions cause the computer to analyze the distribution by analyzing a pattern of the connections at the transport layer so as to detect a potential attack, and analyzing the distribution of the application states so as to make a determination that the potential attack is the attack.

50. The product according to claim 35, wherein the instructions cause the computer to analyze the distribution by:

measuring a time-related property of traffic entering the network,

transforming the time-related property of the traffic into a frequency domain, and

analyzing the distribution of the application states and the property in the frequency domain so as to detect the attack.

51. The product according to claim 50, wherein the instructions cause the computer to measure the time-related property by measuring arrival times of packets, and to transform the time-related property by determining a spectrum of packet arrival frequency.