Stateful attack protection

• US 20060137009A1
• Filed: 12/22/2004
• Published: 06/22/2006
• Est. Priority Date: 12/22/2004
• Status: Active Grant

• Current Assignee: Radware Limited
• Original Assignee: V-Secure Technologies, Inc. (Radware Limited)
• Inventors: Chesla, Avi
• Granted Patent: US 7,607,170 B2
• Time in Patent Office: Days
• US Class Current: 726/22
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00

Abstract

A method for detecting an attack in a computer network includes monitoring communication traffic transmitted over connections on the network that are associated with a stateful application protocol so as to detect respective states of the connections, and analyzing a distribution of the states so as to detect the attack.

Claims

1. A method for detecting an attack in a computer network, comprising:

monitoring communication traffic transmitted over connections on the network that are associated with a stateful application protocol so as to detect respective states of the connections; and

analyzing a distribution of the states so as to detect the attack.

2. The method according to claim 1, wherein monitoring comprises detecting respective handshake states of the connections, and wherein analyzing comprises analyzing the distribution of the handshake states so as to detect the attack.

3. The method according to claim 1, wherein the stateful application protocol includes Simple Mail Transfer Protocol (SMTP).

4. The method according to claim 1, wherein the stateful application protocol is selected from a list consisting of:

Post Office Protocol (POP), Internet Message Access Protocol (IMAP), and File Transfer Protocol (FTP).

5. The method according to claim 1, and comprising filtering traffic entering the network in order to block traffic participating in the attack.

6. The method according to claim 1, wherein analyzing the distribution comprises interpreting a number of connections from a single source address that is greater than a threshold value as indicative of the attack.

7. The method according to claim 1, wherein analyzing the distribution comprises interpreting a deviation of the distribution from a baseline distribution of the states as indicative of the attack.

8. The method according to claim 7, wherein analyzing the distribution comprises interpreting a disproportionately large number of connections in one of the states compared to the baseline distribution as indicative of the attack.

9. The method according to claim 7, wherein the baseline distribution is a generally uniform distribution of the states, and wherein analyzing the distribution comprises interpreting the deviation of the distribution from the generally uniform baseline distribution as indicative of the attack.

10. The method according to claim 9, wherein analyzing the distribution comprises interpreting a disproportionately large number of connections in one of the states as indicative of the attack.

11. The method according to claim 10, wherein analyzing the distribution comprises interpreting as indicative of the attack a number of connections in one of the states that is beyond a certain number of standard deviations from an average number of connections in the other states.

12. The method according to claim 1, wherein analyzing the distribution comprises calculating an average number of connections for each of the states.

13. The method according to claim 12, wherein calculating the average number of connections comprises calculating the average number of connections for each of the states on an aggregate basis for all source addresses of the traffic.

14. The method according to claim 12, wherein calculating the average number of connections comprises repeatedly calculating the average number for recent traffic.

15. The method according to claim 14, wherein calculating the average number of connections comprises applying Infinite Impulse Response (IIR) filtering.

16. The method according to claim 14, wherein calculating the average number of connections comprises calculating the average number using a moving window.

17. The method according to claim 1, wherein the connections are additionally associated with a transport layer protocol, and wherein analyzing the distribution comprises:

analyzing a pattern of the connections at the transport layer so as to detect a potential attack; and

analyzing the distribution of the states so as to make a determination that the potential attack is the attack.

18. The method according to claim 1, wherein analyzing the distribution comprises:

measuring a time-related property of traffic entering the network;

transforming the time-related property of the traffic into a frequency domain; and

analyzing the distribution of the states and the property in the frequency domain so as to detect the attack.

19. The method according to claim 18, wherein measuring the time-related property comprises measuring arrival times of packets, and wherein transforming the time-related property comprises determining a spectrum of packet arrival frequency.

20. A method for protecting a server on a communication network, the method comprising:

monitoring messages transmitted over the communication network for forwarding by the server so as to determine a respective number of recipients specified by each of the messages;

identifying a source address of one of the messages for which the number of recipients is equal at least to a threshold;

tracking a cumulative number of the recipients specified in a plurality of the messages from the identified source address; and

determining the source address to be malicious responsively to the cumulative number.

21. The method according to claim 20, wherein tracking the cumulative number comprises tracking a cumulative excess over the threshold of the number of recipients specified in the plurality of the messages.

22. The method according to claim 20, wherein the messages include e-mail messages.

23. The method according to claim 22, wherein the e-mail messages are transmitted according to the SMTP protocol.

24. The method according to claim 20, wherein identifying the source address comprises adaptively setting the threshold such that a predetermined percentage of the messages transmitted over the network have a number of recipients greater than or equal to the threshold.

25. The method according to claim 20, wherein the threshold includes a first threshold, and wherein determining the source address to be malicious comprises determining that the cumulative number is greater than a second threshold.

26. The method according to claim 20, and comprising terminating connections to the source address determined to be malicious.

27. A method for protecting a server on a communication network, the method comprising:

monitoring messages transmitted over the communication network for forwarding by the server so as to determine respective numbers of recipients specified by the messages;

adaptively setting an attack threshold responsively to the numbers of the recipients specified by a plurality of the messages; and

identifying as malicious one of the messages that specifies a number of recipients that is equal at least to the attack threshold.

28. The method according to claim 27, wherein the network connection is a transport-layer connection.

29. The method according to claim 28, wherein the transport-layer connection is a TCP connection.

30. The method according to claim 27, wherein the messages comprise e-mail messages.

31. The method according to claim 30, wherein the e-mail messages are transmitted according to the SMTP protocol.

32. The method according to claim 27, wherein adaptively setting the attack threshold comprises disregarding messages having a number of recipients greater than a maximum value.

33. The method according to claim 27, wherein adaptively setting the attack threshold comprises:

adaptively setting a trigger threshold such that a predetermined percentage of the plurality of messages specify a number of recipients greater than or equal to the trigger threshold; and

setting the attack threshold responsively to the trigger threshold.

34. The method according to claim 33, wherein setting the attack threshold comprises:

for each of the messages specifying a number of recipients greater than or equal to the trigger threshold, calculating a respective difference between the number of recipients specified by the message and the trigger threshold; and

determining the attack threshold responsively to the trigger threshold and an average of the differences.

35. The method according to claim 34, wherein determining the attack threshold comprises setting the attack threshold equal to a sum of (a) the trigger threshold and (b) a product of (i) the average of the differences and (ii) a constant.

36. The method according to claim 33, wherein the predetermined percentage is less than or equal to 10%.

37. The method according to claim 36, wherein the predetermined percentage is less than or equal to 5%.

38. The method according to claim 27, and comprising discarding the message determined to be malicious.

39. Apparatus for detecting an attack in a computer network, the apparatus comprising a network security processor, which is adapted to monitor communication traffic transmitted over connections on the network that are associated with a stateful application protocol so as to detect respective states of the connections, and analyze a distribution of the states so as to detect the attack.

40. The apparatus according to claim 39, wherein the network security processor is adapted to detect respective handshake states of the connections, and to analyze the distribution of the handshake states so as to detect the attack.

41. The apparatus according to claim 39, wherein the stateful application protocol includes Simple Mail Transfer Protocol (SMTP).

42. The apparatus according to claim 39, wherein the stateful application protocol is selected from a list consisting of:

Post Office Protocol (POP), Internet Message Access Protocol (IMAP), and File Transfer Protocol (FTP).

43. The apparatus according to claim 39, wherein the network security processor is adapted to filter traffic entering the network in order to block traffic participating in the attack.

44. The apparatus according to claim 39, wherein the network security processor is adapted to interpret a number of connections from a single source address that is greater than a threshold value as indicative of the attack.

45. The apparatus according to claim 39, wherein the network security processor is adapted to interpret a deviation of the distribution from a baseline distribution of the states as indicative of the attack.

46. The apparatus according to claim 45, wherein the network security processor is adapted to interpret a disproportionately large number of connections in one of the states compared to the baseline distribution as indicative of the attack.

47. The apparatus according to claim 45, wherein the baseline distribution is a generally uniform distribution of the states, and wherein the network security processor is adapted to interpret the deviation of the distribution from the generally uniform baseline distribution as indicative of the attack.

48. The apparatus according to claim 47, wherein the network security processor is adapted to interpret a disproportionately large number of connections in one of the states as indicative of the attack.

49. The apparatus according to claim 48, wherein the network security processor is adapted to interpret as indicative of the attack a number of connections in one of the states that is beyond a certain number of standard deviations from an average number of connections in the other states.

50. The apparatus according to claim 39, wherein the network security processor is adapted to analyze the distribution by calculating an average number of connections for each of the states.

51. The apparatus according to claim 50, wherein the network security processor is adapted to calculate the average number of connections for each of the states on an aggregate basis for all source addresses of the traffic.

52. The apparatus according to claim 50, wherein the network security processor is adapted to repeatedly calculate the average number for recent traffic.

53. The apparatus according to claim 52, wherein the network security processor is adapted to calculate the average number of connections by applying Infinite Impulse Response (IIR) filtering.

54. The apparatus according to claim 52, wherein the network security processor is adapted to calculate the average number of connections using a moving window.

55. The apparatus according to claim 39, wherein the connections are additionally associated with a transport layer protocol, and wherein the network security processor is adapted to analyze the distribution by analyzing a pattern of the connections at the transport layer so as to detect a potential attack, and analyzing the distribution of the states so as to make a determination that the potential attack is the attack.

56. The apparatus according to claim 39, wherein the network security processor is adapted to analyze the distribution by:

measuring a time-related property of traffic entering the network,

transforming the time-related property of the traffic into a frequency domain, and

analyzing the distribution of the states and the property in the frequency domain so as to detect the attack.

57. The apparatus according to claim 56, wherein the network security processor is adapted to measure the time-related property by measuring arrival times of packets, and to transform the time-related property by determining a spectrum of packet arrival frequency.

58. Apparatus for protecting a server on a communication network, the apparatus comprising a network security processor, which is adapted to:

monitor messages transmitted over the communication network for forwarding by the server so as to determine a respective number of recipients specified by each of the messages,

identify a source address of one of the messages for which the number of recipients is equal at least to a threshold,

track a cumulative number of the recipients specified in a plurality of the messages from the identified source address, and

determine the source address to be malicious responsively to the cumulative number.

59. Apparatus for protecting a server on a communication network, the apparatus comprising a network security processor, which is adapted to:

monitor messages transmitted over the communication network for forwarding by the server so as to determine respective numbers of recipients specified by the messages,

adaptively set an attack threshold responsively to the numbers of the recipients specified by a plurality of the messages, and

identify as malicious one of the messages that specifies a number of recipients that is equal at least to the attack threshold.

60. The apparatus according to claim 59, wherein the network connection is a transport-layer connection.

61. The apparatus according to claim 60, wherein the transport-layer connection is a TCP connection.

62. The apparatus according to claim 59, wherein the messages comprise e-mail messages.

63. The apparatus according to claim 62, wherein the e-mail messages are transmitted according to the SMTP protocol.

64. The apparatus according to claim 59, wherein the network security processor is adapted to disregard messages having a number of recipients greater than a maximum value, when adaptively setting the attack threshold.

65. The apparatus according to claim 59, wherein the network security processor is adapted to adaptively set the attack threshold by:

adaptively setting a trigger threshold such that a predetermined percentage of the plurality of messages specify a number of recipients greater than or equal to the trigger threshold, and

setting the attack threshold responsively to the trigger threshold.

66. The apparatus according to claim 65, wherein the network security processor is adapted to set the attack threshold by:

for each of the messages specifying a number of recipients greater than or equal to the trigger threshold, calculating a respective difference between the number of recipients specified by the message and the trigger threshold, and

determining the attack threshold responsively to the trigger threshold and an average of the differences.

67. The apparatus according to claim 66, wherein the network security processor is adapted to determine the attack threshold by setting the attack threshold equal to a sum of (a) the trigger threshold and (b) a product of (i) the average of the differences and (ii) a constant.

68. The apparatus according to claim 65, wherein the predetermined percentage is less than or equal to 10%.

69. The apparatus according to claim 68, wherein the predetermined percentage is less than or equal to 5%.

70. The apparatus according to claim 59, wherein the network security processor is adapted to discard the message determined to be malicious.

71. A computer software product for detecting an attack in a computer network, the product comprising a computer-readable medium in which program instructions are stored, which instructions, when read by a computer, cause the computer to monitor communication traffic transmitted over connections on the network that are associated with a stateful application protocol so as to detect respective states of the connections, and analyze a distribution of the states so as to detect the attack.

72. The product according to claim 71, wherein the instructions cause the computer to detect respective handshake states of the connections, and to analyze the distribution of the handshake states so as to detect the attack.

73. The product according to claim 71, wherein the stateful application protocol includes Simple Mail Transfer Protocol (SMTP).

74. The product according to claim 71, wherein the stateful application protocol is selected from a list consisting of:

Post Office Protocol (POP), Internet Message Access Protocol (IMAP), and File Transfer Protocol (FTP).

75. The product according to claim 71, wherein the instructions cause the computer to filter traffic entering the network in order to block traffic participating in the attack.

76. The product according to claim 71, wherein the instructions cause the computer to interpret a number of connections from a single source address that is greater than a threshold value as indicative of the attack.

77. The product according to claim 71, wherein the instructions cause the computer to interpret a deviation of the distribution from a baseline distribution of the states as indicative of the attack.

78. The product according to claim 77, wherein the instructions cause the computer to interpret a disproportionately large number of connections in one of the states compared to the baseline distribution as indicative of the attack.

79. The product according to claim 77, wherein the baseline distribution is a generally uniform distribution of the states, and wherein the instructions cause the computer to interpret the deviation of the distribution from the generally uniform baseline distribution as indicative of the attack.

80. The product according to claim 79, wherein the instructions cause the computer to interpret a disproportionately large number of connections in one of the states as indicative of the attack.

81. The product according to claim 80, wherein the instructions cause the computer to interpret as indicative of the attack a number of connections in one of the states that is beyond a certain number of standard deviations from an average number of connections in the other states.

82. The product according to claim 71, wherein the instructions cause the computer to analyze the distribution by calculating an average number of connections for each of the states.

83. The product according to claim 82, wherein the instructions cause the computer to calculate the average number of connections for each of the states on an aggregate basis for all source addresses of the traffic.

84. The product according to claim 82, wherein the instructions cause the computer to repeatedly calculate the average number for recent traffic.

85. The product according to claim 84, wherein the instructions cause the computer to calculate the average number of connections by applying Infinite Impulse Response (IIR) filtering.

86. The product according to claim 84, wherein the instructions cause the computer to calculate the average number of connections using a moving window.

87. The product according to claim 71, wherein the connections are additionally associated with a transport layer protocol, and wherein the instructions cause the computer to analyze the distribution by analyzing a pattern of the connections at the transport layer so as to detect a potential attack, and analyzing the distribution of the states so as to make a determination that the potential attack is the attack.

88. The product according to claim 71, wherein the instructions cause the computer to analyze the distribution by:

measuring a time-related property of traffic entering the network,

transforming the time-related property of the traffic into a frequency domain, and

analyzing the distribution of the states and the property in the frequency domain so as to detect the attack.

89. The product according to claim 88, wherein the instructions cause the computer to measure the time-related property by measuring arrival times of packets, and to transform the time-related property by determining a spectrum of packet arrival frequency.

90. A computer software product for protecting a server on a communication network, the product comprising a computer-readable medium in which program instructions are stored, which instructions, when read by a computer, cause the computer to monitor messages transmitted over the communication network for forwarding by the server so as to determine a respective number of recipients specified by each of the messages, identify a source address of one of the messages for which the number of recipients is equal at least to a threshold, track a cumulative number of the recipients specified in a plurality of the messages from the identified source address, and determine the source address to be malicious responsively to the cumulative number.

91. A computer software product for protecting a server on a communication network, the product comprising a computer-readable medium in which program instructions are stored, which instructions, when read by a computer, cause the computer to monitor messages transmitted over the communication network for forwarding by the server so as to determine respective numbers of recipients specified the messages, adaptively set an attack threshold responsively to the numbers of the recipients specified by a plurality of the messages, and identify as malicious one of the messages that specifies a number of recipients that is equal at least to the attack threshold.

92. The product according to claim 91, wherein the network connection is a transport-layer connection.

93. The product according to claim 92, wherein the transport-layer connection is a TCP connection.

94. The product according to claim 91, wherein the messages comprise e-mail messages.

95. The product according to claim 94, wherein the e-mail messages are transmitted according to the SMTP protocol.

96. The product according to claim 91, wherein the instructions cause the computer to disregard messages having a number of recipients greater than a maximum value, when adaptively setting the attack threshold.

97. The product according to claim 91, wherein the instructions cause the computer to adaptively set the attack threshold by:

adaptively setting a trigger threshold such that a predetermined percentage of the plurality of messages specify a number of recipients greater than or equal to the trigger threshold, and

setting the attack threshold responsively to the trigger threshold.

98. The product according to claim 97, wherein the instructions cause the computer to set the attack threshold by:

for each of the messages specifying a number of recipients greater than or equal to the trigger threshold, calculating a respective difference between the number of recipients specified by the message and the trigger threshold, and

determining the attack threshold responsively to the trigger threshold and an average of the differences.

99. The product according to claim 98, wherein the instructions cause the computer to determine the attack threshold by setting the attack threshold equal to a sum of (a) the trigger threshold and (b) a product of (i) the average of the differences and (ii) a constant.

100. The product according to claim 97, wherein the predetermined percentage is less than or equal to 10%.

101. The product according to claim 100, wherein the predetermined percentage is less than or equal to 5%.

102. The product according to claim 91, wherein the instructions cause the computer to discard the message determined to be malicious.