SYSTEMS, METHODS, ANE MEDIA FOR DETECTING NETWORK ANOMALIES

• US 20110167493A1
• Filed: 05/27/2009
• Published: 07/07/2011
• Est. Priority Date: 05/27/2008
• Status: Active Grant

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Song, Yingbo, Keromytis, Angelos D., Stolfo, Salvatore J.
• Granted Patent: US 8,844,033 B2
• Time in Patent Office: Days
• US Class Current: 726/23
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1466 Active attacks involving interception, injection, modification, spoofing of data unit addresses, e.g. hijacking, packet injection or TCP sequence number attacks
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]

Abstract

Systems, methods, and media for detecting network anomalies are provided. In some embodiments, a training dataset of communication protocol messages having argument strings is received. The content and structure associated with each of the argument strings is determined and a probabilistic model is trained using the determined content and structure of each of the argument strings. A communication protocol message having an argument string that is transmitted from a first processor to a second processor across a computer network is received. The received communication protocol message is compared to the probabilistic model and then it is determined whether the communication protocol message is anomalous.

Claims

1. A method for detecting network anomalies, the method comprising:

receiving a training dataset of communication protocol messages having argument strings;

determining a content and a structure associated with each of the argument strings;

training a probabilistic model using the determined content and structure of each of the argument strings;

receiving a communication protocol message having an argument string that is transmitted from a first processor to a second processor across a computer network;

comparing the received communication protocol message to the probabilistic model; and

determining whether the communication protocol message is anomalous.

2. The method of claim 1, wherein the anomalous communication protocol message is caused by a web layer code injection attack.

3. The method of claim 1, further comprising:

receiving a plurality of packets transmitted from the first processor to the second processor across the computer network;

assembling the plurality of packets to form the communication protocol message; and

extracting the argument string from the communication protocol message.

4. The method of claim 1, further comprising assembling the plurality of packets to form the communication protocol message prior to the plurality of packets being received by the second processor.

5. The method of claim 1, further comprising assembling the communication protocol message from the plurality of packets, wherein the assembled communication protocol message is substantially similar to a request received by an application running on the second processor.

6. The method of claim 1, wherein the communication protocol message is a Hypertext Transfer Protocol (HTTP) request message.

7. The method of claim 1, wherein determining the content and the structure associated with each of the argument strings further comprises determining one or more variable names, a corresponding argument value for each of the variable names, and a layout with respect to each of the variable names.

8. The method of claim 1, wherein the probabilistic model is a mixture of Markov chain models trained using a maximum likelihood estimator.

9. The method of claim 1, wherein the probabilistic model comprises one or more Markov models of transition probabilities.

10. The method of claim 1, further comprising issuing an alert in response to determining that the communication protocol message is anomalous.

11. The method of claim 1, further comprising calculating a normality score for the received communication protocol message using the trained probabilistic model.

12. The method of claim 1, further comprising receiving a gram size and a mixture size for the probabilistic model.

13. A system for detecting network anomalies, the system comprising:

a processor that;

receives a training dataset of communication protocol messages having argument strings;

determines a content and a structure associated with each of the argument strings;

trains a probabilistic model using the determined content and structure of each of the argument strings;

receives a communication protocol message having an argument string that is transmitted from a first processor to a second processor across a computer network;

compares the received communication protocol message to the probabilistic model; and

determines whether the communication protocol message is anomalous.

14. The system of claim 13, wherein the anomalous communication protocol message is caused by a web layer code injection attack.

15. The system of claim 13, wherein the processor is further configured to:

receive a plurality of packets transmitted from the first processor to the second processor across the computer network;

assemble the plurality of packets to form the communication protocol message; and

extract the argument string from the communication protocol message.

16. The system of claim 13, wherein the processor is further configured to assemble the plurality of packets to form the communication protocol message prior to the plurality of packets being received by the second processor.

17. The system of claim 13, wherein the processor is further configured to assemble the communication protocol message from the plurality of packets, wherein the assembled communication protocol message is substantially similar to a request received by an application running on the second processor.

18. The system of claim 13, wherein the communication protocol message is a Hypertext Transfer Protocol (HTTP) request message.

19. The system of claim 13, wherein the processor is further configured to determine one or more variable names, a corresponding argument value for each of the variable names, and a layout with respect to each of the variable names.

20. The system of claim 13, wherein the probabilistic model is a mixture of Markov chain models trained using a maximum likelihood estimator.

21. The system of claim 13, wherein the probabilistic model comprises one or more Markov models of transition probabilities.

22. The system of claim 13, wherein the processor is further configured to issue an alert in response to determining that the communication protocol message is anomalous.

23. The system of claim 13, wherein the processor is further configured to calculate a normality score for the received communication protocol message using the trained probabilistic model.

24. The system of claim 13, wherein the processor is further configured to receive a gram size and a mixture size for the probabilistic model.

25. A non-transitory computer-readable medium containing computer-executable instructions that, when executed by a processor, cause the processor to perform a method for detecting network anomalies, the method comprising:

receiving a training dataset of communication protocol messages having argument strings;

determining a content and a structure associated with each of the argument strings;

training a probabilistic model using the determined content and structure of each of the argument strings;

receiving a communication protocol message having an argument string that is transmitted from a first processor to a second processor across a computer network;

comparing the received communication protocol message to the probabilistic model; and

determining whether the communication protocol message is anomalous.

26. The non-transitory computer-readable medium of claim 25, wherein the anomalous communication protocol message is caused by a web layer code injection attack.

27. The non-transitory computer-readable medium of claim 25, wherein the method further comprises:

receiving a plurality of packets transmitted from the first processor to the second processor across the computer network;

assembling the plurality of packets to form the communication protocol message; and

extracting the argument string from the communication protocol message.

28. The non-transitory computer-readable medium of claim 25, wherein the method further comprises assembling the plurality of packets to form the communication protocol message prior to the plurality of packets being received by the second processor.

29. The non-transitory computer-readable medium of claim 25, wherein the method further comprises assembling the communication protocol message from the plurality of packets, wherein the assembled communication protocol message is substantially similar to a request received by an application running on the second processor.

30. The non-transitory computer-readable medium of claim 25, wherein the communication protocol message is a Hypertext Transfer Protocol (HTTP) request message.

31. The non-transitory computer-readable medium of claim 25, wherein the method further comprises determining one or more variable names, a corresponding argument value for each of the variable names, and a layout with respect to each of the variable names.

32. The non-transitory computer-readable medium of claim 25, wherein the probabilistic model is a mixture of Markov chain models trained using a maximum likelihood estimator.

33. The non-transitory computer-readable medium of claim 25, wherein the probabilistic model comprises one or more Markov models of transition probabilities.

34. The non-transitory computer-readable medium of claim 25, wherein the method further comprises issuing an alert in response to determining that the communication protocol message is anomalous.

35. The non-transitory computer-readable medium of claim 25, wherein the method further comprises calculating a normality score for the received communication protocol message using the trained probabilistic model.

36. The non-transitory computer-readable medium of claim 25, wherein the method further comprises receiving a gram size and a mixture size for the probabilistic model.