Network and application attack protection based on application layer message inspection

• US 7,725,934 B2
• Filed: 12/07/2004
• Issued: 05/25/2010
• Est. Priority Date: 12/07/2004
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Jin, Yi, Wiborg, Christopher R., Kumar, Sandeep, Potti, Sunil
• Primary Examiner(s): Brown; Christopher J
• Application Number:
  US11/007,152
• Publication Number:
  US 20060123479A1
• Time in Patent Office: 1,995 Days
• Field of Search: 726/22
• US Class Current: 726/22
• CPC Class Codes: H04L 2463/141 Denial of service attacks against endpoints in a network
• CPC Class Codes: H04L 63/0245 Filtering by information in the payload
• CPC Class Codes: H04L 63/123 received data contents, e.g. message integrity
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1458 Denial of Service

Abstract

A method is disclosed for protecting a network against a denial-of-service attack by inspecting application layer messages at a network element. According to one aspect, when a network element intercepts data packets that contain an application layer message, the network element constructs the message from the payload portions of the packets. The network element determines whether the message satisfies specified criteria. The criteria may indicate characteristics of messages that are suspected to be involved in a denial-of-service attack, for example. If the message satisfies the specified criteria, then the network element prevents the data packets that contain the message from being received by the application for which the message was intended. The network element may accomplish this by dropping the packets, for example. As a result, the application'"'"'s host does not waste processing resources on messages whose only purpose might be to deluge and overwhelm the application.

Claims

1. A method of preventing a network and application denial-of-service attack, the method comprising the computer-implemented steps of:

accepting one or more specified criteria at a network element after receiving and routing one or more previous data packets at the network element, wherein the one or more specified criteria comprise a set of denial-of-service attack detection criteria;

receiving, at the network element, one or more data packets that collectively contain an application layer message;

determining the application layer message, at the network element, from one or more payload portions of the one or more data packets;

determining whether the application layer message satisfies one or more specified criteria; and

if the application layer message satisfies the one or more specified criteria, then preventing the one or more data packets from being received by an application for which the application layer message was intended, wherein the application is at a different network element from the network element;

wherein determining whether the application layer message satisfies the one or more specified criteria comprises;

determining an application layer protocol according to which the application layer message was communicated;

selecting, from among a plurality of firewall mechanisms, a particular firewall mechanism that is mapped to the application layer protocol;

applying the particular firewall mechanism to the application layer message;

determining whether a particular class of messages is being used to mount an attack;

in response to detecting that a particular class of messages is being used to mount an attack, configuring another firewall mechanism, in another network element external to the network element, that allows or denies messages based on header attributes of layer 4 or below;

wherein the method is performed by one or more computing processors.

2. A method as recited in claim 1, wherein receiving the one or more data packets at the network element comprises intercepting the one or more data packets at the network element, and wherein a destination address of the one or more data packets identifies an application that is hosted on a device that is separate from the network element.

3. A method as recited in claim 1, wherein the network element is a network router or switch.

4. A method as recited in claim 1, wherein determining the application layer message from the one or more payload portions of the one or more data packets comprises assembling, at the network element, contents of two or more of the payload portions to determine the application layer message.

5. A method as recited in claim 1, wherein the application layer message is an Extensible Markup Language (XML) document or a non-XML document.

6. A method as recited in claim 1, further comprising the step of:

if the application layer message does not satisfy the one or more specified criteria, then sending the one or more data packets to the application.

7. A method as recited in claim 1, wherein determining the application layer message from the one or more payload portions of the one or more data packets comprises decrypting encrypted contents of the one or more payload portions at the network element.

8. A method as recited in claim 1, wherein determining whether the application layer message satisfies the one or more specified criteria comprises determining whether at least a part of the application layer message is larger than a specified size.

9. A method as recited in claim 1, wherein determining whether the application layer message satisfies the one or more specified criteria comprises determining whether the application layer message fails to conform syntactically and semantically to a specified schema as expected by an application.

10. A method as recited in claim 9, wherein the specified schema is stored at the network element, and wherein the specified schema was obtained from a trusted source.

11. A method as recited in claim 1, wherein determining whether the application layer message satisfies the one or more specified criteria comprises:

determining one or more particular criteria that are associated with an Internet Protocol (IP) address that is indicated in one or more IP headers of the one or more data packets; and

determining whether the application layer message satisfies the one or more particular criteria.

12. A method as recited in claim 11, further comprising the steps of:

receiving user specified content-matching constraints or criteria at the network element after receiving one or more data packets at the network element, wherein the specified content-matching constraints or criteria specify the IP address and the one or more particular criteria; and

in response to receiving the specified content-matching constraints or criteria, establishing, at the network element, an association between the IP address and the one or more particular criteria.

13. A method as recited in claim 1, wherein determining whether the application layer message satisfies the one or more specified criteria comprises:

determining a message format to which the application layer message conforms;

selecting, from among a plurality of firewall mechanisms, a particular firewall mechanism that is mapped to the message format; and

applying the particular firewall mechanism to the application layer message.

14. A method as recited in claim 1, wherein the application layer message conforms to an application layer protocol in the one or more payload portions of the one or more data packets.

15. A method as recited in claim 1, wherein determining whether the application layer message satisfies one or more specified criteria comprises determining whether the application layer message contains one or more specified keywords.

16. A method as recited in claim 1, wherein the application layer message comprises a multi-part MIME message, and further comprising handling each part of the multi-part MIME message separately and independently from each other part of the multi-part MIME message.

17. A method as recited in claim 16, further comprising:

determining a type of a first part of the multi-part MIME message;

determining a type of a second part of the multi-part MIME message;

inspecting the first part using a first inspection mechanism that is associated with the type of the first part; and

inspecting the second part using a second inspection mechanism that is associated with the type of the second part;

wherein the first inspection mechanism differs from the second inspection mechanism.

18. A method as recited in claim 16, further comprising:

decrypting the first part using a first key; and

decrypting the second part using a second key that differs from the first key.

19. A method as recited in claim 16, further comprising allowing the first part to be forwarded out of the network element, and preventing the second part from being forwarded out of the network element.

20. A method as recited in claim 16, wherein the specified criteria are more restrictive than criteria specified at another network element.

21. A method as recited in claim 16, further comprising:

configuring a firewall mechanism to prevent data packets that satisfy specified criteria from reaching a mechanism of the network element that configured the firewall mechanism.

22. A method as recited in claim 1, further comprising:

re-routing one or more data packets to a different network element if data packets are being received at greater than a specified threshold rate.

23. A method as recited in claim 1, further comprising:

performing, at the network element, an operation on a message that is a request message, a response message, an exception processing message, or a message that was not sent between a client application and a server application.

24. A method as recited in claim 1, wherein determining whether the application layer message satisfies the one or more specified criteria comprises determining whether the application layer message is smaller than a specified size.

25. A method as recited in claim 1, further comprising:

if the application layer message satisfies the one or more specified criteria, then modifying the format of the content of the application layer message and then sending the application layer message.

26. A method as recited in claim 1, further comprising:

if the application layer message satisfies the one or more specified criteria, then performing one or more actions from a set of actions comprising quarantining the application layer message, discarding the application layer message, repairing the application layer message, alerting an entity about the application layer message, and logging the application layer message or one or more parts thereof.

27. A method as recited in claim 1, further comprising:

determining whether the application layer message contains embedded processing instructions; and

in response to a determination that the application layer message contains embedded processing instructions, removing the embedded processing instructions from the application layer message prior to sending the application layer message.

28. A method as recited in claim 1, further comprising:

determining whether a Universal Resource Identifier (URI) contained in the application layer message is contained in a list of allowed URIs; and

in response to a determination that the URI is not contained in the list of allowed URIs, performing one or more specified actions.

29. A method as recited in claim 1, further comprising:

determining whether the application layer message has been tampered with; and

in response to a determination that the application layer message has been tampered with, performing one or more specified actions.

30. A method as recited in claim 1, further comprising the steps of:

receiving an IP address in addition to the one or more specified criteria; and

in response to receiving the IP address, establishing, at the network element, an association between the IP address and the one or more specified criteria.

31. A volatile or non-volatile computer-readable medium carrying one or more sequences of instructions for preventing a network or application denial-of-service attack, which instructions, when executed by one or more processors, cause the one or more processors to carry out the steps of:

accepting one or more specified criteria at a network element after receiving and routing one or more previous data packets at the network element, wherein the one or more specified criteria comprise a set of denial-of-service attack detection criteria;

receiving, at the network element, one or more data packets that collectively contain an application layer message;

determining the application layer message, at the network element, from one or more payload portions of the one or more data packets;

determining whether the application layer message satisfies one or more specified criteria; and

if the application layer message satisfies the one or more specified criteria, then preventing the one or more data packets from being received by an application for which the application layer message was intended, wherein the application is at a different network element from the network element;

determining an application layer protocol according to which the application layer message was communicated;

selecting, from among a plurality of firewall mechanisms, a particular firewall mechanism that is mapped to the application layer protocol;

applying the particular firewall mechanism to the application layer message;

determining whether a particular class of messages is being used to mount an attack;

in response to detecting that a particular class of messages is being used to mount an attack, configuring another firewall mechanism, in another network element external to the network element, that allows or denies messages based on header attributes of layer 4 or below.

32. An apparatus for preventing a denial-of-service attack at a network element, the apparatus comprising:

a processor;

means for accepting one or more specified criteria at a network element after receiving and routing one or more previous data packets at the network element, wherein the one or more specified criteria comprise a set of denial-of-service attack detection criteria;

means for receiving, at the network element, one or more data packets that collectively contain an application layer message;

means for determining the application layer message, at the network element, from one or more payload portions of the one or more data packets;

means for determining whether the application layer message satisfies one or more specified criteria; and

means for preventing the one or more data packets from being received by an application for which the application layer message was intended if the application layer message satisfies the one or more specified criteria, wherein the application is at a different network element from the network element;

wherein the means for determining whether the application layer message satisfies the one or more specified criteria comprises;

means for determining an application layer protocol according to which the application layer message was communicated;

means for selecting, from among a plurality of firewall mechanisms, a particular firewall mechanism that is mapped to the application layer protocol;

means for applying the particular firewall mechanism to the application layer message;

means for determining whether a particular class of messages is being used to mount an attack;

means for configuring another firewall mechanism, in another network element external to the network element, that allows or denies messages based on header attributes of layer 4 or below in response to detecting that a particular class of messages is being used to mount an attack.

33. An apparatus for preventing a denial-of-service attack at a network element, comprising:

a network interface that is coupled to a data network for receiving one or more packet flows therefrom;

a processor;

one or more stored sequences of instructions which, when executed by the processor, cause the processor to carry out the steps of;

accepting one or more specified criteria at a network element after receiving and routing one or more previous data packets at the network element, wherein the one or more specified criteria comprise a set of denial-of-service attack detection criteria;

receiving, at the network element, one or more data packets that collectively contain an application layer message;

determining the application layer message, at the network element, from one or more payload portions of the one or more data packets;

determining whether the application layer message satisfies one or more specified criteria; and

if the application layer message satisfies the one or more specified criteria, then preventing the one or more data packets from being received by an application for which the application layer message was intended, wherein the application is at a different network element from the network element;

determining an application layer protocol according to which the application layer message was communicated;

selecting, from among a plurality of firewall mechanisms, a particular firewall mechanism that is mapped to the application layer protocol;

applying the particular firewall mechanism to the application layer message;

determining whether a particular class of messages is being used to mount an attack;

in response to detecting that a particular class of messages is being used to mount an attack, configuring another firewall mechanism, in another network element external to the network element, that allows or denies messages based on header attributes of layer 4 or below.

34. A volatile or non-volatile computer-readable medium as recited in claim 31, wherein the instructions for receiving the one or more data packets at the network element comprise-instructions for intercepting the one or more data packets at the network element, and wherein a destination address of the one or more data packets identifies an application that is hosted on a device that is separate from the network element.

35. A volatile or non-volatile computer-readable medium as recited in claim 31, wherein the network element is a network router or switch.

36. A volatile or non-volatile computer-readable medium as recited in claim 31, wherein instructions for determining the application layer message from the one or more payload portions of the one or more data packets comprise instructions for assembling, at the network element, contents of two or more of the payload portions to determine the application layer message.

37. A volatile or non-volatile computer-readable medium as recited in claim 31, wherein the one or more sequences of instructions further comprise instructions for sending, in response to determining that the application layer message does not satisfy the one or more specified criteria, the one or more data packets to the application.

38. An apparatus as recited in claim 24, wherein the means for receiving the one or more data packets at the network element further comprises means for intercepting the one or more data packets at the network element, and wherein a destination address of the one or more data packets identifies an application that is hosted on a device that is separate from the network element.

39. An apparatus as recited in claim 24, wherein the network element is a network router or switch.

40. An apparatus as recited in claim 24, wherein the means for determining the application layer message from the one or more payload portions of the one or more data packets comprises means for assembling, at the network element, contents of two or more of the payload portions to determine the application layer message.

41. An apparatus as recited in claim 24, further comprising means for sending, in response to determining that the application layer message does not satisfy the one or more specified criteria, the one or more data packets to the application.

42. An apparatus as recited in claim 33, wherein the instructions for receiving the one or more data packets at the network element comprise instructions for intercepting the one or more data packets at the network element, and wherein a destination address of the one or more data packets identifies an application that is hosted on a device that is separate from the network element.

43. An apparatus as recited in claim 33, wherein the network element is a network router or switch.

44. An apparatus as recited in claim 33, wherein instructions for determining the application layer message from the one or more payload portions of the one or more data packets comprise instructions for assembling, at the network element, contents of two or more of the payload portions to determine the application layer message.

45. An apparatus as recited in claim 33, wherein the one or more sequences of instructions further comprise instructions for sending, in response to determining that the application layer message does not satisfy the one or more specified criteria, the one or more data packets to the application.

46. An apparatus as recited in claim 33, wherein the network element is a network router or switch.