Preventing network denial of service attacks by early discard of out-of-order segments

• US 8,074,275 B2
• Filed: 02/01/2006
• Issued: 12/06/2011
• Est. Priority Date: 02/01/2006
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Ramaiah, Anantha, Somasundaram, Mahadev, Sivakumar, Senthil
• Primary Examiner(s): Arani, Taghi
• Assistant Examiner(s): CHANG, KENNETH W
• Application Number:
  US11/345,999
• Publication Number:
  US 20070180533A1
• Time in Patent Office: 2,134 Days
• Field of Search: 726/22
• US Class Current: 726/22
• CPC Class Codes: H04L 45/54 Organization of routing tables
• CPC Class Codes: H04L 45/58 Association of routers
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/43 Assembling or disassembling of packets, e.g. segmentation and reassembly [SAR]
• CPC Class Codes: H04L 63/1458 Denial of Service
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/163 In-band adaptation of TCP data exchange; In-band control procedures

Abstract

A method of preventing network denial of service attacks by early discard of out-of-order segments comprises creating a reassembly queue for a connection between a first network node and a second network node, wherein the connection has been established based on a transport-layer network protocol, the reassembly queue having a size based on a buffer size of an input interface with which the connection is associated. As out-of-order data segments arrive on the connection, and before other processing of the segments, whether the reassembly queue is full is determined, and the out-of-order segments are discarded if the reassembly queue is full. The size of the reassembly queue is automatically changed in response to one or more changes in any of network conditions and device resources.

Claims

1. A computer-implemented method, comprising:

establishing a connection between a first network node and a second network node using a transport-layer network protocol;

creating a dynamically-sized reassembly queue for the connection between the first network node and the second network node, the dynamically-sized reassembly queue having a size based on a buffer size of an input interface with which the connection is associated;

receiving a segment on the connection;

determining whether the received segment is an out-of-order segment;

in response to determining that the received segment is not an out-of-order segment, then performing normal processing on the received segment;

in response to determining that the received segment is an out-of-order segment, then prior to performing processing of the received segment on the connection other than error check processing, determining whether the dynamically-sized reassembly queue is full;

in response to determining that the dynamically-sized reassembly queue is full, then determining, based upon one or more enlargement factors, whether the dynamically-sized reassembly queue should be enlarged, wherein the one or more enlargement factors include one or more of an amount of system load, an amount of available memory, a number of connections on the interface, and information from one or more other attack detection applications;

in response to determining both that the dynamically-sized reassembly queue is full and that the dynamically-sized reassembly queue should be enlarged based upon the one or more enlargement factors, then dynamically enlarging the dynamically-sized reassembly queue to create an enlarged dynamically-sized reassembly queue and queuing the received segment to the enlarged dynamically-sized reassembly queue, and

in response to determining that the received segment is an out-of-order segment, that the dynamically-sized reassembly queue is full and that the dynamically-sized reassembly queue should not be enlarged based upon the one or more enlargement factors, then prior to performing processing of the received segment on the connection other than error check processing, discarding the received segment.

2. The computer-implemented method as recited in claim 1, wherein the transport-layer network protocol is transmission control protocol (TCP) and wherein the dynamically-sized reassembly queue is a dynamically-sized TCP segment reassembly queue.

3. The computer-implemented method as recited in claim 1, wherein:

the transport-layer network protocol is transmission control protocol (TCP), the dynamically-sized reassembly queue is a dynamically-sized TCP segment reassembly queue, and determining whether the dynamically-sized reassembly queue is full is performed prior to performing processing of the received segment on the connection other than checksum validation.

4. A non-transitory computer-readable medium storing instructions for preventing network denial of service attacks by early discard of out-of-order segments, wherein processing of the instructions by one or more processors, causes:

establishing a connection between a first network node and a second network node using a transport-layer network protocol;

creating a dynamically-sized reassembly queue for the connection between the first network node and the second network node, the dynamically-sized reassembly queue having a size based on a buffer size of an input interface with which the connection is associated;

receiving a segment on the connection;

determining whether the received segment is an out-of-order segment;

in response to determining that the received segment is not an out-of-order segment, then performing normal processing on the received segment;

in response to determining that the received segment is an out-of-order segment, then prior to performing processing of the received segment on the connection other than error check processing, determining whether the dynamically-sized reassembly queue is full;

in response to determining that the dynamically-sized reassembly queue is full, then determining, based upon one or more enlargement factors, whether the dynamically-sized reassembly queue should be enlarged, wherein the one or more enlargement factors include one or more of an amount of system load, an amount of available memory, a number of connections on the interface, and information from one or more other attack detection applications;

in response to determining both that the dynamically-sized reassembly queue is full and that the dynamically-sized reassembly queue should be enlarged based upon the one or more enlargement factors, then dynamically enlarging the dynamically-sized reassembly queue to create an enlarged dynamically-sized reassembly queue and queuing the received segment to the enlarged dynamically-sized reassembly queue, and

in response to determining that the received segment is an out-of-order segment, that the dynamically-sized reassembly queue is full and that the dynamically-sized reassembly queue should not be enlarged based upon the one or more enlargement factors, then prior to performing processing of the received segment on the connection other than error check processing, discarding the received segment.

15. The computer-readable medium as recited in claim 4, wherein:

the transport-layer network protocol is transmission control protocol (TCP),

the dynamically-sized reassembly queue is a dynamically-sized TCP segment reassembly queue, and

determining whether the dynamically-sized reassembly queue is full is performed prior to performing processing of the received segment on the connection other than checksum validation.

16. The computer-readable medium as recited in claim 4, wherein the transport-layer network protocol is transmission control protocol (TCP) and wherein the dynamically-sized reassembly queue is a dynamically-sized TCP segment reassembly queue.

5. An apparatus, comprising:

means for establishing a connection between a first network node and a second network node using a transport-layer network protocol;

means for creating a dynamically-sized reassembly queue for the connection between the first network node and the second network node, the dynamically-sized reassembly queue having a size based on a buffer size of an input interface with which the connection is associated;

means for receiving a segment on the connection;

means for determining whether the received segment is an out-of-order segment;

means for in response to determining that the received segment is not an out-of-order segment, then performing normal processing on the received segment;

means for in response to determining that the received segment is an out-of-order segment, then prior to performing processing of the received segment on the connection other than error check processing, determining whether the dynamically-sized reassembly queue is full;

means for in response to determining that the dynamically-sized reassembly queue is full, then determining, based upon one or more enlargement factors, whether the dynamically-sized reassembly queue should be enlarged, wherein the one or more enlargement factors include one or more of an amount of system load, an amount of available memory, a number of connections on the interface, and information from one or more other attack detection applications;

means for in response to determining both that the dynamically-sized reassembly queue is full and that the dynamically-sized reassembly queue should be enlarged based upon the one or more enlargement factors dynamically enlarging the dynamically-sized reassembly queue to create an enlarged dynamically-sized reassembly queue and queuing the received segment to the enlarged dynamically-sized reassembly queue, and

means for in response to determining that the received segment is an out-of-order segment, that the dynamically-sized reassembly queue is full and that the dynamically-sized reassembly queue should not be enlarged based upon the one or more enlargement factors, then prior to performing processing of the received segment on the connection other than error check processing, discarding the received segment.

6. The apparatus as recited in claim 5, wherein the transport-layer network protocol is transmission control protocol (TCP) and wherein the dynamically-sized reassembly queue is a dynamically-sized TCP segment reassembly queue.

7. The apparatus as recited in claim 5, wherein:

the transport-layer network protocol is transmission control protocol (TCP),

the dynamically-sized reassembly queue is a dynamically-sized TCP segment reassembly queue, and

determining whether the dynamically-sized reassembly queue is full is performed prior to performing processing of the received segment on the connection other than checksum validation.

13. The apparatus as recited in claim 5, wherein the means for enlarging the dynamically-sized reassembly queue comprises means for changing a scaling factor associated with the dynamically-sized reassembly queue.

8. An apparatus for preventing network denial of service attacks by early discard of out-of-order segments, comprising:

a network interface that is coupled to a data network for receiving one or more packet flows therefrom;

one or more processors; and

a memory storing instructions which, when processed by the one or more processors, cause;

establishing a connection between a first network node and a second network node using a transport-layer network protocol;

creating a dynamically-sized reassembly queue for the connection between the first network node and the second network node, the dynamically-sized reassembly queue having a size based on a buffer size of an input interface with which the connection is associated;

receiving a segment on the connection;

determining whether the received segment is an out-of-order segment;

in response to determining that the received segment is not an out-of-order segment, then performing normal processing on the received segment;

in response to determining that the received segment is an out-of-order segment, then prior to performing processing of the received segment on the connection other than error check processing, determining whether the dynamically-sized reassembly queue is full;

in response to determining that the dynamically-sized reassembly queue is full, then determining, based upon one or more enlargement factors, whether the dynamically-sized reassembly queue should be enlarged, wherein the one or more enlargement factors include one or more of an amount of system load, an amount of available memory, a number of connections on the interface, and information from one or more other attack detection applications;

in response to determining both that the dynamically-sized reassembly queue is full and that the dynamically-sized reassembly queue should be enlarged based upon the one or more enlargement factors, then dynamically enlarging the dynamically-sized reassembly queue to create an enlarged dynamically-sized reassembly queue and queuing the received segment to the dynamically-sized reassembly queue, and

in response to determining that the received segment is an out-of-order segment, that the dynamically-sized reassembly queue is full and that the dynamically-sized reassembly queue should not be enlarged based upon the one or more enlargement factors, then prior to performing processing of the received segment on the connection other than error check processing, discarding the received segment.

9. The apparatus as recited in claim 8, wherein the transport-layer network protocol is transmission control protocol (TCP) and wherein the dynamically-sized reassembly queue is a dynamically-sized TCP segment reassembly queue.

10. The apparatus as recited in claim 8, wherein:

the transport-layer network protocol is transmission control protocol (TCP),

the dynamically-sized reassembly queue is a dynamically-sized TCP segment reassembly queue, and

determining whether the dynamically-sized reassembly queue is full is performed prior to performing processing of the received segment on the connection other than checksum validation.

11. The apparatus as recited in claim 8, wherein the instructions include instructions which, when processed by the one or more processors, causes enlarging the dynamically-sized reassembly queue by changing a scaling factor associated with the dynamically-sized reassembly queue.

12. The apparatus as recited in claim 8, wherein the apparatus comprises one or more of a router, a switch, and a TCP proxy.

14. A TCP proxy apparatus that prevents network denial of service attacks by early discard of out-of-order segments, comprising:

a network interface that is coupled to a data network for receiving one or more packet flows therefrom;

one or more processors; and

a memory storing instructions which, when processed by the one or more processors, cause;

establishing a connection between a first network node and a second network node using a transport-layer network protocol;

creating a TCP reassembly queue for the connection between the first network node and the second network node, the TCP reassembly queue having a size based on a buffer size of an input interface with which the connection is associated;

receiving a segment on the connection;

determining whether the received segment is an out-of-order segment;

in response to determining that the received segment is not an out-of-order segment, then performing normal processing on the received segment;

in response to determining that the received segment is an out-of-order segment, then prior to performing processing of the received segment on the connection other than error check processing, determining whether the TCP reassembly queue is full;

in response to determining that the TCP reassembly queue is full, then determining, based upon one or more enlargement factors, whether the TCP reassembly queue should be enlarged, wherein the one or more enlargement factors include one or more of an amount of system load, an amount of available memory, a number of connections on the interface, and information from one or more other attack detection applications;

in response to determining both that the TCP reassembly queue is full and that the TCP reassembly queue should be enlarged based upon the one or more enlargement factors, then dynamically enlarging the TCP reassembly queue by changing a scaling factor associated with the TCP reassembly queue and queuing the received segment to the TCP reassembly queue; and

in response to determining that the received segment is an out-of-order segment, that the TCP reassembly queue is full and that the reassembly queue should not be enlarged based upon the one or more enlargement factors, then prior to performing processing of the received segment on the connection other than error check processing, discarding the received segment.