System and method for preventing a spoofed denial of service attack in a networked computing environment

• US 6,772,334 B1
• Filed: 08/31/2000
• Issued: 08/03/2004
• Est. Priority Date: 08/31/2000
• Status: Expired due to Term

• Current Assignee: McAfee, LLC
• Original Assignee: Networks Associates, Inc. (McAfee, LLC)
• Inventors: Glawitsch, Gregor A.
• Primary Examiner(s): Morse, Gregory
• Assistant Examiner(s): ADAMS, JONATHAN R
• Application Number:
  US09/655,459
• Time in Patent Office: 1,433 Days
• Field of Search: 713/150, 713/153, 713/200, 713/201
• US Class Current: 713/153
• CPC Class Codes: H04L 63/126 the source of the received data
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1458 Denial of Service
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/161 Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
• CPC Class Codes: H04L 69/163 In-band adaptation of TCP data exchange; In-band control procedures

Abstract

A system and a method for preventing a spoofed denial of service attack in a networked computing environment is described. A hierarchical protocol stack is defined. The hierarchical protocol stack includes a plurality of communicatively interfaced protocol layers with at least one session-oriented protocol layer. A packet requesting a session with the session-oriented protocol layer is received from the networked computing environment. The request packet includes headers containing a source address of uncertain trustworthiness. The request packet is acknowledged by performing the following operations. First, a checksum is calculated from information included in the request packet headers. A request acknowledgement packet is generated. The request acknowledgement packet includes headers containing the checksum as a pseudo sequence number and the source address in the request packet headers as a destination address. Finally, the request acknowledgement packet is sent into the networked computing environment. An acknowledgement packet is received from the networked computing environment. The acknowledgement packet includes headers containing an acknowledgement number. The acknowledgement packet is validated by performing the following operations. First, a validation checksum is calculated from information included in the acknowledgement packet headers. Then, the validation checksum is compared to the acknowledgement number of the acknowledgement packet. No state is maintained by the authenticating system until the comparison has succeeded.

Claims

1. A system for preventing a spoofed denial of service attack in a networked computing environment, comprising:

a hierarchical protocol stack comprising a plurality of communicatively interfaced protocol layers with at least one session-oriented protocol layer and receiving a packet from the networked computing environment requesting a session with the session-oriented protocol layer, the request packet comprising headers containing a source address of uncertain trustworthiness, the hierarchical protocol stack receiving an acknowledgement packet from the networked computing environment comprising headers containing an acknowledgement number; and

an authentication module acknowledging the request packet and validating the acknowledgement packet, comprising;

a checksumming module calculating a checksum from information included in the request packet headers and calculating a validation checksum from information included in the acknowledgement packet headers;

a packet module generating request acknowledgement packet comprising headers containing the checksum as a pseudo sequence number and the source address in the request packet headers as a destination address and sending the request acknowledgement packet into the networked computing environment; and

a comparison module comparing the validation checksum to the acknowledgement number of the acknowledgement packet.

2. A system according to claim 1, further comprising:

a firewall performing the request packet acknowledgement and the acknowledgement packet validation, the firewall functionally interposed between a server and the networked computing environment; and

the authentication module performing a session handshaking sequence between the firewall and the server.

3. A system according to claim 2, further comprising:

the authentication module forwarding a synchronize packet to the server, receiving a synchronize-acknowledgement packet from the server and forwarding an acknowledgement packet to the server.

4. A system according to claim 2, further comprising:

a translation module translating sequence numbers and acknowledgement numbers on packets transiting through the firewall to and from the server in accordance with the acknowledgement number of the acknowledgement packet.

5. A system according to claim 1, wherein the calculated checksum is a cryptographic checksum.

6. A system according to claim 5, wherein the cryptographic checksum comprises at least one cryptographic processing procedure selected from the group comprising a secure hash algorithm-1 (SHA-1) and a message authorization code (MAC).

7. A system according to claim 1, wherein the information included in the headers of the request packet comprises a source address, a destination address, a source port number, a destination port number, and a sequence number.

8. A system according to claim 1, wherein the session-oriented protocol layer comprises the Transmission Control Protocol (TCP).

9. A method for preventing a spoofed denial of service attack in a networked computing environment, comprising:

defining a hierarchical protocol stack comprising a plurality of communicatively interfaced protocol layers with at least one session-oriented protocol layer;

receiving a packet from the networked computing environment requesting a session with the session-oriented protocol layer, the request packet comprising headers containing a source address of uncertain trustworthiness;

acknowledging the request packet, comprising;

calculating a checksum from information included in the request packet headers;

generating a request acknowledgement packet comprising headers containing the checksum as a pseudo sequence number and the source address in the request packet headers as a destination address; and

sending the request acknowledgement packet into the networked computing environment;

receiving an acknowledgement packet from the networked computing environment comprising headers containing an acknowledgement number; and

validating the acknowledgement packet, comprising;

calculating a validation checksum from information included in the acknowledgement packet headers; and

comparing the validation checksum to the acknowledgement number of the acknowledgement packet.

10. A method according to claim 9, further comprising:

performing the request packet acknowledgement and the acknowledgement packet validation on a firewall functionally interposed between a server and the networked computing environment; and

performing a session handshaking sequence between the firewall and the server.

11. A method according to claim 10, further comprising:

forwarding a synchronize packet to the server;

receiving a synchronize-acknowledgement packet from the server; and

forwarding an acknowledgement packet to the server.

12. A method according to claim 10, further comprising:

translating sequence numbers and acknowledgement numbers on packets transiting through the firewall to and from the server in accordance with the acknowledgement number of the acknowledgement packet.

13. A method according to claim 9, wherein the calculated checksum is a cryptographic checksum.

14. A method according to claim 13, wherein the cryptographic checksum comprises at least one cryptographic processing procedure selected from the group comprising a secure hash algorithm-1 (SHA-1) and a message authorization code (MAC).

15. A method according to claim 9, wherein the information included in the headers of the request packet comprises a source address, a destination address, a source port number, a destination port number, and a sequence number.

16. A method according to claim 9, wherein the session-oriented protocol layer comprises the Transmission Control Protocol (TCP).

17. A storage medium for preventing a spoofed denial of service attack in a networked computing environment, comprising:

defining a hierarchical protocol stack comprising a plurality of communicatively interfaced protocol layers with at least one session-oriented protocol layer;

receiving a packet from the networked computing environment requesting a session with the session-oriented protocol layer, the request packet comprising headers containing a source address of uncertain trustworthiness;

acknowledging the request packet, comprising;

calculating a checksum from information included in the request packet headers;

generating a request acknowledgement packet comprising headers containing the checksum as a pseudo sequence number and the source address in the request packet headers as a destination address; and

sending the request acknowledgement packet into the networked computing environment;

receiving an acknowledgement packet from the networked computing environment comprising headers containing an acknowledgement number; and

validating the acknowledgement packet, comprising;

calculating a validation checksum from information included in the acknowledgement packet headers; and

comparing the validation checksum to the acknowledgement number of the acknowledgement packet.

18. A storage medium according to claim 17, further comprising:

performing the request packet acknowledgement and the acknowledgement packet validation on a firewall functionally interposed between a server and the networked computing environment; and

performing a session handshaking sequence between the firewall and the server.

19. A storage medium according to claim 18, further comprising:

forwarding a synchronize packet to the server;

receiving a synchronize-acknowledgement packet from the server; and

forwarding an acknowledgement packet to the server.

20. A storage medium according to claim 18, further comprising:

translating sequence numbers and acknowledgement numbers on packets transiting through the firewall to and from the server in accordance with the acknowledgement number of the acknowledgement packet.