Identifying applications for intrusion detection systems

• US 8,291,495 B1
• Filed: 08/08/2007
• Issued: 10/16/2012
• Est. Priority Date: 08/08/2007
• Status: Active Grant

• Current Assignee: Juniper Networks Incorporated
• Original Assignee: Juniper Networks Incorporated
• Inventors: Burns, Bryan, Yang, Siying, Sobrier, Julien
• Primary Examiner(s): Flynn, Nathan
• Assistant Examiner(s): Lakhia, Viral
• Application Number:
  US11/835,923
• Time in Patent Office: 1,896 Days
• Field of Search: 726 13- 25, 713153-154, 709/224, 455/410
• US Class Current: 726/23
• CPC Class Codes: H04L 63/0254 Stateful filtering
• CPC Class Codes: H04L 63/1441 Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/168 above the transport layer

Abstract

An intrusion detection system (“IDS”) device is described that includes a flow analysis module to receive a first packet flow from a client and to receive a second packet flow from a server. The IDS includes a forwarding component to send the first packet flow to the server and the second packet flow to the client and a stateful inspection engine to apply one or more sets of patterns to the first packet flow to determine whether the first packet flow represents a network attack. The IDS also includes an application identification module to perform an initial identification of a type of software application and communication protocol associated with the first packet flow and to reevaluate the identification of the type of software application and protocol according to the second packet flow. The IDS may help eliminate false positive and false negative attack identifications.

Claims

1. A method comprising:

receiving, with a network device, a first packet flow within a network from a client to a server;

performing an initial identification of a type of software application and communication protocol associated with the first packet flow, to determine a first type of software application for the first packet flow;

applying a first set of patterns to the first packet flow to determine whether the first packet flow represents a network attack, wherein the first set of patterns are associated with the determined first type of software application and the communication protocol;

buffering the first packet flow to store at least a connection request by the client;

forwarding the first packet flow to the server;

receiving, in response to forwarding the first packet flow to the server, a second packet flow from the server;

associating the first packet flow and the second packet flow as a communication session between the client and the server;

using the first packet flow and the second packet flow, reevaluating the initial identification of the type of software application and protocol associated with the communication session, to determine a second type of software application for the first packet flow;

selecting a second set of patterns based on the determined second type of software application for the first packet flow and the reevaluated communication protocol; and

after receiving the second packet flow from the server, applying the second set of patterns to the buffered connection request of the first packet flow to re-determine whether the first packet flow represents a network attack.

2. The method of claim 1, wherein performing an initial identification comprises:

creating a hierarchical list of applications and associated protocols;

comparing properties of the first packet flow with members of the list; and

selecting a highest ordered member of the list with which the first packet flow shares properties as a currently selected type of application and protocol.

3. The method of claim 2, wherein reevaluating the identification comprises:

comparing properties of the second packet flow with members of the list; and

when the properties of the second packet flow do not match the currently selected type of application and protocol, selecting a next highest ordered member of the list as the currently selected type of application and protocol.

4. The method of claim 2, wherein the compared properties comprise at least one of a packet flow pattern, a destination port number, a source port number, and a protocol.

5. The method of claim 1, wherein the identified type of application and protocol uses a protocol which is at least one of HyperText Transfer Protocol (HTTP), File Transfer Protocol (FTP), Network News Transfer Protocol (NNTP), Telnet, Domain Name System (DNS), Gopher, Finger, Post Office Protocol (POP), POP3, Secure Socket Layer (SSL) protocol, Lightweight Directory Access Protocol (LDAP), Secure Shell (SSH), Server Message Block (SMB) or Simple Mail Transfer Protocol (SMTP).

6. The method of claim 1,

wherein buffering the first packet flow comprises:

storing a packet from the client as part of the first packet flow;

identifying a source internet protocol (“

IP”

) IP address, a destination IP address, a protocol, a source port, and a destination port of the packet from the first packet flow; and

storing subsequent packets from the client with corresponding source IP address, destination IP address, protocol, source port, and destination port as part of the first packet flow as the packet.

7. The method of claim 1, further comprising:

reassembling transmission control protocol (“

TCP”

) data from each of the first packet flow and the second packet flow of the communication session; and

inspecting the TCP data for both the first packet flow and the second packet flow to determine whether the first packet flow represents a network attack.

8. The method of claim 1, wherein performing an initial identification comprises attempting to identify the type of application and protocol after receiving a predefined minimum number of data packets of the first packet flow prior to forwarding the data packets to the server.

9. The method of claim 1, further comprising forwarding the second packet flow to the client upon applying the second set of patterns and confirming that the first packet flow does not represent a network attack.

10. The method of claim 1, wherein, upon failing to identify type of software application and protocol associated with the communication session with the first set of patterns or the second set of patterns, identifying the type of software application and protocol according to at least one of a source port and a destination port of the first packet flow using a default static port mapping that maps port numbers to corresponding applications.

11. The method of claim 1, wherein performing an initial identification comprises inspecting, with a function pointer, one or more fields of packets of the first packet flow and the second packet flow.

12. The method of claim 11, wherein the function pointer invokes a function on one or more fields of payloads carried by packets of at least one of the first packet flow or the second packet flow to determine whether a particular number, a particular text value, a text string matching a pattern, or a particular Boolean value is present within the packet flow at a particular position.

13. The method of claim 1, wherein performing an initial identification comprises:

accessing configuration data to determine a minimum data size; and

performing the initial identification only after receiving a quantity of data from the first packet flow at or exceeding the minimum data size.

14. An intrusion detection system (“

IDS”

) device comprising;

a flow analysis module to receive a first packet flow from a client and a second packet flow from a server in response to the first packet flow;

a data buffer to store packets from each of the first packet flow and the second packet flow;

a forwarding component to send the first packet flow to the server and the second packet flow to the client;

a stateful inspection engine to apply one or more sets of patterns to the first packet flow to determine whether the first packet flow represents a network attack; and

an application identification module to perform an initial identification of a type of software application and communication protocol associated with the first packet flow to determine a first type of software application for the first packet flow, and to reevaluate the identification of the type of software application and protocol according to the second packet flow to determine a second type of software application for the first packet flow,

wherein the stateful inspection engine is configured to apply a first set of patterns associated with the determined first type of software application to the first packet flow following the determination of the first type of software application, and to apply a second set of patterns associated with the determined second type of software application to the first packet flow stored in the data buffer following the reevaluation of the identification of the second type of software application.

15. The IDS device of claim 14, further comprising a flow table to associate the first packet flow and the second packet flow as a communication session between the client and the server.

16. The IDS device of claim 15, wherein the flow table utilizes source and destination internet protocol (“

IP”

) addresses, protocol, and source and destination port numbers of packets from each of the first packet flow and second packet flow to associate the first packet flow and the second packet flow as the communication session.

17. The IDS device of claim 14,

wherein the application identification module is configured to perform the initial identification of the type of software application and communication protocol associated with the first packet flow before the forwarding component forwards the first packet flow, and

wherein the application identification module is configured to extract the first packet flow from the data buffer and to perform a second identification of the type of software application and communication protocol associated with the first packet flow after the flow analysis module receives the second packet flow.

18. The IDS device of claim 14, wherein the application identification module comprises a hierarchically ordered list of applications and associated protocols to permit the application identification module to select a highest ordered member of the list with which the first packet flow shares properties as a currently selected type of application and protocol and, when properties of the second packet flow do not match the currently selected type of application and protocol, to select a next highest ordered member of the list as the currently selected type of application and protocol.

19. The IDS device of claim 18, wherein the hierarchically ordered list of protocols comprises at least one of HyperText Transfer Protocol (HTTP), File Transfer Protocol (FTP), Network News Transfer Protocol (NNTP), Telnet, Domain Name System (DNS), Gopher, Finger, Post Office Protocol (POP), POPS, Secure Socket Layer (SSL) protocol, Lightweight Directory Access Protocol (LDAP), Secure Shell (SSH), Server Message Block (SMB) or Simple Mail Transfer Protocol (SMTP).

20. The IDS device of claim 14, further comprising a reassembly module to reassemble TCP data from each of the first packet flow and the second packet flow, wherein the application identification module is configured to identify the type of application and protocol based on the reassembled TCP data from both the first packet flow and the second packet flow, and

wherein the stateful inspection engine is configured to determine whether the first packet flow represents a network attack according to the identification from the application identification module based on the reassembled TCP data.

21. The IDS device of claim 14, wherein the application identification module comprises a static port mapping list to associate port numbers with types of applications and protocols, wherein the application identification module is configured to select a type of application and protocol for the first packet flow and second packet flow when the application identification module cannot identify the type of application and protocol by other means.

22. The IDS device of claim 14, wherein the application identification module comprises a function pointer to inspect one or more fields of packets of the first packet flow and the second packet flow and to produce an indicator of a network attack.

23. The IDS device of claim 14, wherein the forwarding component is configured to access configuration data to determine a minimum data size and to forward the first packet flow only after receiving a quantity of data from the first packet flow at or exceeding the minimum data size.

24. A non-transitory computer-readable medium comprising instructions for causing a programmable processor to:

receive a first packet flow within a network from a client to a server;

perform an initial identification of a type of software application and communication protocol associated with the first packet flow using a hierarchically ordered list of applications and protocols and a static port mapping, to determine a first type of software application for the first packet flow;

apply a first set of patterns to the first packet flow to determine whether the first packet flow represents a network attack, wherein the first set of patterns are associated with the determined first type of software application and the communication protocol;

store the first packet flow in a data buffer;

forward the first packet flow to the server;

receive, in response to forwarding the first packet flow to the server, a second packet flow from the server;

associate the first packet flow and the second packet flow as a communication session between the client and the server;

store the second packet flow in the data buffer;

reevaluate the identification of the type of software application and protocol associated with the communication session using the list of applications and protocols and the static port mapping with the first packet flow and the second packet flow, to determine a second type of software application for the first packet flow;

select a second set of patterns based on the determined second type of software application for the first packet flow and the reevaluated communication protocol;

apply the second set of patterns to the first packet flow to re-determine whether the first packet flow represents a network attack; and

forward the second packet flow to the client.

25. The non-transitory computer-readable medium of claim 24, further comprising instructions to:

inspect, with a function pointer, one or more fields of packets of the first packet flow and second packet flow; and

perform, with the function pointer, an operation on one or more fields of payloads carried by packets of the first packet flow or the second packet flow to produce an indicator of the network attack.

26. A method comprising:

receiving, with a first network device, a first packet flow within a network;

performing an initial identification of a type of software application and communication protocol associated with the first packet flow, to determine a first type of software application for the first packet flow;

applying a first set of patterns to the first packet flow to determine whether the first packet flow represents a network attack, wherein the first set of patterns are associated with the determined first type of software application and the communication protocol;

buffering the first packet flow to store at least a connection request by the client;

forwarding the first packet flow to a second network device;

receiving, in response to forwarding the first packet flow to the server, a second packet flow from the second network device;

associating the first packet flow and the second packet flow as a communication session between the first network device and the second network device;

using the first packet flow and the second packet flow, reevaluating the initial identification of the type of software application and protocol associated with the communication session, to determine a second type of software application for the first packet flow;

selecting a second set of patterns based on the determined second type of software application for the first packet flow and the reevaluated communication protocol; and

applying the second set of patterns to each of the first packet flow and the second packet flow to re-determine whether the first packet flow or the second packet flow represent a network attack.

27. The method of claim 26, wherein the first network device is a client device and the second network device is a server device.

28. The method of claim 26, wherein the first network device is a server device and the second network device is a client device.

29. The method of claim 26, wherein the first network device is a peer device of the second network device.