Methods and systems for intrusion detection

• US 7,305,708 B2
• Filed: 03/08/2004
• Issued: 12/04/2007
• Est. Priority Date: 04/14/2003
• Status: Expired due to Fees

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Sourcefire Incorporated (Cisco Systems, Inc.)
• Inventors: Roelker, Daniel J., Norton, Marc A.
• Primary Examiner(s): Moazzami; Nasser
• Assistant Examiner(s): Louie; Oscar A
• Application Number:
  US10/793,887
• Publication Number:
  US 20040205360A1
• Time in Patent Office: 1,366 Days
• Field of Search: 713/151, 726/23, 726/22
• US Class Current: 726/23
• CPC Class Codes: H04L 63/0227 Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/14 for detecting or protecting against malicious traffic

Abstract

Performance of an intrusion detection system is enhanced with the addition of rule optimization, set-based rule inspection, and protocol flow analysis. During rule optimization, rule sets are created and selected in such a way that for every incoming packet only a single rule set has to be searched. Set-based rule inspection divides rules into content and non-content type rules. Only search patterns of content type rules are initially compared to a packet. Rules containing matched search patterns are validated with a parameterized search against the packet. Matches are recorded as events. Non-content rules are searched against a packet using a parameterized search. These matches are also recorded as an event. At least one event is selected per packet for logging. Protocol flow analysis determines the direction of flow of network traffic. Based on the direction of flow and the protocol, portions of packets can be eliminated from rule inspection.

Claims

1. A method for optimizing rules in an intrusion detection system, comprising:

creating a plurality of different rule sets from a plurality of intrusion detection system rules, including combining only one or two unique rules with all generic rules for the same transport protocol to form unique rule sets, and combining all generic rules for the same transport protocol to form generic rule sets, wherein each intrusion detection system rule in the unique rule sets or generic rule sets is for a same transport protocol, wherein the only one or two unique rules specify parameters in a transport header or an IP header which are different from parameters of every other rule, wherein generic rules specify content but not parameters in a transport header or an IP header, wherein each rule set indicates a combination of parameters of the combined rules;

determining at least one rule set that matches each packet by comparing parameters but not the content of the packet to the indicated combination of parameters of the at least one rule set; and

inspecting the content and parameters of the packet with only the intrusion detection system rules in the at least one rule set that matches the packet.

2. The method of claim 1, further comprising creating the plurality of rule sets during initialization of the intrusion detection system.

3. The method of claim 1, further comprising updating the plurality of rule sets while the intrusion detection system is executing when a new rule is added to the intrusion detection system.

4. The method of claim 1, further comprising selecting only one rule set for each said packet.

5. The method of claim 1, further comprising selecting two rule sets for each said packet.

6. The method of claim 1, wherein the at least one rule set includes rules for

determining a protocol associated with the packet, wherein the protocol comprises HTTP;

determining a type of data flow associated with the protocol, wherein the type of data flow comprises one of client-to-server data flow and server-to-client data flow; and

processing the packet in accordance with the determined type of data flow.

7. The method of claim 1,

wherein the at least one rule set includes rules for

determining if a transport protocol of the packet comprises a TCP portion;

determining if a source port in a header portion of the TCP portion comprises a Web server port, if the transport protocol comprises the TCP portion;

determining if a destination port of the header portion comprises a Web server port, if the source port comprises a Web server port;

decoding a data portion of the TCP portion, if the destination port does not comprise a Web server port;

determining if the data portion comprises an HTTP response header;

processing the data portion only up to an HTTP maximum response bytes limit and ignoring the remainder of the data portion, if the data portion comprises the HTTP response header; and

ignoring the data portion, if the data portion does not comprise the HTTP response header.

8. The method of claim 7, wherein it is determined if the source port comprises port 80.

9. The method of claim 7, wherein it is determined if the destination port comprises port 80.

10. The method of claim 1, wherein the inspecting is with all of the intrusion detection system rules in the at least one rule set that matches the packet even after it is determined that one of the intrusion detection system rules matches the packet.

11. A method for resolving unique conflicts between at least two rule sets that match parameters of a packet during rule set selection in an intrusion detection system, comprising:

creating defined unique conflict rule sets, wherein each defined unique conflict rule set is created by combining at least two unique rule sets defined by a user as rule sets capable of having a unique conflict together with all generic rules for the same transport protocol, wherein each rule in the unique rule sets is for the same transport protocol, wherein the at least two unique rule sets specify parameters in a transport header or an IP header which are different from parameters of every other rule, wherein each unique conflict rule set indicates a combination of parameters of the combined rule sets;

comparing the parameters but not the content of the packet to the indicated combination of parameters of each defined unique conflict rule set of the defined unique conflict rule sets;

selecting for inspection a defined unique conflict rule set from the list of defined unique conflict rule sets if the parameters of the packet match the defined unique conflict rule set;

selecting for inspection each of the at least two rule sets that match parameters of the packet if the parameters of the packet do not match a defined unique conflict rule set of the defined unique conflict rule sets; and

inspecting the content and parameters of the packet with only the selected rule sets.

12. The method of claim 11, wherein the defined unique conflict rule sets are created after initialization of the rule sets.

13. The method of claim 11, wherein the parameters of the packet are compared to each defined unique conflict rule set of the defined unique conflict rule sets at run time.

14. The method of claim 11, wherein the inspecting comprises inspecting each of the at least two rule sets that match parameters of the packet if the parameters of the packet do not match a defined unique conflict rule set of the defined unique conflict rule sets.

15. The method of claim 11, wherein the inspecting comprises inspecting each of the at least two rule sets that match parameters of the packet if the parameters of the packet do not match a defined unique conflict rule set of the defined unique conflict rule sets, until a rule set generates an intrusion detection event.

16. The method of claim 11, wherein the inspecting is with all of the rules in the at least one rule set that matches the packet even after it is determined that one of the rules matches the packet.

17. A method for resolving unique conflicts between at least two rule sets that match parameters of a packet during rule set selection in an intrusion detection system, comprising:

creating defined unique conflict rule sets, wherein each defined unique conflict rule set is created by combining at least two unique rule sets defined by a user as rule sets capable of having a unique conflict together with all generic rules for the same transport protocol, wherein each rule in the unique rule sets is for the same transport protocol, wherein the at least two unique rule sets specify parameters in a transport header or an IP header which are different from parameters of every other rule, wherein each unique conflict rule set indicates a combination of parameters of the combined rule sets;

comparing the parameters but not the content of the packet to the indicated combination of parameters of each defined unique conflict rule set of the defined unique conflict rule sets;

selecting for inspection a defined unique conflict rule set from the defined unique conflict rule sets if the parameters of the packet match the defined unique conflict rule set;

selecting randomly for inspection one of the at least two rule sets that matches parameters of the packet if the parameters of the packet do not match a defined unique conflict rule set of the defined unique conflict rule sets; and

inspecting the content and parameters of the packet with only the selected rules.

18. The method of claim 17, wherein the inspecting is with all of the rules in the at least one rule set that matches the packet even after it is determined that one of the rules matches the packet.

19. A system for intrusion detection, the system comprising:

a packet acquisition system, wherein the packet acquisition system intercepts a packet transmitted across a computer network and wherein the packet acquisition system decodes the packet;

a preprocessor, wherein the preprocessor receives decoded packet information from the packet acquisition system and wherein the preprocessor preprocesses the packet;

a detection engine, wherein the detection engine receives preprocessed packet information from the preprocessor and wherein the detection engine inspects the packet for intrusions;

a logging system, wherein the logging system receives and stores detected intrusion information from the detection engine; and

a rule optimizer, wherein the rule optimizer creates a plurality of different rule sets from a plurality of intrusion detection rules, including combining only one or two unique rules with all generic rules for the same transport protocol to form unique rule sets, and combining all generic rules for the same transport protocol to form generic rule sets, wherein each intrusion detection system rule in the unique rule sets or generic rule sets is for the same transport protocol, wherein the only one or two unique rules specify parameters in a transport header or an IP header which are different from parameters of every other rule, wherein generic rules specify packet content but not parameters in a transport header or an IP header, wherein each rule set indicates a combination of parameters of the combined rules, and the rule optimizer provides the plurality of rule sets to the detection engine,

wherein, for each packet that the detection engine processes, the detection engine determines at least one rule set that matches each packet by comparing parameters but not the content of the packet to the indicated combination of parameters of the at least one rule set and inspects the content and parameters of the packet with only the intrusion detection system rules in the at least one rule set that matches each packet.

20. The system of claim 19, wherein the detection engine selects one rule set from the plurality of rule sets for inspection for each packet that the detection engine processes.

21. The system of claim 19, wherein the detection engine selects two rule sets from the plurality of rule sets for inspection for each packet that the detection engine processes.

22. The method of claim 19, wherein the inspecting is with all of the rules in the at least one rule set that matches the packet even after it is determined that one of the rules matches the packet.

23. A method for detecting rule matches during packet processing in an intrusion detection system, wherein a packet has content areas and wherein the packet header fields are non-content areas, comprising:

identifying each rule as one of a content rule and a non-content rule, wherein a non-content rule tests packet header fields and does not inspect content, and wherein a content rule tests for a pattern match in the content area and can include rule parameters to compare to a packet;

scanning a search pattern parameter of a rule identified as a content rule against a packet to identify a first match of the search pattern parameter and the content of the packet;

after the scanning, if there is a first match for the packet, comparing the rule parameters of each rule that matched with packet parameters of the packet to identify a second match between the rule parameters and the packet parameters;

if there is a second match, adding the second match to a list of intrusion events;

comparing non-content rule parameters of each rule identified as a non-content rule with the packet parameters to identify a third match;

if there is a third match, adding the third match of to the list of intrusion events; and

selecting at least one intrusion event from the list of intrusion events for the packet for logging.

24. The method of claim 23, wherein the search pattern parameter comprises at least one of a string defined by rule content keyword and a string defined by a rule uricontent keyword.

25. The method of claim 23, further comprising scanning, in parallel, search pattern parameters of at least two rules identified as content rules against the packet.

26. The method of claim 25, wherein scanning search pattern parameters of at least two rules identified as content rules against the packet is performed using a Wu-Manber search algorithm.

27. The method of claim 25, wherein scanning search pattern parameters of at least two rules identified as content rules against the packet is performed using an Aho-Corasick search algorithm.

28. The method of claim 25, wherein scanning search pattern parameters of at least two rules identified as content rules against the packet is performed using a naive set-based Boyer-Moore search algorithm.

29. The method of claim 23, further comprising halting the scanning of the search pattern parameter against the packet, if the search pattern parameter has previously matched the packet.

30. A system for intrusion detection in a packet processing network, wherein a packet has content areas and wherein the packet header fields are non-content areas, the system comprising:

a packet acquisition system, wherein the packet acquisition system intercepts a packet transmitted across a computer network and wherein the packet acquisition system decodes the packet;

a preprocessor, wherein the preprocessor receives decoded packet information from the packet acquisition system and wherein the preprocessor preprocesses the packet;

a detection engine, wherein the detection engine receives preprocessed packet information from the preprocessor, and wherein the detection engine;

identifies each rule as one of a content rule and a non-content rule, wherein a non-content rule tests packet header fields and does not inspect content, wherein a content rule tests for a pattern match in the content area and can include rule parameters to compare to a packet,

scans a search pattern parameter of a rule identified as a content rule against the packet to identify at least one first match of the search pattern parameter and the content of the packet,

after the scanning, if there is at least one first match for the packet, compares the rule parameters of each rule that matched to packet parameters of the packet to identify a second match between the rule parameters and the packet parameters,

if there is a second match, adds each second match to a list of intrusion events,

compares non-content rule parameters of each rule identified as a non-content rule with the packet parameters to identify a third match,

if there is a third match, adds each third match to the list of intrusion events, and

selects for the packet at least one intrusion event from the list of queued intrusion events for logging; and

a logging system, wherein the logging system receives and stores detected intrusion information from the detection engine.

31. A system for intrusion detection in a packet processing network, wherein a packet has content areas and wherein the packet header fields are non-content areas, the system comprising:

a packet acquisition system, wherein the packet acquisition system intercepts a packet transmitted across a computer network and wherein the packet acquisition system decodes the packet;

a preprocessor, wherein the preprocessor receives decoded packet information from the packet acquisition system and wherein the preprocessor preprocesses the packet;

a detection engine,

wherein the detection engine receives preprocessed packet information from the preprocessor,

wherein the detection engine determines a protocol associated with the packet, determines a type of data flow associated with the protocol, and processes the packet in accordance with the determined type of data flow,

wherein the detection engine;

identifies each rule as one of a content rule and a non-content rule,

wherein a non-content rule tests packet header fields and does not inspect content,

wherein a content rule tests for a pattern match in the content area and can include rule parameters to compare to a packet,

scans a search pattern parameter of a rule identified as a content rule against the packet to identify at least one first match of the search pattern parameter and the content of the packet,

after the scanning, if there is at least one first match for the packet, compares the rule parameters of each rule that matched to packet parameters of the packet to identify a second match between the rule parameters and the packet parameters,

if there is a second match, adds each second match to a list of intrusion events,

compares non-content rule parameters of each rule identified as a non-content rule with the packet parameters to identify a third match,

if there is a third match, adds each third match to the list of intrusion events, and

selects for the packet at least one intrusion event from the list of queued intrusion events for logging;

a logging system, wherein the logging system receives and stores detected intrusion information from the detection engine; and

a rule optimizer, wherein the rule optimizer creates a plurality of different rule sets from a plurality of intrusion detection rules, including combining only one or two unique rules with all generic rules for the same transport protocol to form unique rule sets, and combining all generic rules for the same transport protocol to form generic rule sets, wherein each rule in the unique rule sets or generic rule sets is for the same transport protocol, wherein the only one or two unique rules specify parameters in a transport header or an IP header which are different from parameters of every other rule, wherein generic rules specify packet content but not parameters in a transport header or an IP header, wherein each rule set indicates a combination of parameters of the combined rules, and provides the plurality of rules sets to the detection engine,

wherein the detection engine selects at least one rule set from the plurality of rule sets for inspection for each packet that the detection engine processes by comparing parameters but not the content of the packet to the indicated combination of parameters of the at least one rule set,

wherein the detection engine inspects the content and parameters of the packet with only the rules in the at least one rule set that matches the packet.