Target-based SMB and DCE/RPC processing for an intrusion detection system or intrusion prevention system

• US 9,055,094 B2
• Filed: 05/31/2012
• Issued: 06/09/2015
• Est. Priority Date: 10/08/2008
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Wease, Kenneth Todd
• Primary Examiner(s): Rahman, Mohammad L
• Application Number:
  US13/484,628
• Publication Number:
  US 20120246728A1
• Time in Patent Office: 1,104 Days
• Field of Search: 726/23
• US Class Current: 1/1
• CPC Class Codes: H04L 63/0254 Stateful filtering
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1433 Vulnerability analysis
• CPC Class Codes: H04L 63/1441 Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/1466 Active attacks involving interception, injection, modification, spoofing of data unit addresses, e.g. hijacking, packet injection or TCP sequence number attacks

Abstract

A method performed in a processor of an intrusion detection/prevention system (IDS/IPS) checks for valid packets in an SMB named pipe in a communication network. In a processor configured as an IDS/IPS, a packet in a transmission is received and a kind of application of a target of the packet is determined. Also, the data in the packet is inspected by the IDS/IPS as part of the SMB named pipe on only one of a condition that: (a) the FID in an SMB command header of the packet is valid (i) for segments/fragments in the SMB named pipe and (ii) for the determined kind of application of the target of the packet, as indicated by a reassembly table, and (b) the determined kind of application of the target of the packet does not check the FID, as indicated by the reassembly table.

Claims

1. A method performed in a processor of an intrusion detection/prevention system (IDS/IPS), for checking for valid packets in a server message block (SMB) named pipe in a communication network, comprising:

receiving, in a processor configured as an IDS/IPS, a fragment/segment, and determining a kind of application of a target of the fragment/segment in response to receiving the fragment/segment;

separating, in the IDS/IPS, fragments/segments with a same multiplex ID (MID) as part of a same SMB transaction command from fragments/segments with a different MID, the MID being in the SMB frame header, all for fragments/segments in the same SMB named pipe;

accessing a reassembly table that indicates the kind of application to determine whether the kind of application separates SMB transaction commands based on MID; and

responsive to a determination that the kind of application of the target separates SMB transaction commands based on MID, processing, in the IDS/IPS, the same SMB transaction command with the same MID as being in a distinct SMB transaction command instead of with the fragments/segments with the different MID.

2. The method as in claim 1, wherein the reassembly table indicates whether each of the different kinds of applications check the MID in the SMB frame header.

3. The method as in claim 1, wherein the separating further comprises separating, in the IDS/IPS, fragments/segments with the same MID and a same caller process ID (PID) in the SMB frame header as part of the same SMB transaction command, from fragments/segments with the different MID or a different PID, all for fragments/segments in the same SMB named pipe, when the kind of application of the target also separates based on PID.

4. The method as in claim 3, wherein the reassembly table indicates whether each of the different kinds of applications check the MID and the PID in the SMB frame header.

5. The method as in claim 1, wherein the SMB transaction command comprises a main transaction and a secondary transaction.

6. The method as in claim 1, further comprising inspecting, in the IDS/IPS, the data included as part of the same SMB transaction command, to determine whether the data includes an attack spanning plural fragments/segments in the same SMB transaction command.

7. An apparatus comprising:

a network sensor to sense packets in a server message block (SMB) named pipe in a communication network; and

a processor to;

receive via the network sensor, a fragment/segment and determine a kind of application of a target of the fragment/segment in response to receiving the fragment/segment;

separate fragments/segments with a same multiplex ID (MID) as part of a same SMB transaction command from fragments/segments with a different MID, the MID being in the SMB frame header, all for fragments/segments in the same SMB named pipe;

access a reassembly table that indicates the kind of application to determine whether the kind of application separates SMB transaction commands based on MID; and

responsive to a determination that the kind of application of the target separates SMB transaction commands based on MID, process the same SMB transaction command with the same MID as being in a distinct SMB transaction command instead of with the fragments/segments with the different MID.

8. The apparatus of claim 7, wherein the reassembly table indicates whether each of the different kinds of applications check the MID in the SMB frame header.

9. The apparatus of claim 7, wherein the processor is configured to separate fragments/segments with the same MID and a same caller process ID (PID) in the SMB frame header as part of the same SMB transaction command, from fragments/segments with the different MID or a different PID, all for fragments/segments in the same SMB name pipe, when the application of the target also separates based on PID.

10. The apparatus of claim 9, wherein the reassembly table indicates whether each of the different kinds of applications check the MID and the PID in the SMB frame header.

11. The apparatus of claim 7, wherein the SMB transaction command comprises a main transaction and a secondary transaction.

12. The apparatus of claim 7, wherein the processor inspects data included as part of the same SMB transaction command, to determine whether the data includes an attack spanning plural fragments/segments in the same SMB transaction command.

13. One or more non-transitory computer readable media encoded with software comprising computer executable instructions and when the software is executed operable to cause a processor to check for valid packets in a server message block (SMB) named pipe in a communication network by:

receiving a fragment/segment, and determining a kind of application of a target of the fragment/segment in response to receiving the fragment/segment;

separating fragments/segments with a same multiplex ID (MID) as part of a same SMB transaction command from fragments/segments with a different MID, the MID being in the SMB frame header, all for fragments/segments in the same SMB named pipe;

accessing a reassembly table that indicates the kind of application to determine whether the kind of application separates SMB transaction commands based on MID; and

responsive to a determination that the kind of application of the target separates SMB transaction commands based on MID, processing the same SMB transaction command with the same MID as being in a distinct SMB transaction command instead of with the fragments/segments with the different MID.

14. The one or more non-transitory computer readable media of claim 13, wherein the reassembly table indicates whether each of the different kinds of applications check the MID in the SMB frame header.

15. The one or more non-transitory computer readable media of claim 13, wherein the computer executable instructions that cause the processor to separate further comprise computer executable instructions that cause the processor to separate fragments/segments with the same MID and a same caller process ID (PID) in the SMB frame header as part of the same SMB transaction command, from fragments/segments with the different MID or a different PID, all for fragments/segments in the same SMB named pipe, when the kind of application of the target also separates based on PID.

16. The one or more non-transitory computer readable media of claim 15, wherein the reassembly table indicates whether each of the different kinds of applications check the MID and the PID in the SMB frame header.

17. The one or more non-transitory computer readable media of claim 13, wherein the SMB transaction command comprises a main transaction and a secondary transaction.

18. The one or more non-transitory computer readable media of claim 13, further comprising computer executable instructions that cause the processor to inspect the data included as part of the same SMB transaction command, to determine whether the data includes an attack spanning plural fragments/segments in the same SMB transaction command.