Integrated intrusion detection services

• US 7,076,803 B2
• Filed: 01/28/2002
• Issued: 07/11/2006
• Est. Priority Date: 01/28/2002
• Status: Active Grant

• Current Assignee: Trend Micro Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Bruton, III, David Aro, Jakubik, Patricia, LiVecchi, Patrick Michael, Overby, Jr., Linwood Hugh
• Primary Examiner(s): Song, Hosuk
• Assistant Examiner(s): Pich, Ponnoreay
• Application Number:
  US10/058,870
• Publication Number:
  US 20030145226A1
• Time in Patent Office: 1,625 Days
• Field of Search: None
• US Class Current: 726/23
• CPC Class Codes: H04L 63/0263 Rule management
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00

Abstract

Improvements in intrusion detection are disclosed by providing integrated intrusion detection services. Preferably, these services are integrated into a system or server that is the potential target of attack. Stack-based security processing is leveraged for access to cleartext data within the layers of the protocol stack. Layer-specific attacks may therefore be processed efficiently. Evaluation of incoming traffic for an intrusion is preferably performed only after an error condition of some type has been detected. This approach reduces the overhead of intrusion detection by reducing the number of packets to be inspected, and at the same time allows more efficient packet inspection through use of context-specific information that may be used to direct the inspection to particular candidate attacks. Generic attack class capability is also disclosed. Intrusion detection policy information may be used to direct the actions to be taken upon detecting an attack.

Claims

1. A method of improving intrusion detection in a computing network, comprising:

providing intrusion detection processing in an operating system kernel;

providing an application program which makes use of the operating system kernel during execution;

executing the application program; and

selectively evaluating at least one incoming communication of the executing application program for an intrusion using the provided intrusion detection processing in the operating system kernel, wherein the selectively evaluating is carried out in response to identification of errors by error-handling logic of a protocol stack executed in the operating system kernel; and

invoking a response action when the selectively evaluation detects the intrusion.

2. The method according to claim 1, wherein the response action is determined by consulting intrusion detection policy information.

3. The method according to claim 2, wherein the intrusion detection policy information is stored in a network-accessible repository.

4. The method according to claim 1, wherein the invoking a response action occurs in real time, responsive to the detected intrusion.

5. The method according to claim 1, wherein the selectively evaluating step further comprises comparing the incoming communication of the executing application program to one or more attack signatures.

6. The method according to claim 5, wherein at least one of the attack signatures is a class signature representing a class of attacks.

7. The method according to claim 1, wherein the selectively evaluating step further comprises comparing the incoming communication of the executing application program to one or more intrusion detection conditions.

8. The method according to claim 7, wherein the intrusion detection conditions are specified in intrusion detection rules.

9. The method according to claim 8, wherein each of the intrusion detection rules further comprises one or more actions to be taken when the conditions of the rule are matched.

10. The method according to claim 1, wherein the selectively evaluating further comprises comparing current conditions to predetermined conditions which signal a potential intrusion.

11. The method according to claim 1, wherein the selectively evaluating is provided as layer-specific intrusion detection logic within a protocol stack running in the operating system kernel.

12. The method according to claim 1, wherein the selectively evaluating operates in an endpoint of a connection in the computing network.

13. The method according to claim 12, wherein the incoming communication of the executing application program is encrypted while traversing the connection.

14. A system for improving intrusion detection in a computing network, comprising:

an operating system kernel;

an application program which makes use of the operating system kernel during execution;

means for executing the application program;

an embedded processor for executing a protocol stack within the operating system kernel, wherein the protocol stack is augmented to perform intrusion detection processing; and

means for selectively evaluating at least one incoming communication of the executing application program for an intrusion using the intrusion detection processing by comparing current conditions to predetermined conditions which signal a potential intrusion in response to identification of errors by error-handling logic of a protocol stack executed in the operating system kernel.

15. The system according to claim 14, wherein the current conditions comprise contents of the incoming communication.

16. The system according to claim 15, wherein the current conditions further comprise a protocol state of the protocol stack when the incoming communication was evaluated.

19. The computer program product according to claim 15, wherein the current conditions further comprise a protocol state of the protocol stack when the incoming communication was evaluated.

17. A computer program product for improving intrusion detection in a computing network, the computer program product embodied on one or more computer-readable media and comprising:

computer-readable program code that is configured to execute an application program which makes use of an operating system kernel during execution;

computer-readable program code that is configured to execute a protocol stack within the operating system kernel, wherein the protocol stack is augmented to perform intrusion detection processing; and

computer-readable program code that is configured to selectively at least one incoming communication of the executing application program for an intrusion using the intrusion detection processing by comparing current condition to predetermined conditions which signal a potential intrusion in response to identification of errors by error-handling logic of the protocol stack executed by the poeration system kernel.

18. The computer program product according to claim 17, wherein the current conditions comprise contents of the incoming communication.