System and method for detecting buffer overflow attacks

• US 6,826,697 B1
• Filed: 08/30/2000
• Issued: 11/30/2004
• Est. Priority Date: 08/30/1999
• Status: Expired due to Term

• Current Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Original Assignee: Symantec Corporation (NortonLifeLock Inc.)
• Inventors: Moran, Douglas B.
• Primary Examiner(s): Sheikh, Ayaz
• Assistant Examiner(s): Baum, Ronald
• Application Number:
  US09/651,306
• Time in Patent Office: 1,553 Days
• Field of Search: 713/201
• US Class Current: 726/23
• CPC Class Codes: G06F 21/55 Detecting local intrusion or implementing counter-measures
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1491 using deception as countermeasure, e.g. honeypots, honeynets, decoys or entrapment

Abstract

A system and method are disclosed for detecting intrusions in a host system on a network. The intrusion detection system comprises an analysis engine configured to use continuations and apply forward- and backward-chaining using rules. Also provided are sensors, which communicate with the analysis engine using a meta-protocol in which the data packet comprises a 4-tuple. A configuration discovery mechanism locates host system files and communicates the locations to the analysis engine. A file processing mechanism matches contents of a deleted file to a directory or filename, and a directory processing mechanism extracts deallocated directory entries from a directory, creating a partial ordering of the entries. A signature checking mechanism computes the signature of a file and compares it to previously computed signatures. A buffer overflow attack detector compares access times of commands and their associated files. The intrusion detection system further includes a mechanism for checking timestamps to identify and analyze forward and backward time steps in a log file.

Claims

1. A system for detecting intrusions on a host, comprising:

a) a database of commands and files accessed by the commands, including dependencies encoded as classes of objects; and

b) an analysis engine configured to compare an access time of a first command with access and modification times of files expected to be accessed by the first command and identify the first command as suspicious if the files expected to be accessed by the first command were not in fact accessed.

2. The system as recited in claim 1, further comprising a sensor configured to collect an instance of the first command executed by the host.

3. The system as recited in claim 1, wherein the database comprises set user identifier commands.

4. The system as recited in claim 1, wherein the database comprises set group identifier commands.

5. The system as recited in claim 1, further comprising a configuration discovery mechanism configured to return information about logfiles, and wherein the analysis engine is further configured to encode the information about logfiles in the database.

6. The system as recited in claim 5, wherein the configuration discovery mechanism is configured to assign membership of classes.

7. The system as recited in claim 2, wherein the sensor is configured to collect the instance from a logfile.

8. The system as recited in claim 2, wherein the sensor is configured to communicate the first command to the analysis engine.

9. The system as recited in claim 2, wherein the analysis engine is further configured to assign a suspicion level based on the comparison of times.

10. The system as recited in claim 9, wherein the analysis engine includes a time decay function.

11. The system as recited in claim 10, wherein the analysis engine is configured to use the time decay function to downgrade a suspicion level associated with the instance of the first command.

12. The system as recited in claim 11, wherein the time decay function is exponential.

13. The system as recited in claim 9, further comprising a user interface, wherein the analysis engine is configured to provide the user interface with an analysis based on the suspicion value, and information relating to the analysis.

14. The system as recited in claim 9, further comprising a sensor configured to collect a second instance of a second command, corresponding to the first command, executed by a second host.

15. The system as recited in claim 14, wherein the analysis engine includes a time decay function.

16. The system as recited in claim 15, wherein the analysis engine is further configured to use the first instance, second instance, and time decay function in assigning a suspicion value to the first or second instance.

17. A method for detecting intrusions on a host, comprising the steps of:

a) providing a database of commands and files accessed by the commands

b) encoding dependencies as classes of objects in the database;

c) comparing an access time of a first command with access and modification times of files expected to be accessed by the first command; and

d) identifying the first command as suspicious if the files expected to be accessed by the first command were not in fact accessed.

18. A computer program product for detecting intrusions on a host, the computer program product being embodied in a computer readable medium having machine readable code embodied therein for performing the steps of:

a) providing a database of commands and files accessed by the commands

b) encoding dependencies as classes of objects in the database;

c) comparing an access time of a first command with access and modification times of files expected to be accessed by the first command;

d) identifying the first command as suspicious if the files expected to be accessed by the first command were not in fact accessed.