Systems and methods for adaptive model generation for detecting intrusions in computer systems

• US 20070239999A1
• Filed: 05/25/2007
• Published: 10/11/2007
• Est. Priority Date: 01/25/2002
• Status: Active Grant

• Current Assignee: Andrew Honig, Andrew Howard, Eleazar Eskin, Salvatore Stolfo
• Original Assignee: Andrew Honig, Andrew Howard, Eleazar Eskin, Salvatore Stolfo
• Inventors: Honig, Andrew, Eskin, Eleazar, Howard, Andrew, Stolfo, Salvatore
• Granted Patent: US 8,893,273 B2
• Time in Patent Office: Days
• US Class Current: 713/194
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/212 with details for data modelling support
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 21/554 involving event detection and direct action
• CPC Class Codes: G06F 21/566 Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: H04L 63/14 for detecting or protecting against malicious traffic
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1433 Vulnerability analysis

Abstract

A system and methods for detecting intrusions in the operation of a computer system comprises a sensor configured to gather information regarding the operation of the computer system, to format the information in a data record having a predetermined format, and to transmit the data in the predetermined data format. A data warehouse is configured to receive the data record from the sensor in the predetermined data format and to store the data in a SQL database. A detection model generator is configured to request data records from the data warehouse in the predetermined data format, to generate an intrusion detection model based on said data records, and to transmit the intrusion detection model to the data warehouse according to the predetermined data format. A detector is configured to receive a data record in the predetermined data format from the sensor and to classify the data record in real-time as one of normal operation and an attack based on said intrusion detection model. A data analysis engine is configured to request data records from the data warehouse according to the predetermined data format and to perform a data processing function on the data records.

Claims

1. A system for detection intrusions in the operation of a computer system comprising:

(a) a sensor configured to gather information regarding the operation of the computer system, to format the information in a data record having a predetermined format, and to transmit the data in the predetermined data format;

(b) a data warehouse configured to receive the data record from the sensor in the predetermined data format and to store the data in a SQL database;

(c) a detection model generator configured to request data records from the data warehouse in the predetermined data format, to generate an intrusion detection model based on said data records, and to transmit the intrusion detection model to the data warehouse according to the predetermined data format;

(d) a detector configured to receive a data record in the predetermined data format from the sensor and to classify the data record in real-time as one of normal operation and an attack based on said intrusion detection model; and

(e) a data analysis engine configured to request data records from the data warehouse according to the predetermined data format and to perform a data processing function on the data records.

2. The system according to claim 1, wherein the predetermined data format is XML markup language.

3. The system according to claim 1, wherein the detection model generator is configured to periodically generate the detection model, the system further comprising a detection model distributor configured to receive the detection model from the data warehouse and to transmit the detection model to the detector.

4. The system according to claim 1, wherein the sensor is a network sensor.

5. The system according to claim 1, wherein the sensor is a host sensor.

6. The system according to claim 1, wherein the data record comprises information regarding a source of the information.

7. The system according to claim 1, wherein the detection model is a probabilistic model.

8. The system according to claim 7, wherein the detection model generator is configured to generate a parametrization of the probabilistic model.

9. The system according to claim 8, wherein the detector is configured to compute a probability associated with the data record.

10. The system according to claim 1, wherein the detection model is a set of support vectors which correspond to a decision boundary in a feature space.

11. The system according to claim 10, wherein the detection model generator is configured to generate a set of support vectors.

12. The system according to claim 11, wherein the detector is configured to map a data record to the feature space and determine the location of the data record in the feature space with respect to the decision boundary.

13. The system according to claim 1, wherein the data analysis engine is configured to display the data in the data warehouse as selected by a request in the predetermined data format.

14. The system according to claim 1, wherein the data analysis engine is configured to label a data record as one of normal operation and an attack.

15. The system according to claim 14, wherein the data analysis engine is further configured to append the label data to the data records in the SQL database.

16. The system according to claim 1, wherein the data analysis engine is configured to extract a feature from a plurality data records.

17. The system according to claim 16, wherein the data analysis engine is configured to append the feature data to the data records in the SQL database.

18. A distributed system for detection intrusions in the operation of a computer system comprising:

(a) a plurality of sensors configured to gather information regarding the operation of the computer system, to format the information in a data record having a predetermined format, and to transmit the data in the predetermined data format;

(b) a data warehouse configured to receive the data record from the sensor in the predetermined data format and to store the data in a SQL database;

(c) a plurality of detection model generators configured to request data records from the data warehouse in the predetermined data format, each detection model generator configured to generate a respective intrusion detection model based on said data records, and to transmit the respective intrusion detection model to the data warehouse according to the predetermined data format;

(d) a plurality of detectors configured to receive a data record in the predetermined data format from the sensor and to classify the data record in real-time as one of normal operation and an attack based on a respective intrusion detection model; and

(e) a data analysis engine configured to request data records from the data warehouse according to the predetermined data format and to perform a data processing function on the data records.

19. The system according to claim 18, wherein the predetermined data format is XML markup language.

20. A method for detection intrusions in the operation of a computer system comprising:

(a) gathering information regarding the operation of the computer system at a sensor and formatting the information into a data record having a predetermined data format;

(b) transmitting the data record in the predetermined data format to a data warehouse and storing the data in a SQL database;

(c) generating an intrusion detection model comprising requesting data records from the data warehouse according to the predetermined data format and transmitting the intrusion detection model to the data warehouse in the predetermined data format;

(d) classifying a data item in real-time as one of normal operation and an attack based on the intrusion detection model comprising receiving the data records from the sensor and; and

(e) requesting a data record from the data warehouse according to the predetermined data format and performing a data processing function on the data record.