Method and system for detecting unusual events and application thereof in computer intrusion detection

• US 7,475,405 B2
• Filed: 12/27/2000
• Issued: 01/06/2009
• Est. Priority Date: 09/06/2000
• Status: Expired due to Fees

• Current Assignee: Trend Micro Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Manganaris, Stefanos, Hermiz, Keith
• Primary Examiner(s): Lee, Thomas
• Assistant Examiner(s): Wu, Qing-Yuan
• Application Number:
  US09/749,095
• Publication Number:
  US 20020082886A1
• Time in Patent Office: 2,932 Days
• Field of Search: 719/318, 705/39, 705/10, 713/201, 379/114.14, 726 22- 23, 714/26, 707/6, 706 45- 48
• US Class Current: 719/318
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 2221/2101 Auditing as a secondary aspect
• CPC Class Codes: H04L 41/16 using machine learning or artificial intelligence
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1458 Denial of Service

Abstract

An automated decision engine is utilized to screen incoming alarms using a knowledge-base of decision rules. The decision rules are updated with the assistance of a data mining engine that analyzes historical data. “Normal” alarm events, sequences, or patterns generated by sensors under conditions not associated with unusual occurrences (such as intrusion attacks) are characterized and these characterizations are used to contrast normal conditions from abnormal conditions. By identifying frequent occurrences and characterizing them as “normal” it is possible to easily identify anomalies which would indicate a probable improper occurrence. This provides very accurate screening capability based on actual event data.

Claims

1. A computer-implemented method of adaptively generating frequent event patterns as an expected behavior model by processing event data to detect the occurrence of unusual events, said method comprising the steps of:

receiving a historical event data set wherein said historical event data comprises individual event alarms, a series of event alarm groupings or a combination of single event and event groupings as well as context information including historic conditions present when an event occurred;

identifying a context in which each event in said historical event data set occurred and categorizing each event according to its identified context;

performing pattern analysis on said historical event data set and the identified context of the events in said historical data set to generate frequent event patterns based on said historic event data wherein said pattern analysis step comprises;

performing association analysis on said historical event data set and the identified context of the events in said historical event data set to generate association rules and frequent itemsets based on the event occurrences within a window of event activity as said frequent event patterns;

performing sequential pattern analysis on said historical event data set and the identified context of the events in said historical event data set to generate commonly occurring sequence of data events or data patterns as said frequent event patterns, wherein the association analysis and sequential pattern analysis are perform serially;

receiving a current event data set wherein said current event data set comprises new input data;

identifying a context in which each event in said current event data set occurred and categorizing each event according to its identified context;

comparing said frequent event patterns to said current event data set and the identified context of the events in said current event data set to identify event occurrences in said current event data set that do not correspond to any of said frequent event patterns, wherein said comparing step comprises applying said commonly occurring sequence of data events or data patterns, said association rules and frequent itemsets to said current event data set including an analysis of the context in which an event occurred as compared to historic conditions of when similar events occurred; and

outputting an unusual event indication whenever an event occurrence in the current event data set that does not correspond to any of said frequent event patterns is identified.

2. A method as set forth in claim 1, wherein said event occurrences that do not correspond to any of said frequent event patterns comprise the sequences of data occurring in a particular context within said current event data set, where (a) the sequences contain data not usually seen in the particular context, or (b) data frequently seen in the particular context is missing.

3. A method as set forth in claim 1, wherein said event occurrences that do not correspond to any of said frequent event patterns comprise combinations of data occurring in a particular context within said current event data set, where (a) the combinations of data within a window of event activity are not usually seen in the particular context, or (b) data frequently seen in the particular context is missing.

4. A method as set forth in claim 1, wherein said unusual events comprise suspicious intrusions in a computer network.

5. A computer-implemented method of adaptively generating frequent event patterns as an expected behavior model by detecting unusual events, comprising the steps of:

identifying a context in which each event in a historical data set (HDS) occurred and categorizing each event according to its identified context wherein said HDS comprises individual event alarms, a series of event alarm groupings or a combination of single event and event groupings as well as context information including historic conditions present when an event occurred;

performing pattern analysis on said historical data set and the identified context of the events in said historical data set to generate frequent event patterns based on said historic event data wherein said pattern analysis step comprises;

performing association analysis on said historical data set and the identified context of the events in said historical data set to generate association rules and frequent itemsets based on the event occurrences within a window of event activity as said frequent event patterns;

performing sequential pattern analysis on said historical data set and the identified context of the events in said historical data set to generate commonly occurring sequence of data events or data patterns as said frequent event patterns, wherein the association analysis and sequential pattern analysis are perform serially;

detecting event occurrences in a current data set (CDS) that do not correspond to the frequent event patterns by comparing the generated frequent event patterns of said HDS, with events occurring in said CDS and a categorized context in which each event in the current event data set occurred, wherein said CDS comprises new input data; and

outputting an unusual event indication whenever any event occurrences in said CDS that do not correspond to the frequent event patterns are detected.

6. A computer-implemented method of adaptively generating frequent alarm event patterns as an expected behavior model by detecting suspicious intrusions in a computer network, comprising the steps of:

identifying a context in which each event in a historical data set occurred and categorizing each event according to its identified context wherein said historical data set comprises individual event alarms, a series of event alarm groupings or a combination of single event and event groupings as well as context information including historic conditions present when an event occurred;

performing pattern analysis on said historical data set and the identified context of the events in said historical data set to generate frequent alarm event patterns based on said historic event data wherein said pattern analysis step comprises;

performing association analysis on said historical data set and the identified context of the events in said historical event data set to generate association rules and frequent itemsets based on the event occurrences within a window of event activity as said frequent alarm event patterns;

performing sequential pattern analysis on said historical event data set and the identified context of the events in said historical data set to generate commonly occurring sequence of data events or data patterns as said frequent alarm event patterns, wherein the association analysis and sequential pattern analysis are perform serially;

detecting alarm event occurrences in a current data set (CDS) that do not correspond to said frequent alarm event patterns by comparing the generated frequent alarm event patterns of said historical data set, with alarm events occurring in said CDS and a categorized context in which each alarm event in said CDS occurred, wherein said CDS comprises new input; and

outputting an indication of a suspicious intrusion whenever any alarm event occurrences in said CDS that do not correspond to the frequent alarm event patterns are detected.