Advanced Intelligence Engine

• US 20120131185A1
• Filed: 11/23/2011
• Published: 05/24/2012
• Est. Priority Date: 11/24/2010
• Status: Active Grant

• Current Assignee: Logrhythm Incorporated
• Original Assignee: Logrhythm Incorporated
• Inventors: Petersen, Chris, Villella, Phillip, Aisa, Brad
• Granted Patent: US 8,543,694 B2
• Time in Patent Office: Days
• US Class Current: 709/224
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06N 5/025 Extracting rules from data
• CPC Class Codes: H04L 41/069 using logs of notifications; Post-processing of notifications
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

An advanced intelligence engine (AIE) for use in identifying what may be complex events or developments on one or more data platforms or networks from various types of structured or normalized data generated by one or more disparate data sources. The AIE may conduct one or more types of quantitative, correlative, behavioral and corroborative analyses to detect events from what may otherwise be considered unimportant or non-relevant information spanning one or more time periods. Events generated by the AIE may be passed to an event manager to determine whether further action is required such as reporting, remediation, and the like.

Claims

1. A method for use in monitoring one or more platforms of one or more data systems, comprising:

receiving, at a processing engine, structured data generated by one or more platforms over at least one communications network;

first evaluating, at the processing engine using a first rule block, at least some of the data;

first determining, from the first evaluating, whether a result is one of at least first and second outcomes; and

depending upon the determining, second evaluating, at the processing engine using a second rule block, at least some of the data; and

second determining, from the second evaluating, whether a result is one of at least first and second outcomes, wherein the results are analyzed to determine an event of interest.

2. The method of claim 1, wherein the first determining comprises:

ascertaining that the result of the first evaluating is the first outcome.

3. The method of claim 2, further comprising:

ascertaining that the result of the second evaluating is the first outcome; and

generating an event after the result of the second evaluating is determined to be the first outcome.

4. The method of claim 2, wherein each of the first and second evaluating comprises:

making a determination about one or more fields of the at least some of the data, wherein a content of one of the fields in the first evaluating matches a content of one of the fields in the second evaluating.

5. The method of claim 4, wherein the second evaluating comprises:

utilizing the matching content as a key into an index structure of the at least some of the data to determine if the result is one of the first and second outcomes.

6. The method of claim 2, wherein the first outcome of the first evaluating occurs at a first time that is identified by at least one time stamp of the at least some of the data.

7. The method of claim 6, wherein the second evaluating comprises:

considering data of the at least some of the data that is associated with at least one time stamp including a second time that is the same as or after the first time.

8. The method of claim 6, wherein the second evaluating comprises:

considering data of the at least some of the data that is associated with at least one time stamp including a second time that is before the first time.

9. The method of claim 1, wherein the first outcome of at least one of the first and second evaluating comprises data of the at least some of the data matching first criteria.

10. The method of claim 9, wherein the first criteria comprises a particular content of one or more fields of the data of the at least some of the data.

11. The method of claim 9, wherein the first outcome further comprises a particular quantitative value of the data matching the first criteria being equal to or exceeding a threshold for the particular quantitative value.

12. The method of claim 9, wherein the first outcome further comprises a particular field of the data comprising at least a threshold number of unique contents of the particular field.

13. The method of claim 12, wherein the first outcome further comprises the particular field of the data comprising at least the threshold number of unique contents of the particular field within a particular time period.

14. The method of claim 13, wherein the particular time period is measured in relation to a time at which the first outcome of one of the first and second evaluating occurred.

15. The method of claim 1, wherein the result of each of the first and second evaluating comprises the first outcomes, and wherein the method further comprises:

third evaluating, at the processing engine using a third rule block, at least some of the data, wherein the results are analyzed to determine an event of interest.

16. A system for use in monitoring one or more platforms of one or more data systems, comprising:

a processing module; and

a memory module logically connected to the processing module and comprising a set of computer readable instructions that are executable by the processing module to;

receive structured data generated by one or more platforms over at least one communications network;

first evaluate, using a first rule block, at least some of the data;

first determine, from the first evaluating, whether a result is one of at least first and second outcomes; and

depending upon the determining, second evaluate, using a second rule block, at least some of the data; and

second determine, from the second evaluating, whether a result is one of at least first and second outcomes, wherein the results are analyzed to determine an event of interest.

17. An event generating system, comprising:

an event module adapted to be operatively interposed between a user console for manipulating the event module and at least one source of structured data, the structured data being generated by one or more platforms of one or more data systems, the event module comprising;

a user rule module including a plurality of objects for use in allowing a user to define a user version of at least one data processing rule; and

an event table for storing generated events; and

a event processing engine, logically connected between the at least one source of structured data and the event module, for generating events from the at least one source of structured data, the event processing engine comprising;

a receiving module for receiving data related to the at least one source of structured data;

a compiling module for obtaining the plurality of objects from the user rule module and generating a processing version of the at least one data processing rule; and

a processing module for;

evaluating the received data using the processing version of the at least one data processing rule; and

in response to the received data matching the processing version of the at least one data processing rule, writing at least one event to the event table.

18. The event generating system of claim 17, wherein the event module further comprises:

an alarming module for evaluating generated events in the event table using one or more alarm rules, wherein in response to a generated event matching an alarm rule, the alarming module is operable to generate a notification message for transmission to one or more entities.

19. The event generating system of claim 17, further comprising:

at least one structured data manager operatively interposed between the event processing engine and the at least one data source, wherein the structured data manager is operable to parse structured data received from the at least one data source, process the parsed structured data, and forward matching structured data to the receiving module of the event processing engine for further analysis.

20. The event generating system of claim 17, wherein the at least one data processing rule comprises:

first and second rule blocks, wherein the received data matches the processing version of the at least one data processing rule upon a first evaluation of the received data by the processing module using the first rule block comprising a first of at least first and second first rule block outcomes and a second evaluation of the received data by the processing module using the second rule block comprising a first of at least first and second rule block outcomes.

21. The event generating system of claim 20, wherein in response to the first or second evaluations by the processing module comprising one of the first rule block first outcome and second rule block first outcome, the processing module proceeds to determine whether the other of the first and second evaluations comprises the other of the first rule block first outcome and second rule block first outcome.

22. The event generating system of claim 21, wherein the processing module ceases determination of whether the other of the first and second evaluations comprises the other of the first rule block first outcome and second rule block first outcome upon expiration of a predetermined time period.

23. The event generating system of claim 22, wherein the predetermined time period is measured from a determination that the first or second outcomes comprises the one of the first rule block first outcome and second rule block second outcome.

24. The event generating system of claim 20, wherein the first rule block first outcome comprises the received data matching one or more particular criteria and wherein the second rule block first outcome comprises the received data comprising one or more quantitative fields having a value exceeding a particular threshold value.