Systems and methods for validation, completion and construction of event relationship networks

• US 20030074440A1
• Filed: 10/12/2001
• Published: 04/17/2003
• Est. Priority Date: 10/12/2001
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Grabarnik, Genady, Ma, Sheng, Hellerstein, Joseph L., Perng, Chang-Shing, Thoenen, David Hutchinson
• Granted Patent: US 7,631,058 B2
• Time in Patent Office: Days
• US Class Current: 709/224
• CPC Class Codes: H04L 41/0631 using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis

Abstract

Techniques for data-driven validation, completion and construction of event relationship networks (ERNs) are provided. Event relationship networks are widely used in event management system design. To date, ERNs are constructed purely based on human expertise and there is no automatic or event semi-automatic method that validates or completes ERNs. The present invention provides techniques for automatically validating and completing existing ERNs and/or constructing new ERNs, based on collected event data.

Claims

1. A computer-based method for use in accordance with an event management system, the method comprising the steps of:

automatically generating one or more event relationship networks from event data, wherein an event relationship network comprises nodes representing events and links connecting correlated nodes; and

utilizing the one or more generated event relationship networks to construct one or more correlation rules for use by a correlation engine in the event management system.

2. The method of claim 1, further comprising the step of subjecting the one or more generated event relationship networks to human review prior to utilizing the one or more generated event relationship networks to construct the one or more correlation rules.

3. The method of claim 1, wherein, when one or more previously generated event relationship networks are available, the step of automatically generating one or more event relationship networks comprises:

obtaining one or more previously generated event relationship networks;

validating the one or more previously generated event relationship networks by removing any nodes or links included therein that are incorrect for a particular application context;

completing the one or more previously generated event relationship networks by adding any nodes or links thereto that are missing for the particular application context;

outputting the one or more validated and completed event relationship networks as the one or more event relationship networks used to construct the one or more correlation rules.

4. The method of claim 3, wherein the validating and completing steps utilize a statistical correlation analysis.

5. The method of claim 4, wherein the statistical correlation analysis utilizes pairwise correlation analysis, wherein correlation between a pair of events is measured in accordance with one or more statistical measurements.

6. The method of claim 3, wherein the validating step comprises, for a particular event relationship network, determining that links in the event relationship network have a confidence level not less than a given threshold.

7. The method of claim 3, wherein the validating step, for a particular event relationship network, comprises:

splitting the event relationship network into correlation paths;

for every correlation path, removing a node that has the least number of correlated nodes associated therewith until every node is fully correlated with every other node; and

merging correlation paths into one or more event relationship networks such that every path in a resulting event relationship network has every node fully correlated with every other node in the path.

8. The method of claim 1, wherein, when one or more previously generated event relationship networks are not available, the step of automatically generating one or more event relationship networks comprises:

mining patterns from the event data;

utilizing the mined patterns to construct the one or more event relationship networks;

outputting the one or more event relationship networks constructed from the mined patterns as the one or more event relationship networks used to construct the one or more correlation rules.

9. The method of claim 8, wherein the constructing step utilizes a statistical correlation analysis to mine patterns.

10. The method of claim 8, wherein the statistical correlation analysis utilizes pairwise correlation analysis, wherein correlation between a pair of events is measured in accordance with one or more statistical measurements.

11. The method of claim 1, wherein the event data is obtained from an event log representing historical events associated with a particular system being managed by the event management system.

12. The method of claim 1, wherein the one or more event relationship networks comprise annotations relating to statistical correlation between nodes.

13. The method of claim 1, wherein the event data is preprocessed prior to use in generating the one or more event relationship networks by removing at least a portion of any redundant events.

14. Apparatus use in accordance with an event management system, the apparatus comprising:

at least one processor operative to;

(i) automatically generate one or more event relationship networks from event data, wherein an event relationship network comprises nodes representing events and links connecting correlated nodes; and

(ii) utilize the one or more generated event relationship networks to construct one or more correlation rules for use by a correlation engine in the event management system; and

memory, coupled to the at least one processor, which stores at least one of the event data and the one or more event relationship networks.

15. The apparatus of claim 14, wherein the at least one processor is further operative to permit the operation of subjecting the one or more generated event relationship networks to human review prior to utilizing the one or more generated event relationship networks to construct the one or more correlation rules.

16. The apparatus of claim 14, wherein, when one or more previously generated event relationship networks are available, the operation of automatically generating one or more event relationship networks comprises:

obtaining one or more previously generated event relationship networks;

validating the one or more previously generated event relationship networks by removing any nodes or links included therein that are incorrect for a particular application context;

completing the one or more previously generated event relationship networks by adding any nodes or links thereto that are missing for the particular application context;

outputting the one or more validated and completed event relationship networks as the one or more event relationship networks used to construct the one or more correlation rules.

17. The apparatus of claim 16, wherein the validating and completing operations utilize a statistical correlation analysis.

18. The apparatus of claim 17, wherein the statistical correlation analysis utilizes pairwise correlation analysis, wherein correlation between a pair of events is measured in accordance with one or more statistical measurements.

19. The apparatus of claim 16, wherein the validating operation comprises, for a particular event relationship network, determining that links in the event relationship network have a confidence level not less than a given threshold.

20. The apparatus of claim 16, wherein the validating operation, for a particular event relationship network, comprises:

splitting the event relationship network into correlation paths;

for every correlation path, removing a node that has the least number of correlated nodes associated therewith until every node is fully correlated with every other node; and

merging correlation paths into one or more event relationship networks such that every path in a resulting event relationship network has every node fully correlated with every other node in the path.

21. The apparatus of claim 14, wherein, when one or more previously generated event relationship networks are not available, the step of automatically generating one or more event relationship networks comprises:

mining patterns from the event data;

utilizing the mined patterns to construct the one or more event relationship networks;

outputting the one or more event relationship networks constructed from the mined patterns as the one or more event relationship networks used to construct the one or more correlation rules.

22. The apparatus of claim 21, wherein the constructing operation utilizes a statistical correlation analysis to mine patterns.

23. The apparatus of claim 21, wherein the statistical correlation analysis utilizes pairwise correlation analysis, wherein correlation between a pair of events is measured in accordance with one or more statistical measurements.

24. The apparatus of claim 14, wherein the event data is obtained from an event log representing historical events associated with a particular system being managed by the event management system.

25. The apparatus of claim 14, wherein the one or more event relationship networks comprise annotations relating to statistical correlation between nodes.

26. The apparatus of claim 14, wherein the event data is preprocessed prior to use in generating the one or more event relationship networks by removing at least a portion of any redundant events.

27. An article of manufacture for use in accordance with an event management system, the article comprising a machine readable medium containing one or more programs which when executed implement the steps of:

automatically generating one or more event relationship networks from event data, wherein an event relationship network comprises nodes representing events and links connecting correlated nodes; and

utilizing the one or more generated event relationship networks to construct one or more correlation rules for use by a correlation engine in the event management system.