Ignoring redundant symptoms in modular self-healing systems

• US 7,757,120 B2
• Filed: 06/23/2006
• Issued: 07/13/2010
• Est. Priority Date: 06/23/2006
• Status: Active Grant

• Current Assignee: Twitter Inc (X Holdings Corp.)
• Original Assignee: International Business Machines Corporation
• Inventors: Topol, Brad B., Ogle, David M., Subramanian, Balan
• Primary Examiner(s): GUYTON, PHILIP A
• Application Number:
  US11/426,288
• Publication Number:
  US 20070299943A1
• Time in Patent Office: 1,481 Days
• Field of Search: 714/25, 714/26
• US Class Current: 714/26
• CPC Class Codes: G06F 11/0748 in a remote unit communicating with a single-box computer node experiencing an error/fault remote testing G06F11/2294
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/2257 using expert systems

Abstract

Techniques for processing events in a self-healing system. In one embodiment, the system receives system events, one or more of which may indicate a possible fault condition in the system. A hierarchal rule structure is used to generate a correlated event set. A determining operation determines at least one corrective action to be performed in response to the correlated event set. A deactivating operation deactivates all correlation rules within the hierarchal rule structure that are only used to generate the correlated event set. Repeated instances of the same fault condition are prevented from being processed by the self-healing system.

Claims

1. A self-healing system comprising:

an event bus configured to transmit events in the self-healing system;

a correlation engine coupled to the event bus, the correlation engine configured to correlate events transmitted on the event bus from different components of the self-healing system, wherein the correlation engine correlates the events into a correlated event set, the correlated event set indicating a possible fault condition in the system, the correlation engine further configured to evaluate a plurality of correlation rules, the plurality of correlation rules being arranged in a hierarchal rule structure to correlate the events;

an analysis engine coupled to the event bus, the analysis engine configured to perform analysis on the correlated event set that is received from the correlation engine, determine at least one corrective action to be performed in response to the correlated event set, and send a deactivation event to the correlation engine upon initiation of the corrective action, the deactivation event including the correlated event set; and

wherein the correlation engine is further configured to deactivate all correlation rules within the hierarchal rule structure that are used to generate only the correlated event set, and wherein the correlation engine deactivates the correlation rules while recursively traversing the hierarchal rule structure from child rule to parent rule; and

wherein the first, second and third program instructions are stored on the computer readable storage media for execution by the CPU via the computer readable memory.

2. The system of claim 1, wherein the analysis engine is further configured to send a reactivation event set to the correlation engine, the reactivation event set indicating at least one reactivation event required to be received by the correlation engine to reactivate the deactivated correlation rules.

3. The system of claim 1, wherein the correlation engine is further configured to create an activation rule causing the deactivated correlation rules to return to an active state.

4. The system of claim 1, further comprising:

a system component coupled to the event bus, the system component configured to send at least one component event to the correlation engine; and

wherein the correlation engine is further configured to disable transmission of the component event from the system component to the correlation engine upon a determination that the component event is only processed by disabled rules.

5. The system of claim 1, further comprising:

wherein the analysis engine is configured to notify the correlation engine of at least one target event which, if detected by the correlation engine, increases the likelihood of a fault condition in the system; and

wherein the correlation engine is configured to rearrange the hierarchal rule structure such that a target rule set of at least one target rule for evaluating the target event is given processing priority.

6. The system of claim 1, wherein the analysis engine is further configured to compare the correlated event set to a symptom database.

7. The system of claim 1, wherein the correlation engine is further configured to deactivate the correlation rules while recursively traversing the hierarchal rule structure from child rule to parent rule.

8. A method for processing events in a computer system, the method comprising:

receiving at least one system event, the at least one system event indicating a possible fault condition in the system;

utilizing a hierarchal rule structure to generate a correlated event set;

determining at least one corrective action to be performed in response to the correlated event set;

deactivating all correlation rules within the hierarchal rule structure that are only used to generate the correlated event set;

determining at least one target event which, if detected, increases the likelihood of a fault condition; and

rearranging the hierarchal rule structure such that a target rule set of at least one target rule for evaluating the target event is given processing priority.

9. The method of claim 8, further comprising receiving a deactivation event upon initiation of the corrective action, the deactivation event including the correlated event set.

10. The method of claim 8, further comprising receiving a reactivation event set, the reactivation event set indicating at least one reactivation event required to be received to reactivate the deactivated correlation rules.

11. The method of claim 8, further comprising creating an activation rule for causing the deactivated correlation rules to return to an active state.

12. The method of claim 8, further comprising disabling transmission of a component event from a system component upon a determination that the component event is only processed by disabled rules.

13. A method for processing events in a computer system, the method comprising:

receiving at least one system event, the at least one system event indicating a possible fault condition in the system;

utilizing a hierarchal rule structure to generate a correlated event set;

determining at least one corrective action to he performed in response to the correlated event set;

deactivating all correlation rules within the hierarchal rule structure that are only used to generate the correlated event set; and

deactivating the correlation rules while recursively traversing the hierarchal rule structure from child rule to parent rule.

14. The method of claim 13, further comprising receiving a deactivation event upon initiation of the corrective action, the deactivation event including the correlated event set.

15. The method of claim 13, further comprising receiving a reactivation event set, the reactivation event set indicating at least one reactivation event required to be received to reactivate the deactivated correlation rules.

16. The method of claim 13, further comprising creating an activation rule for causing the deactivated correlation rules to return to an active state.

17. The method of claim 13, further comprising disabling transmission of a component event from a system component upon a determination that the component event is only processed by disabled rules.

18. A computer program product for processing events in a computer system, the computer program product comprising:

a computer usable storage medium having computer usable program code embodied therewith, the computer usable program code comprising;

computer usable program code configured to receive at least one system event, the at least one system event indicating a possible fault condition in the system;

computer usable program code configured to utilize a hierarchal rule structure to generate a correlated event set;

computer usable program code configured to determine at least one corrective action to be performed in response to the correlated event set;

computer usable program code configured to deactivate all correlation rules within the hierarchal rule structure that are only used to generate the correlated event set;

computer usable program code configured to determine at least one target event which, if detected, increases the likelihood of a fault condition; and

computer usable program code configured to rearrange the hierarchal rule structure such that a target rule set of at least one target rule for evaluating the target event is given processing priority.

19. The computer program product of claim 18, further comprising computer usable program code configured to receive a deactivation event upon initiation of the corrective action, the deactivation event including the correlated event set.

20. The computer program product of claim 18, further comprising computer usable program code configured to receive a reactivation event set, the reactivation event set indicating at least one reactivation event required to be received to reactivate the deactivated correlation rules.

21. The computer program product of claim 18, further comprising computer usable program code configured to create an activation rule for causing the deactivated correlation rules to return to an active state.

22. The computer program product of claim 18, further comprising computer usable program code configured to disable transmission of a component event from a system component upon a determination that the component event is only processed by disabled rules.

23. The computer program product of claim 18, further comprising:

computer usable program code configured to determine at least one target event which, if detected, increases the likelihood of a fault condition; and

computer usable program code configured to rearrange the hierarchal rule structure such that a target rule set of at least one target rule for evaluating the target event is given processing priority.

24. The computer program product of claim 18, further comprising computer usable program code configured to deactivate the correlation rules while recursively traversing the hierarchal rule structure from child rule to parent rule.