Preventing and servicing system errors with event pattern correlation

• US 9,652,316 B2
• Filed: 03/31/2015
• Issued: 05/16/2017
• Est. Priority Date: 03/31/2015
• Status: Active Grant

• Current Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Original Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Inventors: Gamage, Nimal K K, Whitner, Richard Bennett
• Primary Examiner(s): Riad, Amine
• Application Number:
  US14/674,780
• Publication Number:
  US 20160292028A1
• Time in Patent Office: 777 Days
• Field of Search: 714 473
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0748 in a remote unit communicating with a single-box computer node experiencing an error/fault remote testing G06F11/2294
• CPC Class Codes: G06F 11/0754 by exceeding limits
• CPC Class Codes: G06F 11/076 by exceeding a count or rate limit, e.g. word- or bit count limit
• CPC Class Codes: G06F 11/0772 Means for error signaling, e.g. using interrupts, exception flags, dedicated error registers
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/34 Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: H04L 41/00 Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
• CPC Class Codes: H04L 41/0836 to enhance reliability, e.g. reduce downtime
• CPC Class Codes: H04L 41/147 for predicting network behaviour
• CPC Class Codes: H04L 41/149 for prediction of maintenance

Abstract

A method includes receiving historical monitoring data for components of a system, the historical monitoring data comprising a plurality of events, each triggered by an associated component, and a plurality of alarms, each triggered by detection of a particular type of condition in the components. The method also includes determining common event sequences in the plurality of events, each event sequence culminating in one of the plurality of alarms. The method further includes correlating the common event sequences into an event pattern. The method also includes receiving a plurality of real-time events triggered by the components. The method additionally includes detecting the event pattern in the plurality of real-time events to predict a potential future instance of the particular type of condition in the components of the system.

Claims

1. A method, comprising:

receiving historical monitoring data for components of a system, the historical monitoring data comprising a plurality of events, each triggered by an associated component, and a plurality of alarms, each triggered by detection of a particular type of condition in the components;

determining common event sequences in the plurality of events, each event sequence culminating in one of the plurality of alarms;

correlating the common event sequences into an event pattern;

for each event sequence, determining a relationship between a common system component and each component that generated each event in the event sequence;

receiving a plurality of real-time events triggered by the components;

detecting the event pattern in the plurality of real-time events to predict a potential future instance of the particular type of condition in the components of the system; and

determining a root cause component based on each determined relationship as applied to the detected event pattern.

2. The method of claim 1, further comprising:

determining a root cause inference regarding the particular type of condition based on the associated components for each event in the common event sequences; and

in response to detecting the event pattern in the real-time events, mitigating a root cause of the particular type of condition based on the root cause inference to prevent the potential future instance of the particular type of condition.

3. The method of claim 1, further comprising:

determining a root cause inference regarding the particular type of condition based on the associated components for each event in the common event sequences;

receiving a real time alarm triggered by detection of a first error in the components; and

mitigating a root cause of the first error based on the root cause inference.

4. The method of claim 2, wherein the root cause comprises a configuration setting for a first component, and wherein addressing the root cause comprises modifying the configuration setting.

5. The method of claim 1, wherein the historical monitoring data further comprises information regarding relationships between the components of the system, and wherein the common event sequences are determined based on the relationships between the components.

6. The method of claim 1, wherein the historical monitoring data further comprises attributes for each event in the plurality of events, the attributes comprising an event type, a trigger time, a severity level, an impact level, and an associated component, and wherein the common event sequences are determined based on matching event attributes between each common event sequence.

7. The method of claim 1, wherein the plurality of events in the historical monitoring data are discovered using a component monitoring system.

8. The method of claim 1, further comprising:

accessing a database of predetermined event patterns with associated root cause inferences regarding the particular type of condition; and

modifying the event pattern based on the predetermined event patterns.

9. A computer configured to access a storage device, the computer comprising:

a processor; and

a non-transitory, computer-readable storage medium storing computer-readable instructions that when executed by the processor cause the computer to perform;

receiving historical monitoring data for components of a system, the historical monitoring data comprising a plurality of events, each triggered by an associated component, and a plurality of alarms, each triggered by detection of a particular type of condition in the components;

determining common event sequences in the plurality of events, each event sequence culminating in one of the plurality of alarms;

correlating the common event sequences into an event pattern;

for each event sequence, determining a relationship between a common system component and each component that generated each event in the event sequence;

receiving a plurality of real-time events triggered by the components;

detecting the event pattern in the plurality of real-time events to predict a potential future instance of the particular type of condition in the components of the system; and

determining a root cause component based on each determined relationship as applied to the detected event pattern.

10. The computer of claim 9, wherein the computer-readable instructions further cause the computer to perform:

determining a root cause inference regarding the particular type of condition based on the associated components for each event in the common event sequences; and

in response to detecting the event pattern in the real-time events, mitigating a root cause of the particular type of condition based on the root cause inference to prevent the potential future instance of the particular type of condition.

11. The computer of claim 9, wherein the computer-readable instructions further cause the computer to perform:

determining a root cause inference regarding the particular type of condition based on the associated components for each event in the common event sequences;

receiving a real time alarm triggered by detection of a first error in the components; and

mitigating a root cause of the first error based on the root cause inference.

12. The computer of claim 10, wherein the root cause comprises a configuration setting for a first component, and wherein addressing the root cause comprises modifying the configuration setting.

13. The computer of claim 9, wherein the historical monitoring data further comprises information regarding relationships between the components of the system, and wherein the common event sequences are determined based on the relationships between the components.

14. The computer of claim 9, wherein the historical monitoring data further comprises attributes for each event in the plurality of events, the attributes comprising an event type, a trigger time, a severity level, an impact level, and an associated component, and wherein the common event sequences are determined based on matching event attributes between each common event sequence.

15. The computer of claim 9, wherein the plurality of events in the historical monitoring data are discovered using a component monitoring system.

16. The computer of claim 9, wherein the computer-readable instructions further cause the computer to perform:

accessing a database of predetermined event patterns with associated root cause inferences regarding the particular type of condition; and

modifying the event pattern based on the predetermined event patterns.

17. A computer program product comprising:

a non-transitory computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code comprising;

computer-readable program code configured to receive historical monitoring data for components of a system, the historical monitoring data comprising a plurality of events, each triggered by an associated component, and a plurality of alarms, each triggered by detection of a particular type of condition in the components;

computer-readable program code configured to determine common event sequences in the plurality of events, each event sequence culminating in one of the plurality of alarms;

computer-readable program code configured to correlate the common event sequences into an event pattern;

computer-readable program code configured to determine a relationship between a common system component and each component that generated each event in the event sequence;

computer-readable program code configured to receive a plurality of real-time events triggered by the components;

computer-readable program code configured to detect the event pattern in the plurality of real-time events to predict a potential future instance of the particular type of condition in the components of the system; and

computer-readable program code configured to determine a root cause component based on each determined relationship as applied to the detected event pattern.

18. The computer program product of claim 17, wherein the computer-readable program code further comprises:

computer-readable program code configured to determine a root cause inference regarding the particular type of condition based on the associated components for each event in the common event sequences; and

computer-readable program code configured to, in response to detecting the event pattern in the real-time events, mitigate a root cause of the particular type of condition based on the root cause inference to prevent the potential future instance of the particular type of condition.

19. The computer program product of claim 17, wherein the computer-readable program code further comprises:

computer-readable program code configured to determine a root cause inference regarding the particular type of condition based on the associated components for each event in the common event sequences;

computer-readable program code configured to receive a real time alarm triggered by detection of a first error in the components; and

computer-readable program code configured to mitigate a root cause of the first error based on the root cause inference.

20. The computer program product of claim 18, wherein the root cause comprises a configuration setting for a first component, and wherein addressing the root cause comprises modifying the configuration setting.