System, method and computer program product for optimized root cause analysis

• US 8,463,899 B2
• Filed: 10/29/2009
• Issued: 06/11/2013
• Est. Priority Date: 07/29/2005
• Status: Active Grant

• Current Assignee: BMC Software Incorporated (KKR & Co., Inc.)
• Original Assignee: BMC Software Incorporated (KKR & Co., Inc.)
• Inventors: Scarpelli, Joe, Sodem, Sridhar, LeFaive, Ronald Alexander
• Primary Examiner(s): CHRISTENSEN, SCOTT B
• Application Number:
  US12/608,601
• Publication Number:
  US 20100050023A1
• Time in Patent Office: 1,321 Days
• Field of Search: None
• US Class Current: 709/224
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: H04L 41/0631 using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 43/16 Threshold monitoring

Abstract

Embodiments disclosed herein can significantly optimize a root cause analysis and substantially reduce the overall time needed to isolate the root cause or causes of service degradation in an IT environment. By building on the ability of an abnormality detection algorithm to correlate an alarm with one or more events, embodiments disclosed herein can apply data correlation to data points collected within a specified time window by data metrics involved in the generation of the alarm and the event(s). The level of correlation between the primary metric and the probable cause metrics may be adjusted using the ratio between theoretical data points and actual points. The final Root Cause Analysis score may be modified depending upon the adjusted correlation value and presented for user review through a user interface.

Claims

1. A root cause analysis optimization method, comprising:

at a performance monitoring system, identifying a primary alarm metric and one or more probable cause metrics involved in producing a list of root cause analysis (RCA) candidates, wherein each of the RCA candidates is associated with an RCA score that indicates a level of correlation between each RCA candidate and the primary alarm metric;

determining data points to use from the primary alarm metric and each probable cause metric based on an alarm time of the primary alarm metric;

validating the data points between the primary alarm metric and each probable cause metric to remove discrepancies by aligning the data points in time to compute a correlation coefficient;

applying a data correlation algorithm to the validated data points to compute the correlation coefficient between primary and secondary aligned data points;

calculating a confidence factor by computing a data point ratio between the primary and secondary aligned data points using actual data points and theoretical data points;

adjusting the level of correlation of the validated data points by adjusting the correlation coefficient using the confidence factor including the data point ratio;

adjusting the RCA scores of the RCA candidates based on the computed correlation coefficient including using the adjusted correlation coefficient; and

presenting to a user an optimized RCA candidate list listing a fraction of the RCA candidates sorted based the adjusted RCA score.

2. The root cause analysis optimization method according to claim 1, wherein determining data points to use further comprises:

determining whether abnormal events exist on the primary alarm metric within a user-selected RCA time window.

3. The root cause analysis optimization method according to claim 1, wherein validating the data points further comprises:

discarding erroneous data points.

4. The root cause analysis optimization method according to claim 1, wherein validating the data points further comprises:

discarding misaligned data points.

5. The root cause analysis optimization method according to claim 4, wherein discarding misaligned data points further comprises:

determining an absolute time difference between a primary data point and a secondary data point;

comparing the absolute time difference with half of a maximum poll interval; and

if the absolute time difference is more than half of the maximum poll interval, discarding the secondary data point.

6. The root cause analysis optimization method according to claim 5, wherein the theoretical data points are determined utilizing the maximum poll interval.

7. The root cause analysis optimization method according to claim 6, wherein the actual data points are determined utilizing the validated data points.

8. The root cause analysis optimization method according to claim 1, wherein the correlation coefficient ranges from −

1 to 1, and wherein applying a data correlation algorithm to the validated data points further comprises;

taking an absolute value of the correlation coefficient.

9. The root cause analysis optimization method according to claim 1, wherein adjusting the correlation coefficient further comprises:

multiplying the correlation coefficient with the data point ratio.

10. The root cause analysis optimization method according to claim 1, wherein adjusting the RCA scores of the RCA candidates further comprises:

reducing the RCA score by applying the adjusted correlation coefficient to a corresponding RCA score reduction factor.

11. The root cause analysis optimization method according to claim 1, wherein each of the primary alarm metric and the one or more probable cause metrics collects data utilized by the performance monitoring system to monitor an infrastructure of a computing environment and wherein the infrastructure comprises business services.

12. A computer program product comprising one or more non-transitory computer-readable storage media storing computer instructions translatable by one or more processors of a computer system to perform:

identifying a primary alarm metric and one or more probable cause metrics involved in producing a list of root cause analysis (RCA) candidates, wherein each of the RCA candidates is associated with an RCA score that indicates a level of correlation between each RCA candidate and the primary alarm metric;

determining data points to use from the primary alarm metric and each probable cause metric based on an alarm time of the primary alarm metric;

validating the data points between the primary alarm metric and each probable cause metric to remove discrepancies by aligning the data points in time to compute a correlation coefficient;

applying a data correlation algorithm to the validated data points to compute the correlation coefficient between primary and secondary aligned data points;

calculating a confidence factor by computing a data point ratio between the primary and secondary aligned data points using actual data points and theoretical data points;

adjusting the level of correlation of the validated data points by adjusting the correlation coefficient using the confidence factor including the data point ratio;

adjusting the RCA scores of the RCA candidates based on the computed correlation coefficient including using the adjusted correlation coefficient; and

presenting to a user an optimized RCA candidate list listing a fraction of the RCA candidates sorted based the adjusted RCA score.

13. The computer program product of claim 12, wherein the computer instructions are further translatable to perform:

determining whether abnormal events exist on the primary alarm metric within a user-selected RCA time window.

14. The computer program product of claim 12, wherein the computer instructions are further translatable to perform:

discarding erroneous data points and misaligned data points.

15. The computer program product of claim 12, wherein the computer instructions are further translatable to perform:

determining an absolute time difference between a primary data point and a secondary data point;

comparing the absolute time difference with half of a maximum poll interval; and

if the absolute time difference is more than half of the maximum poll interval, discarding the secondary data point.

16. The computer program product of claim 12, wherein the computer instructions are further translatable to perform:

multiplying the correlation coefficient with the data point ratio.

17. The computer program product of claim 12, wherein the computer instructions are further translatable to perform:

reducing the RCA score by applying the adjusted correlation coefficient to a corresponding RCA score reduction factor.

18. A system, comprising:

one or more processors; and

one or more computer-readable storage media storing computer instructions translatable by the one or more processors to perform;

identifying a primary alarm metric and one or more probable cause metrics involved in producing a list of root cause analysis (RCA) candidates, wherein each of the RCA candidates is associated with an RCA score that indicates a level of correlation between each RCA candidate and the primary alarm metric;

determining data points to use from the primary alarm metric and each probable cause metric based on an alarm time of the primary alarm metric;

validating the data points between the primary alarm metric and each probable cause metric to remove discrepancies by aligning the data points in time to compute a correlation coefficient;

applying a data correlation algorithm to the validated data points to compute the correlation coefficient between primary and secondary aligned data points;

calculating a confidence factor by computing a data point ratio between the primary and secondary aligned data points using actual data points and theoretical data points;

adjusting the level of correlation of the validated data points by adjusting the correlation coefficient using the confidence factor including the data point ratio;

adjusting the RCA scores of the RCA candidates based on the computed correlation coefficient including using the adjusted correlation coefficient; and

presenting to a user an optimized RCA candidate list listing a fraction of the RCA candidates sorted based the adjusted RCA score.

19. The system of claim 18, further comprising:

a user interface, wherein the optimized RCA candidate list is displayed to the user via the user interface.

20. The system of claim 19, further comprising:

a database storing the primary alarm metric and the one or more probable cause metrics.