System and method for monitoring performance metrics

• US 20030110007A1
• Filed: 07/01/2002
• Published: 06/12/2003
• Est. Priority Date: 07/03/2001
• Status: Active Grant

• Current Assignee: Riverbed Technology, LLC (Riverbed Technology Incorporated)
• Original Assignee: Altaworks Corp. (Riverbed Technology Incorporated)
• Inventors: Courtemanche, Michael B., McGee, John, Heath, David M., Beck, Ralph L., Earley, John Michael
• Granted Patent: US 6,643,613 B2
• Time in Patent Office: Days
• US Class Current: 702/179
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0772 Means for error signaling, e.g. using interrupts, exception flags, dedicated error registers
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 17/15 Correlation function computation including computation of convolution operations arithmetic circuits for sum of products per se, e.g. multiply-accumulators G06F7/5443; digital filters, e.g. FIR, IIR, adaptive filters H03H17/00
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: G06F 2201/88 Monitoring involving counting

Abstract

A system and method for monitoring a set of performance metrics is described. The methods include metric correlation and grouping methods that analyze a group of temporally related metrics, and correlate pairs of the metrics in the group. In one embodiment, rank correlation techniques are used to perform this correlation. Methods are also described for grouping metrics using a dynamic correlation pair graph that preserves all of the correlated relationships. This correlation pair graph may be used to determine which metrics are associated with a particular key metric, which may provide information on the cause of an alarm or other event involving the key metric. Another embodiment includes apparatus for correlating, grouping, and identifying the metrics.

Claims

1. A method for dynamically correlating a plurality of metrics to emphasize obscured relationships therebetween, the method comprising the steps of:

receiving data associated with the plurality of metrics;

receiving notification of a plurality of threshold violations associated with a plurality of out-of-tolerance metrics, the plurality of out-of-tolerance metrics being included in the plurality of metrics and further including a first out-of-tolerance metric and a remainder of out-of-tolerance metrics;

synchronizing the data associated with each out-of-tolerance metric; and

calculating, using the synchronized data, a synchronized correlation coefficient between the first out-of-tolerance metric and each of the remainder of out-of-tolerance metrics.

2. The method of claim 1 where the metrics relate to a computer network, and further comprising the step of using the synchronized correlation coefficient to assess performance in an e-commerce system.

3. The method of claim 1, further comprising the step of using the synchronized correlation coefficient to determine a cause of the plurality of threshold violations.

4. The method of claim 1 further comprising the steps of:

offsetting the data associated with each out-of-tolerance metric by an offset amount;

calculating, using the offset data, an offset correlation coefficient between the first out-of-tolerance metric and each of the remainder of out-of-tolerance metrics;

selecting, as a preferred correlation coefficient, one of the synchronized correlation coefficient and the offset correlation coefficient; and

revising the preferred correlation coefficient.

5. The method of claim 1 wherein the step of receiving notification further comprises the step of receiving notification of a frequency of the threshold violations.

6. The method of claim 1 wherein the data associated with each out-of-tolerance metric comprises historical data.

7. The method of claim 1 wherein the step of synchronizing the data further comprises the steps of:

arranging the data associated with each out-of-tolerance metric in to a time-ordered sequence;

aligning each time-ordered sequence along a common time scale, the common time scale including a plurality of time slots; and

determining the existence of missing data within at least one time slot.

8. The method of claim 7 further comprising the step of deleting all data within the at least one time slot exhibiting missing data.

9. The method of claim 7 further comprising the step of deleting data pairwise within the at least one time slot.

10. The method of claim 1 wherein the step of calculating a synchronized correlation coefficient further comprises the step of computing a correlation coefficient based at least in part on a rank correlation technique.

11. The method of claim 10 wherein the rank correlation technique comprises Spearman Rank-Order Correlation.

12. The method of claim 4 wherein the step of offsetting the data further comprises the steps of:

arranging the data associated with each out-of-tolerance metric in to a time-ordered sequence along a time scale having a plurality of time slots;

shifting the time-ordered sequence of data associated with the first out-of-tolerance metric by a number of time slots with respect to the time-ordered sequence of data associated with each of the remainder of out-of-tolerance metrics; and

determining the existence of missing data within at least one time slot.

13. The method of claim 12 further comprising the step of deleting all data within the at least one time slot exhibiting missing data.

14. The method of claim 12 further comprising the step of deleting data pairwise within the at least one time slot.

15. The method of claim 4 wherein the step of calculating an offset correlation coefficient further comprises the step of computing a correlation coefficient based at least in part on a rank correlation technique.

16. The method of claim 15 wherein the rank correlation technique comprises Spearman Rank-Order Correlation.

17. The method of claim 4 wherein the step of selecting the preferred correlation coefficient occurs only when the preferred correlation coefficient exceeds a predetermined value.

18. The method of claim 4 wherein the step of selecting the preferred correlation coefficient further comprises the steps of:

identifying a pair of out-of-tolerance metrics associated with the preferred correlation coefficient; and

storing the identity of the pair with the preferred correlation coefficient and the offset amount.

19. The method of claim 4 wherein the step of revising the preferred correlation coefficient further comprises the step of recalculating the preferred correlation coefficient in response to additional data associated with the out-of-tolerance metrics.

20. The method of claim 18 wherein the step of revising the preferred correlation coefficient further comprises the step of decreasing the preferred correlation coefficient in response to a lack of additional data supporting correlation.

21. The method of claim 20 wherein the step of decreasing the preferred correlation coefficient in response to a lack of additional data supporting correlation comprises the step of decreasing the correlation coefficient in response to a lack of additional data associated with the out-of-tolerance metrics over a predetermined time period.

22. The method of claim 20, further comprising the step of deleting the preferred correlation coefficient and the identity of the pair if the correlation coefficient falls below a predetermined threshold.

23. The method of claim 4, wherein the step of revising the preferred correlation coefficient further comprises the step of increasing the preferred correlation coefficient in response to additional data supporting correlation.

24. A method for identifying at least one metric associated with a key metric to emphasize an obscured relationship therebetween, the method comprising the steps of:

determining a plurality of potential metrics previously correlated with the key metric, the plurality of potential metrics including a first potential metric and a remainder of potential metrics; and

selecting as the at least one associated metric the first potential metric previously correlated to a predetermined percentage of the remainder of the plurality of potential metrics.

25. A method for identifying at least one metric associated with a key metric to emphasize an obscured relationship therebetween, the method comprising the steps of:

determining a plurality of potential metrics previously correlated with the key metric, the plurality of potential metrics including a first potential metric and a remainder of potential metrics; and

selecting as the at least one associated metric a metric previously correlated to a predetermined percentage of the plurality of potential metrics.

26. An article of manufacture comprising a program storage medium having computer readable program code embodied therein for dynamically correlating a plurality of metrics, the computer readable program code in the article of manufacture including:

computer readable code for receiving data associated with the plurality of metrics;

computer readable code for receiving notification of a plurality of threshold violations associated with a plurality of out-of-tolerance metrics, the plurality of out-of-tolerance metrics being included in the plurality of metrics and further including a first out-of-tolerance metric and a remainder of out-of-tolerance metrics;

computer readable code for synchronizing the data associated with each out-of-tolerance metric; and

computer readable code for calculating, using the synchronized data, a synchronized correlation coefficient between the first out-of-tolerance metric and each of the remainder of out-of-tolerance metrics, so as to achieve the dynamic correlation of the plurality of metrics.

27. A program storage medium readable by a computer, tangibly embodying a program of instructions executable by the computer to perform method steps for dynamically correlating a plurality of metrics, the method steps comprising:

receiving data associated with the plurality of metrics;

receiving notification of a plurality of threshold violations associated with a plurality of out-of-tolerance metrics, the plurality of out-of-tolerance metrics being included in the plurality of metrics and further including a first out-of-tolerance metric and a remainder of out-of-tolerance metrics;

synchronizing the data associated with each out-of-tolerance metric; and

calculating, using the synchronized data, a synchronized correlation coefficient between the first out-of-tolerance metric and each of the remainder of out-of-tolerance metrics, so as to achieve the dynamic correlation of the plurality of metrics.

28. An article of manufacture comprising a program storage medium having computer readable program code embodied therein for identifying at least one metric associated with a key metric, the computer readable program code in the article of manufacture including:

computer readable code for determining a plurality of potential metrics previously correlated with the key metric, the plurality of potential metrics including a first potential metric and a remainder of potential metrics; and

computer readable code for selecting as the at least one associated metric the first potential metric previously correlated to a predetermined percentage of the remainder of the plurality of potential metrics.

29. A program storage medium readable by a computer, tangibly embodying a program of instructions executable by the computer to perform method steps for identifying at least one metric associated with a key metric, the method steps comprising:

determining a plurality of potential metrics previously correlated with the key metric, the plurality of potential metrics including a first potential metric and a remainder of potential metrics; and

selecting as the at least one associated metric the first potential metric previously correlated to a predetermined percentage of the remainder of the plurality of potential metrics.

30. An article of manufacture comprising a program storage medium having computer readable program code embodied therein for identifying at least one metric associated with a key metric, the computer readable program code in the article of manufacture including:

computer readable code for determining a plurality of potential metrics previously correlated with the key metric, the plurality of potential metrics including a first potential metric and a remainder of potential metrics; and

computer readable code for selecting as the at least one associated metric a metric previously correlated to a predetermined percentage of the plurality of potential metrics.

31. A program storage medium readable by a computer, tangibly embodying a program of instructions executable by the computer to perform method steps for identifying at least one metric associated with a key metric, the method steps comprising:

determining a plurality of potential metrics previously correlated with the key metric, the plurality of potential metrics including a first potential metric and a remainder of potential metrics; and

selecting as the at least one associated metric a metric previously correlated to a predetermined percentage of the plurality of potential metrics.

32. A system for dynamically correlating a plurality of metrics, the system comprising:

means for receiving data associated with the plurality of metrics;

means for receiving notification of a plurality of threshold violations associated with a plurality of out-of-tolerance metrics, the plurality of out-of-tolerance metrics being included in the plurality of metrics and further including a first out-of-tolerance metric and a remainder of out-of-tolerance metrics;

means for synchronizing the data associated with each out-of-tolerance metric; and

means for calculating, using the synchronized data, a synchronized correlation coefficient between the first out-of-tolerance metric and each of the remainder of out-of-tolerance metrics.

33. A system for identifying at least one metric associated with a key metric, the system comprising:

means for determining a plurality of potential metrics previously correlated with the key metric, the plurality of potential metrics including a first potential metric and a remainder of potential metrics; and

means for selecting as the at least one associated metric the first potential metric previously correlated to a predetermined percentage of the remainder of the plurality of potential metrics.

34. A system for identifying at least one metric associated with a key metric, the system comprising:

means for determining a plurality of potential metrics previously correlated with the key metric, the plurality of potential metrics including a first potential metric and a remainder of potential metrics; and

means for selecting as the at least one associated metric a metric previously correlated to a predetermined percentage of the plurality of potential metrics.

35. Apparatus for dynamically correlating a plurality of metrics, the apparatus comprising:

a first receiver that receives data associated with the plurality of metrics;

a second receiver that receives notification of a plurality of threshold violations associated with a plurality of out-of-tolerance metrics, the plurality of out-of-tolerance metrics being included in the plurality of metrics and further including a first out-of-tolerance metric and a remainder of out-of-tolerance metrics;

a synchronizer that synchronizes the data associated with each out-of-tolerance metric; and

a calculator that calculates, using the synchronized data, a synchronized correlation coefficient between the first out-of-tolerance metric and each of the remainder of out-of-tolerance metrics.

36. The apparatus of claim 35 further comprising means for interfacing with at least one component of a computer network, the plurality of metrics being indicative of network performance.

37. Apparatus for identifying at least one metric associated with a key metric, the apparatus comprising:

a discriminator that determines a plurality of potential metrics previously correlated with the key metric, the plurality of potential metrics including a first potential metric and a remainder of potential metrics; and

a selector that selects as the at least one associated metric the first potential metric previously correlated to a predetermined percentage of the remainder of the plurality of potential metrics.

38. The apparatus of claim 37 further comprising means for interfacing with at least one component of a computer network, the key metric being indicative of network performance.

39. Apparatus for identifying at least one metric associated with a key metric, the apparatus comprising:

a discriminator that determines a plurality of potential metrics previously correlated with the key metric, the plurality of potential metrics including a first potential metric and a remainder of potential metrics; and

a selector that selects as the at least one associated metric a metric previously correlated to a predetermined percentage of the plurality of potential metrics.

40. The apparatus of claim 39 further comprising means for interfacing with at least one component of a computer network, the key metric being indicative of network performance.