System and methods for adaptive threshold determination for performance metrics

• US 20030079160A1
• Filed: 07/18/2002
• Published: 04/24/2003
• Est. Priority Date: 07/20/2001
• Status: Active Grant

• Current Assignee: Riverbed Technology, LLC (Riverbed Technology Incorporated)
• Original Assignee: Altaworks Corp. (Riverbed Technology Incorporated)
• Inventors: Matthews, James W., McGee, John, Earley, John Michael
• Granted Patent: US 7,076,695 B2
• Time in Patent Office: Days
• US Class Current: 714/39
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0772 Means for error signaling, e.g. using interrupts, exception flags, dedicated error registers
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/327 Alarm or error message display
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 17/15 Correlation function computation including computation of convolution operations arithmetic circuits for sum of products per se, e.g. multiply-accumulators G06F7/5443; digital filters, e.g. FIR, IIR, adaptive filters H03H17/00
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet

Abstract

A system and method for dynamically generating alarm thresholds for performance metrics, and for applying those thresholds to generate alarms is described. Statistical methods are used to generate one or more thresholds for metrics that may not fit a Gaussian or normal distribution, or that may exhibit cyclic behavior or persistent shifts in the values of the metrics. The statistical methods used to generate the thresholds may include statistical process control (SPC) methods, normalization methods, and heuristics.

Claims

1. A method for dynamically generating at least one metric threshold indicating alarm conditions in a monitored system, the method comprising the steps of:

establishing at least one default alarm threshold associated with a metric;

repeatedly receiving data associated with the metric;

statistically analyzing the received data to establish at least one updated alarm threshold; and

triggering an alarm on receipt of received data that violate the at least one updated alarm threshold.

2. The method of claim 1, wherein the metric relates to a computer network, and further comprising the step of using the alarm to assess performance in an e-commerce system.

3. The method of claim 1 further comprising the steps of (i) repeating the analysis step and (ii) adjusting the at least one updated alarm threshold based on previously established updated alarm limits.

4. The method of claim 1 wherein the step of statistically analyzing the received data further comprises the step of categorizing the received data as normal.

5. The method of claim 1 wherein the step of statistically analyzing the received data further comprises the step of categorizing the received data as normalizable.

6. The method of claim 1 wherein the step of statistically analyzing the received data further comprises the step of categorizing the received data as non-normal.

7. The method of claim 3 further comprising the steps of:

computing at least one value;

filtering the at least one value; and

equating the at least one updated alarm threshold to the at least one value.

8. The method of claim 4 wherein the step of categorizing the received data further comprises the step of applying a chi-square test to the received data.

9. The method of claim 4 wherein the step of categorizing the received data further comprises the step of applying an Anderson-Darling test to the received data.

10. The method of claim 5 wherein the step of categorizing the received data further comprises the steps of:

operating on the received data with a function representing the estimated cumulative distribution of the received data, producing a first result; and

operating on the first result with a quantile function of a normal distribution.

11. The method of claim 7 wherein the at least one value is computed using statistical process control techniques.

12. The method of claim 7 wherein the at least one value is computed using at least one heuristic technique.

13. The method of claim 12 wherein the at least one heuristic technique comprises at least one of quantile function and weighted linear regression techniques.

14. The method of claim 7 wherein the step of filtering further comprises computing a weighted sum of the received data.

15. The method of claim 14 wherein the received data comprises historical data.

16. The method of claim 14 wherein the received data comprises a statistical summarization of raw metric data.

17. The method of claim 14 wherein the received data are associated with at least one predetermined time period.

18. The method of claim 1 wherein the step of triggering an alarm further comprises the step of comparing the received data with a fixed threshold.

19. The method of claim 1 wherein the step of triggering an alarm further comprises the step of comparing the mean and standard deviation of the received data with the at least one updated alarm threshold.

20. The method of claim 1 wherein the step of triggering an alarm further comprises the steps of:

normalizing the received data; and

comparing the mean and standard deviation of the normalized received data with the at least one updated alarm threshold.

21. The method of claim 1 wherein the step of triggering an alarm further comprises the step of comparing the mean of the received data with the at least one updated alarm threshold.

22. An article of manufacture comprising a program storage medium having computer readable program code embodied therein for dynamically generating at least one metric threshold indicating alarm conditions in a monitored system, the computer readable program code in the article of manufacture including:

computer readable code for establishing at least one default alarm threshold associated with a metric;

computer readable code for repeatedly receiving data associated with the metric;

computer readable code for statistically analyzing the received data to establish at least one updated alarm threshold; and

computer readable code for triggering an alarm on receipt of received data that violate the at least one updated alarm threshold so as to achieve the dynamic generation of at least one metric threshold.

23. A program storage medium readable by a computer, tangibly embodying a program of instructions executable by the computer to perform method steps for dynamically generating at least one metric threshold indication alarm conditions in a monitored system, the method steps comprising:

establishing at least one default alarm threshold associated with a metric;

repeatedly receiving data associated with the metric;

statistically analyzing the received data to establish at least one updated alarm threshold; and

triggering an alarm on receipt of received data that violate the at least one updated alarm threshold so as to achieve the dynamic generation of at least one metric threshold.

24. A system for dynamically generating at least one metric threshold indicating alarm conditions in a monitored system, the system comprising:

means for establishing at least one default alarm threshold associated with a metric;

means for repeatedly receiving data associated with the metric;

means for statistically analyzing the received data to establish at least one updated alarm threshold; and

means for triggering an alarm on receipt of received data that violate the at least one updated alarm threshold.

25. Apparatus for dynamically generating at least one metric threshold indicating alarm conditions in a monitored system, the apparatus comprising:

a limit store that establishes at least one default alarm threshold associated with a metric;

a data manager that repeatedly receives data associated with the metric;

a threshold computation module that statistically analyzes the received data to establish at least one updated alarm threshold; and

an alarm manager that triggers an alarm on receipt of received data that violate the at least one updated alarm threshold.

26. The apparatus of claim 25 further comprising means for interfacing with at least one component of a computer network, the alarm conditions being indicative of network performance.