Method and apparatus for determining time varying thresholds for monitored metrics

• US 20040088406A1
• Filed: 10/31/2002
• Published: 05/06/2004
• Est. Priority Date: 10/31/2002
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Corley, Carole Rhoads, Johnson, Mark Wallace
• Granted Patent: US 8,635,328 B2
• Time in Patent Office: Days
• US Class Current: 709/224
• CPC Class Codes: H04L 41/0213 Standardised network management protocols, e.g. simple network management protocol [SNMP]
• CPC Class Codes: H04L 41/064 involving time analysis
• CPC Class Codes: H04L 41/147 for predicting network behaviour
• CPC Class Codes: H04L 43/16 Threshold monitoring

Abstract

A method and apparatus for determining time-varying thresholds for measured metrics are provided. With the method and apparatus, values of a given metric are captured over time. The behavior of the metric is analyzed to determine its seasonality. Correlated historical values of the metric and additional related metrics (cross-correlation) are used as inputs to a feed-forward back propagation neural network, in order to train the network to generalize the behavior of the metric. From this generalized behavior, point-by-point threshold values are calculated. The metric is monitored and the monitored values are compared with the threshold values to determine if the metric has violated its normal time-varying behavior. If so, an event is generated to notify an administrator of the error condition.

Claims

1. A method of monitoring the performance of a computing system, comprising:

generating a series of predictions of at least one performance parameter for a series of future time points;

determining a time-varying threshold value based on the series of predictions of the at least one performance parameter, wherein the time-varying threshold value is adjusted from one threshold value to another in correspondence with the series of predictions of the performance parameter;

measuring a value of the at least one performance parameter at a future time point; and

comparing the measured value to a value of the time-varying threshold value at the future time point to determine if a potential error condition is present.

2. The method of claim 1, wherein generating a series of predictions of at least one performance parameter includes:

training a neural network to generate a prediction of the at least one performance parameter;

providing inputs to the neural network; and

generating the series of predictions of the at least one performance parameter based on the inputs provided.

3. The method of claim 2, wherein generating the series of predictions of the at least one performance parameter further includes:

analyzing a time series of values for the at least one performance parameter to determine a seasonality of the time series of values; and

determining the inputs to be provided to the neural network based on the seasonality of the time series of values.

4. The method of claim 2, wherein generating the series of predictions of the at least one performance parameter further includes:

analyzing a time series of values for the at least one performance parameter to determine an autocorrelation of the time series of values; and

determining the inputs to be provided to the neural network based on the autocorrelation of the time series of values.

5. The method of claim 2, wherein generating the series of predictions of the at least one performance parameter further includes:

analyzing a time series of values of at least one other performance parameter to determine a cross-correlation of the time series of values of the at least one other performance parameter with the time series of values of the at least one performance parameter; and

determining the inputs to be provided to the neural network based on the cross-correlation.

6. The method of claim 1, wherein determining a time-varying threshold value based on the series of predictions of the at least one performance parameter includes calculating the time-varying threshold based on a standard deviation of a residual error of a prediction of the at least one performance parameter.

7. The method of claim 1, wherein determining a time-varying threshold value based on the series of predictions of the at least one performance parameter includes calculating a threshold magnitude based on the following formula and adding or subtracting the magnitude to or from a prediction of the at least one performance parameter:

T_t=A*(((e_t−

u)²+(e<sub>t−

1</sub>−

u)²)/2)^1/2+B*w

wherein T_t is a magnitude of a threshold at time t, A is a constant, e_t is a residual error at time t, e<sub>t−

1</sub> is a residual error at time t−

1, u is a mean error for a time period in which the threshold is being calculated, B is a constant, and w is a mean prediction value for the time period in which the threshold is being calculated.

8. The method of claim 1, further comprising:

storing the time-varying threshold value in a lookup table indexed by timestamp, wherein comparing the measured value to the value of the time-varying threshold includes performing a lookup of the value of the time-varying threshold in the lookup table using a current timestamp.

9. The method of claim 1, further comprising:

determining, based on results of comparing the measured value to the value of the time-varying threshold, whether to send a notification to a user; and

sending the notification to the user if it is determined that a notification should be sent.

10. The method of claim 1, wherein generating a series of predictions of the at one performance parameter further includes:

analyzing a time series of values for the at least one performance parameter to determine a seasonality of the time series of values;

analyzing the time series of values for the at least one performance parameter based on the determined seasonality of the time series of values, to determine an autocorrelation of the time series of values;

analyzing a time series of values of at least one other performance parameter based on the determined seasonality of the time series of values, to determine a cross-correlation of the time series of values of the at least one other performance parameter with the time series of values of the at least one performance parameter; and

determining the inputs to be provided to the neural network based on the autocorrelation and cross-correlation.

11. A computer program product in a computer readable medium for monitoring the performance of a computing system, comprising:

first instructions for generating a series of predictions of at least one performance parameter for a series of future time points;

second instructions for determining a time-varying threshold value based on the series of predictions of the at least one performance parameter, wherein the time-varying threshold value is adjusted from one threshold value to another in correspondence with the series of predictions of the performance parameter;

third instructions for measuring a value of the at least one performance parameter at a future time point; and

fourth instructions for comparing the measured value to a value of the time-varying threshold value at the future time point to determine if a potential error condition is present.

12. The computer program product of claim 11, wherein the first instructions for generating a series of predictions of at least one performance parameter include:

instructions for training a neural network to generate a prediction of the at least one performance parameter;

instructions for providing inputs to the neural network; and

instructions for generating the series of predictions of the at least one performance parameter based on the inputs provided.

13. The computer program product of claim 12, wherein the first instructions for generating the series of predictions of the at least one performance parameter further include:

instructions for analyzing a time series of values for the at least one performance parameter to determine a seasonality of the time series of values; and

instructions for determining the inputs to be provided to the neural network based on the seasonality of the time series of values.

14. The computer program product of claim 12, wherein the first instructions for generating the series of predictions of the at least one performance parameter further include:

instructions for analyzing a time series of values for the at least one performance parameter to determine an autocorrelation of the time series of values; and

instructions for determining the inputs to be provided to the neural network based on the autocorrelation of the time series of values.

15. The computer program product of claim 12, wherein the first instructions for generating the series of predictions of the at least one performance parameter further include:

instructions for analyzing a time series of values of at least one other performance parameter to determine a cross-correlation of the time series of values of the at least one other performance parameter with the time series of values of the at least one performance parameter; and

instructions for determining the inputs to be provided to the neural network based on the cross-correlation.

16. The computer program product of claim 11, wherein the second instructions for determining a time-varying threshold value based on the series of predictions of the at least one performance parameter include instructions for calculating the time-varying threshold based on a standard deviation of a residual error of a prediction of the at least one performance parameter.

17. The computer program product of claim 11, wherein the second instructions for determining a time-varying threshold value based on the series of predictions of the at least one performance parameter include instructions for calculating a threshold magnitude based on the following formula and adding or subtracting the magnitude to or from a prediction of the at least one performance parameter:

T_t=A*(((e_t−

u)²+(e<sub>t−

1</sub>−

u)²)/2)^1/2+B*w

wherein T_t is a magnitude of a threshold at time t, A is a constant, e_t is a residual error at time t, e<sub>t−

1</sub> is a residual error at time t−

1, u is a mean error for a time period in which the threshold is being calculated, B is a constant, and w is a mean prediction value for the time period in which the threshold is being calculated.

18. The computer program product of claim 11, further comprising:

fifth instructions for storing the time-varying threshold value in a lookup table indexed by timestamp, wherein comparing the measured value to the value of the time-varying threshold includes performing a lookup of the value of the time-varying threshold in the lookup table using a current timestamp.

19. The computer program product of claim 11, further comprising:

fifth instructions for determining, based on results of comparing the measured value to the value of the time-varying threshold, whether to send a notification to a user; and

sixth instructions for sending the notification to the user if it is determined that a notification should be sent.

20. The computer program product of claim 11, wherein the first instructions for generating a series of predictions of the at one performance parameter further include:

instructions for analyzing a time series of values for the at least one performance parameter to determine a seasonality of the time series of values;

instructions for analyzing the time series of values for the at least one performance parameter based on the determined seasonality of the time series of values, to determine an autocorrelation of the time series of values;

instructions for analyzing a time series of values of at least one other performance parameter based on the determined seasonality of the time series of values, to determine a cross-correlation of the time series of values of the at least one other performance parameter with the time series of values of the at least one performance parameter; and

instructions for determining the inputs to be provided to the neural network based on the autocorrelation and cross-correlation.

21. An apparatus for monitoring the performance of a computing system, comprising:

means for generating a series of predictions of at least one performance parameter for a series of future time points;

means for determining a time-varying threshold value based on the series of predictions of the at least one performance parameter, wherein the time-varying threshold value is adjusted from one threshold value to another in correspondence with the series of predictions of the performance parameter;

means for measuring a value of the at least one performance parameter at a future time point; and

means for comparing the measured value to a value of the time-varying threshold value at the future time point to determine if a potential error condition is present.