Automated and adaptive threshold setting

• US 20060276995A1
• Filed: 09/28/2005
• Published: 12/07/2006
• Est. Priority Date: 06/07/2005
• Status: Active Grant

• Current Assignee: ServiceNow Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Breitgand, David, Shehory, Onn, Henis, Ealan
• Granted Patent: US 8,086,708 B2
• Time in Patent Office: Days
• US Class Current: 702/179
• CPC Class Codes: H04L 41/00 Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
• CPC Class Codes: H04L 41/147 for predicting network behaviour
• CPC Class Codes: H04L 41/149 for prediction of maintenance
• CPC Class Codes: H04L 41/5003 Managing SLA; Interaction between SLA and QoS
• CPC Class Codes: H04L 41/5009 Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 43/16 Threshold monitoring

Abstract

A method for managing a computer system includes monitoring first violations of a service level objective (SLO) of a service running on the computer system so as to determine a first statistical behavior of the first violations. Second violations of a component performance threshold of a component of the computer system are monitored so as to determine a second statistical behavior of the second violations. A model that predicts the second statistical behavior based on the first statistical behavior is produced. The component performance threshold is automatically adjusted responsively to the model, so as to improve a prediction of the first violations by the second violations.

Claims

1. A method for managing a computer system, comprising:

monitoring first violations of a service level objective (SLO) of a service running on the computer system so as to determine a first statistical behavior of the first violations;

monitoring second violations of a component performance threshold of a component of the computer system so as to determine a second statistical behavior of the second violations;

producing a model that predicts the second statistical behavior based on the first statistical behavior; and

automatically adjusting the component performance threshold responsively to the model, so as to improve a prediction of the first violations by the second violations.

2. The method according to claim 1, wherein the computer system comprises a Storage Area Network (SAN).

3. The method according to claim 1, wherein monitoring the first and second violations comprises estimating false-positive and false-negative rates of the second violations with respect to the first violations.

4. The method according to claim 3, wherein automatically adjusting the threshold comprises causing the estimated false-positive and false-negative rates to converge to predetermined target values.

5. The method according to claim 3, wherein automatically adjusting the threshold comprises searching for a threshold value that minimizes the estimated false-positive and false-negative rates.

6. The method according to claim 1, wherein producing the model comprises fitting a first sequence comprising historical values of the first violations and a second sequence comprising historical values of the component performance threshold with a probability of component performance threshold violations, and wherein automatically adjusting the component performance threshold comprises calculating an updated threshold value based on the fitted sequences.

7. The method according to claim 6, wherein fitting the sequences comprises applying at least one of a logistic regression process and a polynomial fitting process.

8. The method according to claim 6, wherein producing the model comprises estimating a fitting quality responsively to the model and the sequences, and wherein automatically adjusting the threshold comprises determining whether to update the threshold responsively to the fitting quality.

9. The method according to claim 6, wherein fitting the sequences comprises inserting dummy data points into the sequences, so as to reduce a bias in the calculated updated threshold value.

10. The method according to claim 6, wherein fitting the sequences comprises using only a part of the historical values in the sequences corresponding to at least one of recent events, periodic events, events in which the SLO is almost violated and events in which a performance metric value is within a predetermined interval.

11. The method according to claim 6, wherein fitting the sequences comprises applying weights to at least some of the historical values corresponding to at least one of recent events, periodic events, rare events, events in which the SLO is almost violated and events in which a performance metric value is within a predetermined interval.

12. Apparatus for managing a computer system, comprising:

an interface, which is coupled to receive inputs indicative of first violations of a service level objective (SLO) of a service running on the computer system and of second violations of a component performance threshold of a component of the computer system; and

a processor, which is arranged to determine a first statistical behavior of the first violations and a second statistical behavior of the second violations, to produce a model that predicts the second statistical behavior based on the first statistical behavior, and to automatically adjust the component performance threshold responsively to the model, so as to improve a prediction of the first violations by the second violations.

13. The apparatus according to claim 12, wherein the computer system comprises a Storage Area Network (SAN).

14. The apparatus according to claim 12, wherein the processor is arranged to estimate false-positive and false-negative rates of the second violations with respect to the first violations.

15. The apparatus according to claim 14, wherein the processor is arranged to adjust the threshold so as to cause the estimated false-positive and false-negative rates to converge to predetermined target values.

16. The apparatus according to claim 14, wherein the processor is arranged to search for a threshold value that minimizes the estimated false-positive and false-negative rates.

17. The apparatus according to claim 12, wherein the processor is arranged to fit a first sequence comprising historical values of the first violations and a second sequence comprising historical values of the component performance threshold with a probability of component performance threshold violations in order to produce the model, and to calculate an updated threshold value based on the fitted sequences.

18. The apparatus according to claim 17, wherein the processor is arranged to apply at least one of a logistic regression process and a polynomial fitting process in order to fit the sequences.

19. The apparatus according to claim 17, wherein the processor is arranged to estimate a fitting quality responsively to the model and the sequences, and to determine whether to update the threshold responsively to the fitting quality.

20. The apparatus according to claim 17, wherein the processor is arranged to insert dummy data points into the sequences when producing the model, so as to reduce a bias in the calculated updated threshold value.

21. The apparatus according to claim 17, wherein the processor is arranged to produce the model using only a part of the historical values in the sequences corresponding to at least one of recent events, periodic events, events in which the SLO is almost violated and events in which a performance metric value is within a predetermined interval.

22. The apparatus according to claim 17, wherein the processor is arranged to apply weights to at least some of the historical values corresponding to at least one of recent events, periodic events, rare events, events in which the SLO is almost violated and events in which a performance metric value is within a predetermined interval.

23. A computer software product for managing a computerized system, the product comprising a computer-readable medium, in which program instructions are stored, which instructions, when read by a computer, cause the computer to monitor first violations of a service level objective (SLO) of a service running on the computerized system so as to determine a first statistical behavior of the first violations, to monitor second violations of a component performance threshold of a component of the computerized system so as to determine a second statistical behavior of the second violations, to produce a model that predicts the second statistical behavior based on the first statistical behavior, and to automatically adjust the component performance threshold responsively to the model, so as to improve a prediction of the first violations by the second violations.

24. The product according to claim 23, wherein the computerized system comprises a Storage Area Network (SAN).

25. The product according to claim 23, wherein the instructions cause the computer to estimate false-positive and false-negative rates of the second violations with respect to the first violations, and to adjust the threshold so as to cause the estimated false-positive and false-negative rates to converge to predetermined target values.

26. The product according to claim 25, wherein the instructions cause the computer to search for a threshold value that minimizes the estimated false-positive and false-negative rates.

27. The product according to claim 23, wherein the instructions cause the computer to fit a first sequence comprising historical values of the first violations and a second sequence comprising historical values of the component performance threshold with a probability of component performance threshold violations in order to produce the model, and to calculate an updated threshold value based on the fitted sequences.

28. The product according to claim 27, wherein the instructions cause the computer to apply at least one of a logistic regression process and a polynomial fitting process in order to fit the sequences.

29. The product according to claim 27, wherein the instructions cause the computer to produce the model using only a part of the historical values in the sequences corresponding to at least one of recent events, periodic events, events in which the SLO is almost violated and events in which a performance metric value is within a predetermined interval.

30. The product according to claim 27, wherein the instructions cause the computer to apply weights to at least some of the historical values corresponding to at least one of recent events, periodic events, rare events, events in which the SLO is almost violated and events in which a performance metric value is within a predetermined interval.

31. A method for performing an interactive analysis of a computer system to devise an information technology solution applicable to the computer system, the method comprising:

monitoring first violations of a service level objective (SLO) of a service running on the computer system so as to determine a first statistical behavior of the first violations;

monitoring second violations of a component performance threshold of a component of the computer system so as to determine a second statistical behavior of the second violations;

producing a model that predicts the second statistical behavior based on the first statistical behavior; and

automatically adjusting the component performance threshold responsively to the model, so as to improve a prediction of the first violations by the second violations.

32. The method according to claim 31, wherein the computer system comprises a Storage Area Network (SAN).

33. The method according to claim 31, wherein monitoring the first and second violations comprises estimating false-positive and false-negative rates of the second violations with respect to the first violations, and wherein automatically adjusting the threshold comprises causing the estimated false-positive and false-negative rates to converge to predetermined target values.

34. The method according to claim 33, wherein automatically adjusting the threshold comprises searching for a threshold value that minimizes the estimated false-positive and false-negative rates.

35. The method according to claim 31, wherein producing the model comprises fitting a first sequence comprising historical values of the first violations and a second sequence comprising historical values of the component performance threshold with a probability of component performance threshold violations, and wherein automatically adjusting the component performance threshold comprises calculating an updated threshold value based on the fitted sequences.