Predictive monitoring and problem identification in an information technology (IT) infrastructure

• US 7,107,339 B1
• Filed: 03/29/2002
• Issued: 09/12/2006
• Est. Priority Date: 04/07/2001
• Status: Expired due to Term

• Current Assignee: Software AG USA Incorporated (Software AG)
• Original Assignee: WebMethods Incorporated (Software AG)
• Inventors: Wolters, Timothy Jerome
• Primary Examiner(s): Jaroenchonwanit, Bunjob
• Assistant Examiner(s): Chang, Julian
• Application Number:
  US10/112,015
• Time in Patent Office: 1,628 Days
• Field of Search: 709/224, 709/223, 345/763, 714/47, 714/33
• US Class Current: 709/224
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 41/147 for predicting network behaviour
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 43/0888 Throughput

Abstract

An information technology (IT) infrastructure may be monitored, and the data thus collected may be used to infer problems and predict future conditions. In one embodiment, the present invention may include receiving a plurality of component metrics, each component metric related to a corresponding component of an (IT) infrastructure of an enterprise, each component being associated with one or more gross-level rules, and generating an indicator set by comparing each received component metric to relevant historical values of the component metric. In one embodiment, the present invention may also include determining that a gross-level rule is out of compliance, comparing the indicator set to one or more historical indicator sets to determine whether the indicator set resembles any of the one or more historical indicator sets, and performing an appropriate action based on the result of the comparison.

Claims

1. A method comprising:

receiving a plurality of component metrics, each component metric corresponding to a component of a plurality of components of an information technology (IT) infrastructure of an enterprise, and each component of the plurality of components being associated with one or more gross-level rules;

generating an indicator set representative of a current state of the IT infrastructure by comparing each received component metric to relevant historical values of the component metric;

using the indicator set to diagnose an existing condition or predict a future condition within the IT infrastructure;

determining that a gross-level rule is out of compliance;

comparing the indicator set to one or more historical indicator sets to determine whether the indicator set resembles any of the one or more historical indicator sets;

performing an appropriate action based on the result of the comparison; and

wherein, if the indicator set resembles at least one of historical indicator set, performing the appropriate action comprises inferring a problem associated with the at least one historical indicator set.

2. The method of claim 1, further comprising:

receiving information confirming whether the inferred problem has occurred; and

adjusting a plurality of weights used for comparing the indicator set to one or more historical indicator sets, each weight being associated with one component indicator of the indicator set, based on the received information.

3. The method of claim 1, wherein, if the indicator set does not resemble any of the one or more historical indicator sets, performing the appropriate action comprises:

storing the indicator set; and

associating the indicator set with a problem.

4. The method of claim 1, wherein the comparing the indicator set to one or more historical indicator sets comprises comparing a plurality of component indicators of the indicator set with a plurality of component indicators of each historical indicator set using a plurality of weights.

5. The method of claim 4, wherein the indicator set resembles any of the one or more historical indicator sets if a critical number of component indicators are outside a historical range in both the indicator set and any one of the one or more historical indicator sets.

6. The method of claim 5, wherein the critical number of component indicators have a threshold weight associated with each component indicator.

7. The method of claim 1, wherein performing the appropriate action comprises predicting that the gross-level rule may be out of compliance in the future if the indicator set resembles at least one of the one or more historical indicator sets, the at least on historical indicator set being associated with the gross-level rule being out of compliance.

8. The method of claim 1, wherein performing the appropriate action comprises predicting that a condition may occur in the future if the indicator set resembles at least one of the one or more historical indicator sets, the at least one historical indicator set being associated with the condition.

9. The method of claim 1, wherein the plurality of component metrics are synchronized in time.

10. The method of claim 1, wherein the plurality of component metrics are synchronized in time with the determination that the gross-level rule is out of compliance.

11. The method of claim 1, wherein comparing each component metric to relevant historical values of the component metric comprises calculating a measure of normal variation using the relevant historical values.

12. The method of claim 11, wherein the measure of normal variation comprises one or more of a standard deviation, percent deviation, from-mean deviation, trend analysis.

13. The method of claim 1, wherein the relevant historical values of the component metric depend on at least one of a time of a day, a day of a week, a period of a month, a period of a year.

14. The method of claim 1, wherein generating the indicator set comprises:

calculating a plurality of component statistics by comparing each component metric to relevant historical values of the component metric; and

generating the indicator set by calculating a set of component indicators by normalizing each calculated statistic.

15. A machine-readable medium having stored thereon data representing instructions that, when executed by a processor, cause the processor to perform operations comprising:

receiving a plurality of component metrics, each component metric corresponding to a component of a plurality of components of an information technology (IT) infrastructure of an enterprise, and each component of the plurality of components being associated with one or more gross-level rules;

generating an indicator set representative of a current state of the IT infrastructure by comparing each received component metric to relevant historical values of the component metric;

using the indicator set to diagnose an existing condition or predict a future condition within the IT infrastructure;

determining that a gross-level rule is out of compliance;

comparing the indicator set to one or more historical indicator sets to determine whether the indicator set resembles any of the one or more historical indicator sets;

performing an appropriate action based on the result of the comparison; and

wherein, if the indicator set resembles at least one of historical indicator set, performing the appropriate action comprises inferring a problem associated with the at least one historical indicator set.

16. The machine-readable medium of claim 15, wherein the instructions further cause the processor to perform operations comprising:

receiving information confirming whether the inferred problem has occurred; and

adjusting a plurality of weights used for comparing the indicator set to one or more historical indicator sets, each weight being associated with one component indicator of the indicator set, based on the received information.

17. The machine-readable medium of claim 15, wherein, if the indicator set does not resemble any of the one or more historical indicator sets, performing the appropriate action comprises:

storing the indicator set; and

associating the indicator set with a problem.

18. The machine-readable medium of claim 15, wherein the comparing the indicator set to one or more historical indicator sets comprises comparing a plurality of component indicators of the indicator set with a plurality of component indicators of each historical indicator set using a plurality of weights.

19. The machine-readable medium of claim 18, wherein the indicator set resembles any of the one or more historical indicator sets if a critical number of component indicators are outside a historical range in both the indicator set and any one of the one or more historical indicator sets.

20. The machine-readable medium of claim 19, wherein the critical number of component indicators have a threshold weight associated with each component indicator.

21. The machine-readable medium of claim 15, wherein performing the appropriate action comprises predicting that the gross-level rule may be out of compliance in the future if the indicator set resembles at least one of the one or more historical indicator sets, the at least on historical indicator set being associated with the gross-level rule being out of compliance.

22. The machine-readable medium of claim 15, wherein performing the appropriate action comprises predicting that a condition may occur in the future if the indicator set resembles at least one of the one or more historical indicator sets, the at least one historical indicator set being associated with the condition.

23. The machine-readable medium of claim 15, wherein the plurality of component metrics are synchronized in time.

24. The machine-readable medium of claim 15, wherein the plurality of component metrics are synchronized in time with the determination that the gross-level rule is out of compliance.

25. The machine-readable medium of claim 15, wherein comparing each component metric to relevant historical values of the component metric comprises calculating a measure of normal variation using the relevant historical values.

26. The machine-readable medium of claim 25, wherein the measure of normal variation comprises one or more of a standard deviation, percent deviation, from-mean deviation, trend analysis.

27. The machine-readable medium of claim 15, wherein the relevant historical values of the component metric depend on at least one of a time of a day, a day of a week, a period of a month, a period of a year.

28. The machine-readable medium of claim 15, wherein generating the indicator set comprises:

calculating a plurality of component statistics by comparing each component metric to relevant historical values of the component metric; and

generating the indicator set by calculating a set of component indicators by normalizing each calculated statistic.

29. An information technology (IT) infrastructure monitoring apparatus comprising:

a storage medium having stored thereon one or more routines configured to receive a plurality of component metrics each corresponding to a component of a plurality of components of an information technology (IT) infrastructure of an enterprise, generate an indicator set representative of a current state of the IT infrastructure, and use the indicator set to diagnose an existing condition or predict a future condition within the IT infrastructure; and

a processor coupled to the storage medium to execute one or more routines, where;

each component of the plurality of components are associated with one or more gross-level rules;

the indicator set is generated by comparing each received component metric to relevant historical values of the component metric;

determine that a gross-level rule is out of compliance;

compare the indicator set to one or more historical indicator sets to determine whether the indicator set resembles any of the one or more historical indicator sets;

perform an appropriate action based on the result of the comparison; and

wherein, if the indicator set resembles at least one of historical indicator set, performing the appropriate action comprises inferring a problem associated with the at least one historical indicator set.

30. The apparatus of claim 29, wherein the one or more routines are further configured to:

receive information confirming whether the inferred problem has occurred; and

adjust a plurality of weights used for comparing the indicator set to one or more historical indicator sets, each weight being associated with one component indicator of the indicator set, based on the received information.

31. The apparatus of claim 29, wherein, if the indicator set does not resemble any of the one or more historical indicator sets, performing the appropriate action comprises:

storing the indicator set; and

associating the indicator set with a problem.

32. The apparatus of claim 29, wherein the comparing the indicator set to one or more historical indicator sets comprises comparing a plurality of component indicators of the indicator set with a plurality of component indicators of each historical indicator set using a plurality of weights.

33. The apparatus of claim 32, wherein the indicator set resembles any of the one or more historical indicator sets if a critical number of component indicators are outside a historical range in both the indicator set and any one of the one or more historical indicator sets.

34. The apparatus of claim 33, wherein the critical number of component indicators further have greater than or equal to a threshold weight associated with each component indicator.

35. The apparatus of claim 29, wherein performing the appropriate action comprises predicting that the gross-level rule may be out of compliance in the future if the indicator set resembles at least one of the one or more historical indicator sets, the at least on historical indicator set being associated with the gross-level rule being out of compliance.

36. The apparatus of claim 29, wherein performing the appropriate action comprises predicting that a condition may occur in the future if the indicator set resembles at least one of the one or more historical indicator sets, the at least one historical indicator set being associated with the condition.

37. The apparatus of claim 29, wherein the plurality of component metrics are synchronized in time.

38. The apparatus of claim 29, wherein the plurality of component metrics are synchronized in time with the determination that the gross-level rule is out of compliance.

39. The apparatus of claim 29, wherein comparing each component metric to relevant historical values of the component metric comprises calculating a measure of normal variation using the relevant historical values.

40. The apparatus of claim 39, wherein the measure of normal variation comprises one or more of a standard deviation, percent deviation, from-mean deviation, trend analysis.

41. The apparatus of claim 29, wherein the relevant historical values of the component metric depend on at least one of a time of a day, a day of a week, a period of a month, a period of a year.

42. The apparatus of claim 29, wherein generating the indicator set comprises:

calculating a plurality of component statistics by comparing each component metric to relevant historical values of the component metric; and

generating the indicator set by calculating a set of component indicators by normalizing each calculated statistic.

43. A method comprising:

maintaining a database of a plurality of historical fingerprints, each of the plurality of historical fingerprints corresponding to an out-of-compliance condition of a gross-level rule of a plurality of gross-level rules, each of the plurality of gross-level rules representing an expected or desired functioning of an information technology (IT) infrastructure of an enterprise, each of the plurality of historical fingerprints associated with one or more of;

an indication of one or more problems that existed in the IT infrastructure when the historical fingerprint was generated and that were believed to have been responsible for or contributed to the corresponding out-of-compliance condition of the gross-level rule, and

an indication of one or more fixes or solutions that were performed during the corresponding out-of-compliance condition of the gross-level rule and that were believed to have been responsible for or contributed to bringing the gross-level rule back into compliance;

determining that a gross-level rule of the plurality of gross-level rules is out of compliance;

responsive to said determining that a gross-level rule of a plurality of gross-level rules is out of compliance, generating a current fingerprint for the out-of-compliance gross-level rule by gathering data from a plurality of component monitors regarding a plurality of components of the IT infrastructure, each of the plurality of components being associated with the out-of-compliance gross-level rule; and

inferring a problem, a fix or a solution associated with the out-of-compliance gross-level rule by comparing the current fingerprint to the plurality of historical fingerprints and identifying a historical fingerprint of the plurality of historical fingerprints that sufficiently resembles the current fingerprint by performing a weighted comparison of a plurality of component indicators of the current fingerprint and a plurality of component indicators of one or more of the plurality of historical fingerprints.

44. The method of claim 43, further comprising responsive to receiving confirmation regarding correctness of the inferred problem, the inferred fix or the inferred solution, increasing the statistical significance of one or more component indicators by adjusting weightings of the component indicators within the identified historical fingerprint.

45. The method of claim 43, further comprising responsive to receiving confirmation regarding incorrectness of the inferred problem, the inferred fix or the inferred solution, decreasing the statistical significance of one or more component indicators by adjusting weightings of the component indicators within the identified historical fingerprint.

46. The method of claim 43, further comprising causing information regarding the inferred problem, fix or solution to be conveyed to a technician having responsibility for maintaining the IT infrastructure.

47. The method of claim 43, further comprising programmatically taking corrective action to remedy the inferred problem.