System and method for generating performance models of complex information technology systems

• US 6,311,175 B1
• Filed: 03/06/1998
• Issued: 10/30/2001
• Est. Priority Date: 03/06/1998
• Status: Expired due to Term

• Current Assignee: NTT Data Services Corporation (Nippon Telegraph and Telephone Corporation)
• Original Assignee: Perot Systems Corporation (Dell Technologies Inc.)
• Inventors: Adriaans, Pieter Willem, Gathier, Marc, Knobbe, Arno Jan
• Primary Examiner(s): Davis, George B.
• Assistant Examiner(s): Holmes, Michael B.
• Application Number:
  US09/036,393
• Time in Patent Office: 1,334 Days
• Field of Search: 706/25
• US Class Current: 706/25
• CPC Class Codes: G06F 11/3447 Performance evaluation by modeling
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/81 Threshold

Abstract

A system and method for automatically creating performance models of complex information technology (IT) systems. System components and elements are subject to periodic monitoring associated with performance thresholds. A continuity analysis is performed by synchronizing testing functions associated with the predetermined system performance thresholds. Resulting data is accumulated and data mined for component and functional relations within the IT system. Models of the system may then be adapted with results generated from the analysis.

Claims

1. In an information technology system having a multiplicity of interconnected nodes, a method for optimizing performance monitoring of said system, said method comprising the steps of:

(a) performing a continuity analysis on said system;

(b) automatically generating a plurality of performance models of said system based on periodic measurements of predefined performance levels;

(c) continuously monitoring, at a plurality of said nodes, the performance of said system at the respective plurality of said nodes;

(d) collecting, periodically, performance data on said system at said respective nodes;

(e) applying a plurality of data mining techniques to said periodically collected system performance data and its associated test program data;

(f) generating a decision tree using said periodically collected system performance data, said decision tree having a multiplicity of decision nodes, each said decision node corresponding to a component of said system;

(g) comparing a plurality of relationships within said system between said system performance data and said test program data;

(h) automatically modifying said steps of continuously monitoring and periodically collecting said system performance data at a plurality of said nodes, whereby said autonomous modification iteratively optimizes said continuous performance monitoring of said system; and

(i) automatically updating an adaptive system model according to newly discovered relationships.

2. The method according to claim 1, wherein said steps (a)-(i) are repeated a plurality of times, whereby said continuous performance monitoring of said system is further optimized.

3. The method according to claim 1, further comprising, prior to said steps of continuous monitoring and periodic collecting, the step of:

generating a test program pursuant to at least one service level agreement, said plurality of nodes for continuous monitoring and periodic performance data collection being selected pursuant to said at least one service level agreement.

4. The method according to claim 3, wherein said test program targets a target component within said system, said target component being selected from the group consisting of a system hardware resource and a system software application.

5. The method according to claim 4, wherein said target component substantially corresponds to a root decision node of said decision tree.

6. The method according to claim 4, wherein said target component targeted by said test program is an underperforming system component, whereby said step of automatic modifying modifies said steps of continuously monitoring and periodically collecting said performance data on said underperforming system component.

7. The method according to claim 1, wherein said step of automatic modifying modifies the periodicity of said continuous monitoring and periodic collection of said performance data.

8. The method according to claim 7, wherein said periodicity increases after said modification.

9. The method according to claim 7, wherein said periodicity decreases after said modification.

10. The method according to claim 1, further comprising the step of:

storing said performance data.

11. The method according to claim 10, wherein said performance data is stored with an associated time stamp.

12. The method according to claim 10, wherein said performance data is stored in a relational database.

13. The method according to claim 1, wherein said step of generating said decision tree comprises decision tree induction.

14. The method according to claim 1, wherein said performance data comprises a respective plurality of state information at said plurality of said nodes.

15. The method according to claim 14, wherein said performance data further comprises a respective plurality of system information at said plurality of said nodes.

16. The method according to claim 15, wherein said system information comprises a plurality of service level agreements.

17. The method according to claim 1, wherein said performance data in said step of periodic collecting is collected periodically at specific time intervals.

18. The method according to claim 17, wherein said specific time intervals for periodically collecting said performance data is selected from the group consisting of days, hours, minutes and seconds.

19. The method according to claim 1, wherein said performance data periodically collected in said step of periodic collecting has a value selected from the group consisting of real numbers, integers and Booleans.

20. The method according to claim 1, wherein said performance data periodically collected in said step of periodic collecting is converted to a plurality of Boolean values.

21. The method according to claim 20, wherein said plurality of Boolean values correspond to a plurality of performance threshold conditions.

22. The method according to claim 21, wherein said performance threshold conditions are predetermined.

23. The method according to claim 21, wherein said performance threshold conditions are variable.

24. The method according to claim 1, wherein said performance data periodically collected in said step of periodic collecting is averaged at specific time intervals.

25. The method according to claim 24, wherein said averaged performance data is converted to at least one Boolean value.

26. The method according to claim 25, wherein said at least one Boolean value corresponds to at least one service level agreement within said system.

27. The method according to claim 1, wherein in said step of generating, regression analysis is performed on at least one target component corresponding to a target node of said decision tree, whereby the performance of said at least one target component is predicted at a future time from a plurality of parameter data within a plurality of said decision nodes.

57. A program storage device readable by a machine and encoding a program of instructions for executing the method steps of claim 1.

58. The method according to claim 1, wherein said newly discovered relationships are uncovered or unexpected relations.

28. An information technology system having a multiplicity of interconnected nodes, said system comprising:

performance means for performing a continuity analysis on said system;

generating means for automatically generating a plurality of performance models of said system based on periodic measurements of predefined performance levels;

monitor means, for continuously monitoring, at a plurality of said nodes, the performance of said system at the respective nodes;

collection means for periodically collecting, at said plurality of said nodes, performance data for said system at specific time intervals;

data mining technique application means for applying a plurality of data mining techniques to said collected system performance data and its associated test program data;

decision tree generation means for generating a decision tree using said periodically collected system performance data, said decision tree having a multiplicity of decision nodes, each said decision node corresponding to a component of said system;

comparison means for comparing a plurality of relationships within said system between said system performance data and said test program data;

modification means for automatically modifying said monitor and collection means for the continuous monitoring and periodic collection, respectively, of said system performance data; and

updating means for automatically updating an adaptive system model according to newly discovered relationships.

29. The system according to claim 28, further comprising:

test program generation means for generating a test program pursuant to at least one service level agreement.

30. The system according to claim 29, wherein said test program targets a target component within said system.

31. The system according to claim 30, wherein said target component substantially corresponds to a root decision node of said decision tree.

32. The system according to claim 31, wherein said target component is selected from the group consisting of a system hardware resource and a system software application.

33. The method according to claim 31, wherein said target component substantially corresponds to a root decision node of said decision tree.

34. The method according to claim 33, wherein said target component targeted by said test program is an underperforming system component, whereby said modification means automatically modifies the continuous monitoring and periodic collecting of said performance data on said underperforming system component by said monitor and collection means, respectively.

35. The system according to claim 28, wherein said modification means automatically modifies the periodicity of said performance data continuous monitoring and periodic collection by said monitor and collection means, respectively.

36. The system according to claim 35, wherein said periodicity increases after said autonomous modification.

37. The system according to claim 35, wherein said periodicity decreases after said autonomous modification.

38. The system according to claim 28, further comprising:

storage means for storing said periodically collected performance data.

39. The system according to claim 38, wherein said performance data is stored with an associated time stamp.

40. The system according to claim 38, wherein said storage means is a relational database.

41. The system according to claim 28, wherein said decision tree generation means generates said decision tree using decision tree induction.

42. The system according to claim 28, wherein said performance data comprises a respective plurality of state information at said plurality of said nodes.

43. The system according to claim 42, wherein said performance data further comprises a respective plurality of system information at said plurality of said nodes.

44. The system according to claim 43, wherein said system information comprises a plurality of service level agreements.

45. The system according to claim 28, wherein said collection means periodically collects said performance data periodically at specific time intervals.

46. The system according to claim 45, wherein said specific time intervals for periodically collecting said performance data is selected from the group consisting of days, hours, minutes and seconds.

47. The system according to claim 28, wherein said performance data periodically collected by said collection means has a value selected from the group consisting of real numbers, integers and Booleans.

48. The system according to claim 28, wherein said performance data collected periodically by said collection means is converted to a plurality of Boolean values.

49. The system according to claim 48, wherein said plurality of Boolean values correspond to a plurality of performance threshold conditions.

50. The method according to claim 49, wherein said performance threshold conditions are predetermined.

51. The method according to claim 50, wherein said performance threshold conditions are variable.

52. The system according to claim 28, wherein said performance data periodically collected by such collection means is averaged at specific time intervals.

53. The system according to claim 52, wherein said averaged performance data is converted to at least one Boolean value.

54. The system according to claim 53, wherein said at least one Boolean value corresponds to at least one service level agreement within said system.

55. The system according to claim 28, wherein said decision tree generation means employs regression analysis on at least one component of said decision tree, whereby the performance of said at least one target component is predicted at a future time from a plurality of parameter data within a plurality of said decision nodes.

59. The system according to claim 28, wherein said newly discovered relationships are uncovered or unexpected relations.

56. An article of manufacture comprising a computer usable medium having computer readable program code means embodied thereon for optimizing performance monitoring of at least one node in an information technology system, the computer readable program code means in said article of manufacture comprising:

computer readable program code means for;

(a) performing a continuity analysis on said system;

(b) automatically generating a plurality of performance models of said system based on periodic measurements of predefined performance levels;

(c) continuously monitoring, at a plurality of said nodes, the performance of said system at the respective plurality of said nodes;

(d) collecting, periodically, performance data on said system at said respective nodes;

(e) applying a plurality of data mining techniques to said periodically collected system performance data and its associated test program data;

(f) generating a decision tree using said periodically collected system performance data, said decision tree having a multiplicity of decision nodes, each said decision node corresponding to a component of said system;

(g) comparing a plurality of relationships within said system between said system performance data and said test program data;

(h) automatically modifying said steps of continuously monitoring and periodically collecting said system performance data at a plurality of said nodes, whereby said autonomous modification iteratively optimizes said continuous performance monitoring of said system; and

(i) automatically updating an adaptive system model according to newly discovered relationships.