Performance modeling for information systems

• US 20040220792A1
• Filed: 04/30/2003
• Published: 11/04/2004
• Est. Priority Date: 04/30/2003
• Status: Abandoned Application

• Current Assignee: Martingale Corp.
• Original Assignee: Martingale Corp.
• Inventors: Gallanis, Peter Thomas, Teflian, Mark Samuel, Holloran, Thomas Joseph, Bacon, Bruce Allen, Sleeper, Scott Thurston
• Application Number:
  US10/427,121
• Publication Number:
  US 20040220792A1
• Time in Patent Office: Days
• US Class Current: 703/13
• CPC Class Codes: G06F 11/3447 Performance evaluation by modeling
• CPC Class Codes: G06F 8/77 Software metrics

Abstract

A method for monitoring performance of an IT-based computing system over the life cycle of the system is disclosed. The life cycle includes a conception phase, an analysis phase, an architecture design phase, a system design phase, a construction phase, a testing phase, a deployment phase, and finally, an operational phase. Performance requirements for the IT-based computing system are defined during the analysis phase. In the architecture design phase, the appropriate hardware and software infrastructure are selected for use in designing the base architecture for the system. In the system design phase, application specific system components are defined for the system. In the construction phase, the hardware platform is assembled and the software design specifications are turned into executable code. The constructed system is then tested for functionality and performance prior to being deployed for subsequent operation. A performance model is created during the analysis phase using the performance requirements. The model, which is refined during each phase in the life cycle to more accurately reflect the system being constructed, may be executed at any phase in the life cycle to render optimal levels of performance for the system at that phase. Furthermore, the refined model may be used during the operational phase of the life cycle to test the effect of enhancing a pre-existing IT-based computing system.

Claims

1. A method for monitoring performance of an information technology (IT) based computing system during a time period beginning with conception of the IT-based computing system and ending at a predefined time occurring after deployment of the IT-based computing system into a production environment, the method comprising:

creating a model of the IT-based computing system using one or more performance requirements defined for the IT-based computing system, wherein the model is a computer generated representation of the IT-based computing system at the time of deployment into the production environment; and

refining the model after deployment of the IT-based computing system into the production environment based on information collected from the IT-based computing system during operation in the production environment.

2. A method as defined in claim 1, wherein the performance requirements are defined during an analysis phase occurring during the time period after conception of the IT-based computing system and prior to any creation phases wherein the IT-based computing system is designed and constructed, the creating act comprising:

creating the model during the analysis phase.

3. A method as defined in claim 2, wherein the creation phases comprise an initial design phase wherein various software and hardware components used to construct the system are defined, the method further comprising:

refining the model after the initial design phase such that the model reflects the various software and hardware components defined during the design phase.

4. A method as defined in claim 3, wherein the creation phases comprise a subsequent design phase wherein application specific components used to construct the system are defined, the method further comprising:

refining the model after the subsequent design phase such that the model reflects the application specific components defined during the subsequent design phase.

5. A method as defined in claim 4, wherein the creation phases comprise a construction phase wherein the software components, hardware components and application specific components are interconnected and evaluated to determine whether additional components are needed to render the IT-based computing system in completed form, the additional components being integrated into the system during the construction phase to complete the IT-based computing system, wherein the method further comprises:

refining the model after the construct phase such that the model accurately represents the completed IT-based computing system.

6. A method as defined in claim 1, further comprising:

executing the model to render performance levels indicative of the performance requirements defined for the IT-based computing system;

executing the IT-based computing system to render performance metrics associated with performance of the IT-based computing system in the production environment; and

comparing the performance levels to the performance metrics to determine whether implementation of the IT-based system complies with the defined performance requirements.

7. A method as defined in claim 6, further comprising:

determining that implementation of the of the IT-based system complies with the defined performance requirements if the performance metrics are within a predetermined range of the performance levels.

8. A method as defined in claim 6, further comprising:

determining that implementation of the of the IT-based system complies with the defined performance requirements if the performance metrics are greater than or equal to the performance levels.

9. A method as defined in claim 6, wherein prior to the refining act, the IT-based computing system is enhanced with one or more components if implementation of the IT-based system is determined to not comply with the defined performance requirements.

10. A method as defined in claim 9, wherein the one or more components used to enhance the IT-based computing system are selected from the group consisting of hardware components, software components and middleware components.

11. A method as defined in claim 10, wherein the refining act comprises:

updating the model to reflect that the IT-based computing system has been enhanced with the one or more components.

12. A method as defined in claim 1, wherein after deployment and prior to the refining act, the IT-based computing system is enhanced with one or more components during implementation of the IT-based computing system in the production environment, the one or more components being selected from the group consisting of hardware components, software components and middleware components.

13. A method as defined in claim 11, wherein the refining act comprises:

updating the model to reflect that the IT-based computing system has been enhanced with the one or more components.

14. A method as defined in claim 1, further comprising:

refining the performance model to reflect desired enhancements to the IT-based computing system.

15. A method as defined in claim 14, wherein the desired enhancements relate to desired performance levels, wherein the refined performance model is operable for use in determining additional components required for the IT-based computing system to meet the desired enhancements.

16. A method as defined in claim 15, wherein the desired enhancements relate to additional components that are to be added to the IT-based computing system, the additional components being selected from a group consisting of hardware components, software components and middleware components, wherein the refined performance model is operable for use in predicting whether implementation of the IT-based computing system enhanced with the additional components will comply with the defined performance requirements.

57. The computer program product of claim 14, wherein the desired enhancements relate to desired performance levels, wherein the refined performance model is operable for use in determining additional components required for the IT-based computing system to meet the desired enhancements.

58. The computer program product of claim 15, wherein the desired enhancements relate to additional components that are to be added to the IT-based computing system, the additional components being selected from a group consisting of hardware components, software components and middleware components, wherein the refined performance model is operable for use in predicting whether implementation of the IT-based computing system enhanced with the additional components will comply with the defined performance requirements.

17. A method for monitoring performance of an information technology (IT) based computing system during a time period beginning with conception of the IT-based computing system and ending at a predefined time occurring after deployment of the IT-based computing system into a production environment, the method comprising:

constructing a performance model of the IT-based computing system prior to deployment into the production environment, wherein the performance model is a computer generated representation of the IT-based computing system;

following deployment of the IT-based computing system into the production environment, executing the performance model to render performance levels indicative of the performance requirements defined for the IT-based computing system;

executing the IT-based computing system to render performance metrics associated with performance of the IT-based computing system in the production environment; and

comparing the performance levels to the performance metrics to determine whether implementation of the IT-based system complies with the defined performance requirements.

18. A method as defined in claim 17, wherein the constructing act comprises:

creating the performance model using one or more performance requirements defined for the IT-based computing system; and

refining the performance model based on information collected from the IT-based computing system during one or more creation phases occurring prior to deployment of the IT-based computing system into the production environment.

19. A method as defined in claim 17, further comprising:

determining that implementation of the of the IT-based system complies with the defined performance requirements if the performance metrics are within a predetermined range of the performance levels.

20. A method as defined in claim 17, further comprising:

determining that implementation of the of the IT-based system complies with the defined performance requirements if the performance metrics are greater than or equal to the performance levels.

21. A method as defined in claim 17, wherein prior to the refining act, the IT-based computing system is enhanced with one or more components if implementation of the IT-based system is determined to not comply with the defined performance requirements.

22. A method as defined in claim 21, wherein the one or more components used to enhance the IT-based computing system are selected from the group consisting of hardware components, software components and middleware components.

23. A method as defined in claim 22, further comprising:

updating the performance model to reflect that the IT-based computing system has been enhanced with the one or more components.

24. A method as defined in claim 17, wherein the constructing act is accomplished using a discrete event simulation software application program.

25. A method as defined in claim 17, further comprising:

refining the performance model to reflect desired enhancements to the IT-based computing system.

26. A method as defined in claim 25, wherein the desired enhancements relate to desired performance levels, wherein the refined performance model is operable for use in determining additional components required for the IT-based computing system to meet the desired enhancements.

27. A method as defined in claim 26, wherein the desired enhancements relate to additional components that are to be added to the IT-based computing system, the additional components being selected from a group consisting of hardware components, software components and middleware components, wherein the refined performance model is operable for use in predicting whether implementation of the IT-based computing system enhanced with the additional components will comply with the defined performance requirements.

28. A method for developing an information technology (IT) based computing system for use in a production environment, the method comprising:

creating a computer-implemented model of the IT-based computing system using one or more performance requirements defined for the IT-based computing system, wherein the computer-implemented model renders performance levels for the IT-based computing system when executed;

comparing operating parameters of the IT-based computing system after each of a plurality of phases during a period in time associated with creation of the IT-based computing system to performance levels rendered by the executed computer-implemented model to predict whether performance resulting from implementation of the IT-based computing system in the production environment will satisfy the one or more performance requirements; and

if it is predicted that performance resulting from implementation of the IT-based computing system in the production environment will satisfy the one or more performance requirements, refining the computer-implemented model after each of the plurality of phases such that the computer-implemented model reflects components and interconnectivity thereof for the IT-based computing system after each of the plurality of phases.

29. A method as defined in claim 28, wherein the performance requirements are defined during an analysis phase, the analysis phase occurring after conception of the IT-based computing system and prior to any creation phases wherein the IT-based computing system is designed and constructed, the creating act comprising:

creating the computer-implemented model during the analysis phase.

30. A method as defined in claim 29, wherein the plurality of phases comprises an architecture design phase occurring immediately after the analysis phase, wherein various software and hardware components used to construct the system are defined during the architecture design phase, the refining act comprising:

refining the computer-implemented model after the architecture design phase such that the computer-implemented model reflects the various software and hardware components defined during the architecture design phase.

31. A method as defined in claim 30, wherein the plurality of phases comprises a system design phase occurring immediately after the architecture design phase, wherein application specific components used to construct the system are defined during the system design phase, the refining act further comprising:

refining the computer-implemented model after the system design phase such that the computer-implemented model reflects the application specific components defined during the system design phase.

32. A method as defined in claim 31, wherein the plurality of phases comprises a construction phase occurring immediately after the system design phase, wherein the software components, hardware components and application specific components are interconnected and evaluated to determine whether additional components are needed to render the IT-based computing system in completed form, the additional components being integrated into the system during the construction phase to complete the IT-based computing system, wherein the refining act further comprises:

refining the computer-implemented model after the construction phase such that the computer-implemented model accurately reflects the completed IT-based computing system.

33. A method as defined in claim 28, further comprising:

refining the computer-implemented model after deployment of the IT-based computing system into the production environment based on information collected from the IT-based computing system during operation in the production environment.

34. A method as defined in claim 33, further comprising:

executing the computer-implemented model to render performance levels indicative of the performance requirements defined for the IT-based computing system;

executing the IT-based computing system to render performance metrics associated with performance of the IT-based computing system in the production environment; and

comparing the performance levels to the performance metrics to determine whether implementation of the IT-based system complies with the defined performance requirements.

35. A method as defined in claim 34, further comprising:

determining that implementation of the IT-based system complies with the defined performance requirements if the performance metrics are within a predetermined range of the performance levels.

36. A method as defined in claim 34, further comprising:

determining that implementation of the IT-based system complies with the defined performance requirements if the performance metrics are greater than or equal to the performance levels.

37. A method as defined in claim 34, wherein prior to the act of refining the computer-implemented model after deployment of the IT-based computing system, the IT-based computing system is enhanced with one or more components if implementation of the IT-based system is determined to not comply with the defined performance requirements.

38. A method as defined in claim 37, wherein the one or more components used to enhance the IT-based computing system are selected from the group consisting of hardware components, software components and middleware components.

39. A method as defined in claim 38, wherein the refining act comprises:

updating the computer-implemented model to reflect that the IT-based computing system has been enhanced with the one or more components.

40. A method as defined in claim 28, further comprising:

returning creation of the IT-based computing system to an earlier phase in the period of time if it is predicted that performance resulting from implementation of the IT-based computing system in the production environment will not satisfy the one or more performance requirements.

41. A method as defined in claim 28, further comprising:

moving creation of the IT-based computing system to a subsequent phase in the period of time if it is predicted that performance resulting from implementation of the IT-based computing system in the production environment will satisfy the one or more performance requirements.

42. A method as defined in claim 28, wherein the refining act comprises:

if it is predicted that performance resulting from implementation of the IT-based computing system in the production environment will not satisfy the one or more performance requirements, refining the computer-implemented model to reflect components and interconnectivity thereof for the IT-based computing system after each of the plurality of phases if performance resulting from implementation of the IT-based computing system in the production environment is more desirable than the one or more defined performance requirements.

43. A computer program product encoding a computer program for executing on a computer system a computer process for monitoring performance of an information technology (IT) based computing system during a time period beginning with conception of the IT-based computing system and ending at a predefined time occurring after deployment of the IT-based computing system into a production environment, the method comprising:

creating a model of the IT-based computing system using one or more performance requirements defined for the IT-based computing system, wherein the model is a computer generated representation of the IT-based computing system at the time of deployment into the production environment; and

refining the model after deployment of the IT-based computing system into the production environment based on information collected from the IT-based computing system during operation in the production environment.

44. The computer program product of claim 43, wherein the performance requirements are defined during an analysis phase occurring during the time period after conception of the IT-based computing system and prior to any creation phases wherein the IT-based computing system is designed and constructed, the creating act comprising:

creating the model during the analysis phase.

45. The computer program product of claim 44, wherein the creation phases comprise an initial design phase wherein various software and hardware components used to construct the system are defined, the method further comprising:

refining the model after the initial design phase such that the model reflects the various software and hardware components defined during the design phase.

48. The computer program product of claim 43, the computer process further comprising:

executing the model to render performance levels indicative of the performance requirements defined for the IT-based computing system;

executing the IT-based computing system to render performance metrics associated with performance of the IT-based computing system in the production environment; and

comparing the performance levels to the performance metrics to determine whether implementation of the IT-based system complies with the defined performance requirements.

49. The computer program product of claim 48, the computer process further comprising:

determining that implementation of the of the IT-based system complies with the defined performance requirements if the performance metrics are within a predetermined range of the performance levels.

50. The computer program product of claim 48, the computer process further comprising:

determining that implementation of the IT-based system complies with the defined performance requirements if the performance metrics are greater than or equal to the performance levels.

51. The computer program product of claim 48, wherein prior to the refining act, the IT-based computing system is enhanced with one or more components if implementation of the IT-based system is determined to not comply with the defined performance requirements.

52. The computer program product of claim 51, wherein the one or more components used to enhance the IT-based computing system are selected from the group consisting of hardware components, software components and middleware components.

53. The computer program product of claim 52, wherein the refining act comprises:

updating the model to reflect that the IT-based computing system has been enhanced with the one or more components.

54. The computer program product of claim 43, wherein after deployment and prior to the refining act, the IT-based computing system is enhanced with one or more components during implementation of the IT-based computing system in the production environment, the one or more components being selected from the group consisting of hardware components, software components and middleware components.

55. The computer program product of claim 53, wherein the refining act comprises:

updating the model to reflect that the IT-based computing system has been enhanced with the one or more components.

56. The computer program product of claim 43, the computer process further comprising:

refining the performance model to reflect desired enhancements to the IT-based computing system.

46. The computer program product of 45, wherein the creation phases comprise a subsequent design phase wherein application specific components used to construct the system are defined, the method further comprising:

refining the model after the subsequent design phase such that the model reflects the application specific components defined during the subsequent design phase.

47. The computer program product of claim 46, wherein the creation phases comprise a construction phase wherein the software components, hardware components and application specific components are interconnected and evaluated to determine whether additional components are needed to render the IT-based computing system in completed form, the additional components being integrated into the system during the construction phase to complete the IT-based computing system, wherein the method further comprises:

refining the model after the construct phase such that the model reflects the completed IT-based computing system.

59. A computer program product encoding a computer program for developing an information technology (IT) based computing system for use in a production environment, the computer process comprising:

creating a computer-implemented model of the IT-based computing system using one or more performance requirements defined for the IT-based computing system, wherein the computer-implemented model renders performance levels for the IT-based computing system when executed;

comparing operating parameters of the IT-based computing system after each of a plurality of phases during a period in time associated with creation of the IT-based computing system to performance levels rendered by the executed computer-implemented model to predict whether performance resulting from implementation of the IT-based computing system in the production environment will satisfy the one or more performance requirements; and

if it is predicted that performance resulting from implementation of the IT-based computing system in the production environment will satisfy the one or more performance requirements, refining the computer-implemented model after each of the plurality of phases such that the computer-implemented model reflects components and interconnectivity thereof for the IT-based computing system after each of the plurality of phases.

60. The computer program product of claim 59, wherein the performance requirements are defined during an analysis phase occurring after conception of the IT-based computing system and prior to any creation phases wherein the IT-based computing system is designed and constructed, the creating act comprising:

creating the computer-implemented model during the analysis phase.

61. The computer program product of claim 60, wherein the plurality of phases comprises an architecture design phase occurring immediately after the analysis phase, wherein various software and hardware components used to construct the system are defined during the architecture design phase, the refining act comprising:

refining the computer-implemented model after the architecture design phase such that the computer-implemented model reflects the various software and hardware components defined during the architecture design phase.

62. The computer program product of claim 61, wherein the plurality of phases comprises a system design phase occurring immediately after the architecture design phase, wherein application specific components used to construct the system are defined during the system design phase, the refining act further comprising:

refining the computer-implemented model after the system design phase such that the computer-implemented model reflects the application specific components defined during the system design phase.

63. The computer program product of claim 62, wherein the plurality of phases comprises a construction phase occurring immediately after the system design phase, wherein the software components, hardware components and application specific components are interconnected and evaluated to determine whether additional components are needed to render the IT-based computing system in completed form, the additional components being integrated into the system during the construction phase to complete the IT-based computing system, wherein the refining act further comprises:

refining the computer-implemented model after the construction phase such that the computer-implemented model accurately represents the completed IT-based computing system.

64. The computer program product of claim 59, the computer process further comprising:

refining the computer-implemented model after deployment of the IT-based computing system into the production environment based on information collected from the IT-based computing system during operation in the production environment.

65. The computer program product of claim 64, the computer process further comprising:

executing the computer-implemented model to render performance levels indicative of the performance requirements defined for the IT-based computing system;

executing the IT-based computing system to render performance metrics associated with performance of the IT-based computing system in the production environment; and

comparing the performance levels to the performance metrics to determine whether implementation of the IT-based system complies with the defined performance requirements.

66. The computer program product of claim 65, the computer process further comprising:

determining that implementation of the IT-based system complies with the defined performance requirements if the performance metrics are within a predetermined range of the performance levels.

67. The computer program product of claim 65, the computer process further comprising:

determining that implementation of the IT-based system complies with the defined performance requirements if the performance metrics are greater than or equal to the performance levels.

68. The computer program product of claim 65, wherein prior to the act of refining the computer-implemented model after deployment of the IT-based computing system, the IT-based computing system is enhanced with one or more components if implementation of the IT-based system is determined to not comply with the defined performance requirements.

69. The computer program product of claim 68, wherein the one or more components used to enhance the IT-based computing system are selected from the group consisting of hardware components, software components and middleware components.

70. The computer program product of claim 69, wherein the refining act comprises:

updating the computer-implemented model to reflect that the IT-based computing system has been enhanced with the one or more components.

71. The computer program product of claim 59, the computer process further comprising:

returning creation of the IT-based computing system to an earlier phase in the period of time if it is predicted that implementation of the IT-based computing system in the production environment will not satisfy the one or more performance requirements.

72. The computer program product of claim 59, the computer process further comprising:

moving creation of the IT-based computing system to a subsequent phase in the period of time if it is predicted that implementation of the IT-based computing system in the production environment will satisfy the one or more performance requirements.

73. The computer program product of claim 59, wherein the refining act comprises:

if it is predicted that performance resulting from implementation of the IT-based computing system in the production environment will not satisfy the one or more performance requirements, refining the computer-implemented model to reflect components and interconnectivity thereof for the IT-based computing system after each of the plurality of phases if it is predicted that performance resulting from implementation of the IT-based computing system in the production environment is more desirable than the one or more defined performance requirements.