Method for using statistical analysis to monitor and analyze performance of new network infrastructure or software applications for deployment thereof

• US 20050278703A1
• Filed: 06/15/2005
• Published: 12/15/2005
• Est. Priority Date: 06/15/2004
• Status: Abandoned Application

• Current Assignee: K5 Systems Incorporated
• Original Assignee: K5 Systems Incorporated
• Inventors: Chung, Richard Y., Lo, Kevin H.
• Application Number:
  US11/153,120
• Publication Number:
  US 20050278703A1
• Time in Patent Office: Days
• US Class Current: 717/126
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3447 Performance evaluation by modeling
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/835 Timestamp
• CPC Class Codes: G06F 2201/865 Monitoring of software
• CPC Class Codes: H04L 41/0213 Standardised network management protocols, e.g. simple network management protocol [SNMP]
• CPC Class Codes: H04L 41/0681 Configuration of triggering conditions
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 41/147 for predicting network behaviour
• CPC Class Codes: H04L 41/5009 Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 41/5032 Generating service level reports
• CPC Class Codes: H04L 41/5054 Automatic deployment of services triggered by the service manager, e.g. service implementation by automatic configuration of network components
• CPC Class Codes: H04L 41/5064 Customer relationship management
• CPC Class Codes: H04L 43/00 Arrangements for monitoring or testing data switching networks
• CPC Class Codes: H04L 43/028 : by filtering
• CPC Class Codes: H04L 43/065 : related to network devices
• CPC Class Codes: H04L 43/067 : using time frame reporting
• CPC Class Codes: H04L 43/16 : Threshold monitoring
• CPC Class Codes: H04L 67/535 : Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

Methods for using statistical analysis to monitor performance of new network infrastructure and applications for deployment thereof. A method monitors a release of executing software applications or execution infrastructure to detect deviations in performance. A first set of time-series data is acquired from executing software applications and execution infrastructure. A first statistical description of expected behavior is derived from the first set of acquired data. A second set of time-series data is acquired from the monitored release of executing software applications and execution infrastructure. A second statistical description of behavior is derived from the second set of acquired data. The first and second statistical descriptions are compared to identify instances where the first and second statistical descriptions deviate sufficiently to indicate a statistically significant probability that an operating anomaly exists within the monitored release of executing software applications and execution infrastructure.

Claims

1. A method of monitoring a release of executing software applications or execution infrastructure to detect deviations in performance, said method comprising:

acquiring a first set of time-series data from executing software applications and execution infrastructure;

deriving a first statistical description of expected behavior from said first set of acquired data;

acquiring a second set of time-series data from the monitored release of executing software applications and execution infrastructure;

deriving a second statistical description of behavior from said second set of acquired data;

comparing the first and second statistical descriptions to identify instances where the first and second statistical descriptions deviate sufficiently to indicate a statistically significant probability that an operating anomaly exists within the monitored release of executing software applications and execution infrastructure.

2. The method of claim 1 performed before deployment of the release into a production environment.

3. The method of claim 1 performed when the release has been deployed into a limited production environment.

4. The method of claim 1 wherein executing software applications or execution infrastructure are grouped and defined as managed units and wherein the deriving and comparing is performed on a managed unit basis.

5. The method of claim 4 wherein a first and second managed unit are non-mutually exlcusive.

6. The method of claims 5 wherein the first and second managed unit each include a new version of a software application or execution infrastructure.

7. The method of claim 1 wherein deriving the first and second statistical descriptions of behavior includes deriving at least statistical means and standard deviations of at least a subset of data elements within the acquired time-series data.

8. The method of claim 1 wherein deriving the first and second statistical descriptions of behavior includes deriving covariance matrices of at least a subset of data elements within the acquired time-series data.

9. The method of claim 1 wherein deriving the first and second statistical descriptions of behavior includes deriving principal component analysis (PCA) data for at least a subset of data elements within the acquired time-series data.

10. The method of claim 1 wherein said acquired data includes monitored data.

11. The method of claim 10 wherein the monitored data includes SNMP data.

12. The method of claim 10 wherein the monitored data includes transactional response values.

13. The method of claim 10 wherein the monitored data includes trapped data.

14. The method of claim 1 wherein said acquired data includes business process data.

15. The method of claim 14 wherein the business process data describes a specified end-user process.

16. The method of claim 1 further including logic to pre-process data received from the at least one managed unit and to provide pre-processed data to the logic to acquire time-series data.

17. The method of claim 1 wherein comparing the first and second statistical descriptions produces a single difference measurement.

18. The method of claim 1 wherein the software applications and execution infrastructure can be an arbitrary, unconstrained selection of software applications and execution infrastructure.

19. The method of claim 1 wherein acquiring time-series data is an in-band process.

20. The method of claim 1 wherein acquiring time-series data is an out-of-band process.