Automatic Baselining Of Metrics For Application Performance Management

• US 20110098973A1
• Filed: 10/23/2009
• Published: 04/28/2011
• Est. Priority Date: 10/23/2009
• Status: Abandoned Application

• Current Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Original Assignee: Computer Associates Think Inc. (Broadcom, Inc.)
• Inventors: Seidman, David Isaiah
• Application Number:
  US12/605,087
• Publication Number:
  US 20110098973A1
• Time in Patent Office: Days
• US Class Current: 702/179
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0754 by exceeding limits
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/865 Monitoring of software
• CPC Class Codes: G06F 2201/87 Monitoring of transactions
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet

Abstract

An application monitoring system monitors one or more applications to generate and report application performance data for transactions. Actual performance data for one or more metrics is compared with a baseline metric value(s) to detect anomalous transactions or components thereof. Automatic baselining for a selected metric is provided using variability based on a distribution range and arithmetic mean of actual performance data to determine an appropriate sensitivity for boundaries between comparison levels. A user-defined sensitivity parameter allows adjustment of baselines to increase or decrease comparison sensitivity for a selected metric. The system identifies anomalies in transactions, components of transaction based on a comparison of actual performance data with the automatically determined baseline for a corresponding metric. The system reports performance data and other transactional data for identified anomalies.

Claims

1. A computer-implemented method of determining a normal range of behavior for an application, comprising:

accessing performance data associated with a metric for a plurality of transactions of an application;

accessing an initial range multiple for the metric;

calculating a variability measure for the metric based on a maximum value, minimum value and arithmetic mean of the performance data;

modifying the initial range multiple based on the calculated variability measure for the metric; and

automatically establishing a baseline for the metric based on the modified range multiple.

2. The method of claim 1, further comprising:

automatically instrumenting object code of the application to monitor the plurality of transactions.

3. The method of claim 1, wherein accessing an initial range multiple for the metric comprises establishing the initial range multiple based on a default value.

4. The method of claim 1, further comprising:

determining a standard deviation of the performance data for the metric;

determining an average value of the performance data for the metric;

determining a product of the standard deviation and the modified range multiple;

determining a sum of the average value and the product;

determining a difference of the average value and the product; and

wherein the baseline for the metric includes a comparison threshold for the metric based on the sum and the difference.

5. A method according to claim 4, wherein automatically establishing the baseline for the metric, includes:

establishing a first comparison threshold for the metric when the variability of the metric is at a first value; and

establishing a larger comparison threshold when the variability of the metric is at a second value that is less than the first value.

6. A method according to claim 1, further comprising:

receiving a user-defined desired sensitivity for the metric; and

wherein establishing the baseline for the metric is based on the modified range multiple and the user-defined sensitivity for the metric.

7. A method according to claim 6, further comprising:

determining a sensitivity multiple based on the user-defined sensitivity, a maximum sensitivity and a default sensitivity;

wherein establishing the baseline metric includes adjusting the modified range multiple using the sensitivity multiple.

8. A method according to claim 1, further comprising:

monitoring the application to determine additional performance data for the metric after establishing the baseline for the metric;

comparing the additional performance data for the metric to the baseline for the metric;

determining if the metric for the application is anomalous based on the comparing; and

reporting, responsive to the determining.

9. A method according to claim 8, further comprising:

updating the established baseline for the metric using the additional performance data.

10. A method according to claim 1, wherein:

the range multiple is a number of standard deviations for the metric.

11. An apparatus, comprising:

a communication interface;

a storage device; and

one or more processors in communication with the storage device and the communication interface, the one or more processors adapted to access performance data associated with a metric for a plurality of transactions of an application, access an initial range multiple for the metric, calculate a variability measure for the metric based on a maximum value, minimum value and arithmetic mean of the performance data, modify the initial range multiple based on the calculated variability measure for the metric, and automatically establish a baseline for the metric based on the modified range multiple.

12. An apparatus according to claim 11, further comprising:

one or more agents, said one or more agents collect data about the plurality of transactions; and

an enterprise manager implemented by the one or more processors to communicate with the one or more agents and establish the baseline for the metric.

13. An apparatus according to claim 11, wherein the one or more processors are adapted to:

determine a standard deviation of the performance data for the metric;

determine an average value of the performance data for the metric;

determine a product of the standard deviation and the modified range multiple;

determine a sum of the average value and the product;

determine a difference of the average value and the product; and

wherein the baseline for the metric includes a comparison threshold for the metric based on the sum and the difference.

14. An apparatus according to claim 11, wherein the one or more processors are adapted to:

receive a user-defined desired sensitivity parameter for the metric; and

establish the baseline for the metric based on the modified range multiple and the user-defined sensitivity for the metric.

15. An apparatus according to claim 14, wherein the one or more processors are adapted to:

determine a sensitivity multiple based on the user-defined sensitivity, a maximum sensitivity and a default sensitivity; and

establish the baseline metric by adjusting the modified range multiple using the sensitivity multiple.

16. An apparatus according to claim 11, wherein the one or more processors are adapted to:

monitor the application to determine additional performance data for the metric after establishing the baseline for the metric;

compare the additional performance data for the metric to the baseline for the metric;

determine if the metric for the application is anomalous based on the comparing; and

report, responsive to the determining.

17. One or more processor readable storage devices having process readable code embodied thereon, said processor readable code for programming one or more processors to perform a method comprising:

monitoring a plurality of transactions associated with an application;

generating performance data for the plurality of transactions of the application, the performance data corresponding to a selected metric;

establishing a default deviation threshold for the selected metric;

modifying the default deviation threshold using a calculated variability measure for the selected metric based on the performance data;

automatically establishing a baseline for the selected metric using the modified deviation threshold;

comparing the generated performance data for the plurality of transactions to the baseline for the metric; and

reporting one or more transactions having performance data outside of the baseline for the selected metric.

18. One or more processor readable storage devices according to claim 17, wherein reporting the one or more transactions includes displaying a user interface with one or more indications that the one or more transactions contain an anomaly.

19. One or more processor readable storage devices according to claim 17, wherein the method further comprises:

calculating a sensitivity multiple based on a user-defined sensitivity parameter;

wherein automatically establishing a baseline for the selected metric includes combining the sensitivity multiple with the modified deviation threshold and determining at least one comparison threshold based on the combination of the sensitivity multiple and the modified deviation.

20. One or more processor readable storage devices according to claim 17, wherein the method further comprises:

dynamically updating the baseline for the selected metric in response to additional performance data generated for one or more additional transactions of the application.

21. One or more processor readable storage devices according to claim 17, wherein generating performance data for the plurality of transactions of the application includes reporting transaction events to an agent by monitoring code added to object code for the application.

22. A computer-implemented method of application performance management, comprising:

accessing performance data associated with a metric of an application;

establishing an initial baseline for the metric;

modifying the initial baseline based on a calculated variability of the performance data associated with the metric;

determining at least one comparison threshold for the metric using the modified baseline for the metric;

generating additional performance data associated with the metric of the application;

comparing the additional performance data with the at least one comparison threshold; and

reporting one or more anomalies associated with the application responsive to the comparing.

23. The method of claim 22, wherein comparing the additional performance data with the at least one comparison threshold includes:

identifying a range of performance data values for the application; and

determining if the additional performance data is contained within the identified range.