ASSESSING APPLICATION PERFORMANCE WITH AN OPERATIONAL INDEX

• US 20130036122A1
• Filed: 04/11/2012
• Published: 02/07/2013
• Est. Priority Date: 12/04/2011
• Status: Active Grant

• Current Assignee: Riverbed Technology, LLC (Riverbed Technology Incorporated)
• Original Assignee: OPNET Technologies Incorporated (Riverbed Technology Incorporated)
• Inventors: COHEN, Alain J., Ding, Yiping
• Granted Patent: US 8,725,741 B2
• Time in Patent Office: Days
• US Class Current: 707/741
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/865 Monitoring of software

Abstract

The present invention relates to a system and method for assessing application performance and user satisfaction. In one embodiment, the system and method calculates an Operational Index (OPdex) representing user satisfaction with an application. The OPdex may be a number quantifying user satisfaction with an application, such as a web application, and system performance. The OPdex may be based on one or more measurable metrics having a range of values that may affect user satisfaction or performance of an application. The OPdex may comprise calculating the index based on a soft threshold, a hard threshold, and measurements indicating a perceived application response time. The OPdex calculation may also account for sensitivity of user satisfaction to response time. Based on the OPdex, the system and methods also provide information indicating the relationship among application response time thresholds set by the users, the user satisfaction level, and the mean response time.

Claims

1. A computer-implemented method of identifying a source of a problem in an application running on in a computer system based on an index, said method comprising:

receiving data related to a plurality of types of metrics associated with the application;

determining, for each type of metric, a respective index that ranges between a first value corresponding to unsatisfactory performance and a second value corresponding to a completely satisfactory performance; and

providing information identifying a set of the indexes to indicate a possible source of the problem.

2. The method of claim 1, wherein receiving data comprises receiving network data.

3. The method of claim 1, wherein receiving data comprises receiving application performance data.

4. The method of claim 1, wherein receiving data comprises receiving application response time data.

5. The method of claim 1, wherein receiving data comprises receiving link utilization data.

6. The method of claim 1, wherein receiving data comprises receiving processor utilization data.

7. The method of claim 1, wherein providing information identifying the set of indexes comprises providing information ranking the set of indexes based on a severity of the corresponding metric.

8. The method of claim 1, wherein providing information identifying the set of indexes comprises providing the indexes with a color indicating a severity of the index in relation to the possible source of the problem.

9. The method of claim 1, wherein the respective indexes range in value from 0 to 1.

10. The method of claim 1, wherein the respective indexes comprise different ranges in value based on the type of metric.

11. A computer-implemented method for calculating an index indicating performance of an application running on a computer system, wherein the index ranges between a first value corresponding to unsatisfactory performance and a second value corresponding to a completely satisfactory performance, said method comprising:

receiving, from at least one instrument coupled to the computer system, data indicating at least one metric having a range of values related to a performance of an application;

determining a first threshold for the at least one metric indicating a value of the metric beyond which performance of an application is categorized as declining;

determining a second threshold for the at least one metric value indicating a value of the metric beyond which performance of an application is categorized as unsatisfactory;

determining a set of parameters indicating a sensitivity of the index to a change in the value of the metric between the first and second threshold; and

calculating an index indicating performance of the application between the first threshold and second threshold based on the first threshold, the second threshold, and the set of parameters indicating the sensitivity of the index.

12. The method of claim 11, wherein the index ranges between 0 corresponding to unsatisfactory performance and 1 corresponding to completely satisfactory performance.

13. The method of claim 11, wherein determining the set of parameters indicating the sensitivity of the index comprises receiving at least one desired value for the index based on a value for the metric between the first and second thresholds.

14. The method of claim 11, wherein calculating the index is based on a quadratic function of the first and second thresholds and the set of parameters.

15. The method of claim 11, wherein calculating the index comprises calculating the index based on respective functions corresponding to different intervals of values between the first and second thresholds.

16. A method of modeling performance of an application based on an index in a computer system, wherein the index ranges between a first value corresponding to unsatisfactory performance and a second value corresponding to a completely satisfactory performance, said method comprising:

receiving a set of desired values for the index between the first and second thresholds;

determining a sensitivity of the index based on the received set of desired values; and

determining a function for the index based on the received set of desired values, wherein the function calculates the index for response times between the first threshold and second threshold based on the sensitivity.

17. The method of claim 16, further comprising determining a response time for a desired value of the index.

18. The method of claim 16, further comprising determining a maximum response time for a desired value of the index.

19. The method of claim 16, wherein the index ranges between 0 corresponding to unsatisfactory performance and 1 corresponding to completely satisfactory performance.

20. The method of claim 16, wherein the function comprises a quadratic function.

21. A computer-implemented method of developing an index scoring a performance of an application running on a computer system, said method comprising:

determining, by a computer, a model comprising respective index scores for ranges of at least one metric measuring performance of the application based on application data collected from the computer system running the application;

calculating, by the computer, estimated scores for the application based on the range of metrics based on the model; and

determining, by the computer-based system, an index for the application based on the calculated model.

22. The method of claim 21, wherein the index ranges between 0 and 1.

23. The method of claim 21, wherein determining the model comprises selecting at least one metric from the collected application data that is most indicative of application performance.

24. The method of claim 21, wherein determining the model comprises a statistical analysis of the collected application data over a given time.

25. The method of claim 21, further comprising providing a display indicating the index for the application.