System and method for automated analysis of load testing results

• US 6,694,288 B2
• Filed: 08/06/2002
• Issued: 02/17/2004
• Est. Priority Date: 08/06/2001
• Status: Expired due to Term

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Mercury Interactive Corp. (HP Inc.)
• Inventors: Smocha, Itzhak, Landan, Gideon
• Primary Examiner(s): Hoff, Marc S
• Assistant Examiner(s): BARBEE, MANUEL L
• Application Number:
  US10/214,421
• Publication Number:
  US 20030074161A1
• Time in Patent Office: 560 Days
• Field of Search: 702/108, 702/119, 702/122, 702/123, 702/176, 702/179-183, 702/186-188, 709/224, 714/43, 714/47
• US Class Current: 702/186
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3457 Performance evaluation by simulation
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 2201/87 Monitoring of transactions
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: G06F 2201/885 Monitoring specific for caches

Abstract

A system for monitoring and evaluating the performance of a network accessible application comprises one or more load servers, each of which is capable of simulating the load imposed upon the application server by one or more clients. The load servers are configured to execute a particular sequence of server requests in order to evaluate the operation of the server under the specified load. Various performance metrics associated with the operation of the network and the application server are measured during the testing of the server, and these metrics are stored for later access by an analysis module. The analysis module identifies those portions of the test data which are statistically significant and groups these significant parameters to suggest possible relationships between the conditions of the load test and the observed performance results.

Claims

1. A method for analyzing the performance of a server under load, the method comprising:

providing at least one client configured to send a series of requests to a server;

measuring a plurality of parameters of the server over the period of time during which the server receives requests from the at least one client;

storing a series of values representing these measured parameters;

associating a time at which each value was measured with each stored value;

identifying correlations between the measured parameters based upon the stored values for the parameters, wherein identifying the correlations comprises calculating a correlation coefficient between the series of values for one parameter and the series of values for another parameter; and

selecting parameters which may be related to one another based upon the correlation between the measured values of the parameters.

2. The method of claim 1, wherein selecting parameters which may be related comprises comparing the correlation coefficient associated with a pair of parameters to a lower threshold value and selecting the pair of parameters if the correlation coefficient is greater than the lower threshold value.

3. The method of claim 1, wherein calculating a correlation coefficient between the series of values comprises calculating a Pearson correlation coefficient.

4. The method of claim 1, further comprising applying a clustering algorithm to said correlations to group together parameters that represent similar behavior.

5. The method of claim 1, further comprising generating a tree which shows parameters that, based on a clustering analysis, are deemed to demonstrate similar behavior.

6. The method of claim 1, wherein identifying correlations between the measured parameters comprises detecting an offset in time between a transition point of a first monitor and a transition point of a second monitor.

7. The method of claim 1, further comprising performing a sampling analysis of a series of values of a selected parameter to identify a significant portion of said series of values of said selected parameter.

8. The method of claim 1, further comprising applying a re-sampling method to a series of values of a first parameter to facilitate a comparison with a series of values of a second parameter.

9. The method of claim 8, wherein applying the re-sampling method comprises generating interpolated values for said first parameter.

10. The method of claim 1, wherein selecting parameters which may be related to one another comprises determining whether a pair of parameters are so closely correlated to be deemed to merely represent redundant information.

11. The method of claim 1, wherein measuring the plurality of parameters comprises measuring said parameters over a time period in which a controlled load is applied to the server.

12. The method of claim 1, wherein measuring the plurality of parameters comprises measuring said parameters over a time period in which a load on said server is controllably increased.

13. The method of claim 1, further comprising displaying to a user graphs representing measurements of first and second parameters over time, and providing to the user an option to select a region of said graphs on which to perform an automated correlation analysis.

14. The method of claim 1, further comprising presenting to a user a graph of selected parameters that are deemed to be related to each other, together with numerical data indicating why the selected parameters were selected as being related.

15. The method of claim 1, wherein identifying correlations between the measured parameters comprises analyzing data values for at least the following types of parameters:

(a) server transactions executed per unit time, (b) server load, and (c) server response time.

16. A method for analyzing the performance of a server under load, the method comprising:

providing at least one client configured to send a series of requests to a server;

measuring a plurality of parameters of the server over the period of time during which the server receives requests from the at least one client;

storing a series of values representing these measured parameters;

associating a time at which each value was measured with each stored value;

identifying correlations between the measured parameters based upon the stored values for the parameters, wherein identifying correlations comprises performing a sampling analysis on each series of values in order to identify significant portions thereof and

selecting parameters which may be related to one another based upon the correlation between the measured values of the parameters.

17. The method of claim 16, wherein identifying correlations further comprises comparing a significant portion of a first series of values to a significant portion of a second series of values in order to determine a correlation between the first series and the second series.

18. The method of claim 16, wherein performing the sampling analysis comprises evaluating whether collected parameter values exhibit a statistically significant trend.

19. The method of claim 16, wherein performing the sampling analysis comprises identifying at least one parameter that is deemed uninformative.

20. The method of claim 16, wherein performing the sampling analysis comprises determining whether a sufficient number of values have been collected for a given parameter to perform a statistically significant analysis.

21. The method of claim 16, wherein performing the sampling analysis comprises evaluating whether measurement values for different parameters correspond sufficiently in time.

22. The method of claim 16, wherein performing the sampling analysis comprises identifying at least one segment of time for which data values collected for said parameters are sufficient for performing a statistically meaningful correlation analysis.

23. The method of claim 16, further comprising dividing the series of measurement values into multiple statistically significant segments based on said sampling analysis, each segment representing a different period of time.

24. The method of claim 16, further comprising identifying to a user segments of parameter data that are deemed to be significant, and providing to the user an option to perform an automated correlation analysis of said segments.

25. The method of claim 16, wherein identifying the correlations comprises generating correlation coefficients for selected pairs of parameters.

26. The method of claim 16, wherein measuring the plurality of parameters comprises measuring said parameters over a time period in which a controlled load is applied to the server.

27. The method of claim 16, wherein measuring the plurality of parameters comprises measuring said parameters over a time period in which a load on said server is controllably increased.

28. The method of claim 16, wherein performing the sampling analysis comprises analyzing data values of at least the following types of parameters:

(a) server transactions executed per unit time, (b) server load, and (c) server response time.

29. A system for facilitating the analysis of a server system, comprising:

a data collection component that collects sequences of data values of each of a plurality of performance metrics reflective of the performance of a server system; and

an automated analysis component that analyses the sequences of data values, and generates correlation coefficients for specific pairs of the performance metrics, to identify monitors that are related, to thereby facilitate identification of causal relationships that affect server performance.

30. The system of claim 29, wherein the automated analysis component evaluates whether a pair of performance metrics are related by comparing a correlation coefficient associated with the pair to a threshold value.

31. The system of claim 29, wherein the automated analysis component calculates Pearson correlation coefficients for the pairs of performance metrics.

32. The system of claim 29, wherein the automated analysis component uses a clustering algorithm to group together performance metrics that are deemed related.

33. The system of claim 29, wherein the automated analysis component generates a tree indicating performance metrics that, based on a clustering analysis, are deemed to demonstrate similar behavior.

34. The system of claim 29, wherein the automated analysis component detects offsets in time between transition points in a pair of performance metrics.

35. The system of claim 29, wherein the automated analysis component performs a sampling analysis on the sequences of measurement values to identify significant portions of said sequences.

36. The system of claim 35, wherein the automated analysis component performs the sampling analysis at least in-part by evaluating whether collected performance metric values exhibit a statistically significant trend.

37. The system of claim 35, wherein the automated analysis component further identifies performance metrics that, based on the sampling analysis, are deemed uninformative.

38. The system of claim 35, wherein the automated analysis component performs the sampling analysis at least in-part by determining whether a sufficient number of values of a given performance metric have been collected to perform a statistically meaningful analysis.

39. The system of claim 35, wherein the automated analysis component performs the sampling analysis at least in-part by evaluating whether measurement values for different performance metrics correspond sufficiently in time.

40. The system of claim 35, wherein the automated analysis components divides sequences of measurement values into multiple statistically significant segments based on said sampling analysis, each segment representing a different period of time.

41. The system of claim 29, wherein the automated analysis component identifies a segment of time for which collected data values of the performance metrics are sufficient for performing a statistically meaningful correlation analysis.

42. The system of claim 29, further comprising a user interface component that displays graphs representing sequences of data values of selected performance metrics, and provides a user option to select portions of said graphs on which to apply an automated correlation analysis.

43. The system of claim 29, wherein the data collection component collects, and the automated analysis component analyzes, data values of each of the following types of performance metrics:

(a) server transactions executed per unit time, (b) server load, and (c) server response time.

44. The system of claim 29, further comprising a component that enables a user to control a load applied to the server system during collection of the data values by said data collection component.