Real-time, multi-tier load test results aggregation

• US 9,436,579 B2
• Filed: 07/19/2010
• Issued: 09/06/2016
• Est. Priority Date: 07/19/2010
• Status: Active Grant

• Current Assignee: Akamai Technologies, Inc.
• Original Assignee: Soasta Incorporated (Akamai Technologies, Inc.)
• Inventors: Broda, Tal, Hemmert, Michael, Solnit, Matthew
• Primary Examiner(s): SUN, LI P
• Application Number:
  US12/804,338
• Publication Number:
  US 20120017156A1
• Time in Patent Office: 2,241 Days
• Field of Search: 715/736
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet

Abstract

A method for real-time analysis of results from a load test performed on a target website includes periodically computing first-level aggregated test results within each of a plurality of load server instances that generate a load on the target website. The first-level aggregated test results are computed from data received by each of the load server instances from the target website every first time interval. The first-level aggregated test results are then periodically sent from each of the load server instances to an associated one of a plurality of analytic server instances every second time interval. The first-level aggregated test results are aggregated by each of the analytic server instances to produce second-level aggregated test results, which test results may then further aggregated to produce third-level aggregated test results at a data storage instance in real-time.

Claims

1. A computer-implemented method for real-time analysis of results from a load test as the load test is being performed on a target website, comprising:

(a) computing first-level aggregated test results within each of a plurality of load server instances that generate a load on the target website, the load including requests of a plurality of virtual users, the first-level aggregated test results including first-level response time statistics computed from data received by each of the plurality of load server instances from the target website in response to the generated load, the first-level aggregated test results including statistics computed by an embedded component within each of the plurality of load server instances every first time interval, the statistics including an average response time across a subset of the plurality of virtual users;

(b) sending the first-level aggregated test results from each of the plurality of load server instances to an associated one of a plurality of analytic server instances every second time interval, the second time interval being longer than the first time interval;

(c) computing second-level aggregated test results by each of the plurality of analytic server instances in response to receiving the first-level aggregated test results from the plurality of load server instances every second time interval, the second-level aggregated test results including second-level response time statistics computed from the first-level aggregated test results;

(d) storing the second-level aggregated test results in a data storage instance;

(e) running a query to the data storage instance to perform one or more statistical functions on the second-level aggregated test results, thereby producing third-level aggregated test results, the third-level aggregated test results including a website response time statistic across all of the virtual users; and

(f) communicating the third-level aggregated test results to a browser application for providing an analytic dashboard with a graphical user interface that outputs the website response time statistic,

wherein steps (a)-(f) are each performed in real-time as the load test is being performed on the target website.

2. The computer-implemented method of claim 1 wherein the plurality of load server instances and the plurality of analytic server instances are provided by at least one cloud provider.

3. The computer-implemented method of claim 1 wherein the first time interval is approximately one second.

4. The computer-implemented method of claim 1 wherein the second time interval is approximately five seconds.

5. The computer-implemented method of claim 1 further comprising changing, in real-time responsive to user input, a time format of a chart displayed on the analytic dashboard of the third-level aggregated test results.

6. The computer-implemented method of claim 1 wherein the periodically sending of the first-level aggregated test results comprises periodically sending a batch message to the associated one of the plurality of analytic server instances, the batch message comprising all of the first-level aggregated test results computed during the second time interval.

7. A non-transitory computer-readable storage medium encoded with a computer program product, which, when executed, is operable to:

(a) compute first-level aggregated test results within each of a plurality of load server instances during execution of a load test that generates a load on a target website, the load including requests of a plurality of virtual users, the first-level aggregated test results including first-level response time statistics computed from data received by each of the plurality of load server instances from the target website in response to the generated load, the first-level aggregated test results including statistics computed by an embedded component within each of the plurality of load server instances every first time interval, the statistics including an average response time across a subset of the plurality of virtual users;

(b) send the first-level aggregated test results from each of the plurality of load server instances to an associated one of a plurality of analytic server instances every second time interval, the second time interval being longer than the first time interval;

(c) compute second-level aggregated test results by each of the plurality of analytic server Instances in response to receiving the first-level aggregated test results from the plurality of load server instances every second time interval, the second-level aggregated test results including second-level response time statistics computed from the first-level aggregated test results;

(d) store the second-level aggregated test results in a data storage instance;

(e) run a query to the data storage instance to perform one or more statistical functions on the second-level aggregated test results, thereby producing third-level aggregated test results, the third-level aggregated test results including a website response time statistic across all of the virtual users; and

(f) communicate the third-level aggregated test results to a browser application for providing an analytic dashboard with a graphical user interface that outputs the website response time statistic,

wherein (a)-(f) are each performed in real-time during the running of the load test.

8. The non-transitory computer-readable storage medium of claim 7 wherein the computer program product, when executed, is further operable to periodically send one or more messages to the associated one of the plurality of analytic server instances, each message comprising the first-level aggregated test results computed in one of the first time intervals.

9. The non-transitory computer-readable storage medium of claim 7 wherein the computer program product, when executed, is further operable to store the website response time statistic in the data storage instance, the website response time statistic being available on a real-time basis to the browser application.

10. The non-transitory computer-readable storage medium of claim 7 wherein the plurality of load server instances and the plurality of analytic server instances are provided by more than one cloud provider.

11. The non-transitory computer-readable storage medium of claim 7 wherein the first time interval is approximately one second and the second time interval is approximately five seconds.

12. The non-transitory computer-readable storage medium of claim 7 wherein the computer program product, when executed, is further operable to change, in real-time responsive to user input, a time format of a chart displayed on the analytic dashboard of the third-level aggregated test results.

13. A computer-implemented method for real-time analysis of results from a load test as the load test is being performed on a target website, comprising:

computing first-level aggregated test results within each of a plurality of load server instances, each load server instance simulating a plurality of virtual users that generate a load on the target website, the first-level aggregated test results including statistics computed by an embedded component within each of the plurality of load server instances as the load test is being performed from data received by each of the plurality of load server instances from the target website in response to the generated load, the first-level aggregated test results being computed every N seconds, where N is a positive integer, the statistics including an average response time across a subset of the plurality of virtual users;

sending, as the load test is being performed, a batch message that includes all of the first-level aggregated test results computed by each of the plurality of load server instances to an associated one of a plurality of analytic server instances every M seconds, where M is an integer greater than N;

computing, as the load test is being performed, second-level aggregated test results by each of the plurality of analytic server instances in response to receiving the first-level aggregated test results from the plurality of load server instances every M seconds, the second-level aggregated test results including second-level response time statistics computed from the first-level aggregated test results;

storing, as the load test is being performed, the second-level aggregated test results in a data storage instance;

running, as the load test is being performed, a query to the data storage instance to perform one or more statistical functions on the second-level aggregated test results, thereby producing third-level aggregated test results, the third-level aggregated test results including a website response time statistic across all of the virtual users; and

communicating, as the load test is being performed, the third-level aggregated test results to a browser application for providing an analytic dashboard with a graphical user interface that outputs the website response time statistic in real-time.

14. The computer-implemented method of claim 13 wherein the plurality of load server instances and the plurality of analytic server instances are provided by more than one cloud provider.