Load test charts with standard deviation and percentile statistics

• US 10,067,850 B2
• Filed: 02/01/2016
• Issued: 09/04/2018
• Est. Priority Date: 07/19/2010
• Status: Active Grant

• Current Assignee: Akamai Technologies, Inc.
• Original Assignee: Akamai Technologies, Inc.
• Inventors: Vazac, Charles A., Broda, Tal, Hemmert, Michael, Sharma, Mukul
• Primary Examiner(s): To, Jennifer N
• Assistant Examiner(s): Li, Liang Y
• Application Number:
  US15/012,839
• Publication Number:
  US 20160147632A1
• Time in Patent Office: 946 Days
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

A method for real-time analysis of results from a load test performed on a target website includes calculating first-level aggregated test results within each of a plurality of load server instances that generate a load on the target website. The first-level aggregated test results are calculated from data points received by each of the load server instances from the target website. The first-level aggregated test results include a sum of the data points, a count of the number of the data points, a sum of squares of the data points, and an average of the data points. A standard deviation result (STDEV) is calculated and chart is generated on a display via a graphical user interface. The chart provides a visual representation of a performance metric for the load test based on the standard deviation result.

Claims

1. A computer-implemented method for real-time analysis of results obtained from a load test as the load test is being performed on a target website, comprising:

(a) computing first-level aggregated test results within each of a plurality of load server instances that generate a load on the target website, the first-level aggregated test results including first-level statistics computed from a set of data points received over a time period by each of the load server instances from the target website, the first-level statistics being computed by an embedded component coupled with a memory within each of the load server instances, the first-level statistics including a sum of the data points, a count of the number of the data points, a sum of squares of the data points, and an average of the data points;

(b) sending the first-level aggregated test results over a network from each of the load server instances to an associated one of a plurality of analytic server instances;

(c) aggregating, in one or more analytic server instances, all of the first-level aggregated test results computed by the load server instances resulting in second-level aggregated test results;

(d) aggregating in a main instance computer, which includes a processor coupled with a memory, all of the second-level aggregated test results resulting in a cumulative sum, a total count, a total sum of squares, and a cumulative average, of all of the data points across an entire data set;

(e) calculating by the main instance computer a standard deviation result (STDEV) as

STDEV=SQRT[{(X₁²+X₂²+ . . . +X_N²)+NA²−

2A(X₁+X₂+ . . . +X_N)}/N];

where N is the total count, A is the cumulative average, X₁, X₂ . . . X_N are the data points across the entire data set, (X₁²+X₂²+ . . . +X_N²) is the total sum of squares, and (X₁+X₂+ . . . +X_N) is the cumulative sum; and

wherein steps (a)-(e) are each performed in real-time as the load test is being performed on the target website.

2. The computer-implemented method of claim 1 further comprising:

(f) providing on the main instance computer a graphical user interface (GUI) that displays a chart on a screen, the chart providing a visual representation of a performance metric for the load test based on the standard deviation result.

3. The computer-implemented method of claim 2 further comprising:

communicating the standard deviation result to a browser application running on the main instance computer, the browser application providing an analytic dashboard with the graphical user interface.

4. The computer-implemented method of claim 2 wherein the chart is generated in real-time responsive to an input command of the graphical user interface.

5. The computer-implemented method of claim 1 wherein the performance metric comprises a confidence interval percentile (X).

6. The computer-implemented method of claim 1 further comprising calculating by the main instance computer the confidence interval percentile (X) as

X={(A+(Z*STDEV)};

where Z is a constant used for calculating a certain percentile.

7. The computer-implemented method of claim 1 further comprising calculating by the main instance computer a delta STDEV based on incremental changes occurring as the load test is running in real-time.