Method to configure monitoring thresholds using output of load or resource loadings

• US 10,198,348 B2
• Filed: 08/12/2016
• Issued: 02/05/2019
• Est. Priority Date: 08/13/2015
• Status: Active Grant

• Current Assignee: Spirent Communications Incorporated
• Original Assignee: Spirent Communications Incorporated
• Inventors: Buege, Brian
• Primary Examiner(s): Bonzo, Bryce P
• Assistant Examiner(s): Arcos, Jeison C
• Application Number:
  US15/236,262
• Publication Number:
  US 20170046254A1
• Time in Patent Office: 907 Days
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0754 by exceeding limits
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3457 Performance evaluation by simulation
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

The technology disclosed enables the automatic definition of monitoring alerts for a web page across a plurality of variables such as server response time, server CPU load, network bandwidth utilization, response time from a measured client, network latency, server memory utilization, and the number of simultaneous sessions, amongst others. This is accomplished through the combination of load or resource loading and performance snapshots, where performance correlations allow for the alignment of operating variables. Performance data such as response time for the objects retrieved, number of hits per second, number of timeouts per sec, and errors per second can be recorded and reported. This allows for the automated ranking of tens of thousands of web pages, with an analysis of the web page assets that affect performance, and the automatic alignment of performance alerts by resource participation.

Claims

1. A method of identifying and load testing resources invoked by a web app (inclusive of a web application or a web page), the method including:

initiating a load test of a subject web app that requests resources from a system under test, including,

for each testable resource in a set of multiple testable resources that are objects on a web page or objects provided by a back end system, causing a test load to vary for the testable resource in a test segment, and

collecting from the test segment at least one measure of load-dependent results related to expected user experience as the test load varies;

wherein the set of testable resources is a selected subset of resource objects, including nested resource objects, requested by a user device upon invoking the subject web app;

correlating the load-dependent results and identifying co-variant load-dependent results;

responsive to receiving user input selecting a first alert threshold for a first testable resource or for a first load-dependent result, using the identified co-variant load-dependent results to suggest or set a second alert threshold for a second load-dependent result; and

storing the first alert threshold and the second alert threshold to be applied by a monitoring device.

2. The method of claim 1, wherein the first and second alert threshold represent intervention points when measured conditions are expected to impact or begin impacting the expected user experience from invoking the subject web app.

3. The method of claim 1, further including conducting a hybrid test using a simulated user load generator to request the testable resources and an emulated browser to invoke the web app and measure response time of invoking components of the web app as the measure of load-dependent results.

4. The method of claim 1, further including conducting a hybrid test using a plurality of simulated user load generators to request the testable resources from distinct and separated locations and a plurality of emulated browsers at distinct and separated locations to invoke the web app and measure response time of invoking components of the web app as the measure of load-dependent results.

5. The method of claim 4, wherein at least some of the simulated user load generators and emulated browsers run on virtual machines using resources of a hosted cloud service provider.

6. The method of claim 1, further including:

transmitting for display at least one performance graph of the varying load for the testable resource and a dependent performance measure, wherein the display includes a control for user selection of one or more points from the performance graph as a first alert rule; and

receiving data representing user selection of the first alert rule.

7. The method of claim 1, further including, prior to the selection of a first alert rule, analyzing the dependent results to identify a knee or elbow in a trend line representing the dependent results and transmitting to a user a suggested first alert rule based on a load level relative to the knee or elbow in the trend line.

8. The method of claim 1, further including receiving user input that indicates that two or more testable resources are causally interrelated and confirming the indicated causal relationship based on the correlated testable resources or the correlated dependent results.

9. The method of claim 1, further including receiving user input that indicates that two or more testable resources are not causally interrelated and rejecting the indicated causal relationship based on the correlated testable resources or the correlated dependent results.

10. The method of claim 8, wherein the causal relationship is indicated to be that the two or more testable resources are compiled by a common database server.

11. The method of claim 8, wherein the causal relationship is indicated to be that the two or more testable resources are served by a common content delivery network.

12. The method of claim 1, at least one of a first alert rule or a second alert rule specifies a threshold and is triggered when a load for the testable resource or for a dependent performance measure reaches or crosses the threshold.

20. At least one device, including a processor, a network interface, and memory storing computer instructions that, when executed on the processor, cause the device to carry out the method of claim 1.

21. A non-transitory computer readable medium storing instructions that, when executed on a processor device, cause the device to carry out the method of claim 1.

13. A method of identifying and load testing resources critical to invoking a web app, the method including:

parsing a subject web app to identify resources, including nested resource objects, loaded upon invoking the subject web app that are objects on a web page or objects provided by a back end system;

conducting a hybrid load test of a selected plurality of the identified resources, including;

load testing response to requests for particular resource objects among the identified resources, wherein the load testing simulates a multiplicity of user sessions requesting the particular resource objects and a number of the user sessions varies during the load testing; and

experience testing load time of at least components of the subject web app during the load testing, wherein the experience testing

emulates a browser,

requests the subject web app,

requests the identified resources and nested resource objects, and

records at least response times for loading the identified resources and nested resource objects;

identifying from the load testing and experience testing some of the particular resource objects as more sensitive than others to how many user sessions request the particular resource object; and

generating data for display that includes the identified more sensitive particular resource objects.

14. The method of claim 13 further including iterating through the selected identified resources and load testing requests for individual particular resource objects while experience testing the subject web app.

15. The method of claim 13, further including:

iterating through the selected identified resources;

load testing requests for pairs particular resource objects while experience testing the subject web app; and

correlating the pairs of particular resource objects using results of the experience testing.

16. The method of claim 13, further including:

receiving at least one grouping of the selected identified resources as being loaded from a common server; and

load testing requests for the grouping of resources from the common server while experience testing the subject web app.

17. The method of claim 13, further including varying a number of simulated user sessions through a predetermined range.

18. The method of claim 13, further including varying a number of the simulated user sessions to produce a predetermined load time of the subject web app.

19. The method of claim 13, wherein component response times are divided into at least an initial response time and a component load time.

22. At least one device, including a processor, a network interface, and memory storing computer instructions that, when executed on the processor, cause the device to a hybrid load test of a selected plurality of identified resources that are objects on a web page or objects provided by a back end system, including:

load testing response to requests for particular resource objects among the identified resources, wherein the load testing simulates a multiplicity of user sessions requesting the particular resource objects and a number of the user sessions varies during the load testing; and

experience testing load time of at least components of a subject web app during the load testing, wherein the experience testing

emulates a browser,

requests the subject web app,

requests the identified resources and nested resource objects, and

records at least response times for loading the identified resources and nested resource objects;

identifying from the load testing and experience testing some of the particular resource objects as more sensitive than others to how many user sessions request the particular resource object; and

generating data for display that includes the identified more sensitive particular resource objects.

23. A non-transitory computer readable medium storing instructions that, when executed on a processor device, cause the device to a hybrid load test of a selected plurality of identified resources that are objects on a web page or objects provided by a back end system, including:

load testing response to requests for particular resource objects among the identified resources, wherein the load testing simulates a multiplicity of user sessions requesting the particular resource objects and a number of the user sessions varies during the load testing; and

experience testing load time of at least components of a subject web app during the load testing, wherein the experience testing

emulates a browser,

requests the subject web app,

requests the identified resources and nested resource objects, and

records at least response times for loading the identified resources and nested resource objects;

identifying from the load testing and experience testing some of the particular resource objects as more sensitive than others to how many user sessions request the particular resource object; and

generating data for display that includes the identified more sensitive particular resource objects.