System load testing coordination over a network

• US 6,601,020 B1
• Filed: 05/03/2000
• Issued: 07/29/2003
• Est. Priority Date: 05/03/2000
• Status: Expired due to Fees

• Current Assignee: Eureka Software Solutions, Inc.
• Original Assignee: Eureka Software Solutions, Inc.
• Inventors: Myers, Monty G.
• Primary Examiner(s): Hoff, Marc S.
• Assistant Examiner(s): Raymond, Edward
• Application Number:
  US09/565,062
• Time in Patent Office: 1,182 Days
• Field of Search: 702/186, 702/187, 702/57, 702/81, 702/84, 702/108, 702/117, 702/118, 702/176, 702/178, 702/179, 702/182, 340/286.01, 340/286.02, 455/405, 455/423, 455/67.4, 709/223, 709/224, 709/105, 709/225, 709/226, 714/25, 714/26, 714/27, 714/31
• US Class Current: 702/186
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet

Abstract

A method and system using one or more load cells coordinates load testing of one or more sites under test over a network such as the Internet in accordance with one or more load test requests. In one embodiment, such method and system facilitates an economic, efficient, and accurate evaluation of the capacity of the SUTs to support concurrent users of the SUTs. A load test request is received from a User and provides information concerning a load test to be performed on a site under test (SUT). A load test is generated, and the load test is allocated to one or more load cells. The load cells perform the load test by sending load test data to the SUT and receiving responses from the SUT. Portions of a load test may be allocated to different load cells, which may be geographically remote from the SUT and/or geographically diverse. Multiple concurrent pending load tests may be processed for the same SUT. A load cell may concurrently perform all or a portion of multiple different load tests for multiple different SUT'"'"'s simultaneously. A reporting function generates a report based on the responses received from the SUT. The reporting function may in addition or alternatively allow the User to view real-time results during load test execution.

Claims

1. A method of load testing at least one site under test (SUT) in accordance with at least one load test request which provides information for load testing at least one SUT, the method comprising:

reviewing at least one load test request;

generating at least one load test corresponding to at least one load test request;

allocating the load test to a plurality of load cells;

sending respective load test data in accordance with the load test from the load cells to at least one SUT to collectively conduct the load test; and

receiving responses with the load cells to the load test data from the SUT.

2. The method as in claim 1 wherein allocating the load test to the plurality of load cells comprises:

determining an availability of a sufficient number of the plurality of load cells which are available to collectively conduct the load test of at least one SUT; and

distributing the load test to the number of the available load cells until the load test is fully allocated among the plurality of load cells.

4. The method as recited in claim 1 wherein the at least one load test request comprises a test type field that includes a value corresponding to one of a plurality of test types.

5. The method as recited in claim 4 wherein the plurality of test types includes a first load test request type, a second load test request type, and a one-click load test request type.

6. The method as recited in claim 5 wherein the first load test request type comprises:

an SUT address field;

a requested test start date field;

a requested test start time field;

a requested test duration field;

a requested simulated user multiplier field; and

a requested user scenario field.

7. The method as recited in claim 6 wherein the first load test request type further comprises:

a browser type;

and a browser version.

8. The method as recited in claim 5 wherein the second load test request type comprises:

an SUT address field;

a requested test start time field;

a requested test duration field;

a requested test start date field;

a requested maximum number of users to simulate field;

a requested user scenario field; and

a target average response time field.

9. The method as in claim 1 further comprising:

generating a load test report based on the received response.

10. A load test report generated in accordance with the method recited in claim 9.

11. The method as in claim 1 further comprising:

receiving a plurality of load test requests;

reviewing the plurality of load test requests;

generating a plurality of load tests corresponding to the load test requests;

allocating the load tests to the plurality of load cells; and

conducting a plurality of load tests of at least one SUT in accordance with respective load test requests, wherein at least one load test is conducted by more than one of the load cells.

12. The method as in claim 11 further comprising:

conducting a plurality of load tests of a plurality of SUTs in accordance with respective load test requests, wherein at least one load test is conducted by more than one of the load cells.

3. The method as in 1 wherein at least one of the number of the available load cells is geographically remote from the at least one SUT.

13. A network system to load test at least one site under test (SUT) in accordance with at least one load test request submitted by a user system, the system comprising:

at least one computer having a processor and a memory coupled to the processor, wherein the processor includes circuitry to receive a load test request from a user system to load test at least one SUT; and

program code stored by the memory and executable by the processor, wherein the program code comprises instructions for;

reviewing the request to load at least one SUT;

generating at least one load test based upon at least one load test request;

allocating the load test to a plurality of load cells;

sending respective load test data in accordance with the load test to at least one SUT from the load cells;

receiving responses to load test data from at least one SUT, wherein the load test data correlates with the load test; and

generating a load test report based on the received responses.

14. The network system of claim 13 wherein each of the plurality of load cells comprises:

a load cell manager;

a load cell test executive; and

a data buffer coupled to the load cell manager and the load cell test executive.

15. The network system of claim 13 wherein the processor further includes circuitry to receive, via the Internet, a load test request from a user system to load test at least one SUT.

16. The network system of claim 13 wherein the program code is logically partitioned into a plurality of functional subsystems, the functional subsystems comprising:

a public web site layer;

a private web site layer;

a load test database;

a load test system manager;

a work allocator; and

a data collection system.

17. A load test report generated by the network system recited in claim 13.

18. The network system of claim 16 wherein the load test database is capable of storing login information, security information, captured user simulation scripts, and scenarios.

19. The network system of claim 16 wherein the load test database includes a relational database server.

20. The network system of claim 16 wherein the load test database includes a hierarchical database server.

21. The network system of claim 16 wherein the load test database includes an object store database server.

22. A computer-readable medium having a computer program accessible therefrom, the computer program comprising instructions for:

reviewing a request to load at least one site under test (SUT);

generating at least one load test based upon at least one load test request;

allocating the load test to a plurality of load cells;

sending respective load test data in accordance with the load test to at least one SUT from the load cells;

receiving responses to load test data from at least one SUT, wherein the load test data correlates with the load test; and

generating a load test report based on the received responses.

23. A load test report generated in accordance with the computer program recited in claim 22.

24. The computer program product of claim 22 wherein the computer readable medium includes any of a magnetic storage medium, including disk and tape storage medium;

an optical storage medium, including compact disk memory and a digital video disk storage medium;

a nonvolatile memory storage memory;

a volatile storage medium; and

data transmission medium including packets of electronic data and electromagnetic waves modulated in accordance with the instructions.

25. An apparatus comprising:

means for reviewing at least one load test request, wherein the load test request includes information for load testing at least one site under test (SUT);

means for generating at least one load test based upon at least one load test request;

means for allocating the at least one load test to a plurality of load cells;

means for sending respective load test data in accordance with the load test from the load cells to at least one SUT to collectively conduct the load test; and

means for receiving responses with the load cells to the load test data from the SUT to the respective load test data.

26. An apparatus for load testing at least one site under test (SUT) in accordance with at least one load test request generated with a user data processing system, the apparatus comprising:

a plurality of load cells capable of communicating with the at least one SUT;

a load test system capable of communicating with the load cells and the user data processing system, the load test system having a work allocator capable of (i) receiving the at least one load test request, (ii) querying the plurality of load cells to select a group of the load cells which is available to collectively process the at least one load test request, and (iii) selecting the group of load cells to allocate load test tasks to the select group of load cells to collectively process the at least one load test request.

27. A method of load testing a plurality of sites under test (SUTs) in accordance with at least one load test request that provides information for load testing the plurality of SUTs the method comprising:

reviewing at least one load test request;

generating at least one load test based on one load test request;

allocating at least one load test to at least one load cell;

sending respective load test data in accordance with the load test from the load cells to the SUTs to conduct the load test; and

receiving responses with at least one load cell to the load test data from the SUTs.

28. The method as in claim 27 further comprising:

generating a load test report based on the received responses.

29. A load test report generated in accordance with the method recited in claim 28.

30. The method as in claim 27 further comprising:

receiving a plurality of load test requests;

reviewing the plurality of load test requests;

generating a plurality of load tests corresponding to the load test requests;

allocating the plurality of load tests to at least one load cell;

sending respective load test data in accordance with the plurality of load tests from the at least one load cell to the SUTs to conduct the load tests; and

receiving responses with the at least one load cell to the respective load test data from the SUTs.

31. A network system to load test a plurality of sites under test (SUTs) in accordance with at least one load test request submitted by a user system, the system comprising:

at least one computer having a processor and a memory coupled to the processor, wherein the processor includes circuitry to receive a load test request from a user system to load test at least one SUT; and

program code stored by the memory and executable by the processor, wherein the program code comprises instructions for;

reviewing at least one load test request;

generating at least one load test based on one load test request;

allocating at least one load test to at least one load cell;

sending respective load test data in accordance with the load test from the load cells to the SUTs to conduct the load test;

receiving responses with at least one load cell to the load test data from the SUTs; and

generating at least one load test report based on the received responses.

32. The network system of claim 31 wherein the at least one SUT comprises one or more application servers.

33. The network system of claim 31 wherein the at least one SUT comprises one or more database servers.

34. A load test report generated by the network system recited in claim 31.

35. A computer-readable medium having a computer program accessible therefrom, the computer program comprising instructions for:

reviewing at least one load test request;

generating at least one load test based on one load test request;

allocating at least one load test to at least one load cell;

sending respective load test data in accordance with the load test from the load cells to at least one of a plurality of sites under test (SUTs) to conduct the load test;

receiving responses with at least one load cell to the load test data from the SUTs; and

generating at least one load test report based on the received responses.

36. The computer program product of claim 35 wherein the computer readable medium includes any of an magnetic storage medium, including disk and tape storage medium;

an optical storage medium, including compact disk memory and a digital video disk storage medium;

a nonvolatile memory storage memory;

a volatile storage medium; and

data transmission medium including packets of electronic data and electromagnetic waves modulated in accordance with the instructions.

37. A load test report generated in accordance with the computer program recited in claim 35.

38. An apparatus comprising:

a means for reviewing at least one load test request;

a means for generating at least one load test based on one load test request;

a means for allocating at least one load test to at least one load cell;

a means for sending respective load test data in accordance with the load test from the load cells to at least one of a plurality of sites under test (SUTs) to conduct the load test;

a means for receiving responses with at least one load cell to the load test data from the SUTs; and

a means for generating at least one load test report based on the received responses.

39. The apparatus recited in claim 38 wherein the means for generating at least one load test report based on the received responses further comprises:

a means for generating a real-time report of the received responses that is accessible to the a user during the load test.

40. The apparatus recited in claim 38 wherein the means for generating at least one load test report based on the received responses further comprises:

a means for generating a post-test report of the received responses that is accessible to a user after the load test has completed execution.

41. A load test report generated by the apparatus recited in claim 38.

42. An apparatus for load testing a plurality of sites under test (SUTs) in accordance with at least one load test request generated with a user data processing system, the apparatus comprising:

at least one load cell capable of communicating with the SUTs;

a load test system capable of communicating with the at least one load cell and the user data processing system, the load tests system having a work allocator capable of (i) receiving the load test request, (ii) querying the at least one load cell to select a group of the load cells which is available to collectively process the load test request, and (iii) selecting the group of load cells to allocate load test tasks to the select group of load cells to collectively process the load test request.

43. The apparatus of claim 42 wherein the at least one load cell further comprises:

a load cell manager;

a load cell test executive; and

a data buffer coupled to the load cell manager and to the load cell test executive.

44. The apparatus of claim 42 wherein the work allocator is capable of (i) receiving a plurality of load test requests, (ii) querying the at least one load cell to select a group of the load cells which is available to collectively process the plurality of load test requests, and (iii) selecting the group of load cells to allocate load test tasks to the select group of load cells to collectively process the plurality of load test requests.

45. A method of load testing at least one site under test (SUT) in accordance with a plurality of concurrently pending load test requests, wherein each of the load test requests provides respective information for load testing at least one SUT, the method comprising:

reviewing the plurality of load test requests;

generating a plurality of load tests corresponding to respective one load test requests;

allocating the load tests to at least one load cell;

for each of the load tests, sending respective load test data in accordance with the respective load test from the load cells to at least one SUT; and

receiving responses with the load cells to the respective load test data from at least one SUT.

46. The method as in claim 45 further comprising:

generating a respective load test report for at least one SUT based on the received responses.

47. A load test report generated in accordance with the method recited in claim 46.

48. The method as in claim 45 further comprising:

for each of the load tests, sending respective load test data in accordance with the respective load test from the load cells to a plurality of SUTs; and

receiving responses with the load cells to the respective load test data from the plurality of SUTs.

49. The method as in claim 45 wherein sending further comprises:

for each of the load tests, serially sending respective load test data in accordance with the respective load test from the load cells to the at least one SUT.

50. The method as in claim 45 wherein sending further comprises:

for each of the load tests, concurrently sending respective load test data in accordance with the respective load test from the load cells to the at least one SUT.

51. A network system to load test at least one site under test (SUT) in accordance with a plurality of concurrently pending load test requests submitted by at least one user system, the system comprising:

at least one computer having a processor and a memory coupled to the processor, wherein the processor includes circuitry to receive a plurality of load test requests from at least one user system to load test at least one SUT; and

program code stored by the memory and executable by the processor, wherein the program code comprises instructions for;

reviewing the plurality of load test requests;

generating a plurality of load tests corresponding to respective one load test requests;

allocating the load tests to at least one load cell;

for each of the load tests, sending respective load test data in accordance with the respective load test from the load cells to at least one SUT; and

receiving responses with the load cells to the respective load test data from at least one SUT.

52. The network system of 51 wherein the processor includes circuitry to receive a plurality of load test requests from a plurality of user systems to load test a plurality of SUTs.

53. The network system of 51 wherein the processor includes circuitry to receive a plurality of load test requests from a plurality of user systems to load test at least one SUT.

54. The network system of 51 wherein the at least one load cell further comprises:

a load cell manager;

a load cell test executive; and

a data buffer coupled to the load cell manager and to the load cell test executive.

55. A computer-readable medium having a computer program accessible therefrom, the computer program comprising instructions for:

reviewing a plurality of load test requests;

generating a plurality of load tests corresponding to respective one load test requests;

allocating the load tests to at least one load cell;

for each of the load tests, sending respective load test data in accordance with the respective load test from the load cells to at least one site under test (SUT); and

receiving responses with the load cells to the respective load test data from at least one SUT.

56. The computer program product of claim 55 wherein the computer readable medium includes any of an magnetic storage medium, including disk and tape storage medium;

an optical storage medium, including compact disk memory and a digital video disk storage medium;

a nonvolatile memory storage memory;

a volatile storage medium; and

data transmission medium including packets of electronic data and electromagnetic waves modulated in accordance with the instructions.

57. An apparatus comprising:

a means for reviewing the plurality of load test requests;

a means for generating a plurality of load tests corresponding to respective one load test requests;

a means for allocating the load tests to at least one load cell;

a means sending, for each of the load tests, respective load test data in accordance with the respective load test from the load cells to at least one site under test (SUT); and

;

a means for receiving responses with the load cells to the respective load test data from at least one SUT.

58. An apparatus for load testing at least one site under test (SUT) in accordance with a plurality of concurrently pending load test requests generated with a user data processing system, the apparatus comprising:

a plurality of load cells capable of communicating with the SUT;

a load test system capable of communicating with the load cells and the user data processing system, the load test system having a work allocator capable of (i) receiving the plurality of load test requests, (ii) generating a plurality of load tests corresponding to respective one load test requests, (iii) allocating the load tests to at least one load cell, (iv) sending, for each of the load tests, respective load test data in accordance with the respective load test from the load cells to at least one SUT, and (v) receiving responses with the load cells to the respective load test data from at least one SUT.

59. A system for load testing at least one site under test (SUT) in accordance with at least one load test request provided by at least one user system, the apparatus comprising:

a means for receiving communications from the at least one user system, where the means for receiving communications includes a security means for preventing a user system from causing load to be provided to the at least one SUT when the user system is not authorized to load the at least one SUT and also includes a Run Load Test web page;

a means for setting up an account corresponding to the user system, where the means for setting up an account includes a log-in page and also includes one or more Account Setup transaction pages;

a means for creating one or more simulated user scripts based upon input from the user system, the means for creating simulated user scripts including a capture interface and a probe interface, the means for creating simulated user scripts also including a means for saving the simulated user scripts in a memory;

a means for creating one or more scenarios, where the one or more scenarios include one or more of the simulated user scripts;

a means for creating one or more load tests based on the one or more scenarios;

a means for scheduling the one or more load tests for execution;

a means for executing the one or more scheduled load tests;

a means for receiving load test results based on the execution of the one or more scheduled load tests;

a means for processing the load test results to provide report data; and

a means for reporting the report data to the user system.

60. The system of claim 59 wherein the means for reporting the report data to the user system comprises a status home page that includes a means for generating a real-time report of the load test results that is accessible to the user system during the load test.

61. The system of claim 59 wherein the means for reporting the report data to the user system comprises a status home page that includes a means for generating a post-test report of the load test results that is accessible to the user system after the load test has completed execution.

62. The system of claim 59 wherein the capture interface further comprises a means for capturing, cataloguing, and managing simulated user actions without requiring any software modification of the user system.

63. The system of claim 59 wherein the probe interface further comprises a means for automatically probing the at least one SUT to create a random sequence of simulated user actions.

64. The system of claim 59 wherein the capture interface further comprises a capture means for capturing, cataloguing, and managing simulated user actions, the capture means including a software application that is installed on a hardware server associated with the at least one SUT.

65. The system of claim 59 wherein the means for creating one or more simulated user scripts includes a means for editing a previously-created simulated user script saved in a memory.

66. The system of claim 59 wherein the means for creating one or more simulated user scripts includes a means for selecting a previously-created simulated user script saved in a memory.

67. The system of claim 59 wherein:

the means for receiving communications is capable of receiving communications from a plurality of user systems;

the means for creating one or more simulated user scripts is capable of creating a plurality of simulated user scripts associated with each of the plurality of user systems;

the means for creating one or more scenarios is capable of creating a plurality of sets of scenarios, each set of scenarios being associated with one of the plurality of user systems;

the means for creating one or more load tests is capable of creating a plurality of load tests, each of the plurality of load tests being based on one of the scenarios included in the plurality of sets of scenarios;

the means for scheduling is capable of scheduling the plurality of load tests for execution;

the means for executing is capable of executing the plurality of scheduled load tests; and

the means for receiving load test results is capable of receiving load test results based on the execution of the plurality of scheduled load tests.

68. The system of claim 59 wherein the means for reporting the report data to the user system is capable of providing real-time review of the user data.

69. The system of claim 59 wherein the means for reporting the report data to the user system provides post-test review of the user data.

70. The system of claim 59 wherein the means for reporting the report data further comprises a Status Home Page.

71. A load test report generated by the system for load testing recited in claim 59.

72. An apparatus that performs at least one load test on at least one site under test (SUT) in accordance with at least one load test request submitted by a user system, the apparatus comprising:

at least one computer having a processor and a memory coupled to the processor, wherein the processor includes circuitry to receive the load test request, the circuitry further including;

a capture subsystem, the capture subsystem being capable of generating at least one captured simulated user script;

a probe subsystem, the probe subsystem being capable of generating at least one random simulated user script;

a load test database that is capable of receiving the at least one captured simulated user script from the capture subsystem and that is also capable of receiving the at least one random simulated user script from the probe subsystem;

a setup subsystem that is configured to receive information from the user system and that is capable of receiving the at least one captured simulated user script from the load test database and that is also capable of receiving the at least one random simulated user script from the load test database;

a FIFO load test request queue that is configured to receive the at least one load test request from the setup subsystem;

a load test system manager that receives the at least one load test request from the FIFO load test request queue and that is configured to send confirmation information corresponding to the at least one load test request to a scheduled/in process database;

a work allocator that is configured to receive the confirmation information from the scheduled/in process database and that is configured to allocate the load test to at least one load cell;

a data collection system that is configured to receive response data from the at least one load cell and to generate formatted data using the response data;

a load test results database that is configured to receive the formatted data from the data collection system; and

an internal web control site that is configured to manage at least one internal process that occurs within the circuitry.

73. The apparatus of claim 72 wherein the work allocator is further configured to allocate the load test among a plurality of load cells.

74. The apparatus of claim 73 wherein a computer processing system is logically partitioned to include two or more of the plurality of load cells.

75. The apparatus of claim 73 wherein at least one of the load cells comprising the plurality of load cells comprises more than one computer processing system.

76. The apparatus of claim 72 wherein the capture subsystem is further configured to:

receive a time-sequenced series of actions;

record the time-sequenced series of actions as a captured simulated user script;

save the captured simulated user script in the memory; and

assign the saved simulated user script a unique script identifier.

77. The apparatus of claim 76 wherein the capture subsystem is further configured to use the unique script identifier to retrieve the saved simulated user script from the memory.

78. The apparatus of claim 72 wherein the processor includes circuitry to receive the load test request via the Internet.

79. The apparatus of claim 72 wherein the work allocator is further configured to allocate the load test among one or more measurement/QA cells.

80. The apparatus of claim 79 wherein the one or more measurement/QA cells are configured to communicate with the SUT at a connection speed that the user system uses to communicate with the SUT.

81. The apparatus of claim 72 wherein the data collection system is further configured to receive one or more packets of measured data from a load cell manager.