EXECUTING SOFTWARE PERFORMANCE TEST JOBS IN A CLUSTERED SYSTEM
First Claim
1. A computer-implemented method for collecting performance statistics for an application running in a multiple-host system, comprising the steps of:
- receiving, at a testing framework, input that specifies test details, wherein the test details comprise a test plan;
based on said input, said testing framework executing said test plan on an execution host, wherein executing the test plan comprises;
initiating execution of at least a portion of an application on each of a plurality of hosts;
receiving, from each host in the plurality of hosts, performance data indicating performance statistics for a particular period of time;
based on said performance data, the testing framework generating a test result comprising a plurality of data reports for a plurality of performance metrics; and
the testing framework presenting said test result to a user.
3 Assignments
0 Petitions
Accused Products
Abstract
Using a testing framework, developers may create a test module to centralize resources and results for a software test plan amongst a plurality of systems. With assistance from the testing framework, the test module may facilitate the creation of test cases, the execution of a test job for each test case, the collection of performance statistics during each test job, and the aggregation of collected statistics into organized reports for easier analysis. The test module may track test results for easy comparison of performance metrics in response to various conditions and environments over the history of the development process. The testing framework may also schedule a test job for execution when the various systems and resources required by the test job are free. The testing framework may be operating system independent, so that a single test job may test software concurrently on a variety of systems.
173 Citations
52 Claims
-
1. A computer-implemented method for collecting performance statistics for an application running in a multiple-host system, comprising the steps of:
-
receiving, at a testing framework, input that specifies test details, wherein the test details comprise a test plan; based on said input, said testing framework executing said test plan on an execution host, wherein executing the test plan comprises; initiating execution of at least a portion of an application on each of a plurality of hosts; receiving, from each host in the plurality of hosts, performance data indicating performance statistics for a particular period of time; based on said performance data, the testing framework generating a test result comprising a plurality of data reports for a plurality of performance metrics; and the testing framework presenting said test result to a user. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42)
-
-
17. A computer-implemented method for executing a test job in a test cluster comprising a plurality of systems, comprising the steps of, at a testing framework:
-
(1) receiving input specifying a test details for a test job, wherein the test details include a test plan for said test job; (2) determining one or more systems in said test cluster which may be used to implement one or more steps of said test plan for said test job; (3) determining whether any of said one or more systems are reserved for any other test job; (4) if none of said one or more systems are reserved any other test job, executing said test job according to said test plan; and (5) if any one of said one or more systems are reserved for another test job, waiting for a period of time and then repeating steps 3-5. - View Dependent Claims (18, 19, 20, 43, 44, 45, 46)
-
-
21. A computer-implemented method for executing a test job for testing software performance in a test cluster comprising a plurality of systems, comprising the steps of:
-
at said testing framework, performing the steps of; receiving input specifying a test details for said test job, wherein the test details include a test plan for said test job; determining one or more resources required to implement said test plan; determining one or more systems in said test cluster that may be said one or more resources to implement the steps of said test plan for said test job; determining that a particular system of said one or more systems does not comprise at least one of said one or more resources; causing said one or more resources to be installed on said particular system; executing said test job according to said test plan. - View Dependent Claims (22, 23, 24, 25, 26, 47, 48, 49, 50, 51, 52)
-
Specification