System and method for testing software using data-driven test variations
First Claim
1. A system for testing a software application comprising:
- a processor;
a testing runtime executing on the processor, the testing runtime directing the software application to perform operations based on a test definition and storing results related to performance of operations of the software application;
a variation data provider identified by the test definition and executing on the processor, the variation data provider generating a plurality of variations from a template and a plurality of values associated with the parameter, the template being a set of operations including a parameter and each variation being the template set of operations using a different one of the plurality of values for the parameter; and
the variation data provider supplying the plurality of variations to the testing runtime.
2 Assignments
0 Petitions
Accused Products
Abstract
The integrated test framework of the present invention includes a test framework runtime that executes test scripts and that also allows a script to identify a variation tool called a “data provider” that is responsible for providing data to the test framework runtime and controlling the iteration of the runtime through the variations. The script also identifies the type of data set from which the data provider should obtain the data from which the variations are derived and the location of the data set. Multiple variation data providers may be used in conjunction with an adapter layer that coordinates the iteration of each variation data provider and creates an aggregated variation over multiple varying parameters of different scope.
91 Citations
20 Claims
-
1. A system for testing a software application comprising:
-
a processor;
a testing runtime executing on the processor, the testing runtime directing the software application to perform operations based on a test definition and storing results related to performance of operations of the software application;
a variation data provider identified by the test definition and executing on the processor, the variation data provider generating a plurality of variations from a template and a plurality of values associated with the parameter, the template being a set of operations including a parameter and each variation being the template set of operations using a different one of the plurality of values for the parameter; and
the variation data provider supplying the plurality of variations to the testing runtime. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A computer readable medium having computer-executable instructions for performing a method comprising:
-
a) initializing software on a computer system;
b) initializing a variation data provider;
c) requesting from the variation data provider, a variation comprising at least one operation to be performed by the software;
d) receiving a variation from the variation data provider, the variation including an initial configuration for the software and a set of one or more operations;
e) configuring the software to the initial configuration;
f) directing the software to perform the set of one or more operations;
g) recording information describing the performance of the software of the set of one or more operations;
h) repeating c) through g) wherein a different variation is received from the variation data provider each time until the variation data provider returns a message indicating there are no further variations. - View Dependent Claims (10, 11, 12, 13, 14)
-
-
15. A computer readable medium having computer-executable instructions for performing a method comprising:
-
a) initializing software on a computer system;
b) initializing a first variation data provider and a second variation data provider and an adapter layer;
c) identifying the scope of each variation data provider to the adapter layer;
d) requesting from the adapter layer, a variation comprising at least one operation to be performed by the software;
e) receiving a variation from the adapter layer, the variation including an initial configuration for the software and a set of one or more operations, wherein a first value in the set of one or more operations is provided by the first variation data provider and a second value in the set of one or more operations is provided by the second variation data provider;
f) configuring the software to the initial configuration;
g) directing the software to perform the set of one or more operations;
h) recording information describing the performance of the software of the set of one or more operations;
i) repeating d) through h) wherein a different variation is received from the adapter layer each time until the variation data provider returns a message indicating there are no further variations. - View Dependent Claims (16)
-
-
17. A computer readable medium as defined in claim 17 further comprising computer-executable instructions for performing:
reading a test script, the test script identifying the software, the first variation data provider, and the second variation data provider to be initialized and associating a first scope with the first variation data provider and a second scope with the second variation data provider.
-
18. A computer readable medium having computer-executable instructions for performing a method comprising:
-
receiving a request for a variation from a testing runtime executing a test script directing the testing of a software application;
delivering to the testing runtime, in response to the request, a revert command, directing the testing runtime to configure the software application to the initial configuration;
storing a template set of operations, the template set of operations including one or more operations to be performed by the software application, at least one operation having a parameter requiring a value;
generating a variation based on the template set of operations by obtaining a next value from a variation data set and using the next value as the parameter; and
delivering to the testing runtime, in response to the request, the variation to the testing runtime. - View Dependent Claims (19, 20)
-
Specification