Testing software using verification data defined independently of the testing code

• US 20070028217A1
• Filed: 07/29/2005
• Published: 02/01/2007
• Est. Priority Date: 07/29/2005
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rashevsky, Vladislav, Mishra, Rajeshdutta
• Application Number:
  US11/193,294
• Publication Number:
  US 20070028217A1
• Time in Patent Office: Days
• US Class Current: 717/124
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

Systems, methods, data structures and software for testing software, in which the testing code and the verification data are independent from each other A variation map may specify one or more testing (e.g., automation) modules, and may specify the verification data to apply to the testing module(s). For example, the variation map may specify: specific value(s) for variables (e.g., parameters) of a testing module; type(s) of data patterns to apply to the variables; a number of testing iterations to perform using the testing module; and how to record the results of each iteration. A testing framework may be provided that is configured to use a variation map to test software. The testing framework may be configured to generate verification data using this recognition, and to control the execution of the one or more testing modules specified in the variation map, including supplying the generated verification data to the testing module(s).

Claims

1. A system for executing a software test on at least a portion of a software application, comprising:

at least one testing module defining one or more functions to perform on the at least portion of software code; and

a data structure separate and distinct from the at least one testing module, the data structure specifying verification data to be applied by the at least one testing module.

2. The system of claim 1, wherein the data structure is formatted in accordance with an extensible markup language.

3. The system of claim 1, further comprising:

a testing framework operative to execute the software test based on the data structure, wherein the testing framework is operative to receive an instruction to execute a software test using the data structure, and wherein the testing framework controls execution of the software test based on the data structure in response to receiving the instruction.

4. The system of claim 3, wherein the instruction specifies one or more parameters corresponding to the data structure, and

wherein the testing framework is operative to control execution of the at least one testing module based at least in part on a value of the one or more parameters.

5. The system of claim 4, wherein the data structure includes a reference to the at least one testing module, and

wherein the testing framework is operative to interpret the data reference, including generating values of the verification data and identifying the at least one testing module from the reference and operative to execute the at least one testing module using the generated values.

6. The system of claim 1, wherein the at least one testing module includes a plurality of testing modules, and wherein the data structure defines an order in which the testing modules are to be executed and/or a number of times each testing module is to be executed.

7. The system of claim 1, wherein the data structure specifies a data pattern to be used to generate values of the verification data, the system further comprising:

a testing framework operative to execute the software test based on the data structure, including generating values of the verification data based on the specified data pattern.

8. The system of claim 7, wherein the testing framework is operative to generate random values based on the specified data pattern.

9. The system of claim 7, wherein the specified data pattern is one of the following types of data patterns:

a simple name-value pair;

a regular expression;

a permutation of multi-key value pairs;

a Pairwise Independent Combinatorial Testing pattern;

a complex data pattern.

10. A method of executing a software test on at least a portion of a software application, comprising acts of:

(A) interpreting a data structure specifying verification data to be applied by at least one testing module, the at least one testing module defining one or more functions to perform on the at least portion of software code;

(B) in response to the act (A), executing the at least one testing module one or more times using the verification data.

11. The method of claim 10, wherein the data structure includes a reference to the at least one testing module,

wherein the act (A) comprises interpreting the data reference, generating values of the verification data and identifying the at least one testing module from the reference, and

wherein the act (B) comprises executing the at least one testing module using the generated values.

12. The method of claim 10, wherein the data structure specifies a data pattern to be used to generate values of the verification data, the method further comprising:

(C) generating values of the verification data based on the specified data pattern.

13. The method of claim 12, wherein the act (C) comprises generating random values based on the specified data pattern.

14. The method of claim 12, wherein the specified data pattern is one of the following types of data patterns:

a simple name-value pair;

a regular expression;

a permutation of multi-key value pairs;

a Pairwise Independent Combinatorial Testing pattern;

a complex data pattern, and

wherein the act (C) comprises generating values of the verification data based on the type of the specified data pattern.

15. The method of claim 10, further comprising:

(C) receiving an instruction to execute a software test using the data structure,

wherein the act (A) is performed in response to the act (C), and

wherein the act (B) comprises executing the at least one testing module based at least in part on a value of the one or more parameters.

16. A computer-readable medium having computer-readable signals stored thereon that define a data structure for testing at least a portion of software code, the data structure comprising:

a reference to at least one testing module including one or more functions to perform on the at least portion of software code; and

one or more definitions of verification data to be applied by at least one testing module when performing the one or more functions.

17. The computer-readable medium of claim 16, wherein the data structure is formatted in accordance with an extensible markup language.

18. The computer readable medium of claim 16, wherein the at least one testing module includes a plurality of testing modules, and wherein the data structure defines an order in which the testing modules are to be executed and/or a number of times each testing module is to be executed.

19. The computer readable medium of claim 16, wherein the data structure specifies a data pattern to be used to generate values of the verification data.

20. The computer readable medium of claim 19, wherein the specified data pattern is one of the following types of data patterns:

a simple name-value pair;

a regular expression;

a permutation of multi-key value pairs;

a Pairwise Independent Combinatorial Testing pattern;

a complex data pattern.