Unit test generalization

• US 20070033443A1
• Filed: 12/30/2005
• Published: 02/08/2007
• Est. Priority Date: 08/04/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Tillmann, Nikolai, Schulte, Wolfram, Grieskamp, Wolfgang
• Granted Patent: US 7,587,636 B2
• Time in Patent Office: Days
• US Class Current: 714/45
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases

Abstract

A computer system provides a test program and one or more unit tests, such as a traditional unit test and or a parameterized unit test. The system also includes a constraint solver, a theorem prover, an implementation under test, a symbolic executor, a generalizor, and generated test cases. The generalizor receives a traditional unit tests as input, and modifies the traditional unit test into a parameterized unit test. The modification includes replacing plural concrete values in the traditional unit test with symbols, and exporting the symbols into a signature of the parameterized unit test. A symbolic executor identifies constraints while symbolically executing the created parameterized unit test of the implementation under test. A constraint solver and or theorem prover generates a set of test cases by solving for values that satisfy the series of constraints. The test program executes the automatically generated test cases.

Claims

1. A computerized method comprising:

receiving a unit test comprising a test of an implementation;

creating a parameterized unit test from the unit test comprising replacing plural concrete values in the unit test with symbolic values and exporting the symbolic values into a signature of the parameterized unit test;

symbolically executing the parameterized unit test to identify path constraints of the implementation; and

creating test cases that conform to the path constraints identified during symbolic execution.

2. The method of claim 1 wherein replacing plural concrete values in the unit test with symbolic values further comprises replacing an object with a symbolic object or mock object.

3. The method of claim 1 further comprising:

upon identifying a test case comprising a successful path through the parameterized unit test, associating the test case with the parameterized unit test; and

upon identifying a test case that violates an assert statement of the parameterized unit test, modifying an altered parameterized unit test, and associating the test that violates the assert statement with the altered parameterized unit test.

4. The method of claim 3 wherein modifying the altered parameterized unit test comprises adding a program statement to the altered parameterized unit test, the added program statement comprising an expected exception statement.

5. The method of claim 3 wherein modifying the altered parameterized unit test comprises removing a violated assert statement from the altered parameterized unit test.

6. The method of claim 1 wherein the test cases are associated with the parameterized unit test as an inline database.

7. The method of claim 1 wherein the test cases associated with parameterized unit tests comprise relations of symbolic values provided as assume statements.

8. The method of claim 3 wherein identifying a test case that violates an assert statement comprises catching a thrown exception, and modifying the altered parameterized unit tests comprises annotating the altered parameterized unit test with an expected exception statement of a type corresponding to the thrown exception.

9. The method of claim 1 further comprising:

receiving user inputs indicating concrete values for generalization; and

while creating the parameterized unit test, determining at least one of the plural concrete values to replace based at least in part upon the received user inputs.

10. The method of claim 1 wherein exporting the symbolic values into the signature comprises exporting all replaced values.

11. The method of claim 1 further comprising receiving user inputs comprising a user indication of at least the unit test, a concrete value, and a coverage criteria.

12. The method of claim 11, wherein the coverage criteria is code coverage.

13. The method of claim 1 wherein the test cases comprise code coverage for the implementation under test.

14. A system for testing programs comprising:

a digital processor; and

computer memory comprising,

a traditional unit test comprising an implementation under test,

a test generalizor for modifying the traditional unit test into a parameterized unit test, the modification comprising,

replacing plural concrete values in the traditional unit test with symbols, and

exporting the symbols into a parametric signature of the parameterized unit test,

a symbolic executor for identifying constraints while symbolically executing the created parameterized unit test of the implementation under test, and

a constraint solver that generates a set of test cases by solving for values that satisfy the series of constraints.

15. The system of claim 14 wherein the computer memory further comprises a graphical component for receiving user inputs comprising an indication of at least one of the plural concrete values to replace with symbols.

16. The system of claim 15 wherein receiving user inputs further comprises a user indication of at least the unit test and a coverage criteria.

17. The system of claim 14 wherein exporting the symbolic values into a parametric signature of the parameterized unit test comprises exporting all replaced values.

18. A computer readable medium comprising computer executable instructions comprising:

instructions for receiving a unit test of an implementation under test;

instructions for creating a parameterized unit test from the unit test comprising replacing plural concrete values in the unit test with symbolic values and exporting the symbolic values into a signature of the parameterized unit test;

instructions for symbolically executing the parameterized unit test;

instructions for identifying one or more path conditions during symbolic execution;

instructions for generating test cases comprising inputs for the parameters of the parameterized unit test; and

instructions for verifying an implementation under test by executing the generated test cases on the inputs to the parameterized unit test.

19. The computer readable medium of claim 18 further comprising:

instructions for identifying a successful test case comprising a successful path through the parameterized unit test, and for associating the successful test case with the parameterized unit test; and

instructions for identifying a test case that violates an assert statement of the parameterized unit test, and for associating the test case that violates the assert statement with an altered parameterized unit test.

20. The computer readable medium of claim 19 wherein the altered parameterized unit test comprises adding an expected exception statement to the altered parameterized unit test responsive to identifying the test case that violates the assert statement.