Software application test coverage analyzer

• US 20040025088A1
• Filed: 08/01/2002
• Published: 02/05/2004
• Est. Priority Date: 08/01/2002
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Chin, Philip A., Dhanikonda, Uday S., Avvari, Madhava V., Nandigama, Murali K.
• Granted Patent: US 6,978,401 B2
• Time in Patent Office: Days
• US Class Current: 714/38
• CPC Class Codes: G06F 11/3676 for coverage analysis

Abstract

A method for analyzing test coverage of a software application is provided. The method includes profiling an executable of the software application to generate application call trees. The method also includes profiling test cases used to test a production Java VM so as to generate test case call trees. Also included is comparing the application call trees with the test case call trees so as to identify gaps in the application call tree not covered by the test case call trees.

Claims

1. A method for analyzing test coverage of a software application, the method comprising:

profiling an executable of the software application to generate application call trees;

profiling test cases used to test a production Java VM, the profiling of the test cases generating test case call trees; and

comparing the application call trees with the test case call trees, the comparing configured to identify gaps in the application call tree not covered by the test case call trees.

2. A method for analyzing test coverage of a software application as recited in claim 1, the method further including,

generating gap filling test cases to test the production Java VM;

and

executing the production Java VM using the gap filling test cases.

3. A method for analyzing test coverage of a software application as recited in claim 2, the method further including,

profiling the gap filling test cases to generate new test case call trees, the new test case call trees being combined with existing test case call trees.

4. A method for determining a source of an error during an execution of a software application using Java production VM, the method comprising:

profiling an executable of the software application to generate application call trees;

profiling test cases used to test the production Java VM, the profiling of the test cases generating test case call trees; and

comparing the application call trees with the test case call trees, the comparing configured to identify gaps in the application call trees not covered by the test case call trees;

reproducing the application call tree;

executing a reproduced application call tree using the production Java VM;

fixing the production Java VM so as to eliminate the error in the production Java VM if the error is generated during the execution of the reproduced software application using the production Java VM; and

confirming that the bug is not generated by the production Java VM if the execution of the production software component by the reproduced data structure does not generate the bug.

5. A method for determining a source of an error during an execution of a software application using Java production VM as recited in claim 4, the method further including,

identifying gaps in the application call trees not covered by the test case call trees;

6. A method for determining a source of an error during an execution of a software application using Java production VM as recited in claim 4, the method further including,

generating gap filling test cases to test the production Java VM;

and

executing the production Java VM using the gap filling test cases.

7. A method for determining a source of an error during an execution of a software application using Java production VM as recited in claim 6, wherein if the execution of the production Java VM using the gap filling test cases causes a replication of an error, the method further includes,

fixing the production Java VM.

8. A method for determining a source of an error during an execution of a software application using Java production VM as recited in claim 6, the method further including,

profiling the gap filling test cases to generate new test case call trees, the new test case call trees being combined with existing test case call trees.

9. A method for determining a source of an error during an execution of a software application using Java production VM as recited in claim 6, the method further including,

reproducing the application call tree using selected ones of the test case call trees and the new test case call trees so as to create a reproduced application call tree.

10. A method for determining a source of an error during an execution of a software application using Java production VM as recited in claim 7, the method further including,

executing the reproduced application call tree using production Java VM.

11. A method for determining a source of an error during an execution of a software application using Java production VM as recited in claim 8, the method further including,

modifying a test case so as to create a modified test case having a substantially identical call tree path as a portion of a path of the application call tree.

12. A method for determining a source of an error during an execution of a software application using Java production VM as recited in claim 11, the method further including,

replicating the application call tree using selected ones of the test case call trees, the new test case call trees, and modified test case call tree so as to create a replicated application call tree.

13. A system for analyzing test coverage of a software application, the system comprising:

a production Java virtual machine (VM);

a profiled Java VM;

a plurality of test cases used to test the production Java VM, the plurality of test cases executed using the profiled Java VM so as to generate test coverage data;

a software application executable configured to be executed using the production Java VM, the application executable configured to be executed using the profiled Java VM so as to generate an application coverage data;

a database to store the test coverage data and the application coverage data;

a compare module configured to compare the test coverage data against the application coverage data so as to locate a gap in the application coverage data.

14. A system for analyzing test coverage of a software application as recited in claim 13, wherein a new test case is generated if the compare module locates a gap in the application coverage data.

15. A system for analyzing test coverage of a software application as recited in claim 14, wherein the new test case is executed using the profiled Java VM so as to generate a new test coverage data.

16. A method for determining a source of a bug generated while a software application is executed using a production Java VM, the method comprising:

providing a production Java VM;

providing a profiled Java VM;

providing a plurality of test cases configured to be executed using the production Java VM;

receiving an application executable;

executing the test cases using the profiled Java VM so as to generate test coverage data;

storing test coverage data in a database;

executing the application executable using the profiled Java VM so as to generate an application coverage data;

storing the application coverage data in the database;

using test coverage data to reproduce the application coverage data;

executing a reproduced application coverage data using the production Java VM;

updating the production Java VM so as to eliminate the bug in the production Java VM, if the bug is generated while executing the test cases generating the reproduced application coverage data using the production Java VM; and

confirming that the production Java VM does not generate the bug, if the execution of the production Java VM by the test cases generating the reproduced application coverage data does not generate the bug.

17. A method as recited in claim 16, wherein using the test case coverage data to reproduce the application coverage data, includes,

determining whether the application coverage data can be copied using the test case coverage data.

18. A method as recited in claim 16, wherein the application coverage data is compared against the test coverage data to determine whether a gap exists in the coverage data.

19. A method as recited in claim 18, wherein determining whether the application coverage data can be copied using the test case coverage data, includes

generating a new test case to traverse a gap in the application coverage data, if a gap in the coverage data exists.

20. A method as recited in claim 19, further comprising:

executing the new test case using the profiled Java VM so as to generate a new test coverage data; and

storing the new test coverage data in the database.

21. A method as recited in claim 20, further comprising,

selecting a plurality of test cases from the existing test cases and the new test cases so as to reproduce the application coverage data.

22. A method as recited in claim 21, further comprising,

modifying a test case so as to arrive at a modified test case, the modified test case traversing a test path substantially identical to a portion of the application coverage data;

generating a modified coverage data; and

replicating the application coverage data using selected ones of a plurality of test cases from the existing test cases, new test cases, and the modified test cases.

23. A method for analyzing a test coverage of a software application being executed using a production Java VM, the method comprising:

providing a profiled Java VM;

providing a plurality of test cases being executed using the production Java VM;

providing an application executable;

executing the test cases using the profiled Java VM so as to generate test coverage data;

storing test coverage data in a database in a call tree format;

executing the application executable using the profiled Java VM so as to generate an application coverage data;

storing the application coverage data in the database in the call tree format;

comparing an application call tree with the test call trees; and

locating a path in the application call tree not having been traversed by the test call trees so as to provide coverage for all paths in the application call tree.

24. A method as recited in claim 23, further comprising, generating a new test case to test the path not traversed by the test case call trees.

25. A method as recited in claim 23, further comprising,

executing the new test case using the production Java VM.

26. A method as recited in claim 23, further comprising,

fixing the production Java VM if a bug is found during an execution of the new test case using the production Java VM.

27. A method as recited in claim 24, further comprising,

executing the new test case using the profiled Java VM so as to generate a new test call tree;

and

store the new test call tree in the database.