Computer software testing method and apparatus
First Claim
1. A computer program testing method for producing incremental coverage information indicating which portions of a computer program have been exercised, the method including the steps of:
- instrumenting the computer program to provide an instrumented computer program; and
executing the instrumented computer program to exercise portions of the instrumented computer program, including the step of producing incremental coverage output indicating exercised portions of the instrumented computer program which were unreached by at least one previous execution of the instrumented computer program, the incremental coverage results being provided without requiring the instrumenting step to be repeated subsequent to said previous execution.
1 Assignment
0 Petitions
Accused Products
Abstract
An automatic testing technique generates an incremental coverage report indicating coverage only for portions of the program that are unreached by previous testing--allowing software testing processes to concentrate upon finding tests which increase the test coverage and reducing the sheer bulk of the coverage output. A computer program to be tested is parsed and "instrument" code is inserted at appropriate parts of the program. The instrumented program is then typically executed multiple times with different input conditions in an attempt to "exercise" and test different parts of the program. For each execution, the instrument code inserted by the instrumentation process causes coverage results to be generated and stored. Such coverage results may include information about which parts of the program have been reached by the current execution that had not previously been reached by earlier executions. This "incremental" coverage results output is useful in specifying the value of a current test without flooding the user with volumes of information specifying all parts of the program reached by the current execution.
-
Citations
29 Claims
-
1. A computer program testing method for producing incremental coverage information indicating which portions of a computer program have been exercised, the method including the steps of:
-
instrumenting the computer program to provide an instrumented computer program; and executing the instrumented computer program to exercise portions of the instrumented computer program, including the step of producing incremental coverage output indicating exercised portions of the instrumented computer program which were unreached by at least one previous execution of the instrumented computer program, the incremental coverage results being provided without requiring the instrumenting step to be repeated subsequent to said previous execution. - View Dependent Claims (2, 3, 4, 5)
-
-
6. A method of software testing including the following steps:
-
(a) maintaining a data structure indicating whether each of plural software paths has been exercised by testing; (b) upon reaching a path, checking the data structure to determine whether the path has previously been exercised; and (c) if the checking step reveals that the path has not previously been exercised, modifying the data structure to indicate that the path has been exercised and writing an indication into an output data structure indicating the path. - View Dependent Claims (7, 8, 9, 10, 11, 12)
-
-
13. A method of iterative software testing comprising:
-
(a) instrumenting a computer program having at least one branch with multiple alternative outcomes, including the step of inserting instrumentation code into the computer program that tests for each of the multiple alternative outcomes of the branch; (b) executing the instrumented computer program to provide branch coverage results; and (c) repeating step (a) in response to the branch coverage results, including the step of omitting at least some instrumentation code inserted by an earlier performance of step (a) based on the branch coverage results indicating that at least one portion of the program has been exercised. - View Dependent Claims (14, 15)
-
-
16. A method of software testing including the steps of:
-
(a) automatically parsing a computer program to locate at least one inline template and/or member function defined inline within at least one template class, wherein the inline function is defined within a header file; and (b) automatically inserting, into said header file, instrument code to provide at least one coverage evaluation function. - View Dependent Claims (17, 18, 19)
-
-
20. A method of software testing comprising:
-
(a) parsing a computer program to locate at least one inline function; (b) compiling the computer program, including the step of expanding the inline function to include instrumentation code; and (c) executing the compiled computer program, including the step of generating incremental coverage information.
-
-
21. Software testing apparatus comprising:
-
an instrumenter that accepts a computer program as an input, parses the computer program to locate at least one insertion point, and inserts instrument code at the insertion point to provide an instrumented computer program; and an executor that executes the instrumented computer program and generates incremental coverage results in response to the inserted instrument code, the incremental coverage results indicating exercised portions of the instrumented computer program that were unreached by at least one previous execution of the instrumented computer program, the incremental coverage results being provided without requiring the instrumenter to re-instrument the computer program subsequent to said previous execution. - View Dependent Claims (22, 23, 24)
-
-
25. Software testing apparatus comprising:
-
instrumenting means including inputting means for accepting a computer program as an input, parsing means for parsing the computer program to locate at least one insertion point, and inserting means for inserting instrumentation code at the insertion point to provide an instrumented version of the computer program; and executing means coupled to the instrumenting means, the executing means for executing the instrumented version of the computer program, the executing means including outputting means for outputting at least incremental coverage results indicating exercised portions of the instrumented computer program that were unreached by at least one previous execution of the instrumented computer program version, the incremental coverage results being provided without requiring the instrumenting means to re-instrument the computer program subsequent to said previous execution.
-
-
26. A storage medium comprising:
-
means for storing an instrumented executable computer program defining plural paths; means for storing a data structure indicating which of the plural paths have been exercised; and means for storing a further data structure indicating which of the plural paths were exercised by the most recent execution of the computer program and which were not exercised by at least one previous execution of the computer program. - View Dependent Claims (27, 28, 29)
-
Specification