Method and system for generating a computer program test suite using dynamic symbolic execution of JAVA programs
First Claim
1. A system for generating a test suite for use in testing a computer program written in the JAVA progamming language, the computer program having been compiled by a JAVA compiler into JAVA bytecodes, comprising:
- means for symbolically executing instructions of the computer program represented by the JAVA bytecodes to determine values of program variables of the computer program at selected points of execution and for finding input values to the computer program resulting in complete test coverage of the computer program according to a predetermined criteria based on the symbolic execution; and
means for storing the input values as a test suite for testing the computer program.
1 Assignment
0 Petitions
Accused Products
Abstract
A method and system for generating a test suite for a computer program written in the JAVA programming language. The JAVA program comprises program statements and program variables represented as JAVA source code and compiled by a JAVA compiler into JAVA bytecodes, including at least one input statement having one or more input variables, that are grouped into code blocks and stored in a program database. The test suite comprises sets of inputs. Each of the sets of inputs corresponds to a pth in the program. The program statements corresponding to a candidate code block are read from the program database. Each of the input variables for each input statement and each of the program variables that depend on them are represented in symbolic form as a symbolic memory value and transforming each program statement dependent on such an input variable into a symbolic expression. A trial set of inputs for each of the input statements is created by finding a solution to the symbolic expression obtained using dynamic symbolic execution. The trial set of inputs are stored into the test suite if coverage of the candidate code block was obtained. A dynamic symbolic execution consists of a symbolic execution of the program performed along the path that corresponds to the trial set of actual inputs. The first input to the program is generated randomly. From that first input, inputs satisfying any coverage criteria can be obtained by performing the above procedure iteratively.
262 Citations
26 Claims
-
1. A system for generating a test suite for use in testing a computer program written in the JAVA progamming language, the computer program having been compiled by a JAVA compiler into JAVA bytecodes, comprising:
-
means for symbolically executing instructions of the computer program represented by the JAVA bytecodes to determine values of program variables of the computer program at selected points of execution and for finding input values to the computer program resulting in complete test coverage of the computer program according to a predetermined criteria based on the symbolic execution; and means for storing the input values as a test suite for testing the computer program. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
-
16. A method using a computer for generating a test suite for a computer program written in the JAVA programming language, the computer program comprising program statements and program variables and being represented as JAVA bytecodes after being compiled by a JAVA compiler, the computer program including at least one input statement having one or more input variables, grouped into code blocks, the test suite comprising sets of inputs, each of the sets of inputs corresponding to each of the input statements, comprising the steps of:
-
reading the program statements corresponding to a candidate code block from the JAVA bytecodes; representing each of the input variables for each input statement and each of the program variables in symbolic form as a symbolic memory value and transforming each program statement dependent on such an input variable into a symbolic expression; creating a trial set of inputs for each of the input statements by finding a solution to the symbolic expression comprising actual input values corresponding to each symbolic memory value using dynamic symbolic execution of the JAVA bytecodes; interpreting the JAVA bytecodes using the trial set of inputs and analyzing results obtained for coverage of the candidate code block; and storing the trial set of inputs into the test suite if coverage of the candidate code block was obtained. - View Dependent Claims (17, 18, 19, 20, 21)
-
-
22. A system for generating a test suite for a computer program written in the JAVA programming language, the computer program comprising program statements and program variables and represented as JAVA bytecodes after being compiled by a JAVA compiler, the computer program including at least one input statement having one or more input variables, grouped into code blocks and stored in a program database, the test suite comprising sets of inputs, each of the sets of inputs corresponding to each of the input statements, comprising:
-
means for reading the program statements corresponding to a candidate code block from the JAVA bytecodes; means for representing each of the input variables for each input statement and each of the program variables in symbolic form as a symbolic memory value and means for transforming each program statement dependent on such an input variable into a symbolic expression; means for creating a trial set of inputs for each of the input statements by finding a solution to the symbolic expression comprising actual input values corresponding to each symbolic memory value using dynamic symbolic execution of the JAVA bytecodes; means for interpreting the JAVA bytecodes using the trial set of inputs and means for analyzing results obtained for coverage of the candidate code block; and means for storing the trial set of inputs into the test suite if coverage of the candidate code block was obtained.
-
-
23. A method of operating a computer system for performing dynamic symbolic execution by a symbolic virtual machine of a computer program written in the JAVA programming language and represented as JAVA bytecodes after being compiled by a JAVA compiler, the computer system having a memory, the method comprising the steps of:
-
reading a JAVA bytecode to be executed; writing memory modifications into selected memory areas of the symbolic virtual machine when the JAVA bytecode being executed modifies a location in computer system memory; resolving any undefined external references included in the JAVA bytecode; and repeating the reading, writing and resolving steps for all JAVA bytecodes in the computer program.
-
-
24. A computer-implemented method of detecting runtime errors in a computer program written in the JAVA programming language, the computer program being represented by JAVA bytecodes after being compiled by a JAVA compiler, the computer-implemented method comprising the steps of:
-
reading the JAVA bytecodes; obtaining at least one input value for the computer program; symbolically executing an instruction of the computer program represented in the JAVA bytecodes; examining all symbolic expressions on which the instruction depends based on the symbolic execution; and storing the at least one input value in a test database and marking the at least one input value as an input that generates a runtime error when a solution to the symbolic expressions generates an error condition.
-
-
25. A computer-implemented method of generating a test suite for a module of an incomplete application program written in the JAVA programming language having undefined external references or a missing main method, the application program being represented by JAVA bytecodes after being compiled by a JAVA compiler, the computer-implemented method comprising the steps of:
-
reading the JAVA bytecodes for the module; determining a sequence of JAVA methods to be called in the module and arguments to be passed to the JAVA methods; performing symbolic interpretation of the sequence of JAVA methods of the JAVA bytecodes by calculating symbolic expressions for the JAVA methods and using the symbolic expressions to determine input values causing a predetermined testing criteria to be satisfied; storing the input values in a test suite database when solutions for the predetermined testing criteria based on the symbolic expressions are found for the input values; and repeating the determining and performing steps when solutions are not found for the predetermined testing criteria based on the symbolic expressions.
-
-
26. A computer-implemented method of inserting instrumentation information into JAVA bytecodes produced by compiling a computer program written in the JAVA programming language, comprising the steps of:
-
reading the JAVA bytecodes and the computer program; creating a parse tree for the computer program and analyzing the parse tree to identify points in the computer program where instrumentation can be inserted; analyzing the JAVA bytecodes and instrumenting the JAVA bytecodes at the identified points; and writing a new set of JAVA bytecodes including the instrumentation information.
-
Specification