Automatic identification of input values that expose output failures in a software object
First Claim
1. In a computer system that is capable of executing a software object, a method for testing the software object to expose failures in the functionality of the software object, the method comprising the following:
- an act of inspecting a binary version of a test case template that includes an object that is to be tested for intended behavior, the inspection to identify annotations annotating one or more individual input fields that are part of the object, the annotations representing applied attributes applied on a per field basis to individually constrain how corresponding values for each individual input field are to be varied, from among a range of available values based on the input field data type, when generating test cases for testing the object for the intended behavior such that test cases can be generated through reference to the annotations;
an act of automatically generating a plurality of test cases for testing the object for the intended behavior, each test case including a value for each of the one or more individual input fields, the combination of values for each test case constrained in accordance with the applied attributes represented in the annotations to the one or more individual input fields of the object;
an act of invoking the object using the values of the one or more input fields specified in the at least one generated test case to attempt to cause the object to exhibit the intended behavior;
an act of accessing execution data for the object for each of the at least one test cases, the execution data indicating whether or not the intended behavior was exhibited for the test case;
an act of identifying test cases that did not cause the intended behavior to be exhibited as failed test cases;
an act of recording input values for the one or more input fields for any failed test cases on physical storage media; and
wherein the act of automatically generating a plurality of test cases comprises;
an act of automatically converting the binary test input to text-based test input comprising an act of serializing the binary test input into XML instructions;
an act of automatically generating a plurality of text-based test cases from the text-based test input;
an act of automatically converting at least the automatically generated plurality of text-based test cases into a corresponding plurality of binary test cases representative of the constraints in the binary test input comprising an act of deserializing XML instructions into binary test cases.
2 Assignments
0 Petitions
Accused Products
Abstract
Computerized testing of a system to identify combinations of input values that expose failures in the system'"'"'s operation. Attributes are applied to fields in a source code version of a test case template corresponding to the object. Applied attributes can indicate fields that are to be used when generating test cases for the object. The source code version of the test case template is subsequently compiled or interpreted into a binary test case template. Metadata in the binary test case template is inspected (e.g., through reflection) to identify the fields, data types, and attributes that are to be used when generating test cases. Based on possible input values for the identified fields and generation rules, a plurality of test cases is generated. The generated test cases are executed to identify combinations of input values that expose failures in the system'"'"'s operation.
62 Citations
35 Claims
-
1. In a computer system that is capable of executing a software object, a method for testing the software object to expose failures in the functionality of the software object, the method comprising the following:
-
an act of inspecting a binary version of a test case template that includes an object that is to be tested for intended behavior, the inspection to identify annotations annotating one or more individual input fields that are part of the object, the annotations representing applied attributes applied on a per field basis to individually constrain how corresponding values for each individual input field are to be varied, from among a range of available values based on the input field data type, when generating test cases for testing the object for the intended behavior such that test cases can be generated through reference to the annotations; an act of automatically generating a plurality of test cases for testing the object for the intended behavior, each test case including a value for each of the one or more individual input fields, the combination of values for each test case constrained in accordance with the applied attributes represented in the annotations to the one or more individual input fields of the object; an act of invoking the object using the values of the one or more input fields specified in the at least one generated test case to attempt to cause the object to exhibit the intended behavior; an act of accessing execution data for the object for each of the at least one test cases, the execution data indicating whether or not the intended behavior was exhibited for the test case; an act of identifying test cases that did not cause the intended behavior to be exhibited as failed test cases; an act of recording input values for the one or more input fields for any failed test cases on physical storage media; and wherein the act of automatically generating a plurality of test cases comprises; an act of automatically converting the binary test input to text-based test input comprising an act of serializing the binary test input into XML instructions; an act of automatically generating a plurality of text-based test cases from the text-based test input; an act of automatically converting at least the automatically generated plurality of text-based test cases into a corresponding plurality of binary test cases representative of the constraints in the binary test input comprising an act of deserializing XML instructions into binary test cases. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
-
16. In a computer system that is capable of executing a software object, a method for testing the software object to expose failures in the functionality of the software object, the method comprising the following:
-
an act of receiving a binary test case template including an object that is to be tested for intended behavior, the binary test case template including annotations annotating one or more individual input fields that are part of the object, the annotations representing applied attributes applied on a per field basis to individually constrain how corresponding values for each individual input field are to be varied, from among a range of available values based on the input field data type, when generating test cases for testing the object for an intended behavior such that test cases can be generated through reference to the annotations; an act of receiving binary generation rules that further constrain combinations of input values for the one or more input fields; an act of automatically generating binary test input for testing the object for the intended behavior, the combination of values for each test case constrained in accordance with the applied attributes represented in the annotations to the individual input fields of the object and in accordance with the binary generation rules; an act of automatically converting the binary test input to text-based test input comprising an act of serializing the binary test input into XML instructions; an act of automatically generating a plurality of text-based test cases from the text-based test input; an act of automatically converting at least the automatically generated plurality of text-based test cases into a corresponding plurality of binary test cases representative of the constraints in the binary test input comprising an act of deserializmnn XML instructions into binary test cases; an act of automatically executing at least one of the binary test cases to attempt to cause the object to exhibit the intended behavior; an act of accessing execution data for the object for the at least one test case, the execution data indicating whether or not the intended behavior was exhibited for the test case; and an act of recording input values for the one or more input fields for test cases that did not cause the object to exhibit the intended behavior on physical storage media. - View Dependent Claims (17, 18, 19, 20, 21, 22, 23, 24, 25, 26)
-
-
27. A computer program product for use in a computer system that is capable of executing a software object, the computer program product for implementing a method for testing the software object to expose failures in the functionality of the software object, the computer program product comprising one or more physical media having stored thereon computer executable instructions that, when executed by a processor, cause the computer system to perform the following:
-
inspect a binary version of a test case template that includes an object that is to be tested for intended behavior, the inspection to identify annotations annotating one or more individual input fields that are part of the object, the annotations representing applied attributes applied on a per field basis to individually constrain how corresponding values for each individual input field are to be varied, from among a range of available values based on the input field data type, when generating test cases for testing the object for an intended behavior such that test cases can be generated through reference to the annotations; automatically generate a plurality of test cases for testing the object for the intended behavior, each test case including a value for each of the one or more individual input fields, the combination of values for each test case constrained in accordance with the applied attributes represented in the annotations to the one or more individual input fields of the object; invoke the object using the values of the one or more input fields specified in the at least one generated test case to attempt to cause the object to exhibit the intended behavior; access execution data for the object for each of the at least one test cases, the execution data indicating whether or not the intended behavior was exhibited for the test case; identify test cases that did not cause the intended behavior to be exhibited as failed test cases; record input values for the one or more input fields for any failed test cases; and wherein the act of automatically generating a plurality of test cases comprises; an act of automatically converting the binary test input to text-based test input comprising an act of serializing the binary test input into XML instructions; an act of automatically generating a plurality of text-based test cases from the text-based test input; an act of automatically converting at least the automatically generated plurality of text-based test cases into a corresponding plurality of binary test cases representative of the constraints in the binary test input comprising an act of deserializing XML instructions into binary test cases. - View Dependent Claims (28, 29, 30, 31)
-
-
32. A computer program product for use in a computer system that is capable of executing an software object, the computer program product for implementing a method for testing the software object to expose failures in the functionality of the software object, the computer program product comprising one or more physical media having stored thereon computer executable instructions that, when executed by a processor, cause the computer system to perform the following:
-
receive a binary test case template foe-the that includes an object that is to be tested for intended behavior, the binary test case template including annotations annotating one or more individual input fields that are part of the object, the annotations representing applied attributes applied on a per field basis to individually constrain how corresponding values for each individual input field are to be varied, from among a range of available values based on the input field data type, when generating test cases for testing the object for an intended behavior such that test cases can be generated through reference to the annotations; receive binary generation rules that further constrain combinations of input values for the one or more input fields; automatically generate binary test input for testing the object for the intended behavior, the combination of values for each test case constrained in accordance with the applied attributes represented in the annotations to the individual input fields of the object and in accordance with the binary generation rules; convert the binary test input to text-based test input by serializing the binary test input into XML instructions; generate a plurality of text-based test cases from the text-based test input; convert at least the automatically generated plurality of text-based test cases into a corresponding plurality of binary test cases representative of the constraints in the binary test input by deserializin XML instructions into binary test cases; execute at least one of the binary test cases to attempt to cause the object to exhibit the intended behavior; access execution data for the object for the at least one test case, the execution data indicating whether or not the intended behavior was exhibited for the test case; and record input values for the one or more input fields for test cases that did not cause the object to exhibit the intended behavior. - View Dependent Claims (33, 34)
-
-
35. In a computer system that is capable of executing a software object, a method for testing the software object to expose failures in the functionality of the software object, the method comprising the following:
-
a step for determining that annotations to one or more individual input fields that are part of an object contain applied attributes that are to be used to test the object, the applied attributes applied on a per field basis to individually constrain how corresponding values for each individual input field are to be varied, from among a range of available values based on the input field data type, when generating test cases for testing the object for an intended behavior such that test cases can be generated through reference to the annotations; an act of automatically generating a plurality of test cases for testing the object for the intended behavior, each test case including a value for each of the one or more individual input fields, the combination of values for each test case constrained in accordance with the applied attributes represented in the annotations to the one or more individual input fields of the object; an act of invoking the object using the values of the one or more input fields specified in the at least one generated test case to attempt to cause the object to exhibit the intended behavior; an act of accessing execution data for the object for each of the at least one test cases, the execution data indicating whether or not the intended behavior was exhibited for the test case; an act of identifying test cases that did not cause the intended behavior to be exhibited as failed test cases; an act of recording input values for the one or more input fields for any failed test cases on physical storage media; and wherein the act of automatically generating a plurality of test cases comprises; an act of automatically converting the binary test input to text-based test input comprising an act of serializing the binary test input into XML instructions; an act of automatically generating a plurality of text-based test cases from the text-based test input; an act of automatically converting at least the automatically generated plurality of text-based test cases into a corresponding plurality of binary test cases representative of the constraints in the binary test input comprising an act of deserializing XML instructions into binary test cases.
-
Specification