Generation of test cases for functional testing of applications

• US 8,683,446 B2
• Filed: 07/09/2007
• Issued: 03/25/2014
• Est. Priority Date: 07/09/2007
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Paradkar, Amitkumar Manoharrao, Sinha, Avik
• Primary Examiner(s): Vu, Tuan A
• Application Number:
  US11/775,090
• Publication Number:
  US 20090018811A1
• Time in Patent Office: 2,451 Days
• Field of Search: 717/100, 717/105, 717/117, 717/109, 717/124, 717/136, 717/104, 717/110, 717/126, 715/234, 715/237, 715/762, 715/809, 709/206, 709/201, 709/220, 719/328, 719/330, 714/25, 714/819, 714/38, 703/17, 707/104.1, 379/298
• US Class Current: 717/126
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases

Abstract

An automated system and method to generate functional conformance tests for applications are provided. The system and method in one aspect use Inputs, Outputs, Preconditions, Effects (IOPEs) paradigm associated with an application for automatically generating test goals. A planner component may accept these testing goals to generate a sequence of operations or invocations as a test case. Verification sequences are also generated. The system and method also allow generation of executable test cases, which can be applied to the various interfaces through which the application may be accessed.

Claims

1. A computer implemented method for generating a testing procedure for an implementation of a function, comprising:

obtaining automatically a model of an application that has been implemented, the model including at least a description of input, output, and one or more state transitions of a function in the application, said application model being an input-output-precondition-effect (IOPE) model of a web service;

refining one or more preconditions of the model based on a fault model that represents one or more ways the application can fail;

generating a testing goal from the model, the testing goal including at least a state condition and a testing function and the one or more of the preconditions refined based on the fault model, wherein the testing goal is generated for each precondition-effect pair in the OIPE model;

wherein the IOPE model is constructed as syntactic ensemble comprising input and output constraints pair for a target operation, using a template having corresponding placeholders that are to be replaced by actual values from an operation under test, and as a pair-set of precondition and effect for said operation, the pair-set defined in part as the testing goal,

the fault model used as a leverage for redefining the testing goal and corresponding to the IOPE constraints, the fault model comprising one or more of boundary values for input parameters, cardinality constraints enforcing a threshold that is to be observed, and fault sensitization serving a behavioral basis for the testing goal;

generating a sequence of one or more actions based on the testing goal, the one or more actions at least transforming a computer system'"'"'s state running the application to meet the one or more preconditions;

generating one or more expected output and a system end state associated with applying the sequence of one or more actions; and

generating a sequence of one or more verification actions based on said one or more expected output and the system end state,

wherein the sequence of one or more actions and the sequence of one or more verification actions are invoked to test the application, and

wherein a functional test case generation is fully automated for the application.

2. The method of claim 1, wherein the sequence of one or more actions includes one or more function calls in the application.

3. The method of claim 1, wherein the step of obtaining a model includes obtaining information describing one or more objects and associated attributes of the application.

4. The method of claim 1, wherein the step of obtaining a model includes obtaining information describing interaction of a system on which the application may be executed with an environment via a set of operations.

5. The method of claim 4, wherein said information further includes describing the inputs from the environment to the system, describing outputs from the system to the environment, describing one or more pre-conditions of the operations in terms of the internal state of the system, and describing one or more effects of an operation on the internal state of the system.

6. The method of claim 1, wherein the model includes a use case description of the application.

7. The method of claim 1, wherein the model includes web service description of the application.

8. The method of claim 7, wherein the web service description is specified in Web Services Description Language (WSDL-SL Ontology Web Language (OWL-SL or combinations thereof.

9. The method of claim 1, wherein the model includes a functional description of the application.

10. The method of claim 1, wherein the second sequence of one or more actions includes automatically created one or more operations that distinguish the state of the system from other states.

11. The method of claim 1, wherein the test objectives validate the application for one or more functional properties as described in the model of the application.

12. A system for generating a testing procedure for an implementation of a function, comprising:

an application model including at least one or more objects, one or more inputs and outputs associated with said one or more objects, one or more preconditions of a system employing the application, and one or more effects on the system applying said one or more objects, said application model being an input-output-precondition-effect (IOPE) model of a web service;

a goal generator module operable to execute on a processor, the goal generator module further operable to refine said one or more preconditions based on a fault model that represents one or more ways the application can fail, the goal generator module further operable to automatically generate one or more testing goals using the application model and the one or more of the preconditions refined based on a fault model, wherein the testing goal is generated for each precondition-effect pair in the OIPE model,

wherein the IOPE model is constructed as syntactic ensemble comprising input and output constraints pair for a target operation, using a template having corresponding placeholders that are to be replaced by actual values from an operation under test, and as a pair-set of precondition and effect for said operation, the pair-set defined in part as the testing goal,

the fault model used as a leverage for redefining the testing goal and corresponding to the IOPE constraints, the fault model comprising one or more of boundary values for input parameters, cardinality constraints enforcing a threshold that is to be observed, and fault sensitization serving a behavioral basis for the testing goal;

a planner module operable to execute on a processor to generate a sequence of one or more actions based on said one or more testing goals, the one or more actions at least transforming a computer system'"'"'s state running the application to meet the one or more preconditions; and

a verification sequence generator module operable to execute on a processor to generate a sequence of one or more verification actions based on one or more expected outputs from the sequence of one or more actions and an expected system end state resulting from the sequence of one or more actions,

wherein the sequence of one or more actions and the sequence of one or more verification actions are invoked to test the application model, and

wherein a functional test case generation is fully automated for the application model.

13. The system of claim 12, further including:

a test script generator module operator to generate a test script for a selected interface using a template, the sequence of one or more actions and the sequence of one or more verification actions.

14. A non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform a method of generating a testing procedure for an implementation of a function, comprising:

obtaining a model of an application, the model including at least one or more objects, one or more inputs and outputs associated with said one or more objects, one or more preconditions of a system employing the application, and one or more effects on the system applying said one or more objects, wherein the testing goal is generated for each precondition-effect pair in the input-output-precondition-effect (IOPE) model;

refining said one or more preconditions based on a fault model that represents one or more ways the application can fail;

generating a testing goal from the model, the testing goal including at least a state condition and a testing function and the one or more of the preconditions refined based on a fault model, wherein the testing goal is generated for each PE pair in the OIPE model;

wherein the IOPE model is constructed as syntactic ensemble comprising input and output constraints pair for a target operation, using a template having corresponding placeholders that are to be replaced by actual values from an operation under test, and as a pair-set of precondition and effect for said operation, the pair-set defined in part as the testing goal,

the fault model used as a leverage for redefining the testing goal and corresponding to the IOPE constraints, the fault model comprising one or more of boundary values for input parameters, cardinality constraints enforcing a threshold that is to be observed, and fault sensitization serving a behavioral basis for the testing goal;

generating a sequence of one or more actions based on the testing goal, the one or more actions at least transforming a computer system'"'"'s state running the application to meet the one or more preconditions;

generating one or more expected output and a system end state associated with applying the sequence of one or more actions; and

generating a sequence of one or more verification actions based on said one or more expected output and the system end state,

wherein the sequence of one or more actions and the sequence of one or more verification actions are invoked to test the application, and

wherein a functional test case generation is fully automated for the application.

15. The non-transitory program storage device of claim 14, further including:

generating a test script implementing the sequence of one or more actions and the sequence of one or more verification actions using a template associated with a selected interface.