System and method for efficient creation and reconciliation of macro and micro level test plans

• US 9,262,736 B2
• Filed: 06/28/2013
• Issued: 02/16/2016
• Est. Priority Date: 09/11/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bassin, Kathryn A., Huang, Sheng, Kagan, Steven, Li, Shao C., Li, Zhong J., Liu, He H., Skrabanek, Susan E., Tan, Hua F., Zhu, Jun
• Primary Examiner(s): Pan, Hang
• Application Number:
  US13/930,870
• Publication Number:
  US 20130290073A1
• Time in Patent Office: 963 Days
• Field of Search: 717/101
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3608 using formal methods, e.g. model checking, abstract interpretation theorem proving G06N5/013
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 8/70 Software maintenance or management
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 10/06375 Prediction of business process outcome or impact based on a proposed change
• CPC Class Codes: G06Q 10/067 Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/103 Workflow collaboration or project management
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06Q 30/0244 Optimization

Abstract

A method includes creating a macro plan for a test project, creating a micro plan for the test project, wherein the micro plan and the macro plan are based on at least one common parameter, and reconciling the macro plan and the micro plan by identifying deviations between the macro plan and the micro plan based on the at least one common parameter.

Claims

1. A method implemented in a computer infrastructure having computer executable code tangibly embodied on a computer readable storage medium having programming instructions operable to:

create a macro plan for a test project for design of a software product that includes estimating an effort distribution and a defect distribution of defects in the software product, and generating a cost overview of the test project based on the effort distribution and the defect distribution;

create a micro plan for the test project, wherein the micro plan and the macro plan are based on at least one common parameter; and

reconcile the macro plan and the micro plan by identifying deviations between the macro plan and the micro plan based on the at least one common parameter,

wherein the estimating of the defect distribution is based on an output of a prediction engine configured to generate an estimate of a testing result of defects in the software product, and

wherein the reconciling of the macro plan and the micro plan includes comparing the macro plan and the micro plan side-by-side to show the deviations therebetween based upon a matrix showing intersections of orthogonal defect classification (ODC) triggers and activities.

2. The method of claim 1, wherein:

the creating the macro plan further includes permitting a user to specify a constraint; and

the effort distribution and the defect distribution are estimated based on the constraint.

3. The method of claim 2, wherein the constraint comprises one of total effort and maximum field defects.

4. The method of claim 1, wherein the programming instructions are further operable to:

permit a user to adjust one of the effort distribution and the defect distribution; and

re-estimate the effort distribution and the defect distribution based on the adjusting.

5. The method of claim 1, wherein the creating the micro plan includes:

creating a system under test (SUT) model based on test requirements;

receiving annotations of the SUT model from a user; and

estimating a test case distribution for the SUT model.

6. The method of claim 5, wherein the test case distribution is based on a same set of triggers and activities as the macro plan.

7. The method of claim 6, wherein the annotations are related to aspects of use cases of the SUT model.

8. The method of claim 7, wherein the reconciling includes comparing at least one of effort, number of test cases, and estimated defects between the macro plan and the micro plan.

9. The method of claim 8, wherein the at least one of effort, number of test cases, and estimated defects are defined in terms of activities and triggers.

10. The method of claim 1, wherein the reconciling includes:

presenting the deviations between the macro plan and the micro plan to a user; and

for at least one of the deviations, receiving from the user one of a confirmation of or an adjustment at least one of the deviations, wherein the adjustment comprises at least one of;

manually changing a number of test cases in the micro plan,

annotating a use case of the micro plan, and

creating a new test case in the micro plan.

11. The method of claim 1, wherein a service provider at least one of creates, maintains, deploys and supports the computing infrastructure.

12. The method of claim 1, wherein the computing infrastructure is provided by a service provider under a subscription and/or fee agreement and/or based on advertising content to one or more third parties.

13. A system comprising a processor and a computer-readable storage device including program instructions that, when executed by the processor, configure the system to provide a test planning optimization workbench that includes:

a macro planning tool operable to create a macro plan for a test project for design of a software product by estimating an effort distribution and a defect distribution of defects in the software product, and generating a cost overview of the test project based on the effort distribution and the defect distribution;

a micro planning tool operable to create a micro plan for the test project; and

a reconciliation manager operable to identify and reconcile deviations between the macro plan and the micro plan,

wherein the estimating of the defect distribution is based on an output of a prediction engine configured to generate an estimate of a testing result of defects in the software product, and

wherein the reconciling of the deviations between the macro plan and the micro plan includes comparing the macro plan and the micro plan side-by-side to show the deviations therebetween based upon a matrix showing intersections of orthogonal defect classification (ODC) triggers and activities.

14. The system of claim 13, wherein the micro planning tool is further operable to estimate a test case distribution over a set of triggers and a set of activities.

15. A computer program product comprising a computer usable storage memory having readable program code embodied in the storage memory, the computer program product includes at least one component operable to:

create a macro plan for a test project for design of a software product based on a set of triggers and activities, wherein the creating the macro plan comprises;

estimating an effort distribution and a defect distribution of defects in the software product for the set of triggers and activities, and

generating a cost overview of the test project based on the effort distribution and the defect distribution;

create a micro plan for the test project based on the set of triggers and activities; and

reconcile the macro plan and the micro plan by identifying deviations between the macro plan and the micro plan based on the set of triggers and activities,

wherein the estimating of the defect distribution is based on an output of a prediction engine configured to generate an estimate of a testing result of defects in the software product, and

wherein the reconciling of the macro plan and the micro plan includes comparing the macro plan and the micro plan side-by-side to show the deviations therebetween based upon a matrix showing intersections of orthogonal defect classification (ODC) triggers and activities.

16. The computer program product of claim 15, wherein the creating the micro plan comprises estimating a test case distribution over the set of triggers and activities.

17. The computer program product of claim 15, wherein the creating the micro plan includes:

creating a system under test (SUT) model based on test requirements;

receiving annotations of the SUT model from a user; and

estimating a test case distribution for the SUT model.

18. The computer program product of claim 17, wherein:

the test case distribution is based on a same set of triggers and activities as the macro plan; and

the annotations are related to aspects of use cases of the SUT model.

19. The computer program product of claim 15, wherein the reconciling includes comparing at least one of effort, number of test cases, and estimated defects between the macro plan and the micro plan.