System and method to produce business case metrics based on defect analysis starter (DAS) results

• US 10,235,269 B2
• Filed: 09/11/2009
• Issued: 03/19/2019
• Est. Priority Date: 09/11/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bassin, Kathryn A., Kagan, Steven, Li, Shao C., Li, Zhong J., Liu, He H., Skrabanek, Susan E., Tan, Hua F., Zhu, Jun
• Primary Examiner(s): Marcus, Leland
• Application Number:
  US12/558,324
• Publication Number:
  US 20110066557A1
• Time in Patent Office: 3,476 Days
• Field of Search: 705 738
• CPC Class Codes: G06F 11/3616 using software metrics
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/067 Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/103 Workflow collaboration or project management

Abstract

A method is implemented in a computer infrastructure having computer executable code tangibly embodied on a computer readable storage medium having programming instructions. The programming instructions are operable to receive data including defect analysis defect analysis starter (DAS)/defect reduction method (DRM) defect analysis data of a software development project and process the data. Additionally, the programming instructions are operable to determine one or more business metrics based on the data and generate at least one report based on the one or more business metrics. The one or more business metrics comprises at least one of a benefit for shifting defect removal earlier, a benefit for preventing an injection of defects, a benefit for reducing a cycle time, a benefit of reducing invalid defects and a benefit for reducing production defects.

Claims

1. A method implemented in a computer infrastructure for determining when to remove defects in a production lifecycle of a software product, the method comprising:

receiving data including defect analysis starter (DAS)/defect reduction method (DRM) defect analysis data of a software development project to develop the software product, the DAS/DRM defect analysis data comprises a quantity of defects in the software product discovered by each of a plurality of defect triggers, each comprised of conditions under which a defect is uncovered during testing of the software product, and an estimated percentage of defect reduction in the software product after preventative actions are taken by each defect trigger;

processing the data to identify improvements which have an impact to reduce defects in the software product, and which of the improvements have a highest impact among the identified improvements to reduce defects in the software product to determine which defects to remove and when to remove them to control the number of defects in the completed software product, the processing comprising;

determining cost versus benefit received for making the improvements to provide an improved software product with a reduced number of defects,

resolving defects in the software product during development of the software product before marketing the software product;

performing a calibration comprising adjusting a defect volume and a distribution by defect trigger in accordance with the received quantity of defects discovered by each defect trigger;

applying the preventative actions, after the calibration has been performed, to determine a reduction in the defect volume and the distribution by each defect trigger; and

calculating a benefit for preventing an injection of defects comprising calculating costs associated with avoidance of discovered defects and costs associated with avoidance of escaped defects according to the determined reduction in the defect volume and the distribution by each defect trigger;

determining one or more business metrics based on the data, wherein the one or more business metrics comprises the benefit for preventing the injection of defects; and

generating at least one report regarding proceeding with the development of the software product based on the one or more business metrics,

wherein;

at least the step of processing the data is implemented using a processor of the computer infrastructure;

the determining the reduction in the defect volume comprises calculating a product of the percentage of defect reduction and a total defect volume of the software development project;

the one or more business metrics further comprise a benefit for shifting defect removal earlier in a life cycle of developing the software product;

for determining the benefit for shifting defect removal earlier;

the receiving the data comprises receiving a potential defect and defect distribution by triggers and the quantity of defects discovered by each trigger; and

the processing the data comprises;

shifting defects by type from a source activity to an optimally-timed discovery, wherein a defect is one of directly shifted and shifted with a defect probability table; and

calculating a shifting benefit as a defect fix cost difference between discovering one or more defects at the source activity and discovering the one or more defects at the optimally-timed discovery;

for the discovered defects, the benefit for preventing the injection of defects is the sum of a testing cost and a fix cost; and

for the escaped defects, the benefit for preventing the injection of defects is determined based on a business case,

the method further comprising;

applying prevention actions to prevent an injection of defects into the software product and to shift defect removal earlier in the lifecycle of developing the software product based upon the benefit calculated for preventing the injection of defects into the software product and the benefit determined for shifting the defect removal earlier in the lifecycle of developing the software product; and

applying prevention actions to remove production defects at a time, in the production lifecycle of the software product, determined by a comparison of a first maximum potential decrease in the number of production defects if production defect removal is made in a first test phase of the production lifecycle of the software product and a second maximum potential decrease in the number of production defects if production defect removal is made in a second test phase of the production lifecycle of the software product which is subsequent to the first test phase.

2. The method of claim 1, wherein the DAS/DRM defect analysis data further comprises a production defect rate after DAS actions taken, an improved valid rate, and a current invalid rate.

3. The method of claim 1, wherein the receiving the data further comprises receiving one or more organization inputs, wherein the determining the one or more business metrics is additionally based on the one or more organization inputs.

4. The method of claim 3, wherein the one or more organization inputs comprises at least one of a test effort, a test efficiency, defect escape probability tables, a current project profile, potential defects and a test process.

5. The method of claim 1, wherein the receiving the data further comprises receiving one or more other inputs, wherein the determining the one or more business metrics is additionally based on the one or more other inputs.

6. The method of claim 5, wherein the one or more other inputs comprises at least one of an average cost to fix defect in different phases of a software development life cycle and daily rate for a human resource.

7. The method of claim 1, wherein the one or more business metrics further comprise a benefit for reducing the cycle time;

and

wherein for determining the benefit for reducing the cycle time;

the receiving the data comprises receiving a test efficiency; and

the processing the data comprises;

determining a defect volume shifting from a source activity to an optimally-timed discovery; and

calculating the benefit for reducing the cycle time as the defect volume shifting from the source activity to the optimally-timed discovery divided by the test efficiency.

8. The method of claim 1, wherein the one or more business metrics further comprise a benefit of reducing invalid defects;

and

wherein for determining the benefit of reducing invalid defects;

the receiving the data comprises;

receiving potential defects and defect distributions by triggers and an improved valid rate and current invalid rate; and

receiving an average fix cost for defects in different activities; and

the processing the data comprises;

calculating a total reduced invalid defect volume as the product of a total defect volume and a difference between the current invalid rate and the improved valid rate; and

calculating the benefit of reducing invalid defects as a product of the total reduced invalid defect volume, the average fix cost for defects in different activities and an adjustment percentage.

9. The method of claim 1, wherein the one or more business metrics further comprise a benefit for reducing production defects;

and

wherein for determining the benefit for reducing production defects;

the receiving the data comprises;

receiving potential defects and defect distributions by triggers, which indicates potential production defects;

receiving a production defect rate after DAS preventative actions are taken; and

receiving an average fix cost for defects in different activities;

determining a reduced production defect volume as a difference between potential production defects and the production defect rate after DAS actions; and

the processing the data comprises;

calculating the benefit for reducing production defects as a product of the average fix cost for defects in different activities and the reduced production defect volume.

10. The method of claim 1, wherein a service provider at least one of creates, maintains, deploys and supports the computer infrastructure.

11. The method of claim 1, wherein steps are provided by a service provider on a subscription, advertising, and/or fee basis.

12. A system implemented in hardware comprising a processor for determining when to remove defects in a lifecycle of developing a software product, comprising:

a data receiving tool operable to receive data including at least one of defect analysis starter (DAS)/defect reduction method (DRM) defect analysis data, organization data and other data of a software development project to develop the software product, the DAS/DRM defect analysis data comprises a quantity of defects in the software product discovered by each of a plurality of defect triggers, each comprised of conditions under which a defect is uncovered during testing of the software product, and an estimated percentage of defects reduction in the software product after preventative actions are taken by each defect trigger;

a data processing tool operable to;

identify improvements which have an impact to reduce defects in the software product, and which of the improvements have a highest impact among the identified improvements to reduce defects in the software product to determine which defects to remove and when to remove them to control the number of defects in the completed software product;

determine cost versus benefit received for making the improvements to provide an improved software product with a reduced number of defects,

resolve defects in the software product during development of the software product before marketing the software product;

perform a calibration comprising adjusting a defect volume and a distribution by trigger in accordance with the received quantity of defects discovered by each defect trigger;

apply the preventative actions, after the calibration has been performed, to determine a reduction in the defect volume and the distribution by each defect trigger;

calculate a benefit for preventing an injection of defects comprising calculating costs associated with avoidance of discovered defects and costs associated with avoidance of escaped defects according to the determined reduction in the defect volume and the distribution by each defect trigger;

shift defects by type from a source activity to an optimally-timed discovery, wherein a defect is one of directly shifted and shifted with a defect probability table; and

calculate a shifting benefit as a defect fix cost difference between discovering one or more defects at the source activity and discovering the one or more defects at the optimally-timed discovery;

determine one or more business metrics based on the data, wherein the one or more business metrics comprises the benefit for preventing the injection of defects and a benefit for shifting defect removal earlier; and

a report generation tool operable to generate at least one report regarding proceeding with the development of the software product based on the one or more business metrics,

the system further being operable to apply prevention actions to prevent an injection of defects into the software product, based upon the benefit calculated for preventing the injection of defects into the software product, and to shift defect removal from a first time in the lifecycle of developing the software product, which is a time used to determine an initial estimate for removing defects, to a second time in the lifecycle of developing the software product, wherein the second time in the lifecycle of developing the software product is earlier in the lifecycle of developing the software product than the first time, based upon the benefit determined for shifting the defect removal earlier in the lifecycle of developing the software product.

13. The system of claim 12, wherein:

the DAS/DRM defect analysis data further comprises a production defect rate after DAS actions taken, an improved valid rate, and a current invalid rate;

the organization data comprises at least one of a test effort, a test efficiency, defect escape probability tables, a current project profile, potential defects and a test process; and

the one or more other inputs comprises at least one of an average cost to fix defect in different phases of a software development life cycle and daily rate for a human resource.

14. The system of claim 12, wherein for determining the benefit for shifting defect removal earlier:

the data receiving tool is further operable to receive a potential defect and defect distribution by triggers and the quantity defects discovered by each trigger.

15. The system of claim 12, wherein the determining the reduction in the defect volume comprises calculating a product of a percentage of defect reduction and a total defect volume of a software development project;

and

wherein;

for discovered defects, the benefit for preventing the injection of defects is the sum of a testing cost and a fix cost; and

for escaped defects, the benefit for preventing the injection of defects is determined based on a business case.

16. The system of claim 12, wherein the one or more business metrics further comprise a benefit for reducing the cycle time;

and

wherein for determining the benefit for reducing the cycle time;

the data processing tool is further operable to;

determine a defect volume shifting from the source activity to the optimally-timed discovery; and

calculate the benefit for reducing the cycle time the processing as the defect volume shifting from the source activity to the optimally-timed discovery divided by a test efficiency.

17. The system of claim 12, wherein the one or more business metrics further comprise a benefit of reducing invalid defects;

and

wherein for determining the benefit of reducing invalid defects;

the data processing tool is further operable to;

calculate a total reduced invalid defect volume as a product of a total defect volume and a difference between a current invalid rate and an improved valid rate; and

calculate the benefit of reducing invalid defects as a product of the total reduced invalid defect volume, an average fix cost for defects in different activities and an adjustment percentage.

18. The system of claim 12, wherein the one or more business metrics further comprise a benefit for reducing production defects;

and

wherein for determining the benefit of reducing production defects;

the data processing tool is further operable to;

determine a reduced production defect volume as a difference between potential production defects and a production defect rate after DAS actions; and

calculate the benefit for reducing production defects as a product of an average fix cost for defects in different activities and the reduced production defect volume.

19. A computer program product comprising a computer usable storage device having readable program code embodied in the storage device for determining when to remove defects in a production lifecycle of a software product, the computer program product includes at least one component operable to perform steps of:

receiving data including defect analysis defect analysis starter (DAS)/defect reduction method (DRM) defect analysis data of a software development project of an organization to develop the software product, the DAS/DRM defect analysis data comprises a quantity of defects in the software product discovered by each of a plurality of defect triggers, each comprised of conditions under which a defect is uncovered during testing of the software product, and an estimated percentage of defects reduction in the software product after preventative actions are taken by each defect trigger;

processing the data to identify improvements which have an impact to reduce defects in the software product, and which of the improvements have a highest impact among the identified improvements to reduce defects in the software product to determine which defects to remove and when to remove them to control the number of defects in the completed software product, the processing comprising;

determining cost versus benefit received for making the improvements to provide an improved software product with a reduced number of defects,

resolving defects in the software product during development of the software product before marketing the software product;

performing a calibration comprising adjusting a defect volume and a distribution by each defect trigger in accordance with the received quantity of defects discovered by each defect trigger;

applying the preventative actions, after the calibration has been performed, to determine a reduction in the defect volume and the distribution by each defect trigger, wherein the determining the reduction in the defect volume comprises calculating a product of the percentage of defect reduction and a total defect volume of the software development project;

calculating a benefit for preventing an injection of defects comprising calculating costs associated with avoidance of discovered defects and costs associated with avoidance of escaped defects according to the determined reduction in the defect volume and the distribution by each defect trigger;

shifting defects by type from a source activity to an optimally-timed discovery, wherein a defect is one of directly shifted and shifted with a defect probability table; and

calculating a shifting benefit as a defect fix cost difference between discovering one or more defects at the source activity and discovering the one or more defects at the optimally-timed discovery;

determining one or more business metrics for the organization based on the data, wherein the one or more business metrics comprises the benefit for preventing the injection of defects and a benefit for shifting defect removal earlier;

generating at least one report regarding proceeding with the development of the software product based on the one or more business metrics,

applying prevention actions to prevent an injection of defects into the software product and to shift defect removal earlier in the lifecycle of developing the software product based upon the benefit calculated for preventing the injection of defects into the software product and the benefit determined for shifting the defect removal earlier in the lifecycle of developing the software product; and

applying prevention actions to remove production defects at a time, in the production lifecycle of the software product, determined by a comparison of a first maximum potential decrease in the number of production defects if production defect removal is made in a first test phase of the production lifecycle of the software product and a second maximum potential decrease in the number of production defects if production defect removal is made in a second test phase of the production lifecycle of the software product which is subsequent to the first test phase.

20. The computer program product of claim 19, wherein:

the data further comprises one or more organization input,

the determining the one or more business metrics is additionally based on the one or more organization inputs, and

the one or more organization inputs comprises at least one of a test effort, a test efficiency, defect escape probability tables, a current project profile, potential defects and a test process.

21. The computer program product of claim 19, wherein:

the data further comprises one or more other inputs,

the determining the one or more business metrics is additionally based on the one or more other inputs, and

the one or more other inputs comprises at least one of an average cost to fix defect in different phases of a software development life cycle and daily rate for a human resource.

22. A computer system for classifying automated code inspection services defect output for determining when to remove defects in a lifecycle of developing a software product, the system comprising:

a CPU, a computer readable memory and a computer readable storage media;

first program instructions to receive data including defect analysis starter (DAS)/defect reduction method (DRM) defect analysis data of a software development project of an organization to develop the software product, the DAS/DRM defect analysis data comprises a quantity of defects in the software product discovered by each of a plurality of defect triggers, each comprised of conditions under which a defect is uncovered during testing of the software product, and an estimated percentage of defects reduction in the software product after preventative actions are taken by each defect trigger, wherein the receiving the data comprises receiving a potential defect and defect distribution by triggers and the quantity of defects discovered by each trigger;

second program instructions to process the data to identify improvements which have an impact to reduce defects in the software product, and which of the improvements have a highest impact among the identified improvements to reduce defects in the software product to determine which defects to remove and when to remove them to control the number of defects in the completed software product, to determine cost versus benefit received for making the improvements to provide an improved software product with a reduced number of defects, to resolve defects in the software product during development of the software product before marketing the software product, and to determine one or more business metrics of the organization based on the data, the one or more business metrics comprises a benefit for preventing an injection of defects and a benefit for shifting defect removal earlier, wherein the processing the data comprises;

performing a calibration comprising adjusting a defect volume and a distribution by each defect trigger in accordance with the received quantity of defects discovered by each defect trigger;

applying the preventative actions, after the calibration has been performed, to determine a reduction in the defect volume and the distribution by each defect trigger, wherein the determining the reduction in the defect volume comprises calculating a product of the percentage of defect reduction and a total defect volume of the software development project;

calculating the benefit for preventing the injection of defects comprising calculating costs associated with avoidance of discovered defects and costs associated with avoidance of escaped defects according to the determined reduction in the defect volume and the distribution by each defect trigger;

for discovered defects, the benefit for preventing the injection of defects is the sum of a testing cost and a fix cost;

for escaped defects, the benefit for preventing the injection of defects is determined based on a business case;

shifting defects by type from a source activity to an optimally-timed discovery, wherein a defect is one of directly shifted and shifted with a defect probability table; and

calculating a shifting benefit as a defect fix cost difference between discovering one or more defects at the source activity and discovering the one or more defects at the optimally-timed discovery;

third program instructions to generate at least one report regarding proceeding with the development of the software product based on the one or more business metrics, and

fourth program instructions to apply prevention actions to prevent an injection of defects into the software product, based upon the benefit calculated for preventing the injection of defects into the software product, and to shift defect removal from a first time in the lifecycle of developing the software product, which is a time used to determine an initial estimate for removing defects, to a second time in the lifecycle of developing the software product, wherein the second time in the lifecycle of developing the software product is earlier in the lifecycle of developing the software product than the first time, based upon the benefit determined for shifting the defect removal earlier in the lifecycle of developing the software product,

wherein the first, second, third and fourth program instructions are stored on the computer readable storage media for execution by the CPU via the computer readable memory.