Method and system for evaluating the testing of a software system having a plurality of components

• US 8,601,441 B2
• Filed: 04/15/2011
• Issued: 12/03/2013
• Est. Priority Date: 07/17/2010
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Kaulgud, Vikrant Shyamkant, Punja, Rohit, Awaskar, Hillel Solomon
• Primary Examiner(s): Chen, Qing
• Application Number:
  US13/087,636
• Publication Number:
  US 20120017195A1
• Time in Patent Office: 963 Days
• Field of Search: 717124-135, 714/37, 714/38.1
• US Class Current: 717/124
• CPC Class Codes: G06F 11/3692 for test results analysis

Abstract

Methods and systems are disclosed for evaluating testing of a software system in a testing project in which the testing is performed on software code in a plurality of components of the software system. Multi-dimensional data related to the testing project, including results of the testing, are automatically collected. Values for metrics related to a quality of testing effort that components have undergone in the testing project are developed. In one embodiment, the measure is based on a measure of amount of software code of the plurality of components that has been tested during the testing project. Projected testing results at completion of the testing are developed by forecasting projected values of at least one metric at completion of the testing project.

Claims

1. A method for evaluating testing of a software system in a testing project in which the testing is performed on software code in a plurality of components of the software system, comprising:

automatically collecting, by a processor, multi-dimensional data related to the testing project, including results of the testing;

developing, by the processor, a Quality of Component Test Effort (QCTE) measure for the testing project,

wherein the QCTE measure comprises a quality of testing effort that the plurality of components of the software system have undergone in the testing project, and

wherein the QCTE measure is based on a measure of an amount of software code of the plurality of components of the software system that has been tested during the testing project and a measure of an extent of completion of the testing project;

developing, by the processor, projected testing results at completion of the testing by forecasting a projected QCTE measure at completion of the testing project; and

analyzing, by the processor, the multi-dimensional data, the measure of the extent of completion of the testing project, a current QCTE measure, and the projected QCTE measure to identify an area of potential concern in the software system.

2. The method of claim 1, further comprising:

developing, by the processor, the measure of the extent of completion of the testing project; and

developing, by the processor, the current QCTE measure comprising the QCTE measure to date for the testing project.

3. The method of claim 2, wherein the forecasting comprises developing a measure of predicted likelihood that the testing will achieve a desired level of component testing according to a target testing schedule or a target testing effort.

4. The method of claim 1, further comprising developing remediation advice for the identified area of potential concern in the software system using best practices extracted from other software system projects.

5. The method of claim 4, wherein the analyzing further comprises:

developing insight from the multi-dimensional data, the measure of the extent of completion of the testing project, and the current QCTE measure; and

coupling the insight with the best practices to develop the remediation advice.

6. The method of claim 1, further comprising at least one of the following:

developing a projected completion date of the testing to a target extent of completion,

developing a measure of a first estimated effort to complete the testing to the target extent of completion, and

developing a second estimated effort to complete the testing according to a target testing schedule.

7. The method of claim 1, wherein the QCTE measure is based on a count of components having amounts of tested software code above a threshold level.

8. The method of claim 1, wherein the QCTE measure is generated from:

an amount of scheduled effort consumed by the testing, and

a percentage of components determined to have amounts of tested software code above a threshold level.

9. The method of claim 1, wherein the QCTE measure comprises a value generated based on the equation:

10. The method of claim 9, wherein the extent of unhealthiness of code coverage is based on a count of components in the plurality of components of the software system in which an amount of tested software code is below a selected threshold level.

11. The method of claim 10, wherein the extent of unhealthiness of code coverage comprises a weighted measure of an extent of code coverage that is exhibited by the plurality of components of the software system undergoing testing.

12. The method of claim 9, wherein a measure of an extent of code coverage for a selected component is weighted by an associated code coverage level.

13. The method of claim 1, wherein, for a selected component, an amount of software code that has been tested during the testing project comprises a count of lines of software code in the selected component that have been executed during the testing.

14. The method of claim 1, wherein for a selected component, an amount of software code that has been tested during the testing project comprises a count of software code features in the selected component that have been tested during the testing.

15. A method for evaluating testing of a software system in a testing project in which the testing is performed on software code in a plurality of components of the software system, comprising:

automatically collecting, by a processor, multi-dimensional data related to the testing project, including results of the testing;

developing, by the processor for a component in the plurality of components of the software system, a value for a metric for the testing project, wherein the value of the metric is related to a quality of testing effort that the component has undergone in the testing project, and wherein the quality of testing effort for the component is based on a measure of an amount of software code of the component that has been tested during the testing project and a measure of an extent of completion of the testing project;

developing, by the processor, projected testing results at completion of the testing by forecasting a projected value of the metric at completion of the testing project; and

analyzing, by the processor, the multi-dimensional data, the measure of the extent of completion of the testing project, a current value of the metric, and the projected value of the metric to identify an area of potential concern in the software system.

16. The method of claim 15, further comprising developing remediation advice for the identified area of potential concern in the software system using best practices extracted from other software system projects.

17. The method of claim 16, wherein the analyzing further comprises:

developing insight from the multi-dimensional data, the measure of the extent of completion of the testing project, and the current value of the metric; and

coupling the insight with the best practices to develop the remediation advice.

18. The method of claim 15, wherein the value of the metric is further based on a level of complexity of the software code in the component.

19. A system for evaluating testing of a software system in a testing project in which the testing is performed on software code in a plurality of components of the software system, comprising:

a processor; and

a non-transitory computer-readable medium encoding instructions for evaluating the testing of the software system and for execution by the processor, the instructions including;

a multi-dimensional data collecting module configured to automatically collect multi-dimensional data related to the testing project, including results of the testing;

a metrics module configured to develop a Quality of Component Test Effort (QCTE) measure for the testing project,

wherein the QCTE measure comprises a quality of testing effort that the plurality of components of the software system have undergone in the testing project,

wherein the QCTE measure is based on a measure of an amount of software code of the plurality of components of the software system that has been tested during the testing project and a measure of an extent of completion of the testing project, and

wherein the metrics module is further configured to develop projected testing results at completion of the testing by forecasting a projected QCTE measure at completion of the testing project; and

an analyzer module configured to analyze the multi-dimensional data, the measure of the extent of completion of the testing project, a current QCTE measure, and the projected QCTE measure to identify an area of potential concern in the software system.

20. The system of claim 19, wherein the metrics module is further configured to base the QCTE measure on a count of components in the plurality of components of the software system having amounts of tested software code above a threshold level.

21. The system of claim 19, wherein the metrics module is further configured to generate the QCTE measure from:

an amount of scheduled effort consumed by the testing, and

a percentage of components in the plurality of components of the software system determined to have amounts of tested software code above a threshold level.

22. The system of claim 19, wherein the metrics module is further configured to generate the QCTE measure based on the equation:

23. The system of claim 22, wherein the extent of unhealthiness of code coverage is based on a count of components in the plurality of components of the software system in which an amount of tested software code is below a selected threshold level.

24. The system of claim 23, wherein the extent of unhealthiness of code coverage comprises a weighted measure of an extent of code coverage that is exhibited by the plurality of components of the software system undergoing testing.

25. The system of claim 22, wherein a measure of an extent of code coverage for a selected component is weighted by an associated code coverage level.

26. A system for evaluating testing of a software system in a testing project in which the testing is performed on software code in a plurality of components of the software system, comprising:

a processor; and

a non-transitory computer-readable medium encoding instructions for evaluating the testing of the software system and for execution by the processor, the instructions including;

a multi-dimensional data collecting module configured to automatically collect multi-dimensional data related to the testing project, including results of the testing;

a metrics module configured to develop, for a component in the plurality of components of the software system, a value for a metric for the testing project, wherein the value of the metric is related to a quality of testing effort that the component has undergone in the testing project, and wherein the quality of testing effort for the component is based on a measure of an amount of software code of the component that has been tested during the testing project and a measure of an extent of completion of the testing project;

a results projecting module configured to develop projected testing results at completion of the testing by forecasting a projected value of the metric at completion of the testing project; and

an analyzer module configured to analyze the multi-dimensional data, the measure of the extent of completion of the testing project, a current value of the metric, and the projected value of the metric to identify an area of potential concern in the software system.

27. The system of claim 26, further comprising an advisor module configured to develop remediation advice for the identified area of potential concern in the software system using best practices extracted from other software system projects.

28. The system of claim 27, wherein the advisor module is further configured to:

develop insight from the multi-dimensional data, the measure of the extent of completion of the testing project, and the current value of the metric; and

couple the insight with the best practices to develop the remediation advice.

29. The system of claim 26, wherein the metrics module is further configured to base the value of the metric on a level of complexity of the software code in the component.

30. A computer program embodied on a non-transitory computer readable medium for evaluating testing of a software system in a testing project in which the testing is performed on software code in a plurality of components of the software system,

wherein the computer program comprises modules encoding interrelated and interdependent processes, including a multi-dimensional data collecting module, a metrics module, a results projecting module, and an analyzer module, and

wherein the computer program is configured to perform a method comprising:

automatically collecting, by the multi-dimensional data collecting module, multi-dimensional data related to the testing project, including results of the testing;

developing, by the metrics module, for a component in the plurality of components of the software system, a value for a metric for the testing project, wherein the value of the metric is related to a quality of testing effort that the component has undergone in the testing project, and wherein the quality of testing effort for the component is based on a measure of an amount of software code of the component that has been tested during the testing project and a measure of an extent of completion of the testing project;

developing, by the results projecting module, projected testing results at completion of the testing by forecasting a projected value of the metric at completion of the testing project; and

analyzing, by the analyzer module, the multi-dimensional data, the measure of the extent of completion of the testing project, a current value of the metric, and the projected value of the metric to identify an area of potential concern in the software system.

31. A computer program embodied on a non-transitory computer readable medium for evaluating testing of a software system in a testing project in which the testing is performed on software code in a plurality of components of the software system,

wherein the computer program comprises modules encoding interrelated and interdependent processes, including a multi-dimensional data collecting module, a metrics module, a results projecting module, and an analyzer module, and

wherein the computer program is configured to perform a method comprising:

automatically collecting, by the multi-dimensional data collecting module, multi-dimensional data related to the testing project, including results of the testing;

developing, by the metrics module, a Quality of Component Test Effort (QCTE) measure for the testing project,

wherein the QCTE measure comprises a quality of testing effort that the plurality of components of the software system have undergone in the testing project, and

wherein the QCTE measure is based on a measure of an amount of software code of the plurality of components of the software system that has been tested during the testing project and a measure of an extent of completion of the testing project;

developing, by the metrics module, projected testing results at completion of the testing by forecasting a projected QCTE measure at completion of the testing project; and

analyzing, by the analyzer module, the multi-dimensional data, the measure of the extent of completion of the testing project, a current QCTE measure, and the projected QCTE measure to identify an area of potential concern in the software system.