Method and System for Evaluating the Testing of a Software System Having a Plurality of Components

• US 20120017195A1
• Filed: 04/15/2011
• Published: 01/19/2012
• Est. Priority Date: 07/17/2010
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Kaulgud, Vikrant Shyamkant, Punja, Rohit, Awaskar, Hillel Solomon
• Granted Patent: US 8,601,441 B2
• Time in Patent Office: Days
• US Class Current: 717/101
• CPC Class Codes: G06F 11/3692 for test results analysis

Abstract

Methods and systems are disclosed for evaluating testing of a software system in a testing project in which the testing is performed on software code in a plurality of components of the software system. Multi-dimensional data related to the testing project, including results of the testing, are automatically collected. Values for metrics related to a quality of testing effort that components have undergone in the testing project are developed. In one embodiment, the measure is based on a measure of amount of software code of the plurality of components that has been tested during the testing project. Projected testing results at completion of the testing are developed by forecasting projected values of at least one metric at completion of the testing project.

Claims

1. A method for evaluating testing of a software system in a testing project in which the testing is performed on software code in a plurality of components of the software system, comprising:

automatically collecting, by a processor, multi-dimensional data related to the testing project, including results of the testing; and

developing, by the processor, a QCTE measure for the testing project,

wherein QCTE comprises a quality of testing effort that the plurality of components have undergone in the testing project, and

wherein the QCTE measure is based on a measure of amount of software code of the plurality of components that has been tested during the testing project and a measure of extent of completion of the testing project.

2. The method of claim 1, further comprising:

developing, by the processor, the measure of the extent of completion of the testing project;

developing, by the processor, a current QCTE measure comprising the QCTE measure to date for the testing project; and

developing, by the processor, projected testing results at completion of the testing, by forecasting a projected QCTE measure at completion of the testing project.

3. The method of claim 2, further comprising analyzing, by the processor, the multi-dimensional data, the measure of the current extent of completion of the testing, the current QCTE measure, and the projected QCTE measure to identify an area of potential concern in the software system.

4. The method of claim 3, further comprising developing remediation advice for the identified area of concern using best practices extracted from other software system projects.

5. The method of claim 4, wherein the analyzing further comprises:

developing insight from the multi-dimensional data, the measure of the current extent of testing, and the current QCTE measure; and

coupling the insight with the best practices to develop the remediation advice.

6. The method of claim 1, further comprising at least one of the following:

developing a projected completion date of the testing to a target extent of completion,

developing a measure of a first estimated effort to complete the testing to the target extent, and

developing a second estimated effort to complete the testing according to a target testing schedule.

7. The method of claim 2, wherein the forecasting comprises developing a measure of predicted likelihood that the testing will achieve a desired level of component testing according to a target testing schedule or targeted testing effort.

8. The method of claim 1, wherein the QCTE measure is based on a count of components having amounts of tested software code above a threshold level.

9. The method of claim 1, wherein the QCTE measure is generated from:

an amount of scheduled effort consumed by the testing, and

a percentage of components determined to have amounts of tested software code above a threshold level.

10. The method of claim 1, wherein the QCTE measure comprises a value generated based on the equation:

11. The method of claim 10, wherein the extent of unhealthiness of code coverage is based on a count of components in which an amount of tested software code is below a selected threshold level.

12. The method of claim 11, wherein the extent of unhealthiness of code coverage comprises a weighted measure of the extent of code coverage that is exhibited by the components of the computer system undergoing testing.

13. The method of claim 10, wherein a measure of the extent of code coverage for a selected component is weighted by an associated code coverage level.

14. The method of claim 1, wherein, for a selected component, an amount of software code that has been tested during the testing project comprises a count of lines of software code in the selected component that have been executed during the testing.

15. The method of claim 1, wherein, for a selected component, an amount of software code that has been tested during the testing project comprises a count of software code features in the selected component that have been tested during the testing.

16. A method for evaluating testing of a software system in a testing project in which the testing is performed on software code in a plurality of components of the software system, comprising:

automatically collecting, by a processor, multi-dimensional data related to the testing project, including results of the testing; and

developing, by the processor for a component in the plurality of components, a value for a metric related to a quality of testing effort that the component has undergone in the testing project, wherein the quality of testing effort for the component is based on a measure of amount of software code of the component that has been tested during the testing project; and

developing, by the processor, projected testing results at completion of the testing, by forecasting a projected value of the metric at completion of the testing project.

17. The method of claim 16, further comprising analyzing, by the processor, the multi-dimensional data, a measure of the current extent of completion of the testing, and the metric to identify an area of potential concern in the software system.

18. The method of claim 17, further comprising developing remediation advice for the identified area of concern using best practices extracted from other software system projects.

19. The method of claim 18, wherein the analyzing further comprises:

developing insight from the multi-dimensional data, the measure of the current extent of testing, and the metric; and

coupling the insight with the best practices to develop the remediation advice.

20. The method of claim 16, wherein the metric is further based on a level of complexity of the software code in the component.

21. A system for evaluating testing of a software system in a testing project in which the testing is performed on software code in a plurality of components of the software system, comprising:

a processor; and

a non-transitory computer-readable medium encoding instructions for evaluating the testing of the software system and for execution by the processor, the instructions including;

a multi-dimensional data collecting module configured to automatically collect multi-dimensional data related to the testing project, including results of the testing; and

a metrics module configured to develop a QCTE measure for the testing project,

wherein QCTE comprises a quality of testing effort that the plurality of components have undergone in the testing project, and

wherein the QCTE measure is based on a measure of amount of software code of the plurality of components that has been tested during the testing project and a measure of extent of completion of the testing project.

22. The system of claim 21, wherein the metrics module is further configured to base the QCTE measure on a count of components having amounts of tested software code above a threshold level.

23. The system of claim 21, wherein the metrics module is further configured to generate the QCTE measure from:

an amount of scheduled effort consumed by the testing, and

a percentage of components determined to have amounts of tested software code above a threshold level.

24. The system of claim 21, wherein the metrics module is further configured to generate the QCTE measure based on the equation:

25. The system of claim 24, wherein the extent of unhealthiness of code coverage is based on a count of components in which an amount of tested software code is below a selected threshold level.

26. The system of claim 25, wherein the extent of unhealthiness of code coverage comprises a weighted measure of the extent of code coverage that is exhibited by the components of the computer system undergoing testing.

27. The system of claim 24, wherein a measure of the extent of code coverage for a selected component is weighted by an associated code coverage level.

28. A system for evaluating testing of a software system in a testing project in which the testing is performed on software code in a plurality of components of the software system, comprising:

a processor; and

a non-transitory computer-readable medium encoding instructions for evaluating the testing of the software system and for execution by the processor, the instructions including;

a multi-dimensional data collecting module configured to automatically collect multi-dimensional data related to the testing project, including results of the testing; and

a metrics module configured to develop, for a component in the plurality of components, a value for a metric related to a quality of testing effort that the component has undergone in the testing project, wherein the quality of testing effort for the component is based on a measure of amount of software code of the component that has been tested during the testing project; and

a results projecting module configured to develop projected testing results at completion of the testing, by forecasting a projected value of the metric at completion of the testing project.

29. The system of claim 28, further comprising an analyzer configured to analyze the multi-dimensional data, a measure of the current extent of completion of the testing, and the metric to identify an area of potential concern in the software system.

30. The system of claim 29, further comprising an advisor module configured to developing remediation advice for the identified area of concern using best practices extracted from other software system projects.

31. The system of claim 30, wherein the advisor module is further configured to:

develop insight from the multi-dimensional data, the measure of the current extent of testing, and the metric; and

couple the insight with the best practices to develop the remediation advice.

32. The system of claim 28, wherein a metrics module is further configured to base the metric on a level of complexity of the software code in the component.

33. A computer program embodied on a non-transitory computer readable medium for evaluating testing of a software system in a testing project in which the testing is performed on software code in a plurality of components of the software,

wherein the computer program comprises modules encoding interrelated and interdependent processes, including a multi-dimensional data collecting module, a metrics module, and a results projecting module, and

wherein the computer program is configured to perform a method comprising:

automatically collecting, by the multi-dimensional data collecting module, multi-dimensional data related to the testing project, including results of the testing; and

developing, by the metrics module, for a component in the plurality of components, a value for a metric related to a quality of testing effort that the component has undergone in the testing project, wherein the quality of testing effort for the component is based on a measure of amount of software code of the component that has been tested during the testing project; and

developing, by the results projecting module, projected testing results at completion of the testing, by forecasting a projected value of the metric at completion of the testing project.

34. A computer program embodied on a non-transitory computer readable medium for evaluating testing of a software system in a testing project in which the testing is performed on software code in a plurality of components of the software,

wherein the computer program comprises modules encoding interrelated and interdependent processes, including a multi-dimensional data collecting module, a metrics module, and a results projecting module, and

wherein the computer program is configured to perform a method comprising:

automatically collecting, by the multi-dimensional data collecting module, multi-dimensional data related to the testing project, including results of the testing; and

developing, by the metrics module, a QCTE measure for the testing project,

wherein QCTE comprises a quality of testing effort that the plurality of components have undergone in the testing project, and

wherein the QCTE measure is based on a measure of amount of software code of the plurality of components that has been tested during the testing project and a measure of extent of completion of the testing project.