Assessment system for choosing maintenance approaches for GUI-directed test scripts

• US 9,009,672 B2
• Filed: 07/25/2013
• Issued: 04/14/2015
• Est. Priority Date: 06/11/2009
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Grechanik, Mark, Xie, Qing, Fu, Chen
• Primary Examiner(s): Zhen, Wei
• Assistant Examiner(s): UNG, LANNY N
• Application Number:
  US13/950,554
• Publication Number:
  US 20150007141A1
• Time in Patent Office: 628 Days
• Field of Search: 717/124, 717/125
• US Class Current: 717/124
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

A graphical user interface (GUI) tool analysis system helps determine whether to purchase or license automated testing tools. The system provides guidance, e.g., to test managers, for making decisions on expenditures for the automated test tools. As a result, the test managers need not make purchasing decisions ad hoc, based on their own personal experience and perceived benefits of implementing a tool based automatic testing approach versus a manual testing approach.

Claims

1. A method for recommending graphical user interface (GUI) tools comprising:

identifying statistical variables for evaluating allocation of testing resources for test scripts;

partitioning the statistical variables into input variables and output variables;

defining in a memory statistical constraint equations with at least some of the statistical variables;

defining in the memory an evaluation function, with at least some of the statistical variables, for evaluating a testing cost of allocation of testing resources for test scripts;

executing with a processor coupled to the memory an optimization program stored in the memory,

the optimization program comprising instructions that when executed cause the processor to;

obtain a workplace dependent value for each of the input variables apply the constraint equations to a parameter space defined by the output variables to define a curve in the parameter space;

execute an optimization algorithm to obtain an optimum point on the curve that achieves a defined goal for the evaluation function, by the optimization algorithm determining the optimum point on the curve by determining a random point on the curve for a node and one or more neighboring points to the random point;

determine a recommendation based on the optimum point, the random point and the one or more neighboring points to the random point; and

store the recommendation in the memory.

2. The method of claim 1, where the optimization algorithm determines the optimum point on the curve based on returned values of the evaluation function.

3. The method of claim 1, where the evaluation function determines, as the testing cost, a minimum value of:

a first equation defining a manual testing cost; and

a second equation defining an automated testing cost.

4. The method of claim 1, where the evaluation function determines, as the testing cost, a minimum value of:

a first equation defining a first monetary cost of manual testing of a graphical user interface (GUI), the first equation comprising a first subset of the statistical variables;

a second equation representing a second monetary cost of automated testing of the GUI with manual maintenance, the second equation comprising a second subset of the plurality of statistical variables; and

a third equation representing a third monetary cost of automated testing of the GUI with tool based maintenance, the third equation comprising a third subset of the plurality of statistical variables.

5. The method of claim 1, where the statistical variables include any one or more of:

manual testing time (MTT), number of test engineers needed (NTEN), number of manual testers needed (NMT), test script composition time (TSCT), test script running time (TSRT), test script maintenance time (TSMT), and tool based maintenance speed up factor (MS), and the statistical constraint equations comprise any one or more of MTT, NTEN, NMT, TSCT, TSRT, and TSMT.

6. The method of claim 5, where the statistical constraint equations include any one or more of:

(MTT×

NTEN)/(NMT×

TSCT);

(MTT×

NTEN)/(NMT×

TSRT);

(TSMT=TSCT×

87%×

(ER−

1); and

MS.

7. The method of claim 6, where any one or more of the constraint equations are set to constrain values obtained from controlled experiments on prior test script testing projects.

8. A system for recommending graphical user interface (GUI) tools comprising:

a processor; and

stored on a computer readable memory coupled to the processor;

statistical variables for evaluating allocation of testing resources for test scripts, the statistical variables partitioned into input variables and output variables;

statistical constraint equations with at least some of the statistical variables;

an evaluation function, with at least some of the statistical variables, for evaluating a testing cost of allocation of testing resources for test scripts,

an optimization program comprising instructions that when executed cause the processor to;

obtain a workplace dependent value for each of the input variables apply the constraint equations to a parameter space defined by the output variables to define a curve in the parameter space;

execute an optimization algorithm to obtain an optimum point on the curve that achieves a defined goal for the evaluation function, by the optimization algorithm determining the optimum point on the curve by determining a random point on the curve for a node and one or more neighboring points to the random point;

determine a recommendation based on the optimum point, the random point and the one or more neighboring points to the random point; and

store the recommendation in the memory.

9. The system of claim 8, where the optimization algorithm determines the optimum point on the curve based on returned values of the evaluation function.

10. The system of claim 8, where the evaluation function determines, as the testing cost, a minimum value of:

a first equation defining a manual testing cost; and

a second equation defining an automated testing cost.

11. The system of claim 8, where the evaluation function determines, as the testing cost, a minimum value of:

a first equation defining a first monetary cost of manual testing of a graphical user interface (GUI), the first equation comprising a first subset of the statistical variables;

a second equation representing a second monetary cost of automated testing of the GUI with manual maintenance, the second equation comprising a second subset of the plurality of statistical variables; and

a third equation representing a third monetary cost of automated testing of the GUI with tool based maintenance, the third equation comprising a third subset of the plurality of statistical variables.

12. The system of claim 8, where the statistical variables include any one or more of:

manual testing time (MTT), number of test engineers needed (NTEN), number of manual testers needed (NMT), test script composition time (TSCT), test script running time (TSRT), test script maintenance time (TSMT), and tool based maintenance speed up factor (MS), and the statistical constraint equations comprise any one or more of MTT, NTEN, NMT, TSCT, TSRT, and TSMT.

13. The system of claim 12, where the statistical constraint equations include any one or more of:

(MTT×

NTEN)/(NMT×

TSCT);

(MTT×

NTEN)/(NMT×

TSRT);

(TSMT=TSCT×

87%×

(ER−

1); and

MS.

14. The system of claim 13, where any one or more of the constraint equations are set to constrain values obtained from controlled experiments on prior test script testing projects.

15. An article of manufacture, comprising:

a non-transitory computer readable medium;

and stored on the non-transitory computer readable medium;

statistical variables for evaluating allocation of testing resources for test scripts, the statistical variables partitioned into input variables and output variables;

statistical constraint equations with at least some of the statistical variables;

an evaluation function, with at least some of the statistical variables, for evaluating a testing cost of allocation of testing resources for test scripts;

an optimization program comprising instructions that when executed cause a processor to;

obtain a workplace dependent value for each of the input variables apply the constraint equations to a parameter space defined by the output variables to define a curve in the parameter space;

execute an optimization algorithm to obtain an optimum point on the curve that achieves a defined goal for the evaluation function, by the optimization algorithm determining the optimum point on the curve by determining a random point on the curve for a node and one or more neighboring points to the random point;

determine a recommendation based on the optimum point, the random point and the one or more neighboring points to the random point; and

store the recommendation in the memory.

16. The article of manufacture of claim 15, where the optimization algorithm determines the optimum point on the curve based on returned values of the evaluation function.

17. The article of manufacture of claim 15, where the evaluation function determines, as the testing cost, a minimum value of:

a first equation defining a manual testing cost; and

a second equation defining an automated testing cost.

18. The article of manufacture of claim 15, where the evaluation function determines, as the testing cost, a minimum value of:

a first equation defining a first monetary cost of manual testing of a graphical user interface (GUI), the first equation comprising a first subset of the statistical variables;

a second equation representing a second monetary cost of automated testing of the GUI with manual maintenance, the second equation comprising a second subset of the plurality of statistical variables; and

a third equation representing a third monetary cost of automated testing of the GUI with tool based maintenance, the third equation comprising a third subset of the plurality of statistical variables.

19. The article of manufacture of claim 15, where the statistical variables include any one or more of:

manual testing time (MTT), number of test engineers needed (NTEN), number of manual testers needed (NMT), test script composition time (TSCT), test script running time (TSRT), test script maintenance time (TSMT), and tool based maintenance speed up factor (MS), and the statistical constraint equations comprise any one or more of MTT, NTEN, NMT, TSCT, TSRT, and TSMT.

20. The article of manufacture of claim 19, where the statistical constraint equations include any one or more of:

(MTT×

NTEN)/(NMT×

TSCT);

(MTT×

NTEN)/(NMT×

TSRT);

(TSMT=TSCT×

87%×

(ER−

1); and

MS.