Adaptive Framework Automatically Prioritizing Software Test Cases

• US 20160162392A1
• Filed: 12/09/2014
• Published: 06/09/2016
• Est. Priority Date: 12/09/2014
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Hu, Ziheng, Chang, Longhui, Li, Jing, Li, Xin, Zhou, Chen
• Granted Patent: US 9,489,289 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

An automated, self-adaptive framework prioritizes software testing in a consistent and effective manner. A metric evaluates past test execution information for assigning regression testing priority. The metric may be calculated with reference to one or more of the following factors taken in combination: requirement, coverage, history, and cost. The requirement factor considers customer-assigned priority of testing the code, complexity of implementing the code, and proneness of the code to faults. The coverage factor considers code coverage, feature coverage, and common usage rate. The history factor considers previous bug found rate, case stable rate, and priority to calculate. The cost factor considers test case execution time, and step length. A value of each factor for one test case is measured according to that test case and is not related to other test cases. The calculation result representing the metric for each test case determines a priority of the test case.

Claims

1. A computer-implemented method comprising:

an engine receiving a user input and information about a software test case from a database;

the engine processing the information to calculate in a first testing round, a first metric for the software test case;

the engine assigning a testing priority to the software test case for the first testing round based upon the first metric; and

the engine communicating the testing priority to the database for storage.

2. A method as in claim 1 further comprising:

the engine receiving another user input and updated information about the software test case from the database;

the engine processing the updated information to calculate in a second testing round, a second metric for the software test case;

the engine assigning an adaptive testing priority to the software test case for the second testing round based on the second metric; and

the engine communicating the adaptive testing priority to the database for storage.

3. A method as in claim 2 wherein:

the engine receives the testing priority from the database; and

the engine calculates the second metric based upon the testing priority and the updated information.

4. A method as in claim 2 wherein the updated information comprises at least one of updated history information, updated coverage information, and updated cost information.

5. A method as in claim 1 wherein the information comprises requirement information including at least one of a customer-assigned priority value, an implementation complexity value, and a fault proneness value.

6. A method as in claim 1 wherein the information comprises coverage information including at least one of a code coverage rate, and a feature coverage rate.

7. A method as in claim 1 wherein the information comprises history information including at least one of a case stable rate and a bug found rate.

8. A method as in claim 1 wherein the information comprises cost information including at least one of a step length rate and a test case execution time rate.

9. A method as in claim 1 further comprising the engine communicating the first metric to the database for storage.

10. A non-transitory computer readable storage medium embodying a computer program for performing a method, said method comprising:

an engine receiving a user input and information about a software test case from a database, the information comprising at least one of requirement information, coverage information, history information, and cost information;

the engine processing the information to calculate in a first testing round, a first metric for the software test case;

the engine assigning a testing priority to the software test case for the first testing round based upon the first metric; and

the engine communicating the testing priority to the database for storage.

11. A non-transitory computer readable storage medium as in claim 10 wherein the method further comprises:

the engine receiving another user input and updated information about the software test case from the database;

the engine processing the updated information to calculate in a second testing round, a second metric for the software test case;

the engine assigning an adaptive testing priority to the software test case for the second testing round based on the second metric; and

the engine communicating the adaptive testing priority to the database for storage.

12. A non-transitory computer readable storage medium as in claim 11 wherein:

the engine receives the testing priority from the database; and

the engine calculates the second metric based upon the testing priority and the updated information.

13. A non-transitory computer readable storage medium as in claim 12 wherein the updated information comprises updated history information, updated coverage information, or updated cost information.

14. A non-transitory computer readable storage medium as in claim 10 wherein the information comprises at least one of:

requirement information including at least one of a customer-assigned priority value, an implementation complexity value, and a fault proneness value;

coverage information including at least one of a code coverage rate, and a feature coverage rate;

history information including at least one of a case stable rate and a bug found rate; and

cost information including at least one of a step length rate and a test case execution time rate.

15. A non-transitory computer readable storage medium as in claim 10 wherein the method further comprises the engine communicating the first metric to the database for storage.

16. A computer system comprising:

one or more processors;

a software program, executable on said computer system, the software program configured to;

cause an engine to receive a user input and information about a software test case from a database;

cause the engine to process the information to calculate in a first testing round, a first metric for the software test case;

cause the engine to assign a testing priority to the software test case for the first testing round based upon the first metric;

cause the engine to communicate the testing priority to the database for storage;

cause the engine to receive another user input and updated information about the software test case from the database;

cause the engine to process the updated information to calculate in a second testing round, a second metric for the software test case;

cause the engine to assign an adaptive testing priority to the software test case for the second testing round based on the second metric; and

cause the engine to communicate the adaptive testing priority to the database for storage.

17. A computer system as in claim 16 wherein:

the engine receives the testing priority from the database; and

the engine calculates the second metric based upon the testing priority and the updated information.

18. A computer system as in claim 17 wherein the updated information comprises at least one of updated coverage information, updated history information, and updated cost information.

19. A computer system as in claim 16 wherein the information comprises at least one of:

requirement information including at least one of a customer-assigned priority value, an implementation complexity value, and a fault proneness value;

coverage information including at least one of a code coverage rate, and a feature coverage rate;

history information including at least one of a case stable rate and a bug found rate; and

cost information including at least one of a step length rate and a test case execution time rate.

20. A computer system as in claim 16 wherein the software program further comprises code configured to cause the engine to communicate the first metric to the database for storage.