DYNAMIC REPRIORITIZATION OF TEST CASES DURING TEST EXECUTION

• US 20130159774A1
• Filed: 12/19/2011
• Published: 06/20/2013
• Est. Priority Date: 12/19/2011
• Status: Active Grant

• Current Assignee: Siemens AG
• Original Assignee: Siemens Corp. (Siemens AG)
• Inventors: Budnik, Christof, Subramanyan, Rajesh
• Granted Patent: US 8,527,813 B2
• Time in Patent Office: Days
• US Class Current: 714/33
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

Systems and methods are described that dynamically reprioritize test cases for Model-Based Testing (MBT) during test execution. Test case execution is prioritized according to their potential to detect uncovered failures within a design model.

Claims

1. A method for dynamically reprioritizing unexecuted test cases in a test case set for a Model-Based Testing (MBT) System Under Test (SUT) design model during test case execution comprising:

inputting a design model representing the SUT as an event-flow;

analyzing the design model structure in terms of possible event-flows considering the design model'"'"'s control and data elements;

computing test complexity for the coverage of the event-flows and control and data elements;

selecting test case generation criteria for a desired test objective;

generating a plurality of test cases based on the test case selection criteria wherein each test case is an event sequence;

computing test case capability for the coverage of event-flows and data wherein the more data and flows each test case tests, the greater their capability is to detect faults;

weighting identified test data towards their coverage for selection;

prioritizing the test cases based on the test selection criteria and the weighting of test data;

executing the test cases in their priority against the model;

detecting if a test case failed during execution;

calculating a distance between a failed test case and each remaining test case to be executed; and

based on the distance calculation for each unexecuted test case, executing the test case having the greatest distance.

2. The method according to claim 1 wherein the design model is an XML file.

3. The method according to claim 1 wherein the design model is an activity diagram.

4. The method according to claim 1 wherein test complexity is the complexity to test the model.

5. The method according to claim 4 wherein the test complexity is based on the number of identified flows plus the test data within the design model for the coverage of test data within the flows, and further comprises applying various coverage criteria to decide the test case generation criteria to be selected.

6. The method according to claim 5 wherein the test data represents a set of variations describing under which conditions the design model is tested.

7. The method according to claim 1 wherein calculating the distance between a failed test case and each remaining test case to be executed further comprises:

parsing the failed test case event sequence length k into a plurality of event sequences having an event sequence length seqlen from 2 to the failed test case event sequence length k, wherein the parsed event sequences comprise the failed test case event sequences in their original order;

comparing each failed test case parsed event sequence with each remaining unexecuted test case event sequence;

identifying if a failed test case parsed event sequence is present in any of the unexecuted test cases; and

for each unexecuted test case, summing the number of failed test case parsed event sequence matches as a distance.

8. The method according to claim 7 wherein executing the test case having the greatest distance comprises executing the unexecuted test case that has the smallest number of failed test case parsed event sequence matches.

9. A system for dynamically reprioritizing unexecuted test cases in a test case set for a Model-Based Testing (MBT) System Under Test (SUT) design model during test case execution comprising:

means for inputting a design model representing the SUT as an event-flow;

means for analyzing the design model structure in terms of possible event-flows considering the design model'"'"'s control and data elements;

means for computing test complexity for the coverage of the event-flows and control and data elements;

means for selecting test case generation criteria for a desired test objective;

means for generating a plurality of test cases based on the test case selection criteria wherein each test case is an event sequence;

means for computing test case capability for the coverage of event-flows and data wherein the more data and flows each test case tests, the greater their capability is to detect faults;

means for weighting identified test data towards their coverage for selection;

means for prioritizing the test cases based on the test selection criteria and the weighting of test data;

means for executing the test cases in their priority against the model;

means for detecting if a test case failed during execution;

means for calculating a distance between a failed test case and each remaining test case to be executed; and

based on the distance calculation for each unexecuted test case, means for executing the test case having the greatest distance.

10. The system according to claim 9 wherein the design model is an XML file.

11. The system according to claim 9 wherein the design model is an activity diagram.

12. The system according to claim 9 wherein test complexity is the complexity to test the model.

13. The system according to claim 12 wherein the test complexity is based on the number of identified flows plus the test data within the design model for the coverage of test data within the flows, and further comprises applying various coverage criteria to decide the test case generation criteria to be selected.

14. The system according to claim 13 wherein the test data represents a set of variations describing under which conditions the design model is tested.

15. The system according to claim 9 wherein means for calculating the distance between a failed test case and each remaining test case to be executed further comprises:

means for parsing the failed test case event sequence length k into a plurality of event sequences having an event sequence length seqlen from 2 to the failed test case event sequence length k, wherein the parsed event sequences comprise the failed test case event sequences in their original order;

means for comparing each failed test case parsed event sequence with each remaining unexecuted test case event sequence;

means for identifying if a failed test case parsed event sequence is present in any of the unexecuted test cases; and

for each unexecuted test case, means for summing the number of failed test case parsed event sequence matches as a distance.

16. The system according to claim 15 wherein means for executing the test case having the greatest distance comprises means for executing the unexecuted test case that has the smallest number of failed test case parsed event sequence matches.