Predictive model scoring to optimize test case order in real time

• US 10,176,426 B2
• Filed: 07/07/2015
• Issued: 01/08/2019
• Est. Priority Date: 07/07/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Smith, Kevin B., Thompson, Andrew J., Waddling, David R.
• Primary Examiner(s): Leibovich, Yair
• Application Number:
  US14/792,682
• Publication Number:
  US 20170010959A1
• Time in Patent Office: 1,281 Days
• Field of Search: 714 26
• CPC Class Codes: G06F 11/261 by simulating additional hardware, e.g. fault simulation
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06F 16/156 Query results presentation
• CPC Class Codes: G06F 16/17 Details of further file system functions
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06N 99/00 Subject matter not provided for in other groups of this subclass

Abstract

An approach for predictively scoring test case results in real-time. Test case results associated with a test run are received by a software testing environment. Using predictive statistical models, test case results and attribute relationships are matched against model rules and test case history. A statistical correlation and confidence parameter provide the ability to generate test case relationships for predicting the outcome of other test cases in the test run. The test case relationships are transformed into scoring results and output for the further processing.

Claims

1. A computer-implemented method for predictively scoring test case results in real-time, the computer-implemented method comprising:

receiving, by one or more computer processors, one or more test case results associated with a test run;

determining, by the one or more computer processors, one or more test case relationships based on at least one of one or more predictive statistical models or the one or more predictive statistical models and test case history, wherein the one or more predictive statistical models are operational on the one or more test case results;

transforming, by the one or more computer processors, the one or more test case relationships into one or more scoring results based on a predetermined correlation criteria; and

outputting, by the one or more computer processors, the one or more scoring results for further processing, wherein the one or more scoring results comprise multi-value prioritized output comprising a prioritized plurality of test cases for execution and one or more of single-value output comprising a test case for execution, multi-value output comprising a plurality of test cases and statistical correlation and confidence ranking for execution.

2. The computer-implemented method of claim 1, wherein the one or more test case results comprise attributes associated with the one or more test case relationships.

3. The computer-implemented method of claim 1, wherein the one or more test case relationships are partially determined by one or more attributes associated with the test case history.

4. The computer-implemented method of claim 1, wherein determining comprises data mining historic test results to statistically organize related attributes based on the one or more predictive statistical models comprising a priori, regression, clustering, tree and neural network.

5. The computer-implemented method of claim 1, wherein predictively scoring test case results is based on predetermined association rules and the one or more predictive statistical models.

6. The computer-implemented method of claim 1, wherein the scoring results further comprise single value output.

7. The computer-implemented method of claim 6, wherein the single-value output is a test case for execution.

8. A computer program product for predictively scoring test case results in real-time, the computer program product comprising:

one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising;

program instructions to receive one or more test case results associated with a test run;

program instructions to determine one or more test case relationships based on at least one of one or more predictive statistical models or the one or more predictive statistical models and test case history, wherein the one or more predictive statistical models are operational on the one or more test case results;

program instructions to transform the one or more test case relationships into one or more scoring results based on a predetermined correlation criteria; and

program instructions to output the one or more scoring results for further processing, wherein the one or more scoring results comprise multi-value prioritized output comprising a prioritized plurality of test cases for execution and one or more of single-value output comprising a test case for execution, multi-value output comprising a plurality of test cases and statistical correlation and confidence ranking for execution.

9. The computer program product of claim 8, wherein the one or more test case results comprise attributes associated with the one or more test case relationships.

10. The computer program product of claim 8, wherein the one or more test case relationships are partially determined by one or more attributes associated with the test case history.

11. The computer program product of claim 8, wherein determining comprises data mining historic test results to statistically organize related attributes based on the one or more predictive statistical models comprising a priori, regression, clustering, tree and neural network.

12. The computer program product of claim 8, wherein predictively scoring test case results is based on predetermined association rules and the one or more predictive statistical models.

13. The computer program product of claim 8, wherein the scoring results further comprise single value output.

14. The computer program product of claim 13, wherein the single-value output is a test case for execution.

15. A computer system for predictively scoring test case results in real-time, the computer system comprising:

one or more computer processors;

one or more computer readable storage media;

program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising;

program instructions to receive one or more test case results associated with a test run;

program instructions to determine one or more test case relationships based on at least one of one or more predictive statistical models or the one or more predictive statistical models and test case history, wherein the one or more predictive statistical models are operational on the one or more test case results;

program instructions to transform the one or more test case relationships into one or more scoring results based on a predetermined correlation criteria; and

program instructions to output the one or more scoring results for further processing, wherein the one or more scoring results comprise multi-value prioritized output comprising a prioritized plurality of test cases for execution and one or more of single-value output comprising a test case for execution, multi-value output comprising a plurality of test cases and statistical correlation and confidence ranking for execution.

16. The computer system of claim 15, wherein the one or more test case relationships are partially determined by one or more attributes associated with the test case history.

17. The computer system of claim 15, wherein determining comprises data mining historic test results to statistically organize related attributes based on the one or more predictive statistical models comprising a priori, regression, clustering, tree and neural network.

18. The computer system of claim 15, wherein predictively scoring test case results is based on predetermined association rules and the one or more predictive statistical models.

19. The computer system of claim 15, wherein the scoring results further comprise single value output.

20. The computer system of claim 19, wherein the single-value output is a test case for execution.