Using test workload run facts and problem discovery data as input for business analytics to determine test effectiveness

• US 10,540,265 B2
• Filed: 06/30/2016
• Issued: 01/21/2020
• Est. Priority Date: 06/30/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Conti, Thomas W., Gildein, II, Michael E., Moser, Kyle R.
• Primary Examiner(s): Leibovich, Yair
• Application Number:
  US15/197,831
• Publication Number:
  US 20180004641A1
• Time in Patent Office: 1,300 Days
• Field of Search: 714 381
• CPC Class Codes: G06F 11/36 Preventing errors by testing or debugging software
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis

Abstract

Aspects of the present invention include a method, system and computer program product for utilizing various data to determine the effectiveness of a software testing procedure. The method includes preloading, by a processor, into a database data related to workloads and workload data, data related to analysis points, and customer data. The method also includes determining, by the processor, whether to modify a workload model, to run a workload or to perform one of post processing analytics or run time analytics. The method further includes performing, by the processor, one of post processing analytics or run time analytics of the data preloaded into the database.

Claims

1. A system for improved software testing comprising:

a processor in communication with one or more types of memory, the processor configured to;

preload into a database data related to workloads and workload data, data related to analysis points, and customer data;

determine whether to modify a workload model, to run a workload or to perform one of post processing analytics or run time analytics;

responsive to running a workload, collect workload run facts comprising a workload configuration or model information, workload feedback data, and system feedback data, wherein the workload feedback data comprises one or more of a transaction mix, number of simulated users, a transaction rate, failure rate per transaction and a number and distribution of errors and failures;

perform one of post processing analytics or run time analytics of the data preloaded into the database; and

provide visual workload feedback including rates and errors, or visual system feedback with customer data convergence visualization, or system alerts for out of convergence issues, via a display.

2. The system of claim 1 wherein the processor configured to determine whether to modify a workload model, to run a workload or to perform one of post processing analytics or run time analytics comprises the processor configured to modify a workload model.

3. The system of claim 1 wherein the processor configured to determine whether to modify a workload model, to run a workload or to perform one of post processing analytics or run time analytics comprises the processor configured to run a workload.

4. The system of claim 3 wherein the processor configured to run a workload comprises the processor configured to collect and store workload data and system runfacts data.

5. The system of claim 4 further comprising the processor configured to perform one of post processing analytics or run time analytics of the data preloaded into the database and of the stored workload data and the system runfacts data.

6. The system of claim 5 wherein the processor configured to perform one of post processing analytics or run time analytics further comprises the processor configured to generate business analytics reports.

7. The system of claim 4 wherein the further comprising performing, by the processor, one of post processing analytics or run time analytics of test defects data.

8. A computer program product for improved software testing comprising:

a non-transitory storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method comprising;

preloading, by a processor, into a database data related to workloads and workload data, data related to analysis points, and customer data;

determining, by the processor, whether to modify a workload model, to run a workload or to perform one of post processing analytics or run time analytics;

responsive to running a workload, collecting workload run facts comprising a workload configuration or model information, workload feedback data, and system feedback data, wherein the workload feedback data comprises one or more of a transaction mix, number of simulated users, a transaction rate, failure rate per transaction and a number and distribution of errors and failures;

performing, by the processor, one of post processing analytics or run time analytics of the data preloaded into the database; and

providing visual workload feedback including rates and errors, or visual system feedback with customer data convergence visualization, or system alerts for out of convergence issues, via a display.

9. The computer program product of claim 8 wherein determining, by the processor, whether to modify a workload model, to run a workload or to perform one of post processing analytics or run time analytics further comprises the processor modifying a workload model.

10. The computer program product of claim 8 wherein determining, by the processor, whether to modify a workload model, to run a workload or to perform one of post processing analytics or run time analytics comprises the processor running a workload.

11. The computer program product of claim 10 wherein the processor running a workload comprises the processor collecting and storing workload data and system runfacts data.

12. The computer program product of claim 11 further comprising performing, by the processor, one of post processing analytics or run time analytics of the data preloaded into the database and of the stored workload data and the system runfacts data.

13. The computer program product of claim 12 wherein the processor performing one of post processing analytics or run time analytics further comprises the processor generating business analytics reports.