MEASURING AND OPTIMIZING TEST RESOURCES AND TEST COVERAGE EFFECTIVENESS THROUGH RUN TIME CUSTOMER PROFILING AND ANALYTICS

• US 20180067844A1
• Filed: 02/10/2017
• Published: 03/08/2018
• Est. Priority Date: 09/08/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Conti, Thomas W., Moser, Kyle R.
• Granted Patent: US 10,467,129 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: G06F 11/3676 for coverage analysis
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

Aspects of the present invention include a method, system and computer program product. The method includes a processor performing an accounting of available test resources within one or more organizations; storing data relating to the accounting of available test resources; storing data relating to one or more test resource goals; determining to maintain the data relating to the accounting of available test resources; determining to analyze the data relating to the accounting of available test resources; analyzing the data relating to the accounting of available test resources with respect to the data relating to one or more test resource goals; and determining from the analyzing, by the processor, the data relating to the accounting of available test resources with respect to the data relating to one or more test resource goals that the available test resources within one or more organizations are either optimal or non-optimal.

Claims

1. A computer-implemented method comprising:

performing, by a processor, an accounting of available test resources within one or more organizations;

storing, by the processor, data relating to the accounting of available test resources;

storing, by the processor, data relating to one or more test resource goals;

determining, by the processor, to maintain the data relating to the accounting of available test resources;

determining, by the processor, to analyze the data relating to the accounting of available test resources;

analyzing, by the processor, the data relating to the accounting of available test resources with respect to the data relating to one or more test resource goals; and

determining, by the processor, from the analyzing, by the processor, the data relating to the accounting of available test resources with respect to the data relating to one or more test resource goals that the available test resources within one or more organizations are either optimal or non-optimal.

2. The computer-implemented method of claim 1 wherein the available test resources within one or more organizations comprise available test resources across one or more of each of test organizations, test environments and an enterprise.

3. The computer-implemented method of claim 1 wherein the data relating to one or more test resource goals is selected from the group consisting of enterprise, organizations, test environments, test workloads, test cases, test scenarios, test skills, test planners, test architects, testers, business partners, OEMs, open source foundations, other related factors and metrics that are resources expenses, and limitations or enablement'"'"'s associated with business geography, cultural sensitivities and legal obligations, and time zone contradictions.

4. The computer-implemented method of claim 1 wherein the available test resources comprise human resources and physical resources.

5. The computer-implemented method of claim 1 wherein analyzing, by the processor, the data relating to the accounting of available test resources with respect to the data relating to one or more test resource goals comprises determining via historical and run time customer profiling and analytics a score and ranking of use of resources using one or more factors or metrics.

6. The computer-implemented method of claim 5 wherein the one or more factors or metrics is selected from the group consisting of:

test problems found per resource cost using data from resources data stores along with test execution workload run facts data;

customer detected problems;

environment or configuration optimization;

workload optimization comprising functional coverage, stress and activity levels, and workload characteristics'"'"' attainment;

operational optimization;

duplication of test effort within and across test organizations;

test plan coverage efficiency;

problem and/or critical situation recreate capabilities; and

factors or metrics impacting test resource return on investment.

7. The computer-implemented method of claim 1 wherein determining, by the processor, that the available test resources within one or more organizations are non-optimal comprises changing, by the processor, at least one of the one or more test resource goals to reallocate resources within the one or more organizations to improve productivity.