Measuring and optimizing test resources and test coverage effectiveness through run time customer profiling and analytics

• US 10,467,129 B2
• Filed: 02/10/2017
• Issued: 11/05/2019
• Est. Priority Date: 09/08/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Conti, Thomas W., Moser, Kyle R.
• Primary Examiner(s): Guyton, Philip
• Application Number:
  US15/429,262
• Publication Number:
  US 20180067844A1
• Time in Patent Office: 998 Days
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: G06F 11/3676 for coverage analysis
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

Aspects of the present invention include a method, system and computer program product. The method includes a processor performing an accounting of available test resources within one or more organizations; storing data relating to the accounting of available test resources; storing data relating to one or more test resource goals; determining to maintain the data relating to the accounting of available test resources; determining to analyze the data relating to the accounting of available test resources; analyzing the data relating to the accounting of available test resources with respect to the data relating to one or more test resource goals; and determining from the analyzing, by the processor, the data relating to the accounting of available test resources with respect to the data relating to one or more test resource goals that the available test resources within one or more organizations are either optimal or non-optimal.

Claims

1. A computer-implemented method comprising:

performing, by a processor, an accounting of available test resources within one or more organizations;

storing, by the processor, data relating to the accounting of available test resources;

storing, by the processor, data relating to one or more test resource goals;

determining, by the processor, to analyze the data relating to the accounting of available test resources;

analyzing, by the processor, the data relating to the accounting of available test resources with respect to the data relating to one or more test resource goals;

determining, by the processor, from the analyzing by the processor, the data relating to the accounting of available test resources with respect to the data relating to one or more test resource goals that the available test resources within one or more organizations are either optimal or non-optimal;

responsive to determining ineffective resource utilization, changing the resource utilization goals to reallocate resources to more productive organizations or environments, wherein reallocating resources to more productive organizations or environments comprises providing de-duplication of test efforts; and

providing, by the processor, a visualization of available test resources and test resource deployment recommendations.

2. The computer-implemented method of claim 1 wherein the available test resources within one or more organizations comprise available test resources across one or more of each of test organizations, test environments and an enterprise.

3. The computer-implemented method of claim 1 wherein the data relating to one or more test resource goals is selected from the group consisting of enterprise, organizations, test environments, test workloads, test cases, test scenarios, test skills, test planners, test architects, testers, business partners, OEMs, open source foundations, other related factors and metrics that are resources expenses, and limitations or enablement'"'"'s associated with business geography, cultural sensitivities and legal obligations, and time zone contradictions.

4. The computer-implemented method of claim 1 wherein the available test resources comprise human resources and physical resources.

5. The computer-implemented method of claim 1 wherein analyzing, by the processor, the data relating to the accounting of available test resources with respect to the data relating to one or more test resource goals comprises determining via historical and run time customer profiling and analytics a score and ranking of use of resources using one or more factors or metrics.

6. The computer-implemented method of claim 5 wherein the one or more factors or metrics is selected from the group consisting of:

test problems found per resource cost using data from resources data stores along with test execution workload run facts data;

customer detected problems;

environment or configuration optimization;

workload optimization comprising functional coverage, stress and activity levels, and workload characteristics'"'"' attainment;

operational optimization;

duplication of test effort within and across test organizations;

test plan coverage efficiency;

problem and/or critical situation recreate capabilities; and

factors or metrics impacting test resource return on investment.

7. The computer-implemented method of claim 1, wherein the visualization shows the available test resources and coverage ranked and scored according to a plurality of customizable factors.