METHOD FOR TESTING OPERATION OF SOFTWARE

• US 20120304157A1
• Filed: 04/16/2012
• Published: 11/29/2012
• Est. Priority Date: 05/23/2011
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Yanagisawa, Takashi, Kawashima, Ken
• Granted Patent: US 8,745,588 B2
• Time in Patent Office: Days
• US Class Current: 717/131
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: G06F 11/3692 for test results analysis

Abstract

A method for processing test results from testing operation of software. A test result of pass, fail status, or unperformed is received for each test case of a test performed for each release of the software. A group to which each test belongs is ascertained. A test result stability index is calculated for each test case as being proportional to a total number of consecutive releases that include and are prior to the last release of the software such that the test result for each of the consecutive releases denotes a pass. A module stability index is calculated for each test case as being a summation over a product of a weight at each release and a binary stability indicator of 0 at each release for which the test result is changed from that of the immediately prior release and 1 at every other release.

Claims

1. A method for processing test results from testing operation of software, said method comprising:

receiving, by a processor of a computer system, a test result for each test case of multiple test cases for a test performed for each release of multiple releases of the software, said test result for each test for each release being a first character string denoting a pass status for the test, a second character string denoting a fail status for the test, or a third character string denoting an unperformed status for the test indicating that the test has not been performed;

said processor ascertaining a group to which each test belongs, said ascertaining comprising determining a group identifier of the group to which each test belongs, said group identifier for each test case being a group character string determined from the test results of the tests performed for all releases of the multiple releases for said each test case, said ascertained groups consisting of multiple groups, each group of at least one group of the multiple groups comprising at least two test cases of the multiple test cases;

said processor calculating a test result stability index for each test case as being proportional to a total number of consecutive releases that include and are prior to the last release of the multiple releases of the software such that the test result for each release of the consecutive releases is the first character string denoting the pass status;

said processor calculating a module stability index for each test case as being a summation over a product of a weight at each release and a binary stability indicator of 0 at each release for which the test result is changed from the test result at the immediately prior release and 1 at every other release; and

for each test case, said processor storing the group identifier, the test result stability index, and the module stability index in a hardware storage unit of the computer system.

2. The method of claim 1, wherein the group character string pertaining to the group identifier of each group is a set of single characters sequenced in a sequential order in which the multiple releases of the software are sequenced in a time ordering of the releases, and wherein each single character in the set of single characters has been mapped from the first, second, or third character string that denotes the pass status, the fail status, or the unperformed status, respectively, of the test result for the release of the multiple releases corresponding to a position of the release in the time ordering of the releases.

3. The method of claim 1, wherein the method comprises:

said processor computing the test result stability index for each release (R) of each test case as being equal to T_R/N_R, wherein T_R is a total number of consecutive releases including and prior to said each release (R) for which the test result is the first character string denoting the pass status, wherein N_R is a total number of releases including and prior to said each release (R) starting with the first release of the multiple releases, and wherein said calculating the test result stability index for each test case consists of said computing the test result stability index for the last release of the multiple releases for said each test case.

4. The method of claim 1, wherein the weight at each release after the first release exceeds the weight at each other release that is prior to said each release.

5. The method of claim 1, wherein the method further comprises:

for each group of the multiple groups, said processor determining a group test result stability index from the test result stability indexes of the test cases belonging to said each group;

for each group of the multiple groups, said processor determining a group module stability index from the test result stability indexes of the test cases belonging to said each group;

said processor selecting a plurality of test cases from the multiple test cases such that the plurality of test cases consist of fewer test cases than the multiple test cases, said selecting the plurality of test cases being based on both the determined group test result stability index and the determined group module stability index of all groups of the multiple groups.

6. The method of claim 1, wherein said selecting the plurality of test cases comprises:

said processor receiving a specification of both a group test result stability index threshold and a group module stability index threshold;

for each group, said processor determining whether a condition is satisfied, said condition selected from the group consisting of the group test result stability index not exceeding the specified test result stability index threshold, the group module stability index not exceeding the specified group module stability index threshold, and a combination thereof;

for each group for which the condition is determined to be satisfied, said processor selecting only a single test case from said each group to be included in the plurality of test cases, wherein the selected single test case has a shortest time period of the test cases in the group;

for each group for which the condition is determined to not be satisfied, said processor selecting all test cases from said each group to be included in the plurality of test cases.

7. The method of claim 6, wherein the condition is determined to be satisfied for one or more groups of the multiple groups.

8. The method of claim 6, wherein the condition is determined to not be satisfied for not less than one group of the multiple groups.