Test Selection

• US 20140025997A1
• Filed: 07/19/2012
• Published: 01/23/2014
• Est. Priority Date: 07/19/2012
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kupershtok, Aharon, Yaari, Yaakov, Heiper, Andre
• Granted Patent: US 8,850,270 B2
• Time in Patent Office: Days
• US Class Current: 714/38.1
• CPC Class Codes: G06F 11/3676 for coverage analysis

Abstract

Computer-implemented method, computerized apparatus and a computer program product for test selection. The computer-implemented method comprising: obtaining a test suite comprising a plurality of tests for a Software Under Test (SUT); and selecting a subset of the test suite, wherein the subset provides coverage of the SUT that correlates to a coverage by a workload of the SUT, wherein the workload defines a set of input events to the SUT thereby defining portions of the SUT that are to be invoked during execution.

Claims

1. A computer-implemented method, the method performed by a processor, the method comprising:

obtaining a test suite comprising a plurality of tests for a Software Under Test (SUT); and

selecting a subset of the test suite, wherein the subset provides coverage of the SUT that correlates to a coverage by a workload of the SUT, wherein the workload defines a set of input events to the SUT thereby defining portions of the SUT that are to be invoked during execution.

2. The computer-implemented method of claim 1, further comprises:

determining an execution profile of the workload;

obtaining execution profiles of each test of the test suite; and

wherein said selecting comprises;

determining a similarity measurement between substantially each test and the workload based on the execution profiles thereof; and

selecting the subset of the test suite based on the similarity measurements.

3. The computer-implemented method of claim 2, wherein said selecting the subset based on the similarity measurement comprises selecting most similar tests to the workload based on the similarity measurement.

4. The computer-implemented method of claim 2, wherein said selecting the subset based on the similarity measurement comprises selecting the subset that is associated with a most similar aggregated similarity measurement to that of the workload, wherein an aggregated similarity measurement measures similarity between a set of tests and the workload based on an aggregated execution profile of the set of test and the execution profile of the workload.

5. The computer-implemented method of claim 2, wherein the execution profile is a mapping between program code locations in the SUT and counters of events measured during execution of the program code locations.

6. The computer-implemented method of claim 5, wherein the counters are normalized, thereby the execution profile provides information relating to a distribution of events with respect to program code locations.

7. The computer-implemented method of claim 5, wherein the measured events are selected from a group consisting of:

an execution of the program code;

a branch taken event with respect to a conditional branch statement;

a cache miss event; and

a lock or unlock event.

8. The computer-implemented method of claim 2, wherein said determining the execution profile of the workload comprises:

executing the SUT with respect to the workload and measuring using hardware performance monitors events triggered by the processor during execution of the SUT.

9. The computer-implemented method of claim 2, wherein said selecting comprises:

is having a set of tests, wherein the set is initially empty;

performing repeatedly;

determining which of the tests in the test suite that is not comprised by the set, if added to the set, would induce a highest similarity measurement between an aggregated execution profile of the set and the execution profile of the workload; and

adding the determined test.

10. The computer-implemented method of claim 9, wherein said performing repeatedly is performed until a stop condition is met, wherein the stop condition met in case the similarity measurement is above a predetermined level or in case a size of the set reaches a predetermined threshold.

11. A computerized apparatus having a processor and a memory, the processor being adapted to perform the steps of:

obtaining a test suite comprising a plurality of tests for a Software Under Test (SUT); and

selecting a subset of the test suite, wherein the subset provides coverage of the SUT that correlates to a coverage by a workload of the SUT, wherein the workload defines a set of input events to the SUT thereby defining portions of the SUT that are to be invoked during execution.

12. The apparatus of claim 11, wherein the processor being further adapted to perform:

determining an execution profile of the workload;

obtaining execution profiles of each test of the test suite; and

wherein said selecting comprises;

determining a similarity measurement between substantially each test and the workload based on the execution profiles thereof; and

selecting the subset of the test suite based on the similarity measurements.

13. The apparatus of claim 12, wherein said selecting the subset based on the similarity measurement comprises selecting most similar tests to the workload based on the similarity measurement.

14. The apparatus of claim 12, wherein said selecting the subset based on the similarity measurement comprises selecting the subset that is associated with a most similar aggregated similarity measurement to that of the workload, wherein an aggregated similarity measurement measures similarity between a set of tests and the workload based on an aggregated execution profile of the set of test and the execution profile of the workload.

15. The apparatus of claim 12, wherein the execution profile is a mapping between program code locations in the SUT and counters of events measured during execution of the program code locations.

16. The apparatus of claim 12, wherein said determining the execution profile of the workload comprises:

executing the SUT with respect to the workload and measuring using hardware performance monitors events triggered by the processor during execution of the SUT.

17. The apparatus of claim 12, wherein said selecting comprises:

having a set of tests, wherein the set is initially empty;

performing repeatedly;

determining which of the tests in the test suite that is not comprised by the set, if added to the set, would induce a highest similarity measurement between an aggregated execution profile of the set and the execution profile of the workload; and

adding the determined test.

18. The apparatus of claim 17, wherein said performing repeatedly is performed until a stop condition is met, wherein the stop condition met in case the similarity measurement is above a predetermined level or in case a size of the set reaches a predetermined threshold.

19. A computer program product comprising a non-transitory computer readable medium retaining program instructions, which instructions when read by a processor, cause the processor to perform:

obtaining a test suite comprising a plurality of tests for a Software Under Test (SUT); and

selecting a subset of the test suite, wherein the subset provides coverage of the SUT that correlates to a coverage by a workload of the SUT, wherein the workload defines a set of input events to the SUT thereby defining portions of the SUT that are to be invoked during execution.

20. The computer program product of claim 19, wherein said instructions, when read by the processor, cause the processor to perform:

determining an execution profile of the workload;

obtaining execution profiles of each test of the test suite; and

wherein said selecting comprises;

determining a similarity measurement between substantially each test and the workload based on the execution profiles thereof; and

selecting the subset of the test suite based on the similarity measurements.