Test selection

• US 8,850,270 B2
• Filed: 07/19/2012
• Issued: 09/30/2014
• Est. Priority Date: 07/19/2012
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Heiper, Andre, Kupershtok, Aharon, Yaari, Yaakov
• Primary Examiner(s): LE, DIEU MINH T
• Application Number:
  US13/552,665
• Publication Number:
  US 20140025997A1
• Time in Patent Office: 803 Days
• Field of Search: 714/38.1, 714/37, 717/124, 717/126
• US Class Current: 714/38.1
• CPC Class Codes: G06F 11/3676 for coverage analysis

Abstract

Computer-implemented method, computerized apparatus and a computer program product for test selection. The computer-implemented method comprising: obtaining a test suite comprising a plurality of tests for a Software Under Test (SUT); and selecting a subset of the test suite, wherein the subset provides coverage of the SUT that correlates to a coverage by a workload of the SUT, wherein the workload defines a set of input events to the SUT thereby defining portions of the SUT that are to be invoked during execution.

Claims

1. A computer-implemented method, the method performed by a processor, the method comprising:

obtaining a test suite comprising a plurality of tests for a Software Under Test (SUT);

selecting a subset of the test suite, wherein the subset provides coverage of the SUT that correlates to a coverage by a workload of the SUT, wherein the workload defines a set of input events to the SUT thereby defining portions of the SUT that are to be invoked during execution;

determining an execution profile of the workload;

obtaining execution profiles of each test of the test suite; and

wherein said selecting comprises;

determining a similarity measurement between substantially each test and the workload based on the execution profiles thereof;

selecting the subset of the test suite based on the similarity measurements; and

wherein said selecting comprises selecting the subset that is associated with a most similar aggregated similarity measurement to that of the workload, wherein an aggregated similarity measurement measures similarity between a set of tests and the workload based on an aggregated execution profile of the set of test and the execution profile of the workload.

2. The computer-implemented method of claim 1, wherein said selecting the subset based on the similarity measurement comprises selecting most similar tests to the workload based on the similarity measurement.

3. The computer-implemented method of claim 1, wherein the execution profile is a mapping between program code locations in the SUT and counters of events measured during execution of the program code locations.

4. The computer-implemented method of claim 3, wherein the counters are normalized, thereby the execution profile provides information relating to a distribution of events with respect to program code locations.

5. The computer-implemented method of claim 3, wherein the measured events are selected from a group consisting of:

an execution of the program code;

a branch taken event with respect to a conditional branch statement;

a cache miss event; and

a lock or unlock event.

6. The computer-implemented method of claim 1, wherein said determining the execution profile of the workload comprises:

executing the SUT with respect to the workload and measuring using hardware performance monitors events triggered by the processor during execution of the SUT.

7. A computer-implemented method, the method performed by a processor, the method comprising:

obtaining a test suite comprising a plurality of tests for a Software Under Test (SUT);

selecting a subset of the test suite, wherein the subset provides coverage of the SUT that correlates to a coverage by a workload of the SUT, wherein the workload defines a set of input events to the SUT thereby defining portions of the SUT that are to be invoked during execution;

determining an execution profile of the workload;

obtaining execution profiles of each test of the test suite;

wherein said selecting comprises;

determining a similarity measurement between substantially each test and the workload based on the execution profiles thereof; and

selecting the subset of the test suite based on the similarity measurements; and

wherein said selecting comprises;

having a set of tests, wherein the set is initially empty;

performing repeatedly;

determining which of the tests in the test suite that is not comprised by the set, if added to the set, would induce a highest similarity measurement between an aggregated execution profile of the set and the execution profile of the workload; and

adding the determined test.

8. The computer-implemented method of claim 7, wherein said performing repeatedly is performed until a stop condition is met, wherein the stop condition met in case the similarity measurement is above a predetermined level or in case a size of the set reaches a predetermined threshold.

9. The computer-implemented method of claim 7, wherein the execution profile is a mapping between program code locations in the SUT and counters of events measured during execution of the program code locations.

10. The computer-implemented method of claim 9, wherein the counters are normalized, thereby the execution profile provides information relating to a distribution of events with respect to program code locations.

11. The computer-implemented method of claim 9, wherein the measured events are selected from a group consisting of:

an execution of the program code;

a branch taken event with respect to a conditional branch statement;

a cache miss event; and

a lock or unlock event.

12. The computer-implemented method of claim 7, wherein said determining the execution profile of the workload comprises:

executing the SUT with respect to the workload and measuring using hardware performance monitors events triggered by the processor during execution of the SUT.

13. A computerized apparatus having a processor and a memory, the processor being adapted to perform the steps of:

obtaining a test suite comprising a plurality of tests for a Software Under Test (SUT);

selecting a subset of the test suite, wherein the subset provides coverage of the SUT that correlates to a coverage by a workload of the SUT, wherein the workload defines a set of input events to the SUT thereby defining portions of the SUT that are to be invoked during execution;

determining an execution profile of the workload;

obtaining execution profiles of each test of the test suite; and

wherein said selecting comprises;

determining a similarity measurement between substantially each test and the workload based on the execution profiles thereof; and

selecting the subset of the test suite based on the similarity measurements;

wherein said selecting comprises selecting the subset that is associated with a most similar aggregated similarity measurement to that of the workload, wherein an aggregated similarity measurement measures similarity between a set of tests and the workload based on an aggregated execution profile of the set of test and the execution profile of the workload.

14. The apparatus of claim 13, wherein said selecting the subset based on the similarity measurement comprises selecting most similar tests to the workload based on the similarity measurement.

15. The apparatus of claim 13, wherein the execution profile is a mapping between program code locations in the SUT and counters of events measured during execution of the program code locations.

16. The apparatus of claim 13, wherein said determining the execution profile of the workload comprises:

executing the SUT with respect to the workload and measuring using hardware performance monitors events triggered by the processor during execution of the SUT.

17. The apparatus of claim 13, wherein said selecting comprises:

having a set of tests, wherein the set is initially empty;

performing repeatedly;

determining which of the tests in the test suite that is not comprised by the set, if added to the set, would induce a highest similarity measurement between an aggregated execution profile of the set and the execution profile of the workload; and

adding the determined test.

18. The apparatus of claim 17, wherein said performing repeatedly is performed until a stop condition is met, wherein the stop condition met in case the similarity measurement is above a predetermined level or in case a size of the set reaches a predetermined threshold.