Managing a set of tests based on other test failures

• US 10,452,508 B2
• Filed: 06/15/2015
• Issued: 10/22/2019
• Est. Priority Date: 06/15/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Maple, Elizabeth J., Pringle, Alexander R., Smith, Kevin B., Waddling, David R.
• Primary Examiner(s): Tran, Michael T
• Application Number:
  US14/739,009
• Publication Number:
  US 20160364310A1
• Time in Patent Office: 1,590 Days
• Field of Search: 702123, 702122
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/302 where the computing system component is a software system
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

In an approach for managing a set of tests to run on a set of platforms, a processor identifies a set of tests to run on a set of platforms and one or more priorities associated with the set of tests. A processor sends the set of tests and the one or more priorities associated with the set of tests to the set of platforms. A processor receives information about a first test run of a first test on a first platform of the set of platforms. A processor determines that the first test failed on the first platform. A processor updates the one or more priorities associated with the set of tests. A processor sends the updated one or more priorities associated with the set of tests to at least the second platform.

Claims

1. A method for managing a set of tests to run on a set of platforms, the method comprising:

identifying, by one or more processors, (i) a set of tests to run on a set of networked operating system platforms and (ii) one or more priorities associated with the set of tests, wherein each networked operating system platform comprises a type of device and an operating system;

sending, by one or more processors, the set of tests and the one or more priorities associated with the set of tests to the set of networked operating system platforms;

receiving, by one or more processors, during a first test run on a first networked operating system platform, information about the first test run of a first test on the first networked operating system platform of the set of networked operating system platforms, wherein the first networked operating system platform includes a first operating system;

determining, by one or more processors, during the first test run, that the first test failed on the first networked operating system platform;

updating, by one or more processors, during the first test run, the one or more priorities associated with the set of tests, such that the first test is associated with an increased priority on at least a second networked operating system platform of the set of networked operating system platforms, wherein;

the second networked operating system platform includes the first operating system; and

the first networked operating system platform and the second networked operating system platform are of different device types;

sending, by one or more processors, during the first test run and prior to completion of executing the set of tests, the updated one or more priorities associated with the set of tests to at least the second networked operating system platform to allow the first test to run on the second networked operating system platform with the increased priority during the first test run;

causing, by one or more processors, the second networked operating system platform to run the first test with the increased priority; and

determining, by one or more processors, that the first operating system is a cause of failure of the first test run on the first networked operating system platform, based on a result of the first test on the second networked operating system platform.

2. The method of claim 1, further comprising:

responsive to sending the set of tests and the one or more priorities associated with the set of tests to the set of networked operating system platforms, causing, by one or more processors, the first test to run on the first networked operating system platform; and

responsive to sending the updated one or more priorities associated with the set of tests to at least the second networked operating system platform, causing, by one or more processors, the first test to run on the second networked operating system platform.

3. The method of claim 1, wherein the information about the first test run indicates the test failed.

4. The method of claim 1, further comprising:

associating, by one or more processors, diagnostic information with the first test on the second networked operating system platform, wherein the diagnostic information is based on, at least the information about the first test run of the first test on the first networked operating system platform.

5. The method of claim 1, wherein updating the one or more priorities associated with the set of tests, such that the first test is associated with the increased priority on at least the second networked operating system platform of the set of networked operating system platforms comprises:

updating, by one or more processors, the one or more priorities associated with the set of tests, such that the first test will run sooner than scheduled per the sent one or more priorities.

16. The method of claim 1, wherein sending, during the first test run, the updated one or more priorities associated with the set of tests further comprises enabling additional debugging options that increase an amount of diagnostic information provided by the first test.

6. A computer program product for managing a set of tests to run on a set of platforms, the computer program product comprising:

one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising;

program instructions to identify (i) a set of tests to run on a set of networked operating system platforms and (ii) one or more priorities associated with the set of tests, wherein each networked operating system platform comprises a type of device and an operating system;

program instructions to send the set of tests and the one or more priorities associated with the set of tests to the set of networked operating system platforms;

program instructions to receive, during a first test run on a first networked operating system platform, information about the first test run of a first test on the first networked operating system platform of the set of networked operating system platforms, wherein the first networked operating system platform includes a first operating system;

program instructions to determine, during the first test run, that the first test failed on the first networked operating system platform;

program instructions to update, during the first test run, the one or more priorities associated with the set of tests, such that the first test is associated with an increased priority on at least a second networked operating system platform of the set of networked operating system platforms, wherein;

the second networked operating system platform includes the first operating system; and

the first networked operating system platform and the second networked operating system platform are of different device types;

program instructions to send, during the first test run and prior to completion of executing the set of tests, the updated one or more priorities associated with the set of tests to at least the second networked operating system platform to allow the first test to run on the second networked operating system platform with the increased priority during the first test run;

program instructions to cause the second networked operating system platform to run the first test with the increased priority; and

program instructions to determine that the first operating system is a cause of failure of the first test run on the first networked operating system platform, based on a result of the first test on the second networked operating system platform.

7. The computer program product of claim 6, further comprising:

responsive to sending the set of tests and the one or more priorities associated with the set of tests to the set of networked operating system platforms, program instructions, stored on the one or more computer readable storage media, to cause the first test to run on the first networked operating system platform; and

responsive to sending the updated one or more priorities associated with the set of tests to at least the second networked operating system platform, program instructions, stored on the one or more computer readable storage media, to cause the first test to run on the second networked operating system platform.

8. The computer program product of claim 6, wherein the information about the first test run indicates the test failed.

9. The computer program product of claim 6, further comprising:

program instructions, stored on the one or more computer readable storage media, to associate diagnostic information with the first test on the second networked operating system platform, wherein the diagnostic information is based on, at least the information about the first test run of the first test on the first networked operating system platform.

10. The computer program product of claim 6, wherein program instructions to update the one or more priorities associated with the set of tests, such that the first test is associated with the increased priority on at least the second platform of the set of networked operating system platforms comprise:

program instructions to update the one or more priorities associated with the set of tests, such that the first test will run sooner than scheduled per the sent one or more priorities.

11. A computer system for managing a set of tests to run on a set of platforms, the computer system comprising:

one or more computer processors, one or more computer readable storage media, and program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising;

program instructions to identify (i) a set of tests to run on a set of networked operating system platforms and (ii) one or more priorities associated with the set of tests, wherein each networked operating system platform comprises a type of device and an operating system;

program instructions to send the set of tests and the one or more priorities associated with the set of tests to the set of networked operating system platforms;

program instructions to receive, during a first test run on a first networked operating system platform, information about the first test run of a first test on the first networked operating system platform of the set of networked operating system platforms, wherein the first networked operating system platform includes a first operating system;

program instructions to determine, during the first test run, that the first test failed on the first networked operating system platform;

program instructions to update, during the first test run, the one or more priorities associated with the set of tests, such that the first test is associated with an increased priority on at least a second networked operating system platform of the set of networked operating system platforms, wherein;

the second networked operating system platform includes the first operating system; and

the first networked operating system platform and the second networked operating system platform are of different device types;

program instructions to send, during the first test run and prior to completion of executing the set of tests, the updated one or more priorities associated with the set of tests to at least the second networked operating system platform to allow the first test to run on the second networked operating system platform with the increased priority during the first test run;

program instructions to cause the second networked operating system platform to run the first test with the increased priority; and

program instructions to determine that the first operating system is a cause of failure of the first test run on the first networked operating system platform, based on a result of the first test on the second networked operating system platform.

12. The computer system of claim 11, further comprising:

responsive to sending the set of tests and the one or more priorities associated with the set of tests to the set of networked operating system platforms, program instructions, stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, to cause the first test to run on the first networked operating system platform; and

responsive to sending the updated one or more priorities associated with the set of tests to at least the second networked operating system platform, program instructions, stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, to cause the first test to run on the second networked operating system platform.

13. The computer system of claim 11, wherein the information about the first test run indicates the test failed.

14. The computer system of claim 11, further comprising:

program instructions, stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, to associate diagnostic information with the first test on the second networked operating system platform, wherein the diagnostic information is based on, at least the information about the first test run of the first test on the first networked operating system platform.

15. The computer system of claim 11, wherein program instructions to update the one or more priorities associated with the set of tests, such that the first test is associated with the increased priority on at least the second networked operating system platform of the set of networked operating system platforms comprise:

program instructions to update the one or more priorities associated with the set of tests, such that the first test will run sooner than scheduled per the sent one or more priorities.