Test machine management

• US 9,658,946 B2
• Filed: 08/26/2016
• Issued: 05/23/2017
• Est. Priority Date: 08/20/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bhuiya, Subhajit
• Primary Examiner(s): Chen, Qing
• Assistant Examiner(s): HUDA, MOHAMMED NURUL
• Application Number:
  US15/248,055
• Publication Number:
  US 20170052883A1
• Time in Patent Office: 270 Days
• Field of Search: 717124, 717126
• CPC Class Codes: G06F 11/2205 using arrangements specific to the hardware being tested
• CPC Class Codes: G06F 11/263 Generation of test inputs, e.g. test vectors, patterns or sequences ; with adaptation of the tested hardware for testability with external testers
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis

Abstract

A method includes distributing the plurality of test cases to any available test agents, wherein each test case out of the plurality of test cases does not have any associated preconditions. The method receives event information for a first test case out of the plurality of test cases from a first test agent. Responsive to determining the event information for the first test case includes a satisfied condition for a second test case, the method determines whether the satisfied condition for the second test case relates to a global variable or local variable. The method handles the second test case, wherein handling the second test case includes distributing the second test case to the first test agent subsequent to the first test agent becoming available if the satisfied condition relates to the local variable.

Claims

1. A method comprising:

responsive to reaching a specified execution time for a plurality of test cases, distributing, by one or more processors, the plurality of test cases to any available test agents, wherein each test case out of the plurality of test cases does not have any associated preconditions;

receiving, by one or more processors, event information for a first test case out of the plurality of test cases from a first test agent;

responsive to determining the event information for the first test case includes a satisfied condition for a second test case, determining, by one or more processors, whether the satisfied condition for the second test case relates to a global variable or local variable;

handling, by one or more processors, the second test case, wherein handling the second test case includes distributing the second test case to the first test agent subsequent to the first test agent becoming available if the satisfied condition relates to the local variable and the satisfied condition indicates that the local variable is below, equal, or above a predetermined value, wherein handling the second test case includes distributing the second test case to any idle test agent from the plurality of test agents if the satisfied condition relates to the global variable, and wherein handling the second test case includes queuing the second test case if there are no idle test agents;

responsive to determining that a second test agent has become available, determining, by one or more processors, whether any of the test cases in the queue include satisfied local variables for the second test agent; and

executing, by one or more processors, a third test case with a satisfied local variable for the second test agent.

2. The method of claim 1, further comprising:

creating, by one or more processors, a test suite, wherein the test suite includes a plurality of test cases for execution on a plurality of test agents.

3. The method of claim 1, further comprising:

responsive to determining a number of test cases out of the plurality of test cases with no associated preconditions exceeds a number of test agents currently available, queuing, by one or more processors, a first portion of the test cases out of the plurality of test cases with no associated preconditions and an amount of test code script below or equal to the threshold; and

distributing, by one or more processors, a second portion of the test cases out of the plurality of test cases with no associated preconditions and an amount of test code script above a threshold to any available test agents, wherein the threshold is an amount of test code script for a test case.

4. The method of claim 1, wherein receiving event information for a first test case out of the first portion of test cases includes receiving test results for the first test case.

5. The method of claim 1, wherein the second test case includes associated preconditions, wherein the associated preconditions include a trigger condition.

6. The method of claim 5, wherein the associated preconditions further include a variable name and a variable type.

7. The method of claim 1, further comprising:

receiving, by one or more processors, a request to execute the test suite with the plurality of test cases on the plurality of test agents.

8. A computer program product comprising:

one or more computer readable tangible storage media and program instructions stored on at least one of the one or more storage media, wherein the one or more computer readable tangible storage media are not transitory signals, the program instructions comprising;

program instructions to, responsive to reaching a specified execution time for a plurality of test cases, distribute the plurality of test cases to any available test agents, wherein each test case out of the plurality of test cases does not have any associated preconditions;

program instructions to receive event information for a first test case out of the plurality of test cases from a first test agent;

program instructions to, responsive to determining the event information for the first test case includes a satisfied condition for a second test case, determine whether the satisfied condition for the second test case relates to a global variable or local variable;

program instructions to handle the second test case, wherein handling the second test case includes distributing the second test case to the first test agent subsequent to the first test agent becoming available if the satisfied condition relates to the local variable and the satisfied condition indicates that the local variable is below, equal, or above a predetermined value, wherein handling the second test case includes distributing the second test case to any idle test agent from the plurality of test agents if the satisfied condition relates to the global variable, and wherein handling the second test case includes queuing the second test case if there are no idle test agents;

program instructions to, responsive to determining that a second test agent has become available, determine whether any of the test cases in the queue include satisfied local variables for the second test agent; and

program instructions to execute a third test case with a satisfied local variable for the second test agent.

9. The computer program product of claim 8, wherein the program instructions further comprise:

program instructions to create a test suite, wherein the test suite includes a plurality of test cases for execution on a plurality of test agents.

10. The computer program product of claim 8, wherein the program instructions further comprise:

program instructions to, responsive to determining a number of test cases out of the plurality of test cases with no associated preconditions exceeds a number of test agents currently available, queue a first portion of the test cases out of the plurality of test cases with no associated preconditions and an amount of test code script below or equal to the threshold; and

program instruction to distribute a second portion of the test cases out of the plurality of test cases with no associated preconditions and an amount of test code script above a threshold to any available test agents, wherein the threshold is an amount of test code script for a test case.

11. The computer program product of claim 8, wherein program instructions to receive event information for a first test case out of the first portion of test cases includes program institutions to receive test results for the first test case.

12. The computer program product of claim 8, wherein the second test case includes associated preconditions, wherein the associated preconditions include a trigger condition.

13. The computer program product of claim 12, wherein the associated preconditions further include a variable name and a variable type.

14. The computer program product of claim 8, wherein the program instructions further comprise:

program instructions to receive a request to execute the test suite with the plurality of test cases on the plurality of test agents.

15. A computer system comprising:

one or more computer processors;

one or more computer readable tangible storage media, wherein the one or more computer readable tangible storage media are not transitory signals; and

program instructions stored on the one or more computer readable tangible storage media for execution by at least one of the one or more computer processors, the program instructions comprising;

program instructions to, responsive to reaching a specified execution time for a plurality of test cases, distribute the plurality of test cases to any available test agents, wherein each test case out of the plurality of test cases does not have any associated preconditions;

program instructions to receive event information for a first test case out of the plurality of test cases from a first test agent;

program instructions to, responsive to determining the event information for the first test case includes a satisfied condition for a second test case, determine whether the satisfied condition for the second test case relates to a global variable or local variable;

program instructions to handle the second test case, wherein handling the second test case includes distributing the second test case to the first test agent subsequent to the first test agent becoming available if the satisfied condition relates to the local variable and the satisfied condition indicates that the local variable is below, equal, or above a predetermined value, wherein handling the second test case includes distributing the second test case to any idle test agent from the plurality of test agents if the satisfied condition relates to the global variable, and wherein handling the second test case includes queuing the second test case if there are no idle test agents;

program instructions to, responsive to determining that a second test agent has become available, determine whether any of the test cases in the queue include satisfied local variables for the second test agent; and

program instructions to execute a third test case with a satisfied local variable for the second test agent.

16. The computer system of claim 15, wherein the program instructions further comprise:

program instructions to create a test suite, wherein the test suite includes a plurality of test cases for execution on a plurality of test agents.

17. The computer system of claim 15, wherein the program instructions further comprise:

program instructions to, responsive to determining a number of test cases out of the plurality of test cases with no associated preconditions exceeds a number of test agents currently available, queue a first portion of the test cases out of the plurality of test cases with no associated preconditions and an amount of test code script below or equal to the threshold; and

program instruction to distribute a second portion of the test cases out of the plurality of test cases with no associated preconditions and an amount of test code script above a threshold to any available test agents, wherein the threshold is an amount of test code script for a test case.

18. The computer system of claim 15, wherein program instructions to receive event information for a first test case out of the first portion of test cases includes program institutions to receive test results for the first test case.

19. The computer system of claim 15, wherein the second test case includes associated preconditions, wherein the associated preconditions include a trigger condition.

20. The computer system of claim 19, wherein the associated preconditions further include a variable name and a variable type.