Automated test generation for multi-interface and multi-platform enterprise virtualization management environment

• US 10,552,306 B2
• Filed: 11/08/2017
• Issued: 02/04/2020
• Est. Priority Date: 04/20/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hanif, Tariq, To, Tin H.
• Primary Examiner(s): Contino, Paul
• Application Number:
  US15/806,428
• Publication Number:
  US 20180307575A1
• Time in Patent Office: 818 Days
• CPC Class Codes: G06F 11/1497 Details of time redundant execution on a single processing unit
• CPC Class Codes: G06F 11/1629 Error detection by comparing the output of redundant processing systems
• CPC Class Codes: G06F 11/1641 where the comparison is not performed by the redundant processing components
• CPC Class Codes: G06F 11/165 with continued operation after detection of the error
• CPC Class Codes: G06F 11/183 by voting, the voting not being performed by the redundant components
• CPC Class Codes: G06F 11/187 Voting techniques
• CPC Class Codes: G06F 11/2221 to test input/output devices or peripheral units
• CPC Class Codes: G06F 11/2294 by remote test
• CPC Class Codes: G06F 11/3457 Performance evaluation by simulation
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis

Abstract

Embodiments for automated testing of a virtualization management system are described. An example computer-implemented method for automated testing of a virtualization management system includes sending, by a test server, a test case to a plurality of instances of the system under test, the test case sent to each instance of the system under test via each interface from a plurality of interfaces supported by the system under test. The method further includes, for each instance of the system under test, performing multi-interface comparison. The comparison includes comparing, by the test server, responses to the test case from each of the interfaces. The method also includes in response to the responses from each of the interfaces being identical, storing the responses in an instance-response file corresponding to the instance. The method also includes reporting, by the test server, an error in response to the responses from each interface not being identical.

Claims

1. A computer-implemented method for automated testing of a system under test, the method comprising:

sending, by a test server, a test case to a plurality of instances of the system under test, the test case sent to each instance of the system under test via each interface from a plurality of interfaces supported by the system under test, wherein the test case comprises a plurality of instances of commands executable by the system under test, and wherein the instances of the commands in the test case are executed in sequence;

for each instance of the system under test, performing multi-interface comparison by;

comparing, by the test server, responses to the test case from each of the interfaces;

in response to the responses from each of the interfaces being identical, storing the responses in an instance-response file corresponding to the instance;

generating, by the test server, a response file corresponding to each of the plurality of commands in the test case;

in response to the responses from each interface of the instance of the system under test being identical, storing the responses in the response file corresponding to the command; and

reporting, by the test server, an error in response to the responses from each interface not being identical.

2. The computer-implemented method of claim 1, wherein the instances of the commands in the test case include a first instance of a command and a second instance of said command.

3. The computer-implemented method of claim 2, wherein the responses of the second instance of the command executed via an interface are appended in the response file corresponding to the command, wherein the response file contains results of the first instance of the command executed via said interface.

4. The computer-implemented method of claim 1, further comprising, in response to the responses from each interface of the instance of the system under test being identical:

sending, by the test server, the instance-response file from each of the instances of the system under test to a multi-platform comparison unit;

for each instance of the system under test, performing multi-interface comparison by;

comparing, by the test server, responses to the test case from each of the instances; and

in response to the responses from each of the instances being identical, storing the responses in a system-response file; and

reporting, by the test server, an error in response to the responses from each instance not being identical.

5. The computer-implemented method of claim 4, further comprising:

sending, by the test server, the test case to a result predictor unit;

generating, by the result predictor unit, a predictor-response file comprising simulated responses for the test case;

comparing, by the test server, responses to the test case from the system-response file and the predictor-response file;

in response to the responses from the predictor-response file and the system-response file being identical, indicating that the test case passed; and

reporting, by the test server, an error in response to the responses from the predictor-response file and the system-response file not being identical.

6. The computer-implemented method of claim 1, wherein the interfaces comprise at least two of a command line interface, a graphical user interface, and an application programming interface.

7. The computer-implemented method of claim 1, wherein the plurality of instances of the system under test are being executed on distinct platforms.