Automated test generation for multi-interface enterprise virtualization management environment

• US 9,959,201 B2
• Filed: 03/14/2016
• Issued: 05/01/2018
• Est. Priority Date: 10/30/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Abdirashid, Mohammad, Duale, Ali Y., Hanif, Tariq
• Primary Examiner(s): CONTINO, PAUL F
• Application Number:
  US15/069,211
• Publication Number:
  US 20170123965A1
• Time in Patent Office: 778 Days
• CPC Class Codes: G06F 11/1497 Details of time redundant execution on a single processing unit
• CPC Class Codes: G06F 11/1629 Error detection by comparing the output of redundant processing systems
• CPC Class Codes: G06F 11/1641 where the comparison is not performed by the redundant processing components
• CPC Class Codes: G06F 11/2205 using arrangements specific to the hardware being tested
• CPC Class Codes: G06F 11/2221 to test input/output devices or peripheral units
• CPC Class Codes: G06F 11/263 Generation of test inputs, e.g. test vectors, patterns or sequences ; with adaptation of the tested hardware for testability with external testers
• CPC Class Codes: G06F 11/3457 Performance evaluation by simulation
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

Embodiments for automated testing of a virtualization management system are described. According to one aspect, a method includes generating a test case including a plurality of instances of commands and sending the test case to a plurality of interfaces supported by the virtualization management system. The method also includes generating a response file corresponding to each command in the test case. The method also includes comparing results from each interface to an instance of a command and in response to the results from each interface being identical, storing, the results in the response file corresponding to the command. The method also includes reporting an error in response to the results from each interface of the virtualization management system not being identical. The present document further describes examples of other aspects such as systems, computer products.

Claims

1. A computer-implemented method for automated testing of a virtualization management system, the method comprising:

generating, by a test server, a test case comprising instances of commands executable by the virtualization management system;

sending, by the test server, the test case to each interface of interfaces supported by the virtualization management system, the interfaces comprising a command line interface, a graphical user interface, and an application programming interface, each of which communicates with the virtualization management system;

generating, by the test server, a response file corresponding to each of the instances of commands in the test case;

executing, using the command line interface, the graphical user interface, and the application programming interface, a first instance of a command in the test case by the virtualization management system;

comparing, by the test server, results of the execution for each of the command line interface, the graphical user interface, and the application programming interface of the first instance of the command in the test case in order to determine whether the results are identical for the command line interface, the graphical user interface, and the application programming interface;

in response to the results from each of the command line interface, the graphical user interface, and the application programming interface of the virtualization management system being identical, storing the results from the virtualization management system in the response file corresponding to the first instance of the command;

determining, by the test server, that an error occurred among the command line interface, the graphical user interface, and the application programming interface in response to the results from each of the interfaces of the virtualization management system not being identical; and

the response file corresponding to the first interface of the command includes an entry corresponding to the result of executing each of the instances of the commands via each interface of the virtualization management system.

2. The computer-implemented method of claim 1, wherein the response file corresponding to the first interface of the command is a first response file for the virtualization management system, the method further comprising:

sending, by the test server, the test case to a result predictor unit;

simulating, by the result predictor unit, the instances in the test case;

generating, by the test server, for each interface, a second response file corresponding to each unique command in the test case,

comparing, by the test server, results from each interface to respective ones of the instances of the unique command from the test case upon execution of the instances of the unique command by the result predictor;

in response to the results from each interface being identical, storing, by the test server, in the second response file corresponding to the unique command, the results from the result predictor; and

reporting, by the test server, an error in response to the results from each interface of the result predictor not being identical.

3. The computer-implemented method of claim 2, further comprising:

comparing, by the test server, the results stored in the first response file and the second response file; and

in response to the results from the first response file not being identical to the results from the second response file, reporting, by the test server, an error.

4. The computer-implemented method of claim 1, further comprising executing, by a result predictor unit, the first instance of the command in the test case using the command line interface, the graphical user interface, and the application programming interface in order to obtain other results.

5. The computer-implemented method of claim 1, wherein the instances of the commands in the test case are executed in sequence.

6. The computer-implemented method of claim 1, wherein the instances of the commands in the test case include the first instance of the command and a second instance of said command.

7. The computer-implemented method of claim 6, wherein the results of the second instance executed via a particular interface are appended in the response file corresponding to the particular interface, wherein the response file corresponding to the particular interface contains results of the first instance executed via the particular interface.