Automated test generation for multi-interface enterprise virtualization management environment

• US 9,858,177 B2
• Filed: 10/30/2015
• Issued: 01/02/2018
• Est. Priority Date: 10/30/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Abdirashid, Mohammad, Duale, Ali Y., Hanif, Tariq
• Primary Examiner(s): Kim, Matt
• Assistant Examiner(s): CONTINO, PAUL F
• Application Number:
  US14/927,583
• Publication Number:
  US 20170123951A1
• Time in Patent Office: 795 Days
• CPC Class Codes: G06F 11/1497 Details of time redundant execution on a single processing unit
• CPC Class Codes: G06F 11/1629 Error detection by comparing the output of redundant processing systems
• CPC Class Codes: G06F 11/1641 where the comparison is not performed by the redundant processing components
• CPC Class Codes: G06F 11/2205 using arrangements specific to the hardware being tested
• CPC Class Codes: G06F 11/2221 to test input/output devices or peripheral units
• CPC Class Codes: G06F 11/263 Generation of test inputs, e.g. test vectors, patterns or sequences ; with adaptation of the tested hardware for testability with external testers
• CPC Class Codes: G06F 11/3457 Performance evaluation by simulation
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

Embodiments for automated testing of a virtualization management system are described. According to one aspect, a method includes generating a test case including a plurality of instances of commands and sending the test case to a plurality of interfaces supported by the virtualization management system. The method also includes generating a response file corresponding to each command in the test case. The method also includes comparing results from each interface to an instance of a command and in response to the results from each interface being identical, storing, the results in the response file corresponding to the command. The method also includes reporting an error in response to the results from each interface of the virtualization management system not being identical. The present document further describes examples of other aspects such as systems, computer products.

Claims

1. A system for automated testing of a virtualization management system, the system comprising:

a memory; and

a processor configured to;

determine a number of interfaces supported by the virtualization management system;

generate a test case comprising instances of commands executable by the virtualization management system;

send the test case for execution by the virtualization management system via each of the interfaces of the virtualization management system;

generate a response file for each unique command in the test case, wherein the response file includes an entry corresponding to a result of executing each instance of the unique command via each interface of the virtualization management system;

compare results of execution, by the virtualization management system via each interface, of a first instance of a command in the test case, wherein the interfaces of the virtualization management system comprise a first interface through a last interface;

in response to the results of the first instance of the command via each interface being identical such that the results of the first instance of the command are identical for the first interface though the last interface, append the results to each response file corresponding to the first instance of the command; and

in response to the results of the first instance of the command via each interface not being identical, report an error.

2. The system for automated testing of a virtualization management system of claim 1, wherein the response file is a first response file and the processor is further configured to:

send the test case for execution by a result predictor via each interface of the result predictor, wherein the result predictor supports all the interfaces supported by the virtualization management system;

generate a second response file for each unique command in the test case, wherein the second response file includes an entry corresponding to result of executing each instance of the unique command via each interface of the result predictor;

compare results of execution, by the result predictor via each interface, of the first instance of the command in the test case;

in response to the results of the first instance of the command via each interface being identical, append the results to the second response file corresponding to the command; and

in response to the results of the first instance of the command via each interface not being identical, report an error.

3. The system for automated testing of a virtualization management system of claim 2, wherein the processor is further configured to:

compare the results of the first instance of the command from the first file and the second file; and

in response to the results not being identical, reporting an error.

4. The system for automated testing of a virtualization management system of claim 2, wherein the response file corresponding to the first instance of the command contains a number of entries equal to a product of a number of instances of the command in the test case and a number of instances supported by the virtualization management system.

5. The system for automated testing of a virtualization management system of claim 2,

wherein the interfaces supported by the virtualization management system comprise at least two of a command line interface, a graphical user interface, and an application programming interface.

6. The system for automated testing of a virtualization management system of claim 2, wherein the instances of the commands in the test case are executed in a sequence in which they are included in the test case.

7. The system for automated testing of a virtualization management system of claim 6, wherein the instances of the commands in the test case are selected by a random selection of commands executable by the virtualization management system.

8. A computer product for automated testing of a virtualization management system, the computer product comprising computer readable storage medium, the computer readable storage medium comprising computer executable instructions, wherein the computer readable storage medium comprises instructions to:

determine a number of interfaces supported by the virtualization management system;

generate a test case comprising instances of commands executable by the virtualization management system;

send the test case for execution by the virtualization management system via each of the interfaces of the virtualization management system;

generate a response file for each unique command in the test case, wherein the response file includes an entry corresponding to a result of executing each instance of the unique command via each interface of the virtualization management system;

compare results of execution, by the virtualization management system via each interface, of a first instance of a command in the test case, wherein the interfaces of the virtualization management system comprise a first interface through a last interface;

in response to the results of the first instance of the command via each interface being identical such that the results of the first instance of the command are identical for the first interface through the last interface, append the results to each response file corresponding to the first instance of the command; and

in response to the results of the first instance of the command via each interface not being identical, report an error.

9. The computer product for automated testing of a virtualization management system of claim 8, wherein the response file is a first response file, and wherein the computer readable storage medium further comprises instructions to:

send the test case for execution by a result predictor via each interface of the result predictor, wherein the result predictor supports all the interfaces supported by the virtualization management system;

generate a second response file for each unique command in the test case, wherein the second response file includes an entry corresponding to result of executing each instance of the unique command via each interface of the result predictor;

compare results of execution, by the result predictor via each interface, of the first instance of the command in the test case;

in response to the results of the first instance of the command via each interface being identical, append the results to the second response file corresponding to the command; and

in response to the results of the first instance of the command via each interface not being identical, report an error.

10. The computer product for automated testing of a virtualization management system of claim 9, wherein the computer readable storage medium further comprises instructions to:

compare the results of the first instance of the command from the first file and the second file; and

in response to the results not being identical, reporting an error.

11. The computer product for automated testing of a virtualization management system of claim 8, wherein the response file corresponding to the first instance of the command contains a number of entries equal to a product of a number of instances of the command in the test case and a number of instances supported by the virtualization management system.

12. The computer product for automated testing of a virtualization management system of claim 8, wherein the interfaces supported by the virtualization management system comprise at least two of a command line interface, a graphical user interface, and an application programming interface.