Framework for testing distributed systems

• US 10,642,718 B2
• Filed: 07/31/2018
• Issued: 05/05/2020
• Est. Priority Date: 07/31/2018
• Status: Active Grant

• Current Assignee: Nutanix Incorporated
• Original Assignee: Nutanix Incorporated
• Inventors: Shah, Manan Hemantkumar, Saigal, Jagpreet Kaur, Surjan, Kilol
• Primary Examiner(s): Lee, Marina
• Application Number:
  US16/051,314
• Publication Number:
  US 20200042432A1
• Time in Patent Office: 644 Days
• Field of Search: 717100-135
• CPC Class Codes: G06F 11/263 Generation of test inputs, e.g. test vectors, patterns or sequences ; with adaptation of the tested hardware for testability with external testers
• CPC Class Codes: G06F 11/2733 Test interface between tester and unit under test
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2201/815 Virtual
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/505 considering the load

Abstract

Disclosed is an improved approach to implement testing for distributed computing systems. A switchboard framework is provided that allows tests to be dynamically generated and executed against a system under test.

Claims

1. A method, comprising:

providing a switchboard interface that displays a plurality of test workloads from a workload catalog;

receiving from the switchboard interface a first configuration selection of a test workload to test from available jobs in the workload catalog;

receiving a second configuration selection of system resources to test, wherein the second configuration selection includes a configuration of a protection domain of virtual machines for disaster recover testing;

allocating system resources according to the second configuration selection to form a system under test;

executing the test workload to perform testing in the system under test including resolving configuration inconsistencies and optimizing configuration parameters; and

establishing scheduling times for when backups of the protection domain of virtual machines are done to achieve optimized configuration parameters.

2. The method of claim 1, further comprising generating a third configuration that corresponds to a test-specific configuration, the third configuration applied to configure the system under test or to configure execution of the one or more workloads to perform the testing in the system under test.

3. The method of claim 1, wherein the optimized configuration parameters include a specific input/output operations per second (TOPS) value.

4. The method of claim 1, wherein the first configuration selection for the test workload includes at least one of (a) attributes specific to a particular workflow;

(b) scheduling parameters, (c) failure and success parameters, or (d) iteration parameters.

5. The method of claim 1, further comprising testing different levels of resources to test the system under test with different conditions and resource allocation coverages.

6. The method of claim 1, further comprising allocating processing entities to execute the test workload, wherein the processing entities are child processes managed by a master process.

7. The method of claim 1, wherein a constraint resolver procedure is applied to check for incompatibilities between selected workflow jobs.

8. The method of claim 7, wherein the incompatibilities between the selected workflow jobs are corrected by changing a size of configuration parameters.

9. The method of claim 1, wherein the system under test corresponds to a virtualization system having virtual machines and computing nodes configured using the first configuration.

10. A computer program product embodied on a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method, the method comprising:

providing a switchboard interface that displays a plurality of test workloads from a workload catalog;

receiving from the switchboard interface a first configuration selection of a test workload to test from available jobs in the workload catalog;

receiving a second configuration selection of system resources to test, wherein the second configuration selection includes a configuration of a protection domain of virtual machines for disaster recover testing;

allocating system resources according to the second configuration selection to form a system under test;

executing the test workload to perform testing in the system under test including resolving configuration inconsistencies and optimizing configuration parameters; and

establishing scheduling times for when backups of the protection domain of virtual machines are done to achieve optimized configuration parameters.

11. The computer program product of claim 10, wherein the computer readable medium further includes instructions for generating a third configuration that corresponds to a test-specific configuration, the third configuration applied to configure the system under test or to configure execution of the one or more workloads to perform the testing in the system under test.

12. The computer program product of claim 10, wherein the optimized configuration parameters include a specific input/output operations per second (IOPS) value.

13. The computer program product of claim 10, wherein the first configuration selection for the test workload includes at least one of (a) attributes specific to a particular workflow;

(b) scheduling parameters, (c) failure and success parameters, or (d) iteration parameters.

14. The computer program product of claim 10, wherein the testing is stepped to different levels of resources usage to test the system under test with different conditions and resource allocation coverages.

15. The computer program product of claim 10, further comprising processing entities to execute the test workload, wherein the processing entities are child processes managed by a master process.

16. The computer program product of claim 10, wherein a constraint resolver procedure is applied to check for incompatibilities between selected workflow jobs.

17. The computer program product of claim 16, wherein the incompatibilities between the selected workflow jobs are corrected by changing a size of one or more configuration parameters.

18. The computer program product of claim 10, wherein the system under test corresponds to a virtualization system having virtual machines and computing nodes configured using the first configuration.

19. A system, comprising:

a processor;

a memory for holding programmable code; and

wherein the programmable code includes instructions providing a switchboard interface that displays a plurality of test workloads from a workload catalog, wherein the instructions include;

receiving from the switchboard interface a first configuration selection of a test workload to test from available jobs in the workload catalog;

receiving a second configuration selection of system resources to test, wherein the second configuration selection includes a configuration of a protection domain of virtual machines for disaster recover testing;

allocating system resources according to the second configuration to form the system under test;

executing the test workload to perform testing in the system under test including resolving configuration inconsistencies and optimizing configuration parameters; and

establishing scheduling times for when backups of the protection domain of virtual machines are done to achieve optimized configuration parameters.

20. The system of claim 19, wherein the programmable code further comprises instructions for generating a third configuration that corresponds to a test-specific configuration, the third configuration applied to configure the system under test or to configure execution of the one or more workloads to perform the testing in the system under test.

21. The system of claim 19, wherein the optimized configuration parameters include a specific input/output operations per second (TOPS) value.

22. The system of claim 19, wherein the first configuration selection for the test workload includes at least one of (a) attributes specific to a particular workflow;

(b) scheduling parameters, (c) failure and success parameters, or (d) iteration parameters.

23. The system of claim 19, further comprising processing entities to execute the test workload, wherein the processing entities are child processes managed by a master process.

24. The system of claim 19, wherein a constraint resolver procedure is applied to check for incompatibilities between selected workflow jobs.

25. A computer program product embodied on a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method, the method comprising:

receiving a first configuration selection of a test workload to test from available jobs in a workload catalog;

receiving a second configuration selection of system resources to test, wherein the second configuration selection includes a configuration of virtual machine clusters for disaster recover testing;

allocating system resources according to the second configuration selection to form a system under test;

executing the test workload to perform testing in the system under test including resolving configuration inconsistencies and optimizing configuration parameters; and

establishing scheduling times for when backups of the virtual machine clusters are done to achieve optimized configuration parameters.

26. The computer program product of claim 25, wherein the optimized configuration parameters include a specific input/output operations per second (TOPS) value.

27. The computer program product of claim 25, wherein the first configuration selection for the test workload includes at least one of (a) attributes specific to a particular workflow;

(b) scheduling parameters, (c) failure and success parameters, or (d) iteration parameters.

28. The computer program product of claim 25, wherein the testing is stepped to different levels of resources usage to test the system under test with different conditions and resource allocation coverages.

29. The computer program product of claim 25, wherein a constraint resolver procedure is applied to check for incompatibilities between selected workflow jobs.

30. The computer program product of claim 29, wherein the incompatibilities between the selected workflow jobs are corrected by changing a size of one or more configuration parameters.