PROVISIONING OF DISTRIBUTED COMPUTING CLUSTERS

• US 20130227558A1
• Filed: 02/29/2012
• Published: 08/29/2013
• Est. Priority Date: 02/29/2012
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: XIAO, Jun, DU, Junping, HE, Ying, WAN, Da
• Granted Patent: US 9,268,590 B2
• Time in Patent Office: Days
• US Class Current: 718/1
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal

Abstract

Embodiments perform automated provisioning of a cluster for a distributed computing platform. Target host computing devices are selected from a plurality of host computing devices based on configuration information, such as a desired cluster size, a data set, code for processing the data set and, optionally, a placement strategy. One or more virtual machines (VMs) are instantiated on each target host computing device. Each VM is configured to access a virtual disk that is preconfigured with code for executing functionality of the distributed computing platform and serves as a node of the cluster. The data set is stored in a distributed file system accessible by at least a subset of the VMs. The code for processing the data set is provided to at least a subset of the VMs, and execution of the code is initiated to obtain processing results.

Claims

1. A method for provisioning a cluster for a distributed computing platform, the method comprising:

receiving configuration information for a cluster from a user, wherein the configuration information comprises at least a cluster size, a data set and code for processing the data set from the user;

selecting a plurality of target host computing devices from a plurality of host computing devices based on the configuration information;

instantiating, based on the cluster size, at least one virtual machine (VM) on each of the target host computing devices to serve as a node of the cluster, wherein each instantiated VM is configured to access a virtual disk that is based on a VM template and preconfigured with code for executing functionality of the distributed computing platform;

persistently storing the data set in a distributed file system accessible by at least a subset of the VMs, wherein the distributed file system is accessed by the distributed computing platform during processing of the data set;

providing the code for processing the data set to at least a subset of the VMs; and

initiating execution of the code for processing the data set on the at least subset of VMs to obtain data processing results for the user.

2. The method of claim 1, wherein at least one of the instantiated VMs is configured to utilize local storage in the target host computing device on which the VM has been instantiated, and the local storage supports the distributed file system.

3. The method of claim 2, wherein the step of persistently storing the data set comprises storing different portions of the data set in different local storages corresponding to target host computing devices of the instantiated VMs.

4. The method of claim 1, further comprising the step of selecting the VM template that serves as a basis for virtual disks based on the configuration information.

5. The method of claim 4, wherein at least one virtual disk of an instantiated VM is a linked clone of the VM template.

6. The method of claim 1, further comprising the step of enabling the user to maintain a life cycle of the cluster for future processing of data.

7. The method of claim 1, wherein the configuration information further comprises a placement strategy including one of the following:

operational efficiency, operational robustness, and a combination of operational efficiency and operational robustness, and the step of selecting the target host computing devices is based on the placement strategy and a location of each of the target host computing devices, wherein the location includes one or more of the following;

a physical location and a network location.

8. One or more computer-readable storage media including computer-executable instructions that, when executed by a processor, cause the processor to provision a distributed computing cluster having a plurality of virtual machines (VMs) by:

selecting a plurality of target host computing devices from a plurality of host computing devices based on configuration information including at least a cluster size, a data set and code for processing the data set;

instantiating, based on the cluster size, at least one virtual machine (VM) on each of the target host computing devices to serve as a node of the cluster, wherein each instantiated VM is configured to access a virtual disk that is based on a VM template and preconfigured with code for executing functionality of a distributed computing platform;

persistently storing the data set in a distributed file system accessible by at least a subset of the VMs, wherein the distributed file system is accessed by the distributed computing platform during processing of the data set;

providing the code for processing the data set to at least a subset of the VMs; and

initiating execution of the code for processing the data set on the at least subset of VMs to obtain data processing results.

9. The computer-readable storage media of claim 8, wherein at least one of the instantiated VMs is configured to utilize a local storage in the target host computing device on which the VM has been instantiated, and the local storage supports the distributed file system.

10. The computer-readable storage media of claim 9, wherein the computer-executable instructions cause the processor to persistently store the data set at least in part by storing different portions of the data set in different local storages corresponding to target host computing devices of the instantiated VMs.

11. The computer-readable storage media of claim 8, wherein the computer-executable instructions further cause the processor to select the VM template that serves as a basis for virtual disks based on the configuration information.

12. The computer-readable storage media of claim 11, wherein at least one virtual disk of an instantiated VM is a linked clone of the VM template.

13. The computer-readable storage media of claim 8, wherein the computer-executable instructions further cause the processor to enable a user to maintain a life cycle of the cluster for future processing of data.

14. The computer-readable storage media of claim 8, wherein the configuration information further comprises a placement strategy including one of the following:

operational efficiency, operational robustness, and a combination of operational efficiency and operational robustness, and

the step of selecting the target host computing devices is based on the placement strategy and a location of each of the target host computing devices, wherein the location includes one or more of the following;

a physical location and a network location.

15. A system for provisioning a distributed computing cluster, the system comprising:

a plurality of host computing devices; and

a management device coupled in communication with the host computing devices and configured to;

select a plurality of target host computing devices from the plurality of host computing devices based on configuration information provided by a user, the configuration information including at least a cluster size, a data set and code for processing the data set from the user;

instantiate, based on the cluster size, at least one virtual machine (VM) on each of the target host computing devices to serve as a node of the cluster, wherein each instantiated VM is configured to access a virtual disk that is based on a VM template and preconfigured with code for executing functionality of a distributed computing platform;

persistently store the data set in a distributed file system accessible by at least a subset of the VMs, wherein the distributed file system is accessed by the distributed computing platform during processing of the data set;

provide the code for processing the data set to at least a subset of the VMs; and

initiate execution of the code for processing the data set on the at least subset of VMs to obtain data processing results for the user.

16. The system of claim 15, wherein at least one of the instantiated VMs is configured to utilize a local storage in the target host computing device on which the VM has been instantiated, and the local storage supports the distributed file system.

17. The system of claim 16, wherein the management device is configured to persistently store the data set at least in part by storing different portions of the data set in different local storages corresponding to target host computing devices of the instantiated VMs.

18. The system of claim 15, wherein the management device is further configured to select the VM template that serves as a basis for virtual disks based on the configuration information.

19. The system of claim 18, wherein the management device is further configured to enable the user to maintain a life cycle of the cluster.

20. The system of claim 15, wherein the configuration information further comprises a placement strategy including one of the following:

operational efficiency, operational robustness, and a combination of operational efficiency and operational robustness, and the management device is configured to select the target host computing devices based on the placement strategy and a location of each of the target host computing devices, wherein the location includes one or more of the following;

a physical location and a network location.