SYSTEMS AND METHODS OF HOST-AWARE RESOURCE MANAGEMENT INVOLVING CLUSTER-BASED RESOURCE POOLS

• US 20130055262A1
• Filed: 08/27/2012
• Published: 02/28/2013
• Est. Priority Date: 08/25/2011
• Status: Active Grant

• Current Assignee: VIRTUSTREAM IP HOLDING COMPANY LLC (Dell Technologies Inc.)
• Original Assignee: Vincent G. Lubsey
• Inventors: Reid, Kevin D., LUBSEY, Vincent G., Simpson, Karl J., Rogers, Rodney John
• Granted Patent: US 8,799,920 B2
• Time in Patent Office: Days
• US Class Current: 718/1
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 2209/5011 Pool
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5072 Grid computing

Abstract

Systems and methods are disclosed for managing resources associated with cluster-based resource pool(s). According to illustrative implementations, innovations herein may include or involve one or more of best fit algorithms, infrastructure based service provision, tolerance and/or ghost processing features, dynamic management service having monitoring and/or decision process features, as well as virtual machine and resource distribution features.

Claims

1. A method for processing information related to management of a cluster-based pool of resources, the method comprising:

performing first processing regarding whether more than 1 cluster is required to store a customer'"'"'s required IU (IUreq) including a routine, executed if more than 1 cluster is required, for determining whether to iterate through additional clusters or to execute a bespoke “

stretched”

cluster;

performing second processing regarding determination that the IUreq or a chunk of the IUreq must be allocated across multiple hosts including allocating storage as a function of a quantity of permissible VMs per pool;

performing third processing regarding determination that the IUreq will fit inside a single host including allocating IUs as a function of a VM-to-resource pool value produced and the permissible VMs per pool (a chunk value); and

defining, based on set criteria, a best fit number of chunks for a customer'"'"'s resource requirement or resource pool allocation;

wherein the set of criteria includes a quantity of resource units used, a quantity of virtual machines (VMs) assigned to the customer and/or VM/unit ratio information.

2. The method of claim 1 further comprising determining an acceptable outage time value as a function of infrastructure guarantee information, migration or HA time per virtual machine, and/or the virtual machine per chunk calculation.

3. The method of claim 1 further comprising:

analyzing clusters configured to hold customer data including performing infrastructure speed testing and/or VM memory map sizing of hosts.

4. The method of claim 1 wherein the customer'"'"'s resource requirement is established as a quantity of universal Infrastructure Units (IUs) calculated as a function of analysis of clusters configured to hold customer data, wherein the analysis includes performing infrastructure speed testing and/or VM memory map sizing of hosts.

5. The method of claim 1 wherein a value of the maximum number of virtual machines per resource chunk corresponds to a quantity of virtual machines set to ensure an appropriate speed for high availability (HA) restart or maintenance migration.

6. The method of claim 1 further comprising defining the customer'"'"'s resource pool requirements as a function of accessing/assessing the customer'"'"'s infrastructure.

7. The method of claim 6 further comprising apportioning resources as a function of the defined customer resource pool requirements.

8. The method of claim 6 wherein the accessing/assessing of the customer'"'"'s infrastructure includes obtaining capacity planner results.

9. The method of claim 6 wherein the accessing/assessing of the customer'"'"'s infrastructure includes obtaining perfmon type statistics.

45. One or more computer readable media containing computer readable programs of instructions executable by one or more processing components to implement a method of managing a cluster-based pool of resources, the computer readable programs of instructions comprising instructions for performing one or more steps recited in claims 1-44 and/or other functionality set forth elsewhere in this disclosure.

46. A system comprising:

one or more computing components and/or one or more computer readable media configured to perform one or more steps recited in claims 1-44 and/or other functionality set forth elsewhere in this disclosure.

10. A method for processing information related to management of a cluster-based pool of resources, the method comprising:

determining, based on set criteria, a best fit number of chunks for a customer'"'"'s resource requirement or resource pool allocation, wherein the set of criteria includes a quantity of resource units used, a quantity of virtual machines (VMs) assigned to the customer and/or VM/unit ratio information;

processing a plurality of predefined resource limit values determined for a plurality of customers; and

processing information regarding partitioning hardware resources and/or associated software of a data center of a pool of servers, or of a pool of VM hosts such that the hardware and/or the software resources of the data center units are guaranteed to perform at one or more of the predefined resource limit values.

11. The method of claim 10 wherein the customer'"'"'s resource requirement is established as a quantity of universal Infrastructure Units (IUs) calculated as a function of analysis of clusters configured to hold customer data, wherein the analysis includes performing infrastructure speed testing and/or VM memory map sizing of hosts.

12. The method of claim 10 further comprising:

analyzing clusters configured to hold customer data including performing infrastructure speed testing and/or VM memory map sizing of hosts.

13. The method of claim 10 wherein guaranteed levels of service are provided for each predefined resource limit value from the set of predefined resource limit values.

14. The method of claim 10 wherein guaranteed levels of service are provided for every predefined resource limit value from the set of predefined resource limit values.

15. A method for processing information related to management of a cluster-based pool of resources, the method comprising:

processing first information regarding configuration of the resource clusters with highwater marks to accomplish site and cluster tolerance, the highwater marks reflecting divisions between usable space and tolerance space, and between the tolerance space and host space reserved to ensure host activity;

processing second information regarding distribution/allocation of resource pools throughout the cluster usable space including handling of ghost pools located in unused tolerance space, the ghost pools being defined either on a host not containing a live pool associated with a customer or, in situations where the customer has resource pools on all hosts in a cluster, on multiple hosts within the cluster;

processing third information regarding use of the ghost pools during maintenance windows or node tolerance events;

processing fourth information regarding migration of virtual machines (VM'"'"'s) assigned to the customer'"'"'s resources back to originating pools, upon resumption of normal service.

16. The method of claim 15 further comprising:

analyzing clusters configured to hold customer data including performing infrastructure speed testing and/or VM memory map sizing of hosts.

17. The method of claim 15 wherein the customer'"'"'s resource requirement is established as a quantity of universal Infrastructure Units (IUs) calculated as a function of analysis of clusters configured to hold customer data, wherein the analysis includes performing infrastructure speed testing and/or VM memory map sizing of hosts.

18. The method of claim 15 wherein, as a function of the distribution/allocation and associated failover processing, the VM'"'"'s of the customer are placed together in a limited subset of hosts, to provide faster communication of information and resulting in a more expected/established set of communication paths.

19. The method of claim 18 wherein, as a function of the distribution/allocation and associated failover processing, the cluster is configured such that maintenance windows need only to account for hosts associated with affected locations, rather than entirety of the cluster.

20. The method of claim 15 wherein, in connection with the distribution/allocation and associated failover processing, constraints are configured to remain in place even with loss of management service.

21. A method for processing information related to management of a cluster-based pool of resources, the method comprising:

performing a management service for monitoring and management of the workloads within the cluster, including dynamic shifting of workloads and/or additional resources;

processing information regarding alerts related to highwatermark marks in the cluster groups, the highwater marks reflecting divisions between usable space and tolerance space, and between the tolerance space and host space;

processing information regarding querying the cluster groups for performance information relating to the group and virtual machines running within the group;

processing information regarding migration of a VM based on its workload to an alternative cluster group;

processing information regarding dynamically altering a cluster group limit to provide a burst;

processing information regarding defining and/or destroying a cluster group to provide timed access to a system.

22. The method of claim 21 further comprising:

analyzing clusters configured to hold customer data including performing infrastructure speed testing and/or VM memory map sizing of hosts.

23. The method of claim 21 wherein the customer'"'"'s resource requirement is established as a quantity of universal Infrastructure Units (Ws) calculated as a function of analysis of clusters configured to hold customer data, wherein the analysis includes performing infrastructure speed testing and/or VM memory map sizing of hosts.

24. The method of claim 21 wherein the burst is provided/defined as an expansion of host/cluster/Cgroup limits or space over a short/needed timeframe and/or for other limited circumstances or temporal conditions.

25. The method of claim 21 wherein the processing of information regarding migration of a VM includes providing room for a busy service within a cluster group from which the VM is to be migrated.

26. The method of claim 25 wherein one or more of defining free space in lower watermark pools, defining best fit VM'"'"'s, and/or defining vmotion is performed as a function of whether or not a quantity of pools at highwater mark is less than a total quantity of pools.

27. The method of claim 21 wherein the processing of information regarding migration of a VM includes processing data regarding migration of the VM to a cluster group having sufficient free space to handle operation of the VM task(s).

28. The method of claim 21 wherein the dynamic altering of the cluster group limit to provide a burst involves one or more of dynamically increasing cluster group size across cluster resources, providing a burst into the ghost pool(s), and/or utilizing vmotion as a function of whether or not a quantity of pools at highwater mark equals a total quantity of pools.

29. The method of claim 21 wherein the processing of information regarding defining and/or destroying a cluster group provides “

timed access”

to the system at specified times, such as at month end, at end of a payroll period, etcetera.

30. The method of claim 21 further comprising scheduling a resource allocation to a zeroed cluster group then re-zeroing the cluster group at an end of the scheduled allocation time of the resource.

31. The method of claim 21 wherein a management service or process executing the method is configured to be entirely or substantially stateless in that the cluster groups themselves provide information relating to performance of the cluster groups and/or performance of the virtual machines within the cluster groups.

32. The method of claim 21 wherein a management service or process executing the method is configured to run outside of the cluster.

33. The method of claim 21 wherein a management service or process executing the method is configured to run inside the cluster and is managed as a cluster resource.

34. The method of claim 21 wherein a management service or process is configured to execute the method.

35. The method of claim 34 wherein the management service or process is implemented, at least in part, as a monitoring service/process or as a monitoring application, and the monitoring service/process or the monitoring application is maintained solely as a monitoring process.

36. The method of claim 34 wherein, when a trigger event occurs, the management service/process:

spawns a customer-specific migration decision event or process on the host generating the trigger; and

then returns to its monitoring state.

37. The method of claim 36 wherein the customer-specific migration decision process proceeds to evaluate migration requirements, define a best fit resolution plan, and perform processing related to enacting that plan.

38. The method of claim 36 wherein, if any balance issue exists after the customer-specific migration decision process has performed its function(s), the monitoring service/process generates new customer-specific migration decision process(es).

39. The method of claim 38 wherein the new customer-specific migration decision process(es) result in a per-customer iterative balance.

40. A method for processing information related to management of a cluster-based pool of resources, the method comprising:

processing information regarding a number of infrastructure units for a customer, an indication of a number of virtual machines (VMs) for the customer, and an indication of a maximum number of VMs per resource pool;

associating each infrastructure unit from the infrastructure units for the customer is associated with hardware resources managed based on a set of predefined resource limit values;

calculating a number of resource pools for the customer based on the number of infrastructure units for the customer, the number of VMs for the customer and the maximum number of VMs per resource pool; and

processing information, based on the number of resource pools for the customer, regarding distribution of VMs of the customer among a set of VM hosts such that each resource pool for the customer is associated with VMs of the customer and is distributed to a unique VM host from the set of VM hosts.

41. The method of claim 40 further comprising:

analyzing clusters configured to hold customer data including performing infrastructure speed testing and/or VM memory map sizing of hosts.

42. The method of claim 40 wherein the customer'"'"'s resource requirement is established as a quantity of universal Infrastructure Units (IUs) calculated as a function of analysis of clusters configured to hold customer data, wherein the analysis includes performing infrastructure speed testing and/or VM memory map sizing of hosts.

43. A method for processing information related to management of a cluster-based pool of resources, the method comprising:

processing the number of machines and resources used per virtual machine (VM);

processing information regarding a number of infrastructure units (Ws) for a customer, an indication of a number of virtual machines (VMs) for the customer, and an indication of a maximum number of VMs per resource pool;

associating each infrastructure unit from the infrastructure units for the customer is associated with hardware resources managed based on a set of predefined resource limit values;

calculating a number of resource pools for the customer based on the number of infrastructure units (Us) for the customer, the number of VMs for the customer, and the maximum number of VMs per resource pool;

processing information, based on the number of resource pools for the customer, regarding distribution of VMs of the customer among a set of VM hosts such that each resource pool for the customer is associated with VMs of the customer and is distributed to a unique VM host from the set of VM hosts; and

performing allocation of the customer'"'"'s workload (VMs) to the appropriate resource pools in the infrastructure.

44. The method of claim 43 further comprising:

aggregating the number of machines and resources used per virtual machine (VM) processed into a customer workload with required attributes; and

sending the aggregated information to a capacity planning engine or process.