System and method for providing dynamic provisioning within a compute environment

• US 8,037,475 B1
• Filed: 06/17/2005
• Issued: 10/11/2011
• Est. Priority Date: 06/18/2004
• Status: Active Grant

• Current Assignee: III Holdings 12, LLC (Intellectual Ventures LLC)
• Original Assignee: Adaptive Computing Enterprises, Inc. (Ala Services LLC)
• Inventors: Jackson, David Brian
• Primary Examiner(s): An, Meng
• Assistant Examiner(s): LEE, TAMMY EUNHYE
• Application Number:
  US11/155,091
• Time in Patent Office: 2,307 Days
• Field of Search: 718/102, 718/104, 718/105, 709/226, 709/228, 709/216, 709/223, 709/224
• US Class Current: 718/104
• CPC Class Codes: G06F 2209/5014 Reservation
• CPC Class Codes: G06F 2209/504 Resource capping
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5005 to service a request
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5022 Mechanisms to release resources
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: H04L 41/00 Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
• CPC Class Codes: H04L 41/50 Network service management, e.g. ensuring proper service fulfilment according to agreements
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The disclosure relates to systems, methods and computer-readable media for dynamically provisioning resources within a compute environment. The method aspect of the disclosure comprises A method of dynamically provisioning resources within a compute environment, the method comprises analyzing a queue of jobs to determine an availability of compute resources for each job, determining an availability of a scheduler of the compute environment to satisfy all service level agreements (SLAs) and target service levels within a current configuration of the compute resources, determining possible resource provisioning changes to improve SLA fulfillment, determining a cost of provisioning; and if provisioning changes improve overall SLA delivery, then re-provisioning at least one compute resource.

Claims

1. A method of managing provisioning resources in a compute environment, the method comprising:

analyzing a queue of jobs at a current time to determine an availability of compute resources within the compute environment for each job based on a reservation of compute resources established for each job to yield reserved compute resources, the each job to consume the reserved compute resources at a future time, which is later than the current time, to yield an analysis;

based on the analysis, determining an ability of the compute environment to satisfy all service level agreements that identify promised service levels to users of the compute environment and target service levels within a current configuration of the compute environment;

determining possible resource provisioning changes in the compute environment to improve service level fulfillment for jobs to be processed in the compute environment;

determining a cost of the possible resource provisioning changes; and

if the possible resource provisioning changes improve overall delivery of service levels within the compute environment based on the cost, then provisioning at least one compute resource in the compute environment prior to all the jobs consuming compute resources in the compute environment and according to a generated schedule of times.

2. The method of claim 1, wherein the step of determining if provisioning changes improve overall service level delivery is subject to provisioning costs in both long term and short term.

3. The method of claim 1, further comprising provisioning compute resources as their scheduled time arrives.

4. The method of claim 1, wherein availability is determined in terms of at least one of:

a required OS, hardware architecture, network adapters, real memory, virtual memory, internal and external disk storage, and software applications.

5. The method of claim 1, wherein availability is further constrained by at least one of:

a resource state, a reservation and other political policies.

6. The method of claim 1, further comprising determining a priority of jobs to compute environment objectives and a time frame in which jobs must complete to fulfill SLAs.

7. The method of claim 1, wherein only certain jobs are enabled to receive provisioning services.

8. The method of claim 1, further comprising, if the compute environment will be provisioned:

analyzing a workload schedule associated a node or nodes to be provisioned;

selecting at least one node that can be provisioned with a minimal impact;

when the at least one node becomes available, instructing a provisioning service to provision the at least one node.

9. The method of claim 8, wherein provisioning further comprises creating a reservation for the at least one node to be provisioned.

10. The method of claim 1, wherein provisioning at least one compute resource further comprises provisioning a group of nodes within the compute environment.

11. The method of claim 1, wherein provisioning at least one compute resource further comprises provisioning individual nodes piecemeal within the compute environment.

12. The method of claim 11, wherein provisioning individual nodes piecemeal within the compute environment is based on workload within the compute environment.

13. The method of claim 1, further comprising releasing the at least one compute resource back to a general availability.

14. The method of claim 13, wherein a scheduler releases the at least one compute resource back to a general availability occurs according to one of the following time frames:

when a provisioning manager confirms that the provisioning is complete;

if a provisioning scheduled time is set, ahead of the scheduled time if the provisioning manager confirms that the provisioning is complete; and

if a re-provisioning scheduled time is set, after the scheduled time if the provisioning manager confirms that the provisioning is not complete or failed.

15. The method of claim 13, wherein reservations are used to control resource access and schedule resource provisioning in time.

16. The method of claim 15, wherein a scheduler will release resources from reservations prior to a reservation end time if a provisioning manager confirms successful completion of provisioning.

17. The method of claim 16, wherein the scheduler will extend the reservation while the provisioning manager confirms failure of provisioning.

18. The method of claim 1, further comprising:

analyzing historical information of the compute environment; and

determining whether to provision the compute environment based at least in part on the historical information.

19. The method of claim 18, wherein the historical information is at least one of compute environment performance information, job-submission information, failure rate information and a combination of these types of information.

20. The method of claim 18, wherein the historical information is one of short term or long term historical information.

21. The method of claim 18, wherein instructions to the provisioning service are modified based on the analyzed historical information.

22. The method of claim 21, wherein the modifications to the instructions to the provisioning service are one of speeding up or slowing down the provisioning of the compute environment based on the historical information.

23. The method of claim 21, wherein the modifications to the instructions to the provisioning service modify how the provisioning occurs based on the historical information.

24. The method of claim 1, further comprising:

determining whether a work backlog exists and if so, analyzing the work backlog and including the work backlog analysis in the determination of whether the cost associated with provisioning justifies an overhead amount in comparison to a higher efficiency of the compute environment which would result from the provisioning.

25. The method of claim 1, wherein if the cost to provision crosses a threshold value, then the method further comprises:

analyzing historical information of the compute environment; and

determining whether to provision the compute environment based at least in part on the historical information.

26. The method of claim 25, further comprising analyzing failure probabilities for the provisioning and including the failure probability analysis in the determination of whether the cost associated with provisioning justifies an overhead amount in comparison to a higher efficiency of the compute environment which would result from the provisioning.

27. A non-transitory computer-storage medium storing instructions for controlling a computing device to manage a managing a compute environment, the instructions comprising:

analyzing a queue of jobs at a current time to determine an availability of compute resources within the compute environment for each job based on a reservation of compute resources established for each job to yield reserved compute resources, the each job to consume the reserved compute resources at a future time, which is later than the current time, to yield an analysis;

based on the analysis, determining an ability of the compute environment to satisfy all service level agreements that identify promised service levels to users of the compute environment and target service levels within a current configuration of the compute environment;

determining possible resource provisioning changes in the compute environment to improve service level fulfillment for jobs to be processed in the compute environment;

determining a cost of the possible resource provisioning changes; and

if the possible resource provisioning changes improve overall delivery of service levels within the compute environment based on the cost, then provisioning at least one compute resource in the compute environment prior to all the jobs consuming compute resources in the compute environment and according to a generated schedule of times.

28. A system that manages a compute environment, the system comprising:

a processor;

a first module that controls the processor to analyze a queue of jobs at a current time to determine an availability of compute resources within the compute environment for each job based on a reservation of compute resources established for each job to consume the reserved compute resources at a future time according to the reservation to yield an analysis;

a second module, based on the analysis, that controls the processor to determine an ability of the compute environment to satisfy (1) all service level agreements (SLAs) that identify promised service levels to users of the compute environment and (2) target service levels within a current configuration of the compute environment;

a third module that controls the processor to determine possible resource provisioning changes in the compute environment to improve service level fulfillment for jobs to be processed in the compute environment;

a fourth module that controls the processor to determine a cost of the possible resource provisioning changes; and

a fifth module that controls the processor to provision at least one compute resource in the compute environment prior to all the jobs consuming compute resources in the compute environment and according to a generated schedule of times, if the possible resource provisioning changes improve overall delivery of service levels within the compute environment based on the cost.

29. A method of managing a compute environment, the method comprises:

establishing at a current time a reservation for compute resources in a multi-node compute environment, the reservation reserving at a future time, which is later than the current time, compute resources for a job to consume;

analyzing the compute environment and the reservation to determine whether additional resources can be provisioned to improve how well the multi-node compute environment matches the job;

determining whether a cost of provisioning additional resources outweighs any improvement in how well the multi-node compute environment matches the job; and

if the cost does not outweigh the improvement, then provisioning the additional resources for the job to consume the compute resources, wherein the provisioning occurs prior to the job consuming the compute resources and according to a generated schedule of times.

30. A method of managing a compute environment, the method comprising:

analyzing, at a first time, the compute environment for a job to determine a possibility of re-provisioning at least part of the compute environment for the job, wherein a reservation of compute resources exists for the job to consume reserved compute resources at a second time, which is later than the first time;

if re-provisioning at least part of the compute environment for the job is possible, then determining a cost of the re-provisioning; and

if the cost of the re-provisioning does not outweigh an improvement in the provisioned service level, then reserving new compute resources in the compute environment according to a generated schedule of times to yield reserved new compute resources and re-provisioning at least some of the reserved new compute resources to yield reserved, re-provisioned compute resources, wherein the job consumes the reserved, re-provisioned compute resources.