Dynamic Cost Model Based Resource Scheduling In Distributed Compute Farms

• US 20120290725A1
• Filed: 05/09/2011
• Published: 11/15/2012
• Est. Priority Date: 05/09/2011
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Podila, Sharma R.
• Granted Patent: US 8,583,799 B2
• Time in Patent Office: Days
• US Class Current: 709/226
• CPC Class Codes: G06F 2209/503 Resource availability
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals

Abstract

Systems, methods and apparatuses (i.e., utilities) for use in performing dynamic cost model (DCM)-based scheduling of resources of a distributed computing system (e.g., server farms, computing clouds, and the like). In one arrangement, cost for consumption of a resource or type of resource of the system may be dynamically determined as a function of both a demand for and an availability of the resource. In this regard, changing demand for and/or availability of the resource over time can also adjust a cost for consumption of the resource. For instance, assigning a lower consumption cost to utilizing of a resource in low demand can benefit system operators by realizing increased use on little-used resources as well as users by allowing the users to take advantage of lower usage costs.

Claims

1. A method, comprising:

receiving, at a resource scheduling module of a distributed processing system, at least one request to consume a first resource of the distributed processing system by at least a first workload item of a first project, wherein a plurality of the first resource are distributed throughout the distributed processing system;

for each of the plurality of the first resource;

obtaining a demand for consumption of the resource by workload items;

obtaining an availability for consumption of the resource by workload items; and

establishing a cost for consuming the resource by obtaining, using a processor, a product of a) a cost structure of the resource and b) a quotient of the demand for and availability of the first type of resource;

incorporating the first workload item into a schedule that allocates use of the plurality of the first resource among workload items based at least partially on the consumption costs; and

allocating use of the plurality of the first resource among the workload items according to the schedule.

2. The method of claim 1, wherein a consumption cost for a first of the plurality of the first resource is less than a consumption cost for a second of the plurality of the first resource, and wherein the incorporating comprises:

allocating use of the first of the plurality of the first resource by the first workload item.

3. The method of claim 1, further comprising:

obtaining at least one of an updated demand and an updated availability in relation to at least one of the plurality of the first resource;

establishing an updated cost for consuming the one of the plurality of the first resource by obtaining, using the processor, a product of a) its first cost structure and b) a quotient of the updated demand for and/or updated availability; and

updating the schedule by at least partially utilizing the updated consumption cost.

4. The method of claim 1, wherein the consumption request is received from a first client over at least one network, and wherein the method further comprises:

sending the consumption cost to the first client over the at least one network; and

receiving a message from the first client that approves allocation of the resource by the first workload item according to the schedule.

5. The method of claim 1, wherein the consumption request is in relation to consuming the resource during a first time period, and wherein the demand and availability are in relation to the first time period.

6. The method of claim 1, wherein the first project comprises a first share entitlement of the resource, and wherein the method further comprises:

obtaining, using the processor, a quotient of a) the consumption cost for at least one of the plurality of the first resource and b) the first share entitlement of the first project; and

utilizing the obtained quotient to determine a priority for consumption of the first resource by the first workload item relative to at least one other workload item.

7. The method of claim 6, further comprising:

receiving, at the resource scheduling module of the distributed processing system, a request to consume the first resource by at least a first workload item of a second project;

establishing costs for consuming each of the plurality of the first resource by obtaining, using a processor, a product of a) its cost structure and b) a quotient of the demand for and availability of the resource.

obtaining, using the processor, a quotient of a) a consumption cost for at least one of the plurality of the first resource and b) a second share entitlement of the second project;

determining a lowest of the obtained quotient for the first project and second project; and

assigning priority for consumption of the first resource to the first workload item of the first or second project determined to have the lowest obtained quotient.

8. The method of claim 1, further comprising:

receiving, at the resource scheduling module, at least one request to consume a second resource of the distributed processing system by the first workload item of the first project, wherein a plurality of the second resource are distributed throughout the distributed processing system;

for each of the plurality of the second resource;

obtaining a demand for consumption of the resource by workload items;

obtaining an availability for consumption of the resource by workload items; and

establishing a cost for consuming the resource by obtaining, using a processor, a product of a) a cost structure of the resource and b) a quotient of the demand for and availability of the first type of resource, wherein a cost for consuming resources of the compute farm by the first project comprises at least the costs for consuming the first and second resources.

9. The method of claim 8, wherein cost structures of the plurality of the first resource are different than cost structures of the plurality of the second resource.

10. The method of claim 1, wherein the first resource comprises at least one of memory modules, processing modules, storage devices, network modules, software, and input/output modules.

11. A non-transitory computer readable medium that stores instructions executable by one or more processors to perform a method that comprises:

receiving, at a resource scheduling module of at least a first computing grid, requests to consume a first type of resource of the first computing grid by a plurality of workload items of a first user group and by a plurality of workload items of a second user group of the plurality of user groups;

obtaining, at the resource scheduling module, share entitlements of the first type of resource describing a quantity of use of the first type of resource by the first and second user groups, respectively;

dynamically determining costs for consumption of the first type of resource by each of the workload items of the first and second user groups;

summing the consumption costs of the workload items of the first user group and the consumption costs of the workload items of the second user group to respectively obtain usage costs for the first and second user groups; and

allocating use of the first type of resource between the first and second user groups based on the obtained usage costs and share entitlements.

12. The computer readable medium of claim 11, wherein the dynamically determined costs utilize at least one of a) a demand for the first type of resource by workload items and b) an availability for consumption of the first type of resource by workload items.

13. The computer readable medium of claim 11, wherein the allocating comprises:

obtaining a first quotient of the usage cost and share entitlement of the first user group;

obtaining a second quotient of the usage cost and share entitlement of the second user group;

determining a lower of the first and second quotient; and

allocating use of the first type of resource to the one of the first or second user group associated with the lower of the first and second quotient before allocating use to the other of the first or second user group.

14. The computer readable medium of claim 11, wherein the method further comprises:

receiving, at the resource scheduling module, requests to consume a second type of resource of the first computing grid by a plurality of workload items of the first and second user groups;

obtaining, at the resource scheduling module, share entitlements of a second type of resource of the first computing grid for the first and second user groups, respectively;

dynamically determining costs for consumption of the second type of resource by each of the workload items of the first and second user groups;

summing the costs for consumption of the second type of resource of the workload items of the first user group and the costs for consumption of the second type of resource of the workload items of the second user group to respectively obtain usage costs of the second type of resource for the first and second user groups; and

allocating use of the second type of resource between the first and second user groups based on the obtained usage costs and share entitlements.

15. The computer readable medium of claim 14, wherein determination of allocation of use of the second type of resource between the first and second user groups is free from utilization of the usage costs in relation to the first type of resources for the first and second user groups.

16. The computer readable medium of claim 11, wherein the share entitlements of the first and second user groups for the first type of resource are hierarchically related to share entitlements of other user groups for the first type of resource.

17. The computer readable medium of claim 11, wherein the method further comprises:

providing, to the first and/or second user group, access to the first type of resource of the first computing grid according to the determined allocation.

18. A computing system for use in scheduling usage of a plurality of resources of a computing cloud by a plurality of workload items, comprising

a processing module;

and

a memory module logically connected to the processing module and comprising a set of computer readable instructions executable by the processing module to;

receive a request to consume a first type of resource of the computing cloud by at least a first workload item of a first project;

obtain from the computing cloud a demand for consumption of the first type of resource by workload items;

obtain from the computing cloud an availability for consumption of the first type of resource by workload items;

utilize a quotient of the demand for and availability of the first type of resource to establish a cost for consuming the first type of resource by the first workload item; and

incorporate the first workload item into a schedule that allocates use of the first type of resource among workload items based at least partially on the consumption cost.

19. The computing system of claim 18, wherein the first type of resource is associated with a first cost structure, and wherein the utilizing comprises:

determining a product of a) the first cost structure and b) the quotient to establish the consumption cost of the first type of resource by the first workload item.

20. The computing system of claim 18, wherein the first project comprises a first share entitlement of the first type of resources, and wherein the computer readable instructions are further executable by the processing module to:

obtain a quotient of a) the consumption cost for the first type of resource by the first workload item and b) the first share entitlement of the first project; and

utilize the obtained quotient to determine a priority for consumption of the first type of resource by the first workload item relative to at least one other workload item.