Method and apparatus for utility-based dynamic resource allocation in a distributed computing system

• US 8,352,951 B2
• Filed: 06/30/2008
• Issued: 01/08/2013
• Est. Priority Date: 01/30/2004
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Das, Rajarshi, Kephart, Jeffrey Owen, Tesauro, Gerald James, Walsh, William Edward
• Primary Examiner(s): Tang, Kenneth
• Application Number:
  US12/164,896
• Publication Number:
  US 20080263559A1
• Time in Patent Office: 1,653 Days
• Field of Search: None
• US Class Current: 718/104
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system

Abstract

In one embodiment, the present invention is a method for allocation of finite computational resources amongst multiple entities, wherein the method is structured to optimize the business value of an enterprise providing computational services. One embodiment of the inventive method involves establishing, for each entity, a service level utility indicative of how much business value is obtained for a given level of computational system performance. The service-level utility for each entity is transformed into a corresponding resource-level utility indicative of how much business value may be obtained for a given set or amount of resources allocated to the entity. The resource-level utilities for each entity are aggregated, and new resource allocations are determined and executed based upon the resource-level utility information. The invention is thereby capable of making rapid allocation decisions, according to time-varying need or value of the resources by each of the entities.

Claims

1. An automated method for allocating resources among a plurality of resource-using computational entities in a data processing system, the method comprising:

establishing a service-level utility for each of said plurality of resource-using computational entities; and

transforming said service-level utility into a resource-level utility for each of said plurality of resource-using computational entities,

wherein the resource-level utility is representative of an amount of business value obtained by each of said plurality of resource-using computational entities when a quantity of said resources is allocated to the each of said plurality of resource-using computational entities,

wherein the resource-level utility indicates, for at least one of said plurality of resource-using computational entities, an estimated cumulative discounted or undiscounted future utility starting from current state descriptions of said at least one of said plurality of resource-using computational entities,

wherein the estimated cumulative discounted or undiscounted future utility is trained on a temporal sequence of observed data using an adaptive machine learning procedure,

wherein the machine learning procedure is a reinforcement learning procedure,

and wherein the reinforcement learning procedure is Q-Learning, Temporal Difference Learning, R-Learning or SARSA.

2. The method of claim 1, wherein the resource-level utility is a function of client demand received by one of said plurality of resource-using computational entities and of a service-level agreement governing the performance of said one of said plurality of resource-using computational entities.

3. The method of claim 1, wherein the service-level utility is representative of an amount of business value obtained by each of said plurality of resource-using computational entities for various levels of performance and demand associated with the each of said plurality of resource-using computational entities.

4. The method of claim 1, further comprising steps of:

aggregating said resource-level utilities of all of said plurality of resource-using computational entities to produce aggregated utility information; and

computing a resource allocation from the aggregated utility information.

5. The method of claim 4, further comprising a step of:

executing and conveying to the plurality of resource-using computational entities said resource allocation.

6. The method of claim 4, wherein the aggregating said resource-level utilities of all of said plurality of resource-using computational entities is initiated by said plurality of resource-using computational entities.

7. The method of claim 4, wherein the aggregating said resource-level utilities of all of said plurality of resource-using computational entities is initiated by at least one resource arbiter adapted to compute said resource allocation from the aggregated utility information.

8. The method of claim 4, wherein the computing a resource allocation from the aggregated utility information comprises executing an optimization method to maximize a total utility of said data processing system.

9. The method of claim 8, wherein said optimization method comprises a standard linear or nonlinear algorithm.

10. The method of claim 9, wherein said optimization method is hill climbing, simulated annealing, linear programming or mixed-integer programming.

11. The method of claim 4, wherein the computing a resource allocation from the aggregated utility information comprises computing a cost that may be incurred in reallocating at least one of said resources from one of said plurality of resource-using computational entities to another of said plurality of resource-using computational entities.

12. The method of claim 1, wherein at least one of said plurality of resource-using computational entities operates to set its internal parameters, or an adjustable parameter of the resources the at least one of said plurality of resource-using computational entities has been allocated so as to optimize the service-level utility, the resource-level utility, or both.

13. The method of claim 1, wherein the estimated cumulative discounted or undiscounted future utility is based, for at least one of said plurality of resource-using computational entities, upon predictions of future state descriptions of said at least one of said plurality of resource-using computational entities.

14. A non-transitory computer readable medium containing an executable program for allocating resources among a plurality of resource-using computational entities in a data processing system, where the program performs steps of:

establishing a service-level utility for each of said plurality of resource-using computational entities; and

transforming said service-level utility into a resource-level utility for each of said plurality of resource-using computational entities,

wherein the resource-level utility is representative of an amount of business value obtained by each of said plurality of resource-using computational entities when a quantity of said resources is allocated to the each of said plurality of resource-using computational entities,

wherein the resource-level utility indicates, for at least one of said plurality of resource-using computational entities, an estimated cumulative discounted or undiscounted future utility starting from current state descriptions of said at least one of said plurality of resource-using computational entities,

wherein the estimated cumulative discounted or undiscounted future utility is trained on a temporal sequence of observed data using an adaptive machine learning procedure,

wherein the machine learning procedure is a reinforcement learning procedure,

and wherein the reinforcement learning procedure is Q-Learning, Temporal Difference Learning, R-Learning or SARSA.

15. The non-transitory computer readable medium of claim 14, wherein said program further performs steps of:

aggregating said resource-level utilities of all of said plurality of resource-using computational entities to produce aggregated utility information; and

computing a resource allocation from the aggregated utility information.

16. The non-transitory computer readable medium of claim 15, wherein said program further performs a step of:

executing and conveying to the plurality of resource-using computational entities said resource allocation.

17. The non-transitory computer readable medium of claim 16, wherein computing the resource allocation from the aggregated utility information comprises executing an optimization algorithm to maximize a business value of said data processing system.

18. The non-transitory computer readable medium of claim 14, wherein at least one of said plurality of resource-using computational entities operates to set its internal parameters, or an adjustable parameter of the resources the at least one of said plurality of resource-using computational entities has been allocated so as to optimize the service-level utility, the resource-level utility, or both.

19. A data processing system, comprising:

a plurality of processors adapted for processing client demands;

a plurality of resources adapted for allocation to said plurality of processors; and

at least one resource arbiter adapted for allocating said plurality of resources among said plurality of processors in a manner that optimizes a business value of the data processing system, wherein said at least one resource arbiter performs operations comprising;

establishing a service-level utility for each of said plurality of processors; and

transforming said service-level utility into a resource-level utility for each of said plurality of processors,

wherein the resource-level utility is representative of an amount of business value obtained by each of said plurality of processors when a quantity of said plurality of resources is allocated to the each of said plurality of processors,

wherein the resource-level utility indicates, for at least one of said plurality of processors, an estimated cumulative discounted or undiscounted future utility starting from current state descriptions of said at least one of said plurality of processors,

wherein the estimated cumulative discounted or undiscounted future utility is trained on a temporal sequence of observed data using an adaptive machine learning procedure,

wherein the machine learning procedure is a reinforcement learning procedure,

and wherein the reinforcement learning procedure is Q-Learninq, Temporal Difference Learning, R-Learninq or SARSA.

20. The data processing system of claim 19, wherein said plurality of processors and said at least one resource arbiter are run on a single computer.

21. The data processing system of claim 19, wherein said plurality of processors and said at least one resource arbiter are run on different computers connected by a network.

22. The data processing system of claim 19, wherein said plurality of processors and said at least one resource arbiter are software modules comprising autonomic elements.

23. The data processing system of claim 19, wherein the data processing system is a server, a client computer or a network.