Method and system for multi-tenant resource distribution

• US 10,609,129 B2
• Filed: 04/29/2016
• Issued: 03/31/2020
• Est. Priority Date: 02/19/2016
• Status: Active Grant

• Current Assignee: Huawei Cloud Computing Technologies Company Limited (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Lam, Jason T. S., Chen, Chong, Guo, Lei, Ke, Xiaodi
• Primary Examiner(s): Springer, James E
• Application Number:
  US15/142,371
• Publication Number:
  US 20170244784A1
• Time in Patent Office: 1,432 Days
• Field of Search: 709226
• CPC Class Codes: G06F 9/505 considering the load
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30

Abstract

In a distributed computing network, requests for allocation of resources to tenant workloads and messages identifying resource availability are received and aggregated. Resources are allocated to the workloads in accordance with a distribution policy defining values for resource entitlements of the tenants. The values include pre-emption quantities. In response to determining that a quantity of resources allocated for workloads of a first tenant is less than the tenant'"'"'s pre-emption quantity, processing of another workload from a second tenant is interrupted to re-allocate resources from the second tenant'"'"'s workload to the first tenant'"'"'s workload.

Claims

1. A method of resource allocation in a distributed computing system comprising a plurality of tenants organized in a tiered hierarchy, the plurality of tenants including a root tenant associated with at least two sub-tenants, and each sub-tenant of the two sub-tenants associated with one or more corresponding leaf tenants, the method comprising:

receiving, at a distributed resource manager, a plurality of requests from said plurality of tenants for allocation of quantities of resources to workloads of said tenants;

allocating, at the distributed resource manager, said quantities of resources to said workloads of said plurality of tenants in accordance with a distribution policy, the distribution policy defining a resource entitlement for each of said one or more corresponding leaf tenants and defining a hierarchical resource entitlement for each of the at least two sub-tenants, said resource entitlement for each of the plurality of tenants including a guaranteed quantity of resources for said each of the plurality of tenants; and

in response to determining, at the distributed resource manager, that a first quantity of resources allocated for workloads of a first sub-tenant of said plurality of tenants is less than the guaranteed quantity of resources for said first sub-tenant;

selecting, by the distributed resource manager, a second sub-tenant from among said plurality of tenants based on a comparison of said guaranteed quantity of resources for said second sub-tenant and a quantity of resources allocated to workloads of said second sub-tenant, and

interrupting processing of a workload of said second sub-tenant prior to completing processing of said workload of said second sub-tenant and re-allocating the quantity of resources allocated to said workload of said second sub-tenant to a workload of said workloads of said first sub-tenant.

2. The method of claim 1, wherein said resource entitlement for the each of the plurality of tenants comprises a reserve quantity of resources for the each of the plurality of tenants, and wherein allocating quantities of resources comprises allocating said reserve quantity of resources to workloads of each of said plurality of tenants independently of a quantity of resources requested by said workloads of each of said plurality of tenants.

3. The method of claim 1, wherein said resource entitlement for the each of the plurality of tenants includes a maximum quantity of resources for the each of the plurality of tenants, and wherein said allocating comprises allocating quantities of resources to workloads of each tenant its maximum quantity of resources, wherein said maximum quantity of resources is less than a quantity of idle resources available for allocation to workloads of said plurality of tenant.

4. The method of claim 1, wherein said distribution policy comprises resource allocation weightings for users belonging to ones of said tenants.

5. The method of claim 4, wherein said allocating quantities of resources comprises allocating said quantities of resources to workloads for said users according to said resource allocation weightings.

6. The method of claim 1, wherein said guaranteed quantity of resources for each tenant comprises a proportional share of available resources.

7. The method of claim 1, wherein said guaranteed quantity of resources for each tenant comprises an absolute value defining a number of resource units.

8. The method of claim 1, wherein said guaranteed quantity of resources for each tenant comprises a proportional value of available resources and an absolute value defining a number of resource units.

9. The method of claim 1, wherein said distributed computing system comprises resources in a plurality of resource pools, and wherein said allocating comprises allocating resources of a first resource pool to a first set of said plurality of tenants and allocating resources of a second resource pool to a second set of said plurality of tenants.

10. The method of claim 9, wherein at least one tenant is part of both said first set of said plurality of tenants and said second set of said plurality of tenants.

11. The method of claim 9, further comprising defining a data structure defining a distribution policy for each tenant of each one of said first and second resource pools.

12. A master server of a distributed computing system comprising a plurality of resource servers, the master server comprising:

a resource collection module for receiving messages identifying available resources associated with said resource servers;

a demand collection module for receiving messages identifying resource requests from a plurality of tenants of said distributed computing system, the plurality of tenants organized in a tiered hierarchy, the plurality of tenants including a root tenant associated with at least two sub-tenants, and each sub-tenant of the two sub-tenants associated with one or more corresponding leaf tenants;

a data structure comprising a distribution policy for said available resources, said distribution policy containing a resource entitlement for each of said one or more corresponding leaf tenants and defining a hierarchical resource entitlement for each of the at least two sub-tenants, said resource entitlement for each of the plurality of tenants including a guaranteed quantity of resources for said each of the plurality of tenants;

a distributed resource manager for;

determining that a first quantity of resources allocated for workloads of a first sub-tenant of said plurality of tenants is less than the guaranteed quantity of resources for said first sub-tenant;

selecting a second sub-tenant from among said plurality of tenants based on a comparison of said guaranteed quantity of resources for said second sub-tenant and a quantity of resources allocated to workloads of said second sub-tenant; and

interrupting processing a workload of said second sub-tenant prior to completing processing of said workload of said second sub-tenant and re-allocating the quantity of resources allocated to said workload of said second sub-tenant to a workload of said workloads of said first sub-tenant.

13. The master server of claim 12, wherein said resource entitlement for the each of the plurality of tenants comprises a reserve quantity of resources for the each of the plurality of tenants, and wherein allocating said quantities of resources comprises allocating said reserve quantity of resources to workloads of each of said plurality of tenants independently of a quantity of resources requested by said workloads of each of said tenants.

14. The master server of claim 12, wherein said resource entitlement for the each of the plurality of tenants includes a maximum quantity of resources for the each of the plurality of tenants, wherein said quantities of resources to workloads of each tenant are allocated its maximum quantity of resources, and wherein said maximum quantity of resources is less than a quantity of idle resources available for allocation to workloads of the each of the plurality of tenants.

15. The master server of claim 12, wherein said distribution policy comprises resource allocation weightings for users belonging to ones of said tenants.

16. The master server of claim 12, wherein said guaranteed quantity of resources for each tenant comprises a proportional share of available resources.

17. The master server of claim 12, wherein said guaranteed quantity of resources for each tenant comprises an absolute value defining a number of resource units.

18. The master server of claim 12, wherein said guaranteed quantity of resources for each tenant comprises a proportional value of available resources and an absolute value defining a number of resource units.

19. The master server of claim 12, wherein said available resources comprise resources in a plurality of resource pools.

20. The master server of claim 19, wherein said data structure comprises a distribution policy for each tenant of each one of said plurality of resource pools.

21. The master server of claim 20, wherein said distribution policy for a first resource pool of said plurality of resource pools defines a first set of said plurality of tenants with access to said first resource pool of said plurality of resource pools, and said distribution policy for a second resource pool of said plurality of resource pools defines a second set of said plurality of tenants with access to said second resource pool of said plurality of resource pools.

22. The master server of claim 21, wherein at least one tenant is part of both a first group and a second group.

23. A non-transitory computer-readable medium storing instructions which when executed by at least one processor causes the at least one processor to:

receive a plurality of requests from a plurality of tenants for allocation of quantities of resources to workloads of said tenants, the plurality of tenants organized in a tiered hierarchy, the plurality of tenants including a root tenant associated with at least two sub-tenants, and each sub-tenant of the two sub-tenants associated with one or more corresponding leaf tenants;

allocate said quantities of resources to said workloads of said plurality of tenants in accordance with a distribution policy, the distribution policy defining a resource entitlement for each of said one or more corresponding leaf tenants and defining a hierarchical resource entitlement for each of the at least two sub-tenants, said resource entitlement for each of the plurality of tenants including a guaranteed quantity of resources for said each of the plurality of tenants; and

in response to determining that a first quantity of resources allocated for workloads of a first sub-tenant of said plurality of tenants is less than the guaranteed quantity of resources for said first sub-tenant;

selecting a second sub-tenant from among said plurality of tenants based on a comparison of said guaranteed quantity of resources for said second sub-tenant and a quantity of resources allocated to workloads of said second sub-tenant, and

interrupting processing of a workload of said second sub-tenant prior to completing processing of said workload of said second sub-tenant and re-allocating the quantity of resources allocated to said workload of said second sub-tenant to a workload of said workloads of said first sub-tenant.