Distributed demand-based storage quality of service management using resource pooling

• US 10,686,724 B2
• Filed: 01/25/2016
• Issued: 06/16/2020
• Est. Priority Date: 05/31/2012
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Gulati, Ajay, Shanmuganathan, Ganesha, Varman, Peter Joseph
• Primary Examiner(s): Ghaffari, Abu Zar
• Assistant Examiner(s): Ayers, Michael W
• Application Number:
  US15/005,842
• Publication Number:
  US 20160218994A1
• Time in Patent Office: 1,604 Days
• Field of Search: None
• CPC Class Codes: G06F 2009/45575 Starting, stopping, suspending or resuming virtual machine instances
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources
• CPC Class Codes: H04L 47/805 QOS or priority aware

Abstract

A system and method for providing quality of service (QoS) for clients running on host computers to access a common resource uses a resource pool module and a local scheduler in at least one of the host computers. The resource pool module operates to compute an entitlement of each client for the common resource based on a current capacity for the common resource and demands of the clients for the common resource. In addition, the resource pool module operates to assign a portion of the computed current capacity for the common resource to a particular host computer using the computed entitlement of each client running on the particular host computer. The local scheduler operates to allocate the portion of the computed current capacity among the clients running on the particular host computer.

Claims

1. A method for providing quality of service (QoS) for clients running on host computers to access a common resource, the method comprising:

receiving information defining a hierarchical structure of nodes for resource allocation, each node representing at least one client, wherein a static reservation value of the current capacity of the common resource is assigned to a respective node of the hierarchical structure of nodes, and a dynamic reservation value of the current capacity of the common resource is periodically computed for the respective node, the respective node having a parent node in the hierarchical structure of nodes;

determining that the dynamic reservation value computed for a particular time interval is greater than the static reservation value, the dynamic reservation value being computed by distribution of a global reservation value of the current capacity of the common resource allocated to a root node through the hierarchical structure of nodes, such that a corresponding dynamic reservation value is computed for the particular time interval for each node in the hierarchical structure of nodes, wherein the dynamic reservation value computed for the respective node is based on;

the dynamic reservation value computed for the parent node of the respective node, and an average number of inputs/outputs per second performed at the respective node; and

allocating, for the particular time interval, a portion of the current capacity of the common resource to the respective node based on the dynamic reservation value.

2. The method of claim 1, wherein the current capacity for the common resource is allocated by the common resource being distributed among the clients based on shares assigned to the clients.

3. The method of claim 1, further comprising:

computing demand of each client running on a particular host computer based on;

a local average latency for accessing the common resource from the particular host computer, and the average number of inputs/outputs per second in response to requests to access the common resource from that client.

4. The method of claim 3, further comprising:

computing a current capacity for the common resource based on a global average latency for accessing the common resource by the clients.

5. The method of claim 4, further comprising at least one of:

storing the computed demand in a shared file that is accessible by each of the host computers, or

transmitting the computed demand to other host computers in connection with the particular host computer.

6. The method of claim 4, wherein the current capacity for the common resource is computed using:

the global average latency, a smoothing parameter, and a resource congestion threshold value.

7. The method of claim 1, further comprising:

adjusting a depth of a host queue of a particular host computer, the host queue being used to store outstanding requests for the common resource from the clients running on the particular host computer.

8. A system comprising:

at least one processor;

a plurality of clients operably connected to the at least one processor and running on host computers;

a resource interface with a host queue to store requests from the clients to access a common resource;

a resource pool module operably connected to the at least one processor, the resource pool module, when executed, causing the at least one processor to;

receive information defining a hierarchical structure of nodes for resource allocation, each node representing at least one client, wherein a static reservation value of the current capacity of the common resource is assigned to a respective node of the hierarchical structure of nodes, and a dynamic reservation value of the current capacity of the common resource is periodically computed for the respective node, the respective node having a parent node in the hierarchical structure of nodes;

determine that the dynamic reservation value computed for a particular time interval is greater than the static reservation value, the dynamic reservation value being computed by distribution of a global reservation value of the current capacity of the common resource allocated to a root node through the hierarchical structure of nodes, such that a corresponding dynamic reservation value is computed for the particular time interval for each node in the hierarchical structure of nodes, wherein the dynamic reservation value computed for the respective node is based on;

the dynamic reservation value computed for the parent node of the respective node, and an average number of inputs/outputs per second performed at the respective node; and

a scheduler operably connected to the resource pool module, the scheduler being configured to allocate, for the particular time interval, a portion of the current capacity of the common resource to the respective node based on the dynamic reservation value.

9. The system of claim 8, wherein the current capacity for the common resource is allocated by the common resource being distributed among the clients based on shares assigned to the clients.

10. The system of claim 8, wherein the resource pool module is further configured to compute demand of each client running on a particular host computer based on:

a local average latency for accessing the common resource from the particular host computer, and the average number of inputs/outputs per second in response to requests to access the common resource from that client.

11. The system of claim 10, wherein the resource pool module is further configured to compute the current capacity for the common resource based on a global average latency for accessing the common resource by the clients.

12. The system of claim 11, wherein the resource pool module is further configured to:

store the computed demand in a shared file that is accessible by each of the host computers, or transmit the computed demand to other host computers in connection with the particular host computer.

13. The system of claim 11, wherein the current capacity for the common resource is computed based on:

the global average latency, a smoothing parameter, and a resource congestion threshold value.

14. The system of claim 8, wherein the scheduler is further configured to adjust a depth of a host queue of a particular host computer, the host queue being used to store outstanding requests for the common resource from the clients running on the particular host computer.

15. A non-transitory computer-readable storage medium containing program instructions for providing quality of service (QoS) for clients running on host computers to access a common resource, the program instructions, when executed by one or more processors, causing the one or more processors to:

receive information defining a hierarchical structure of nodes for resource allocation, each node representing at least one client, wherein a static reservation value of the current capacity of the common resource is assigned to a respective node of the hierarchical structure of nodes, and a dynamic reservation value of the current capacity of the common resource is periodically computed for the respective node, the respective node having a parent node in the hierarchical structure of nodes;

determine that the dynamic reservation value computed for a particular time interval is greater than the static reservation value, the dynamic reservation value being computed by distribution of a global reservation value of the current capacity of the common resource allocated to a root node through the hierarchical structure of nodes, such that a corresponding dynamic reservation value is computed for the particular time interval for each node in the hierarchical structure of nodes, wherein the dynamic reservation value computed for the respective node is based on;

the dynamic reservation value computed for the parent node of the respective node, and an average number of inputs/outputs per second performed at the respective node; and

allocate, for the particular time interval, a portion of the current capacity of the common resource to the respective node based on the dynamic reservation value.

16. The non-transitory computer-readable storage medium of claim 15, wherein the current capacity for the common resource is allocated by the common resource being distributed among the clients based on shares assigned to the clients.

17. The non-transitory computer-readable storage medium of claim 15, wherein the program instructions, when executed, further cause the one or more processors to compute demand of each client running on a particular host computer based on:

a local average latency for accessing the common resource from the particular host computer, and the average number of inputs/outputs per second in response to requests to access the common resource from that client.

18. The non-transitory computer-readable storage medium of claim 17, wherein the program instructions, when executed, further cause the one or more processors to compute the current capacity for the common resource based on a global average latency for accessing the common resource by the clients.

19. The non-transitory computer-readable storage medium of claim 18, wherein the program instructions, when executed, further cause the one or more processors to store the computed demand in a shared file that is accessible by each of the host computers or transmit the computed demand to other host computers in connection with the particular host computer.

20. The non-transitory computer-readable storage medium of claim 18, wherein the current capacity for the common resource is computed based on the global average latency, a smoothing parameter and a resource congestion threshold value.

21. The non-transitory computer-readable storage medium of claim 15, wherein the program instructions, when executed, further cause the one or more processors to assign a portion of the current capacity for the common resource to a particular host computer, which includes adjusting a depth of a host queue of the particular host computer, the host queue being used to store outstanding requests for the common resource from the clients running on the particular host computer.