MULTI-STAGE IOPS ALLOCATION

• US 20200133739A1
• Filed: 10/31/2018
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: Nutanix Incorporated
• Original Assignee: Nutanix Incorporated
• Inventors: JAIN, Kshitiz, KAJARIA, Prateek
• Granted Patent: US 10,922,142 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455

Abstract

Systems and methods for policy-based apportionment of input/output operations (IOPS) in computing systems. Embodiments access a policy that specifies IOPS limits. Two or more virtual machines that are associated with the policy and two or more nodes that host those virtual machines are identified. In a first allocation stage, an inter-node policy manager prescribes an initial IOPS limit to the two or more nodes. The allocation amounts sent to the nodes depend at least in part on performance capabilities of respective nodes. In a second allocation stage, for each node that had received a limit amount, that amount is apportioned to the sets of virtual machines that execute on respective host nodes. Each node of the two or more nodes invokes its own node-local IOPS monitoring. Each node reports IOPS usage data to the inter-node policy manager, which in turn adjusts the node-level IOPS apportionments based on the node-level usage.

Claims

1. A method for multi-stage input/output operations (IOPS) allocations, the method comprising:

identifying at least one policy that specifies at least one IOPS limit;

determining two or more virtual machines associated with the at least one policy;

determining two or more nodes that host the two or more virtual machines;

performing a first allocation to apportion the at least one IOPS limit over the two or more nodes, the first allocation resulting in two or more node-level IOPS apportionments that correspond to the two or more nodes; and

performing a second allocation to apportion the node-level IOPS apportionments to the two or more virtual machines, the second allocation resulting in one or more VM-level IOPS apportionments to the two or more virtual machines.

2. The method of claim 1, further comprising:

invoking an IOPS usage monitoring function at the two or more nodes;

monitoring, by the IOPS usage monitoring function at the two or more nodes, node-specific IOPS usage data; and

adjusting at least one of, the two or more node-level IOPS apportionments, or the one or more VM-level IOPS apportionments, the adjusting being based at least in part on the node-specific IOPS usage data.

3. The method of claim 2, wherein the node-specific IOPS usage data comprises at least one predicted IOPS rate.

4. The method of claim 3, wherein the at least one predicted IOPS rate corresponds to at least one of, at least one of the two or more nodes, or at least one of the two or more virtual machines.

5. The method of claim 3, wherein the at least one predicted IOPS rate is derived from one or more average IOPS rates over a respective one or more observation periods.

6. The method of claim 5, wherein at least one of the one or more average IOPS rates is derived from an I/O operations count that corresponds to one of the respective one or more observation periods.

7. The method of claim 1, further comprising:

throttling one or more I/O operations associated with at least one of the two or more virtual machines, the throttling being based at least in part on at least one of, the one or more VM-level IOPS apportionments, or the two or more node-level IOPS apportionments.

8. The method of claim 1, wherein at least one of the two or more virtual machines is permitted to perform a burst of I/Os up to a statically-defined burst limit.

9. The method of claim 1, wherein at least one of the node-level IOPS apportionments is determined based at least in part on one or more performance specifications corresponding to at least one of the two or more nodes.

10. The method of claim 1, wherein the at least one policy comprises a first policy and a second policy that is different from the first policy, and wherein the first policy is applied to a first virtual machine running on a first one of the two or more nodes and wherein the second policy corresponds to a second virtual machine running on the first one of the two or more nodes.

11. A computer readable medium, embodied in a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when stored in memory and executed by one or more processors causes the one or more processors to perform a set of acts for multi-stage input/output operations (IOPS) allocations, the acts comprising:

identifying at least one policy that specifies at least one IOPS limit;

determining two or more virtual machines associated with the at least one policy;

determining two or more nodes that host the two or more virtual machines;

performing a first allocation to apportion the at least one IOPS limit over the two or more nodes, the first allocation resulting in two or more node-level IOPS apportionments that correspond to the two or more nodes; and

performing a second allocation to apportion the node-level IOPS apportionments to the two or more virtual machines, the second allocation resulting in one or more VM-level IOPS apportionments to the two or more virtual machines.

12. The computer readable medium of claim 11, further comprising instructions which, when stored in memory and executed by the one or more processors causes the one or more processors to perform acts of:

invoking an IOPS usage monitoring function at the two or more nodes;

monitoring, by the IOPS usage monitoring function at the two or more nodes, node-specific IOPS usage data; and

adjusting at least one of, the two or more node-level IOPS apportionments, or the one or more VM-level IOPS apportionments, the adjusting being based at least in part on the node-specific IOPS usage data.

13. The computer readable medium of claim 12, wherein the node-specific IOPS usage data comprises at least one predicted IOPS rate.

14. The computer readable medium of claim 13, wherein the at least one predicted IOPS rate corresponds to at least one of, at least one of the two or more nodes, or at least one of the two or more virtual machines.

15. The computer readable medium of claim 13, wherein the at least one predicted IOPS rate is derived from one or more average IOPS rates over a respective one or more observation periods.

16. The computer readable medium of claim 15, wherein at least one of the one or more average IOPS rates is derived from an I/O operations count that corresponds to one of the respective one or more observation periods.

17. The computer readable medium of claim 11, further comprising instructions which, when stored in memory and executed by the one or more processors causes the one or more processors to perform acts of:

throttling one or more I/O operations associated with at least one of the two or more virtual machines, the throttling being based at least in part on at least one of, the one or more VM-level IOPS apportionments, or the two or more node-level IOPS apportionments.

18. The computer readable medium of claim 11, wherein the at least one IOPS limit comprises at least one of, an aggregate IOPS limit, or a provisioned IOPS limit.

19. A system for multi-stage input/output operations (IOPS) allocations, the system comprising:

a storage medium having stored thereon a sequence of instructions; and

one or more processors that execute the instructions to cause the one or more processors to perform a set of acts, the acts comprising,

identifying at least one policy that specifies at least one IOPS limit;

determining two or more virtual machines associated with the at least one policy;

determining two or more nodes that host the two or more virtual machines;

performing a first allocation to apportion the at least one IOPS limit over the two or more nodes, the first allocation resulting in two or more node-level IOPS apportionments that correspond to the two or more nodes; and

performing a second allocation to apportion the node-level IOPS apportionments to the two or more virtual machines, the second allocation resulting in one or more VM-level IOPS apportionments to the two or more virtual machines.

20. The system of claim 19, further comprising instructions to cause the one or more processors to perform a set of acts comprising:

invoking an IOPS usage monitoring function at the two or more nodes;

monitoring, by the IOPS usage monitoring function at the two or more nodes, node-specific IOPS usage data; and

adjusting at least one of, the two or more node-level IOPS apportionments, or the one or more VM-level IOPS apportionments, the adjusting being based at least in part on the node-specific IOPS usage data.