Establishing and maintaining data apportioning for availability domain fault tolerance

• US 10,678,457 B2
• Filed: 11/20/2017
• Issued: 06/09/2020
• Est. Priority Date: 11/22/2016
• Status: Active Grant

• Current Assignee: Nutanix Incorporated
• Original Assignee: Nutanix Incorporated
• Inventors: Jawahar, Anoop, Gupta, Karan, Gundlagutta, Naveen Reddy
• Primary Examiner(s): Giardino, Jr., Mark A
• Application Number:
  US15/818,660
• Publication Number:
  US 20200026446A1
• Time in Patent Office: 932 Days
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0632 by initialisation or re-initialisation of storage systems
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices

Abstract

Systems for distributed computing systems. A topological configuration of computing nodes is selected to manage availability of metadata data in a computing system. A method embodiment commences by accessing a plurality of node topology attributes and using those attributes to map between nodes and availability domains. Resource usage measurements such as computing node load are collected. A plurality of candidate replication configurations are generated, and each candidate replication configuration is scored with respect to several quantitative objectives. Additionally, the candidate replication configurations are given respective resource usage balance scores. One or more candidate replication configurations are selected based on resource usage balance scores and/or a separation skew value. Determination of a selected configuration is dominated by resource usage when there is a tie between best-scoring configurations or when none of the configurations meet a scoring threshold. Recalculation of configurations are triggered by an administrative command or by a topology change.

Claims

1. A method comprising:

accessing an attribute describing a topology of a virtualization environment, wherein the virtualization environment comprises a first node and a second node that belong respectively to a first availability domain and a second availability domain, and the attribute comprises a mapping between a node and an availability domain in the virtualization environment;

collecting resource usage corresponding to the first and the second nodes;

generating multiple candidate replication configurations and a resource usage balance score for each of the multiple candidate replication configurations based on at least in part upon the mapping and the resource usage; and

selecting a replication configuration from the multiple candidate replication configurations based at least in part on the resource usage balance score.

2. The method of claim 1, wherein the resource usage balance score is computed for a candidate replication configuration and describes a quantitative relationship with a resource usage objective.

3. The method of claim 1, further comprising:

deploying a subject node at a subject availability domain, wherein the subject node is identified from the first or the second node, and the subject availability domain is identified from the first or the second availability domain comprising the replication configuration.

4. The method of claim 1, wherein the attribute is accessed responsive to a configuration event.

5. The method of claim 4, wherein the configuration event is triggered by at least one of an initial replication configuration command or a node topology change.

6. The method of claim 1, wherein a configuration score is determined for a candidate replication configuration and describes a quantitative relationship with at least one of a replication factor value, an availability domain aware objective, or a separation skew objective.

7. The method of claim 1, wherein a configuration score corresponding to a candidate replication configuration is determined based at least in part on a count of a set of intervening nodes between a pair of nodes.

8. The method of claim 1, wherein the attribute comprises at least one of a node identifier, an availability domain identifier, a site identifier, a host identifier, an IP address, or a node state.

9. The method of claim 1, wherein the replication configuration is described by at least one of a node identifier, a token identifier, an IP address, a data center, a rack identifier, a predecessor node identifier, or a successor node identifier.

10. The method of claim 1, wherein the first or the second availability domain corresponds to at least a respective one of the node, a block, a host, a site, an appliance, a rack, or a data center.

11. The method of claim 1, wherein at least one of the multiple candidate replication configurations describe a respective logical arrangement of the first and the second nodes as a ring organization.

12. The method of claim 1, further comprising performing a comparison between a configuration score and a configuration score threshold, wherein the configuration score is determined for a candidate replication configuration of the multiple candidate replication configurations.

13. The method of claim 12, wherein the configuration score for the each of the multiple candidate replication configurations surpasses the configuration score threshold.

14. The method of claim 13, wherein the replication configuration is selected from the candidate replication configurations based at least in part on a separation skew value, rather than considering the configuration score for the candidate replication configuration.

15. The method of claim 13, wherein the replication configuration is selected from the candidate replication configurations based at least in part on the resource usage balance score, rather than selecting based on the configuration score.

16. A non-transitory computer readable medium having stored thereon a sequence of instructions which, when stored in memory and executed by a processor, causes the processor to perform a set of acts, the set of acts comprising:

accessing an attribute describing a topology of a virtualization environment, wherein the virtualization environment comprises a first node and a second node that belong respectively to a first availability domain and a second availability domain, and the attribute comprises a mapping between a node and an availability domain in the virtualization environment;

collecting resource usage corresponding to the first and the second nodes;

generating multiple candidate replication configurations and a resource usage balance score for each of the multiple candidate replication configurations based at least in part upon the mapping and the resource usage; and

selecting a replication configuration from the multiple candidate replication configurations based at least in part on the resource usage balance score.

17. The non-transitory computer readable medium of claim 16, wherein the resource usage balance score is computed for a candidate replication configuration and describes a quantitative relationship with a resource usage objective.

18. The non-transitory computer readable medium of claim 16, further comprising instructions which, when stored in the memory and executed by the processor, causes the processor to perform an act of:

deploying a subject node at a subject availability domain, wherein the subject node is identified from the first or the second node, and the subject availability domain is identified from the first or the second availability domain comprising the replication configuration.

19. The non-transitory computer readable medium of claim 16, further comprising instructions which, when stored in the memory and executed by the processor, causes the processor to perform an act of:

determining a third node from the first availability domain for the first node based at least in part upon a topological distance based at least in part upon a configuration score of the replication configuration.

20. The non-transitory computer readable medium of claim 16, wherein the replication configuration is selected based at least in part upon a configuration score for the replication configuration, and the configuration score is determined for the replication configuration based at least in part a first node score that corresponds to the first node in the first availability domain and a separate node in the first availability domain.

21. A system comprising:

a non-transitory storage medium having stored thereon a sequence of instructions; and

a processor that executes the sequence of instructions to cause the one or more processors to perform a set of acts, the acts comprising,

accessing an attribute describing a topology of a virtualization environment, wherein the virtualization environment comprises a first node and a second node that belong respectively to a first availability domain and a second availability domain, and the attribute comprises a mapping between a node and an availability domain in the virtualization environment;

collecting resource usage corresponding to the first and the second nodes;

generating multiple candidate replication configurations and a resource usage balance score for each of the multiple candidate replication configurations based at least in part upon the mapping and the resource usage; and

selecting a replication configuration from the multiple candidate replication configurations based at least in part on the resource usage balance score.

22. The system of claim 21, wherein the resource usage balance score describes a quantitative relationship with a resource usage objective.