Data consistency management in large computing clusters

• US 10,467,115 B1
• Filed: 11/03/2017
• Issued: 11/05/2019
• Est. Priority Date: 11/03/2017
• Status: Active Grant

• Current Assignee: Nutanix Incorporated
• Original Assignee: Nutanix Incorporated
• Inventors: Olderdissen, Jan Ralf Alexander
• Primary Examiner(s): Riad, Amine
• Application Number:
  US15/803,085
• Time in Patent Office: 732 Days
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3058 Monitoring arrangements for monitoring environmental properties or parameters of the computing system or of the computing system component, e.g. monitoring of power, currents, temperature, humidity, position, vibrations thermal management in cooling arrangements of a computing system G06F1/206
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency

Abstract

Storage device groups are formed with respect to data consistency policies and/or quantified probabilities. A method embodiment commences upon identifying a computing system having a plurality of storage devices that are accessed by a plurality of computing nodes. A user interface serves for collecting policies, data loss parameters, and data rebuild parameters. Based on the policies and/or values of the data loss parameters, and values of the data rebuild parameters, sets of storage device groups are formed to achieve particular data loss and rebuild time properties. Data storage containers such as files or virtual disks that hold persistent datasets are assigned to respective storage device groups that are appropriate to the nature of the dataset. The objective pertaining to an acceptable likelihood of data loss as well as the objective of an acceptable time for rebuild are achieved as a result of assignments of certain storage devices into a group.

Claims

1. A method comprising:

determining a rebuild time parameter that characterizes a time for copying stored data from a first storage device to a second storage device of a computing system, the computing system comprising a plurality of computing nodes;

determining a data loss parameter corresponding to a storage device of a computing node of the plurality of computing nodes;

determining a storage device group having a maximum number of storage devices selected from storage devices of the plurality of computing nodes by;

identifying a maximum data loss probability value determined based at least in part on the rebuild parameter and a data loss parameter corresponding to the storage device,

incrementally adding the storage device to the storage device group, and

comparing an estimated data loss probability value of the storage device group having the storage device added against the maximum data loss probability value, wherein the maximum number of storage devices for the storage device group is determined when the estimated data loss probability value exceeds the maximum data loss probability value; and

assigning a dataset to the storage device group, wherein the dataset and a replica of the dataset are stored in the storage device group.

2. The method of claim 1, wherein the rebuild time parameter is based at least in part on a value of a service level agreement.

3. The method of claim 1, wherein the rebuild time parameter is based at least in part on a data throughput of the storage device.

4. The method of claim 1, further comprising implementing a code base that comprises a storage device grouping algorithm, wherein the storage device grouping algorithm is invoked to determine the storage device group.

5. The method of claim 4, wherein the storage device grouping algorithm is invoked based at least in part on a value of the data loss parameter.

6. The method of claim 5, wherein the value of the data loss parameter quantifies at least one of, a maximum data loss probability, a maximum storage device group size, a quantity of computing nodes, a quantity of storage devices, a failure rebuild time, or a drive failure rate.

7. The method of claim 6, wherein the drive failure rate corresponds to a respective storage device.

8. The method of claim 6, wherein the drive failure rate is derived from at least one of, a temperature measurement, or a slot-specific vibration measurement, or a location-specific vibration measurement, or a slot-specific temperature measurement, or a location-specific temperature measurement.

9. The method of claim 1, wherein the storage device group comprises a first storage device from a first computing node and a second storage device from a second computing node.

10. The method of claim 1, wherein the maximum number of storage devices is determined by reducing the maximum number of storage devices by 1 when the estimated data loss probability value exceeds the maximum data loss probability value.

11. The method of claim 1, wherein the storage device group comprises a first storage device of a first computing node and a second storage device of a randomly-identified second node.

12. The method of claim 1, wherein the storage device group comprises a first storage device of a first computing node and a second storage device of a second computing node, and wherein the storage device group is determined based at least in part on a first failure rate of the first storage device and a second failure rate of the second storage device.

13. The method of claim 1, wherein determining a first storage device group is based at least in part on a first data loss parameter, and wherein determining a second storage device group is based at least in part on a second data loss parameter.

14. The method of claim 13, wherein a first storage device grouping algorithm operates over the first data loss parameter and wherein a second storage device grouping algorithm operates over the second data loss parameter.

15. The method of claim 1, wherein the computing system is a cluster of computing nodes.

16. The method of claim 1, wherein the dataset corresponds to a virtual disk.

17. A computer readable medium, embodied in a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when stored in memory and executed by one or more processors causes the one or more processors to perform a set of acts, the acts comprising:

determining a rebuild time parameter that characterizes a time for copying stored data from a first storage device to a second storage device of a computing system, the computing system comprising a plurality of computing nodes;

determining a data loss parameter corresponding to a storage device of a computing node of the plurality of computing nodes;

determining a storage device group having a maximum number of storage devices selected from storage devices of the plurality of computing nodes by;

identifying a maximum data loss probability value determined based at least in part on the rebuild parameter and a data loss parameter corresponding to the storage device,

incrementally adding the storage device to the storage device group, and

comparing an estimated data loss probability value of the storage device group having the storage device added against the maximum data loss probability value, wherein the maximum number of storage devices for the storage device group is determined when the estimated data loss probability value exceeds the maximum data loss probability value; and

assigning a dataset to the storage device group, wherein the dataset and a replica of the dataset are stored in the storage device group.

18. The computer readable medium of claim 17, wherein the maximum number of storage devices is determined by reducing the maximum number of storage devices by 1 when the estimated data loss probability value exceeds the maximum data loss probability value.

19. A system comprising:

a storage medium having stored thereon a sequence of instructions; and

one or more processors that execute the instructions to cause the one or more processors to perform a set of acts, the acts comprising,

determining a rebuild time parameter that characterizes a time for copying stored data from a first storage device to a second storage device of a computing system, the computing system comprising a plurality of computing nodes;

determining a data loss parameter corresponding to a storage device of a computing node of the plurality of computing nodes;

determining a storage device group having a maximum number of storage devices selected from storage devices of the plurality of computing nodes by;

identifying a maximum data loss probability value determined based at least in part on the rebuild parameter and a data loss parameter corresponding to the storage device,

incrementally adding the storage device to the storage device group, and

comparing an estimated data loss probability value of the storage device group having the storage device added against the maximum data loss probability value, wherein the maximum number of storage devices for the storage device group is determined when the estimated data loss probability value exceeds the maximum data loss probability value; and

assigning a dataset to the storage device group, wherein the dataset and a replica of the dataset are stored in the storage device group.

20. The system of claim 19, wherein the maximum number of storage devices is determined by reducing the maximum number of storage devices by 1 when the estimated data loss probability value exceeds the maximum data loss probability value.