Facilitating data consistency in a dispersed storage network

• US 10,402,395 B2
• Filed: 05/14/2018
• Issued: 09/03/2019
• Est. Priority Date: 09/05/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Resch, Jason K., Baptist, Andrew D., McShane, Niall J., Dhuse, Greg R., Shirley, Jr., Thomas F., Palinckx, Gregory, Kennedy, Russell P., Gladwin, S. Christopher, McCammon, Robert C., Sherer, James
• Primary Examiner(s): Pham, Tuan A
• Application Number:
  US15/978,596
• Publication Number:
  US 20180260434A1
• Time in Patent Office: 477 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method for execution by a dispersed storage and task (DST) processing unit that includes a processor includes determining to access a set of storage units; identifying an information dispersal algorithm (IDA) width and a decode threshold number associated with the set of storage units; determining a number of available storage units of the set of storage units; determining a write threshold number and a read threshold number based on the number of available storage units and in accordance with a consistency approach; and accessing at least some of the available storage units utilizing at least one of the write and read threshold numbers.

Claims

1. A method for execution by a processing system of a dispersed storage and task (DST) processing unit that includes a processor, the method comprises:

determining, via the processor, to access a set of storage units;

identifying, via the processor, an information dispersal algorithm (IDA) width and a decode threshold number included in dispersed storage error encoding parameters associated with the set of storage units;

determining, via the processor, a number of available storage units of the set of storage units;

determining, via the processor, a write threshold number and a read threshold number in accordance with a consistency approach based on the number of available storage units, the IDA width, and the decode threshold number, wherein, when the number of available storage units is determined to be equal to a first number, determining the write threshold number and the read threshold number includes establishing the write threshold number and the read threshold number in accordance with strong consistency mode dispersal parameters such that the write threshold number is equal to a first write threshold number and the read threshold number is equal to a first read threshold number, and wherein, when the number of available storage units is determined to be equal to a second number that is lower than the first number, determining the write threshold number and the read threshold number includes establishing the write threshold number and the read threshold number in accordance with impaired mode dispersal parameters such that the write threshold number is equal to a second write threshold number that is lower than the first write threshold number and the read threshold number is equal to a second read threshold number that is lower than the first write threshold number;

issuing, via the processor, the write threshold number of write slice requests to the write threshold number of storage units of the available storage units via a network when the access to the set of storage units includes writing data; and

issuing, via the processor, the read threshold number of read slice requests to the read threshold number of storage units of the available storage units via the network when the access to the set of storage units includes reading data.

2. The method of claim 1, wherein determining to access the set of storage units includes at least one of:

receiving a data access request, determining to store data, or determining to retrieve the data.

3. The method of claim 1, wherein identifying the IDA width and the decode threshold number includes at least one of:

interpreting a portion of a system registry information, receiving the IDA width and decode threshold number, or determining one or more of the IDA width and the decode threshold number.

4. The method of claim 1, wherein determining the number of available storage units includes at least one of:

initiating a query, interpreting a query response, performing a test, interpreting an error message, performing a lookup, or generating an estimate.

5. The method of claim 1, wherein determining the write threshold number and the read threshold number includes establishing each of the write threshold number and the read threshold number to be greater than or equal to the decode threshold number and less than or equal to the number of available storage units.

6. The method of claim 1, wherein, when the IDA width is greater than or equal to twice the decode threshold number, determining the write threshold number and the read threshold number includes establishing the write threshold number and the read threshold number with consistency constraints such that a sum of the write threshold number and the read threshold number is greater than the number of available storage units.

7. The method of claim 1, further comprising:

determining, via the processor, the number of available storage units of the set of storage units has reduced after the read threshold number and write threshold number have been determined;

lowering the write threshold number to a lowered write threshold number in response to determining the number of available storage units of the set of storage units has reduced, wherein the lowered write threshold number is greater than or equal to the decode threshold number;

determining to maintain the read threshold number in response to determining the read threshold number is equal to the decode threshold number.

8. A processing system of a dispersed storage and task (DST) processing unit comprises:

at least one processor;

a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to perform operations including;

determining, via the processor, to access a set of storage units;

identifying, via the processor, an information dispersal algorithm (IDA) width and a decode threshold number included in dispersed storage error encoding parameters associated with the set of storage units;

determining, via the processor, a number of available storage units of the set of storage units;

determining, via the processor, a write threshold number and a read threshold number in accordance with a consistency approach based on the number of available storage units, the IDA width, and the decode threshold number, wherein, when the number of available storage units is determined to be equal to a first number, determining the write threshold number and the read threshold number includes establishing the write threshold number and the read threshold number in accordance with strong consistency mode dispersal parameters such that the write threshold number is equal to a first write threshold number and the read threshold number is equal to a first read threshold number, and wherein, when the number of available storage units is determined to be equal to a second number that is lower than the first number, determining the write threshold number and the read threshold number includes establishing the write threshold number and the read threshold number in accordance with impaired mode dispersal parameters such that the write threshold number is equal to a second write threshold number that is lower than the first write threshold number and the read threshold number is equal to a second read threshold number that is lower than the first write threshold number;

issuing, via the processor, the write threshold number of write slice requests to the write threshold number of storage units of the available storage units via a network when the access to the set of storage units includes writing data; and

issuing, via the processor, the read threshold number of read slice requests to the read threshold number of storage units of the available storage units via the network when the access to the set of storage units includes reading data.

9. The processing system of claim 8, wherein determining to access the set of storage units includes at least one of:

receiving a data access request, determining to store data, or determining to retrieve the data.

10. The processing system of claim 8, wherein identifying the IDA width and the decode threshold number includes at least one of:

interpreting a portion of a system registry information, receiving the IDA width and decode threshold number, or determining one or more of the IDA width and the decode threshold number.

11. The processing system of claim 8, wherein determining the number of available storage units includes at least one of:

initiating a query, interpreting a query response, performing a test, interpreting an error message, performing a lookup, or generating an estimate.

12. The processing system of claim 8, wherein determining the write threshold number and the read threshold number includes establishing each of the write threshold number and the read threshold number to be greater than or equal to the decode threshold number and less than or equal to the number of available storage units.

13. The processing system of claim 8, wherein, when the IDA width is greater than or equal to twice the decode threshold number, determining the write threshold number and the read threshold number includes establishing the write threshold number and the read threshold number with consistency constraints such that a sum of the write threshold number and the read threshold number is greater than the number of available storage units.

14. The processing system of claim 8, wherein the operational instructions, when executed by the at least one processor, further cause the processing system to perform operations including:

determining, via the processor, the number of available storage units of the set of storage units has reduced after the read threshold number and write threshold number have been determined;

lowering the write threshold number to a lowered write threshold number in response to determining the number of available storage units of the set of storage units has reduced, wherein the lowered write threshold number is greater than or equal to the decode threshold number;

determining to maintain the read threshold number in response to determining the read threshold number is equal to the decode threshold number.

15. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by a processing system of a dispersed storage network (DSN) that includes a processor and a memory, causes the processing system to perform operations including;

determining, via the processor, to access a set of storage units;

identifying, via the processor, an information dispersal algorithm (IDA) width and a decode threshold number included in dispersed storage error encoding parameters associated with the set of storage units;

determining, via the processor, a number of available storage units of the set of storage units;

determining, via the processor, a write threshold number and a read threshold number in accordance with a consistency approach based on the number of available storage units, the IDA width, and the decode threshold number, wherein, when the number of available storage units is determined to be equal to a first number, determining the write threshold number and the read threshold number includes establishing the write threshold number and the read threshold number in accordance with strong consistency mode dispersal parameters such that the write threshold number is equal to a first write threshold number and the read threshold number is equal to a first read threshold number, and wherein, when the number of available storage units is determined to be equal to a second number that is lower than the first number, determining the write threshold number and the read threshold number includes establishing the write threshold number and the read threshold number in accordance with impaired mode dispersal parameters such that the write threshold number is equal to a second write threshold number that is lower than the first write threshold number and the read threshold number is equal to a second read threshold number that is lower than the first write threshold number;

issuing, via the processor, the write threshold number of write slice requests to the write threshold number of storage units of the available storage units via a network when the access to the set of storage units includes writing data; and

issuing, via the processor, the read threshold number of read slice requests to the read threshold number of storage units of the available storage units via the network when the access to the set of storage units includes reading data.

16. The non-transitory computer readable storage medium of claim 15, wherein determining the number of available storage units includes at least one of:

initiating a query, interpreting a query response, performing a test, interpreting an error message, performing a lookup, or generating an estimate.

17. The non-transitory computer readable storage medium of claim 15, wherein determining the write threshold number and the read threshold number includes establishing each of the write threshold number and the read threshold number to be greater than or equal to the decode threshold number and less than or equal to the number of available storage units.

18. The non-transitory computer readable storage medium of claim 15, wherein, when the number of available storage units is greater than half of the IDA width, determining the write threshold number and the read threshold number includes establishing the write threshold number and the read threshold number with consistency constraints such that a sum of the write threshold number and the read threshold number is greater than the number of available storage units.