Achieving storage compliance in a dispersed storage network

• US 10,013,203 B2
• Filed: 04/11/2016
• Issued: 07/03/2018
• Est. Priority Date: 01/04/2013
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Resch, Jason K., Leggette, Wesley
• Primary Examiner(s): Kerveros, James C
• Application Number:
  US15/095,558
• Publication Number:
  US 20160224422A1
• Time in Patent Office: 813 Days
• Field of Search: 714763
• CPC Class Codes: G06F 11/1008 in individual solid state devices G06F11/1004 takes precedence
• CPC Class Codes: G06F 11/1044 with specific ECC/EDC distribution
• CPC Class Codes: G06F 11/1068 in sector programmable memories, e.g. flash disk G06F11/1072 takes precedence
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A method begins by a dispersed storage (DS) processing module receiving a request to store data in a dispersed storage network and determining dispersed storage error encoding parameters for encoding the data into sets of encoded data slices. The method continues with the DS processing module determining whether the request includes a desired write reliability indication. When the request includes the desired write reliability indication, the method continues with the DS processing module determining whether storage of the sets of encoded data slices is meeting the desired write reliability indication. When storage of a set of encoded data slices is not meeting the desired write reliability indication, the method continues with the DS processing module determining a storage compliance process for the set of encoded data slices to meet the desired write reliability indication and executing the storage compliance process for the set of encoded data slices.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

receiving, from a device, a request to store data in the DSN, wherein the request includes one of a plurality of desired level of dispersed storage error encoding parameters, wherein the dispersed storage error encoding parameters includes a per set decode threshold, a per set write threshold, a desired write threshold, and a per set total number, wherein the per set decode threshold indicates a number of encoded data slices of a set of encoded data slices required to construct a corresponding segment of the data, the per set write threshold indicates a number of encoded data slices of the set of encoded data slices that are required to be stored in accordance with a write operation, and the per set total number indicates the number of encoded data slices in the set of encoded data slices;

while executing storage of the plurality of sets of encoded data slices in accordance with the dispersed storage error encoding parameters, determining whether the storage of the plurality of sets of encoded data slices is meeting the desired write threshold;

when storage of the set of encoded data slices of the plurality of sets of encoded data slices is not meeting the desired write threshold, determining whether the per set write threshold is being met; and

when the per set write threshold is being met;

determining a storage compliance process for the set of encoded data slices to meet the desired write reliability indication; and

executing the storage compliance process for the set of encoded data slices.

2. The method of claim 1, wherein the desired write threshold comprises:

a value in a range between the per set write threshold and the per set total number.

3. The method of claim 1, wherein the determining the dispersed storage error encoding parameters comprises:

determining a vault based on at least one of the request and the device; and

determining the per set decode threshold, the per set write threshold, the desired write threshold, and the per set total number based on information regarding the vault.

4. The method of claim 1, wherein the executing the storage compliance process comprises:

initiating a rebuilding process for encoded data slices of the set of encoded data slices that were not successfully stored during the execution of storage.

5. The method of claim 1, wherein the executing the storage compliance process comprises:

initiating a storage unit retry process for encoded data slices of the set of encoded data slices that were not successfully stored during the execution of storage.

6. The method of claim 1, wherein the executing the storage compliance process comprises:

sending a message to the device indicating that the storage of the set of encoded data slices met the per set write threshold but did not meet the desired write reliability indication;

receiving a response from the device requesting a storage retry of encoded data slices of the set of encoded data slices that were not successfully stored during the execution of storage; and

retrying storage of the encoded data slices of the set of encoded data slices that were not successfully stored during the execution of storage.

7. The method of claim 1 further comprises:

while executing the storage of the plurality of sets of encoded data slices in accordance with the dispersed storage error encoding parameters, entering a loop that includes;

determining whether the storage of one set of the plurality of sets of encoded data slices is meeting the desired write threshold;

when the storage of the one set of the plurality of sets of encoded data slices is not meeting the desired write reliability indication, flagging the one set of the plurality of sets of encoded data slices and determining whether the one set of the plurality of sets of encoded data slices is a last set of the plurality of sets of encoded data slices;

when the storage of the one set of the plurality of sets of encoded data slices is meeting the desired write reliability indication, determining whether the one set of the plurality of sets of encoded data slices is the last set of the plurality of sets of encoded data slices;

when the one set of the plurality of sets of encoded data slices is not the last set of the plurality of sets of encoded data slices, repeating the loop for another one set of the plurality of sets of encoded data slices;

when the one set of the plurality of sets of encoded data slices is the last set of the plurality of sets of encoded data slices, exiting the loop;

compiling a list of sets of encoded data slices of the plurality of sets of encoded data slices that did not meet the desired write reliability indication to produce a list of sets;

determining the storage compliance process for the list of sets; and

executing the storage compliance process for the sets of encoded data slices of the plurality of sets of encoded data slices that did not meet the desired write reliability indication based on the list of sets.

8. The method of claim 1 further comprises:

when the storage of the set of encoded data slices of the plurality of sets of encoded data slices is meeting the desired write reliability indication, indicating that the set of encoded data slices met the desired write reliability indication.

9. A storage system comprises:

a first module, when operable within a computing device, causes the computing device to;

receive, from a device, a request to store data in a dispersed storage network (DSN), wherein the request includes one of a plurality of desired level of dispersed storage error encoding parameters, wherein the dispersed storage error encoding parameters includes a per set decode threshold, a per set write threshold, a desired write threshold, and a per set total number, wherein the per set decode threshold indicates a number of encoded data slices of a set of encoded data slices required to construct a corresponding segment of the data, the per set write threshold indicates a number of encoded data slices of the set of encoded data slices that are required to be stored in accordance with a write operation, and the per set total number indicates the number of encoded data slices in the set of encoded data slices;

a second module, when operable within the computing device, causes the computing device to;

while executing storage of the plurality of sets of encoded data slices in accordance with the dispersed storage error encoding parameters, determine whether the storage of the plurality of sets of encoded data slices is meeting the desired write threshold;

when storage of the set of encoded data slices of the plurality of sets of encoded data slices is not meeting the desired write threshold, determine whether the per set write threshold is being met; and

a third module, when operable within the computing device, causes the computing device to;

when the per set write threshold is being met;

determine a storage compliance process for the set of encoded data slices to meet the desired write reliability indication; and

execute the storage compliance process for the set of encoded data slices.

10. The storage system of claim 9, wherein the desired write threshold comprises:

a value in a range between the per set write threshold and the per set total number.

11. The storage system of claim 9, wherein the determining the dispersed storage error encoding parameters comprises:

determining a vault based on at least one of the request and the device; and

determining the per set decode threshold, the per set write threshold, the desired write threshold, and the per set total number based on information regarding the vault.

12. The storage system of claim 9, wherein the executing the storage compliance process comprises:

initiating a rebuilding process for encoded data slices of the set of encoded data slices that were not successfully stored during the execution of storage.

13. The storage system of claim 9, wherein the executing the storage compliance process comprises:

initiating a storage unit retry process for encoded data slices of the set of encoded data slices that were not successfully stored during the execution of storage.

14. The storage system of claim 9, wherein the executing the storage compliance process comprises:

sending a message to the device indicating that the storage of the set of encoded data slices met the per set write threshold but did not meet the desired write reliability indication;

receiving a response from the device requesting a storage retry of encoded data slices of the set of encoded data slices that were not successfully stored during the execution of storage; and

retrying storage of the encoded data slices of the set of encoded data slices that were not successfully stored during the execution of storage.

15. The storage system of claim 9 further comprises:

the second module, further operable within the computing device, causing the computing device to;

while executing the storage of the plurality of sets of encoded data slices in accordance with the dispersed storage error encoding parameters, entering a loop that includes;

determine whether the storage of one set of the plurality of sets of encoded data slices is meeting the desired write threshold;

when the storage of the one set of the plurality of sets of encoded data slices is not meeting the desired write reliability indication, flag the one set of the plurality of sets of encoded data slices and determining whether the one set of the plurality of sets of encoded data slices is a last set of the plurality of sets of encoded data slices;

when the storage of the one set of the plurality of sets of encoded data slices is meeting the desired write reliability indication, determine whether the one set of the plurality of sets of encoded data slices is the last set of the plurality of sets of encoded data slices;

when the one set of the plurality of sets of encoded data slices is not the last set of the plurality of sets of encoded data slices, repeat the loop for another one set of the plurality of sets of encoded data slices;

when the one set of the plurality of sets of encoded data slices is the last set of the plurality of sets of encoded data slices, exit the loop;

compile a list of sets of encoded data slices of the plurality of sets of encoded data slices that did not meet the desired write reliability indication to produce a list of sets;

the third module, when operable within the computing device, further causes the computing device to;

determine the storage compliance process for the list of sets; and

execute the storage compliance process for the sets of encoded data slices of the plurality of sets of encoded data slices that did not meet the desired write reliability indication based on the list of sets.

16. The storage system of claim 9, wherein the second module, when operable within the computing device, further causes the computing device to:

when the storage of the set of encoded data slices of the plurality of sets of encoded data slices is meeting the desired write reliability indication, indicate that the set of encoded data slices met the desired write reliability indication.