Multi-level data storage in a dispersed storage network

• US 10,417,253 B2
• Filed: 05/17/2018
• Issued: 09/17/2019
• Est. Priority Date: 09/13/2012
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Kazi, Asimuddin, Resch, Jason K.
• Primary Examiner(s): Giardino, Jr., Mark A
• Application Number:
  US15/982,279
• Publication Number:
  US 20180268043A1
• Time in Patent Office: 488 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 1/00 Arrangements for detecting or preventing errors in the information received correcting synchronisation H04L7/00
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method includes encoding, via the processing system, data using first level dispersal parameters to produce a set of first level encoded data slices, wherein the first level dispersal parameters includes a first level write threshold; outputting the set of first level encoded data slices to a set of distributed storage and task (DST) processing units, wherein each DST processing unit of the set of DST processing units encodes a corresponding one of the set of first level encoded data slices using second level dispersal parameters to produce a set of second level encoded data slices; determining, via the processing system, when a first level write threshold number of favorable storage responses is received from a subset of the set of DST processing units; and generating and outputting, via the processing system, a cancellation message to other DST processing units of the set of DST processing units.

Claims

1. A method for execution by a processing system of a dispersed storage and task (DST) network that includes a processor, the method comprises:

receiving, via the processing system, a request to store data;

encoding, via the processing system, the data using a dispersed storage error coding function and first level dispersal parameters to produce a set of first level encoded data slices, wherein the first level dispersal parameters includes a first level write threshold;

outputting, via the processing system, the set of first level encoded data slices to a set of distributed storage and task (DST) processing units, wherein each DST processing unit of the set of DST processing units encodes one of the set of first level encoded data slices using the dispersed storage error coding function and second level dispersal parameters to produce a set of second level encoded data slices;

determining, via the processing system, when a first level write threshold number of favorable storage responses is received from a subset of the set of DST processing units;

generating and outputting, via the processing system, a cancellation message to other DST processing units of the set of DST processing units not included in the subset of the set of DST processing units; and

updating, via the processing system, a directory to include an association between a data identifier of the data, identifiers of the subset of the set of DST processing units, and identifiers of a first level write threshold number of the set of first level encoded data slices stored via the subset of the set of DST processing units.

2. The method of claim 1, wherein the request to store data includes the data and the data identifier associated with the data.

3. The method of claim 1, wherein the second level dispersal parameters include a second level write threshold.

4. The method of claim 3, wherein the first level write threshold differs from the second level write threshold.

5. The method of claim 1, wherein each DST processing unit of the set of DST processing units, generates and outputs an unfavorable storage response or a favorable storage response.

6. The method of claim 1, wherein the second level dispersal parameters includes a second level write threshold and wherein the favorable storage response indicates that a second level write threshold number of the set of second level encoded data slices have been successfully stored.

7. The method of claim 1, wherein the cancellation message includes a deletion request.

8. The method of claim 1, wherein the cancellation message includes a rollback request.

9. A processing system of a dispersed storage and task (DST) network comprises:

at least one processor;

a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to perform operations including;

receiving, via the processing system, a request to store data;

encoding, via the processing system, the data using a dispersed storage error coding function and first level dispersal parameters to produce a set of first level encoded data slices, wherein the first level dispersal parameters includes a first level write threshold;

outputting, via the processing system, the set of first level encoded data slices to a set of distributed storage and task (DST) processing units, wherein each DST processing unit of the set of DST processing units encodes one of the set of first level encoded data slices using the dispersed storage error coding function and second level dispersal parameters to produce a set of second level encoded data slices;

determining, via the processing system, when a first level write threshold number of favorable storage responses is received from a subset of the set of DST processing units;

generating and outputting, via the processing system, a cancellation message to other DST processing units of the set of DST processing units not included in the subset of the set of DST processing units; and

updating, via the processing system, a directory to include an association between a data identifier of the data, identifiers of the subset of the set of DST processing units, and identifiers of a first level write threshold number of the set of first level encoded data slices stored via the subset of the set of DST processing units.

10. The processing system of claim 9, wherein the request to store data includes the data and the data identifier associated with the data.

11. The processing system of claim 9, wherein the second level dispersal parameters include a second level write threshold.

12. The processing system of claim 9, wherein the first level write threshold differs from the second level write threshold.

13. The processing system of claim 9, wherein each DST processing unit of the set of DST processing units, generates and outputs an unfavorable storage response or a favorable storage response.

14. The processing system of claim 9, wherein the second level dispersal parameters includes a second level write threshold and wherein the favorable storage response indicates that a second level write threshold number of the set of second level encoded data slices have been successfully stored.

15. The processing system of claim 9, wherein the cancellation message includes a deletion request.

16. The processing system of claim 9, wherein the cancellation message includes a rollback request.

17. A computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by a processing system of a dispersed storage network (DSN) that includes a processor and a memory, causes the processing system to perform operations including;

receiving, via the processing system, a request to store data;

encoding, via the processing system, the data using a dispersed storage error coding function and first level dispersal parameters to produce a set of first level encoded data slices, wherein the first level dispersal parameters includes a first level write threshold;

outputting, via the processing system, the set of first level encoded data slices to a set of distributed storage and task (DST) processing units, wherein each DST processing unit of the set of DST processing units encodes one of the set of first level encoded data slices using the dispersed storage error coding function and second level dispersal parameters to produce a set of second level encoded data slices;

determining, via the processing system, when a first level write threshold number of favorable storage responses is received from a subset of the set of DST processing units;

generating and outputting, via the processing system, a cancellation message to other DST processing units of the set of DST processing units not included in the subset of the set of DST processing units; and

updating, via the processing system, a directory to include an association between a data identifier of the data, identifiers of the subset of the set of DST processing units, and identifiers of a first level write threshold number of the set of first level encoded data slices stored via the subset of the set of DST processing units.

18. The computer readable storage medium of claim 17, wherein each DST processing unit of the set of DST processing units, generates and outputs an unfavorable storage response or a favorable storage response.

19. The computer readable storage medium of claim 17, wherein the second level dispersal parameters includes a second level write threshold and wherein the favorable storage response indicates that a second level write threshold number of the set of second level encoded data slices have been successfully stored.

20. The computer readable storage medium of claim 17, wherein the cancellation message includes a deletion request or a rollback request.