Optimizing data storage in a dispersed storage network

• US 10,387,067 B2
• Filed: 12/14/2017
• Issued: 08/20/2019
• Est. Priority Date: 02/27/2015
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Wozniak, Ethan S., Baptist, Andrew D., Dhuse, Greg R., Volvovski, Ilya, Resch, Jason K., Khadiwala, Ravi V., Leggette, Wesley B.
• Primary Examiner(s): Chung, Phung M
• Application Number:
  US15/841,990
• Publication Number:
  US 20180107422A1
• Time in Patent Office: 614 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A method for execution by a computing device includes dispersed storage error encoding a data segment to produce a set of encoded data slices. Dispersed storage network (DSN) configuration information is obtained and a write threshold number of encoded data slices is mapped to storage units to produce a slice storage mapping to facilitate storage of the encoded data slices. Unavailability of at least one encoded data slice is detected, and updated DSN configuration information is obtained. An updated slice storage mapping is produced and storage of an updated write threshold number of encoded data slices is facilitated accordingly. Availability of greater than the write threshold number of encoded data slices is detected and further updated DSN configuration information is obtained. A further updated slice storage mapping is produced and storage of a further updated write threshold number of encoded data slices is facilitated accordingly.

Claims

1. A method for execution by a computing device that includes a processor, the method comprises:

dispersed storage error encoding, by the computing device, a data segment to produce a set of encoded data slices;

obtaining, by the computing device, dispersed storage network (DSN) configuration information by receiving system registry information, wherein the DSN configuration information indicates a plurality of storage units;

mapping, by the computing device, a write threshold number of encoded data slices to a write threshold number of storage units of the plurality of storage units, based on the DSN configuration information, to produce a slice storage mapping;

facilitating, by the computing device, storage of the write threshold number of encoded data slices to the write threshold number of storage units in accordance with the slice storage mapping;

detecting, by the computing device, unavailability of at least one of the write threshold number of encoded data slices by interpreting an error message indicating the at least one of the write threshold number of encoded data slices is unavailable;

obtaining, by the computing device, first updated DSN configuration information by receiving updated system registry information, wherein the first updated DSN configuration information indicates at least one of the plurality of storage units is unavailable;

mapping, by the computing device, a first updated write threshold number of encoded data slices to a first updated write threshold number of storage units to produce a first updated slice storage mapping based on the first updated DSN configuration information;

facilitating, by the computing device, storage of the first updated write threshold number of encoded data slices to the first updated write threshold number of storage units in accordance with the first updated slice storage mapping;

obtaining, by the computing device, second updated DSN configuration information by receiving second updated system registry information, wherein the second updated DSN configuration information indicates a change in availability of at least one of the plurality of storage units indicated as unavailable in the first updated DSN configuration information;

mapping, by the computing device, a second updated write threshold number of encoded data slices to a second updated write threshold number of storage units to produce a second updated slice storage mapping, based on the second updated DSN configuration information, wherein the second updated write threshold number of encoded data slices is determined by altering the first updated write threshold number of encoded data slices based on the second updated DSN configuration information; and

facilitating, by the computing device, storage of the second updated write threshold number of encoded data slices to the second updated write threshold number of storage units in accordance with the second updated slice storage mapping.

2. The method of claim 1, wherein obtaining the DSN configuration information includes interpreting a query response.

3. The method of claim 1, further comprising:

performing a deterministic function on an identifier associated with the data segment, an IDA width number, and a number of available sites to produce the slice storage mapping.

4. The method of claim 3, wherein the deterministic function is performed to produce at least one of:

the first updated slice storage mapping or the second updated slice storage mapping, based on currently available encoded data slices of the write threshold number of encoded data slices.

5. The method of claim 1, wherein producing at least one of:

the first updated slice storage mapping or the second updated slice storage mapping is performed without regards to currently available encoded data slices.

6. The method of claim 1, wherein facilitating storage of the write threshold number of encoded data slices includes:

issuing a plurality of write slice requests to a set of storage units that includes the write threshold number of storage units; and

removing at least one encoded data slice beyond the write threshold number of encoded data slices from the set of encoded data slices; and

retrying storage of the write threshold number of encoded data slices in response to determining that the write threshold number of encoded data slices have not yet been successfully stored.

7. The method of claim 1, wherein detecting the unavailability of the at least one of the write threshold number of encoded data slices includes detecting a storage unit failure responsible for storing the at least one of the write threshold number of encoded data slices.

8. The method of claim 1, wherein facilitating storage of the first updated write threshold number of encoded data slices includes:

identifying a subset of encoded data slices of the first updated write threshold number of encoded data slices that have not been stored;

rebuilding the subset of encoded data slices; and

sending the rebuilt subset of encoded data slices to a corresponding subset of storage units of a set of storage units for storage.

9. The method of claim 1, further comprising:

detecting availability of a previously unavailable storage unit that is responsible for storage of a previously unavailable one of the set of encoded data slices.

10. The method of claim 1, wherein facilitating storage of the second updated write threshold number of encoded data slices includes:

deleting a first at least one previously stored encoded data slices of the set of encoded data slices that is not included in the second updated slice storage mapping; and

determining a second at least one encoded data slice that is included in the second updated slice storage mapping is not available; and

rebuilding the second at least one encoded data slice.

11. A processing system of a computing device comprises:

at least one processor;

a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to;

dispersed storage error encode a data segment to produce a set of encoded data slices;

obtain dispersed storage network (DSN) configuration information by receiving system registry information, wherein the DSN configuration information indicates a plurality of storage units;

map a write threshold number of encoded data slices to a write threshold number of storage units of the plurality of storage units, based on the DSN configuration information, to produce a slice storage mapping;

facilitate storage of the write threshold number of encoded data slices to the write threshold number of storage units in accordance with the slice storage mapping;

detect unavailability of at least one of the write threshold number of encoded data slices by interpreting an error message indicating the at least one of the write threshold number of encoded data slices is unavailable;

obtain first updated DSN configuration information by receiving updated system registry information, wherein the first updated DSN configuration information indicates at least one of the plurality of storage units is unavailable;

map a first updated write threshold number of encoded data slices to a first updated write threshold number of storage units to produce a first updated slice storage mapping based on the first updated DSN configuration information;

facilitate storage of the first updated write threshold number of encoded data slices to the first updated write threshold number of storage units in accordance with the first updated slice storage mapping;

obtain second updated DSN configuration information by receiving second updated system registry information, wherein the second updated DSN configuration information indicates a change in availability of at least one of the plurality of storage units indicated as unavailable in the first updated DSN configuration information;

map a second updated write threshold number of encoded data slices to a second updated write threshold number of storage units to produce a second updated slice storage mapping based on the second updated DSN configuration information, wherein the second updated write threshold number of encoded data slices is determined by altering the first updated write threshold number of encoded data slices based on the second updated DSN configuration information; and

facilitate storage of the second updated write threshold number of encoded data slices to the second updated write threshold number of storage units in accordance with the second updated slice storage mapping.

12. The processing system of claim 11, wherein obtaining the DSN configuration information includes interpreting a query response.

13. The processing system of claim 11, wherein the operational instructions, when executed by the at least one processor, further cause the processing system to:

perform a deterministic function on an identifier associated with the data segment, an IDA width number, and a number of available sites to produce the slice storage mapping.

14. The processing system of claim 13, wherein the deterministic function is performed to produce at least one of:

the first updated slice storage mapping or the second updated slice storage mapping, based on currently available encoded data slices of the write threshold number of encoded data slices.

15. The processing system of claim 11, wherein facilitating storage of the write threshold number of encoded data slices includes:

issuing a plurality of write slice requests to a set of storage units that includes the write threshold number of storage units; and

removing at least one encoded data slice beyond the write threshold number of encoded data slices from the set of encoded data slices; and

retrying storage of the write threshold number of encoded data slices in response to determining that the write threshold number of encoded data slices have not yet been successfully stored.

16. The processing system of claim 11, wherein detecting the unavailability of the at least one of the write threshold number of encoded data slices includes detecting a storage unit failure responsible for storing the at least one of the write threshold number of encoded data slices.

17. The processing system of claim 11, wherein facilitating storage of the first updated write threshold number of encoded data slices includes:

identifying a subset of encoded data slices of the first updated write threshold number of encoded data slices that have not been stored;

rebuilding the subset of encoded data slices; and

sending the rebuilt subset of encoded data slices to a corresponding subset of storage units of a set of storage units for storage.

18. The processing system of claim 11, wherein the operational instructions, when executed by the at least one processor, further cause the processing system to:

detect availability of a previously unavailable storage unit that is responsible for storage of a previously unavailable one of the set of encoded data slices.

19. The processing system of claim 11, wherein facilitating storage of the second updated write threshold number of encoded data slices includes:

deleting a first at least one previously stored encoded data slices of the set of encoded data slices that is not included in the second updated slice storage mapping; and

determining a second at least one encoded data slice that is included in the second updated slice storage mapping is not available; and

rebuilding the second at least one encoded data slice.

20. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by a processing system of a dispersed storage network (DSN) that includes a processor and a memory, causes the processing system to;

dispersed storage error encode a data segment to produce a set of encoded data slices;

obtain dispersed storage network (DSN) configuration information by receiving system registry information, wherein the DSN configuration information indicates a plurality of storage units;

map a write threshold number of encoded data slices to a write threshold number of storage units of the plurality of storage units, based on the DSN configuration information, to produce a slice storage mapping;

facilitate storage of the write threshold number of encoded data slices to the write threshold number of storage units in accordance with the slice storage mapping;

detect unavailability of at least one of the write threshold number of encoded data slices by interpreting an error message indicating the at least one of the write threshold number of encoded data slices is unavailable;

obtain first updated DSN configuration information by receiving updated system registry information, wherein the first updated DSN configuration information indicates at least one of the plurality of storage units is unavailable;

map a first updated write threshold number of encoded data slices to a first updated write threshold number of storage units to produce a first updated slice storage mapping based on the first updated DSN configuration information;

facilitate storage of the first updated write threshold number of encoded data slices to the first updated write threshold number of storage units in accordance with the first updated slice storage mapping;

obtain second updated DSN configuration information by receiving second updated system registry information, wherein the second updated DSN configuration information indicates a change in availability of at least one of the plurality of storage units indicated as unavailable in the first updated DSN configuration information;

map a second updated write threshold number of encoded data slices to a second updated write threshold number of storage units to produce a second updated slice storage mapping based on the second updated DSN configuration information, wherein the second updated write threshold number of encoded data slices is determined by altering the first updated write threshold number of encoded data slices based on the second updated DSN configuration information; and

facilitate storage of the second updated write threshold number of encoded data slices to the second updated write threshold number of storage units in accordance with the second updated slice storage mapping.