Forming a distributed storage network memory without namespace aware distributed storage units

• US 10,310,763 B2
• Filed: 11/27/2017
• Issued: 06/04/2019
• Est. Priority Date: 02/05/2013
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Kazi, Asimuddin, Resch, Jason K.
• Primary Examiner(s): Sun, Scott C
• Application Number:
  US15/822,472
• Publication Number:
  US 20180074744A1
• Time in Patent Office: 554 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1004 to protect a block of data words, e.g. CRC or checksum G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0634 by changing the state or mode of one or more devices
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method for execution by a dispersed storage network (DSN). The method begins by selecting a pillar width number of dispersed storage (DS) units of a DS unit pool for storing data, segmenting the data based on a segmentation scheme to produce a plurality of segments, issuing, for each segment of the plurality of segments, a pillar width number of write slice requests to the pillar width number of DS units, determining that an unfavorable number of write errors have occurred, and for each of the write errors, re-issuing a corresponding write slice request to another DS unit of remaining DS units of the DS unit pool, generating a DSN address for the data based on identities of actual DS units utilized, and updating at least one of a DSN index and a DSN directory to associate the DSN address with a data identifier of the data.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

selecting a pillar width number of dispersed storage (DS) units of a DS unit pool for storing data;

segmenting the data based on a segmentation scheme to produce a plurality of segments;

issuing, for each segment of the plurality of segments, a pillar width number of write slice requests to the pillar width number of DS units;

determining that an unfavorable number of write errors have occurred at one or more of the selected pillar width number of dispersed storage (DS) units, wherein the unfavorable number of write errors has occurred when the one or more processing modules have not received at least a write threshold number of successful write slice responses;

for each of the write errors, re-issuing a corresponding write slice request to another DS unit of remaining DS units of the DS unit pool, wherein the re-issuing a corresponding write slice request to another DS unit includes generating a new slice name for a slice, generating a new slice request to include the new slice name and the slice, and outputting the corresponding write slice request to remaining DS units of the DS unit pool separate from the original pillar width number of DS units;

generating a DSN address for the data based on identities of actual DS units utilized; and

updating at least one of a DSN index and a DSN directory to associate the DSN address with a data identifier of the data.

2. The method of claim 1, wherein the actual DS units utilized includes DS units from the selected pillar width number of dispersed storage (DS) units without write errors and the DS units of the remaining DS units of the DS unit pool utilized during the re-issuing a corresponding write slice request to another DS unit.

3. The method of claim 1, wherein the selecting a pillar width number of dispersed storage (DS) units is based on one or more of storage availability, storage performance history, proximity, or affiliation with a requesting entity.

4. The method of claim 1, wherein the issuing includes encoding each segment using a dispersed storage error coding function to produce a pillar width number of slices and generating the pillar number of write slice requests to include a pillar width number of temporary slice names and the pillar width number of slices.

5. The method of claim 4, wherein the pillar width number of temporary slice names include a unique data identifier of data being stored, a segment identifier, and a pillar identifier.

6. The method of claim 1, wherein the generating a DSN address includes producing a portion of the DSN address based on a deterministic function applied to each identifier of the DS units utilized.

7. The method of claim 1, wherein the generating a DSN address includes concatenating internet protocol addresses of the DS units utilized.

8. The method of claim 1, wherein the updating at least one of a DSN index and a DSN directory includes storing the DSN address in an index entry associated with the data identifier.

9. The method of claim 1, wherein the associate the DSN address with a data identifier further comprises updating the DS units utilized with the DSN addresses to associate the DSN addresses with temporary slice names used during the issuing or re-issuing.

10. A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

select a pillar width number of dispersed storage (DS) units of a DS unit pool for storing data;

segment the data based on a segmentation scheme to produce a plurality of segments;

issue, for each segment of the plurality of segments, a pillar width number of write slice requests to the pillar width number of DS units;

determine that an unfavorable number of write errors have occurred, wherein the unfavorable number of write errors has occurred when the processing module has not received at least a write threshold number of successful write slice responses;

for each of the write errors, re-issuing a corresponding write slice request to another DS unit of remaining DS units of the DS unit pool, wherein the re-issuing a corresponding write slice request to another DS unit includes generating a new slice name for a slice, generating a new slice request to include the new slice name and the slice, and outputting the corresponding write slice request to remaining DS units of the DS unit pool separate from the original pillar width number of DS units;

determine that an unfavorable number of write errors have not occurred;

generate a DSN address for the data based on identities of actual DS units utilized; and

update at least one of a DSN index and a DSN directory to associate the DSN address with a data identifier of the data.

11. The computing device of claim 10, wherein the actual DS units utilized includes DS units from the selected pillar width number of dispersed storage (DS) units without write errors and the DS units of the remaining DS units of the DS unit pool utilized during the re-issuing a corresponding write slice request to another DS unit.

12. The computing device of claim 10, wherein the issue includes encoding each segment using a dispersed storage error coding function to produce a pillar width number of slices and generating the pillar width number of write slice requests to include a pillar width number of temporary slice names and the pillar width number of slices.

13. The computing device of claim 12, wherein the pillar width number of temporary slice names include a unique data identifier of data being stored, a segment identifier, and a pillar identifier.

14. The computing device of claim 10, wherein the generate a DSN address includes concatenating internet protocol addresses of the DS units utilized.

15. A system, the system comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

select a pillar width number of dispersed storage (DS) units of a DS unit pool for storing data;

segment the data based on a segmentation scheme to produce a plurality of segments;

issue, for each segment of the plurality of segments, a pillar width number of write slice requests to the pillar width number of DS units;

determine that an unfavorable number of write errors have occurred, wherein the unfavorable number of write errors has occurred when the processing module has not received at least a write threshold number of successful write slice responses;

for each of the write errors, re-issuing a corresponding write slice request to another DS unit of remaining DS units of the DS unit pool, wherein the re-issuing a corresponding write slice request to another DS unit includes generating a new slice name for a slice, generating a new slice request to include the new slice name and the slice, and outputting the corresponding write slice request to remaining DS units of the DS unit pool separate from the original pillar width number of DS units;

generate a DSN address for the data based on identities of actual DS units utilized; and

update at least one of a DS Network (DSN) index and a DSN directory to associate the DSN address with a data identifier of the data.

16. The system of claim 15, wherein the actual DS units utilized includes DS units from the selected pillar width number of dispersed storage (DS) units without write errors and the DS units of the remaining DS units of the DS unit pool utilized during the re-issuing a corresponding write slice request to another DS unit.