INDIRECT STORAGE OF DATA IN A DISPERSED STORAGE SYSTEM

• US 20110107027A1
• Filed: 08/04/2010
• Published: 05/05/2011
• Est. Priority Date: 10/30/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: GRUBE, GARY W., MARKISON, TIMOTHY W.
• Granted Patent: US 10,509,709 B2
• Time in Patent Office: Days
• US Class Current: 711/114
• CPC Class Codes: G06F 11/2089 Redundant storage control functionality
• CPC Class Codes: G06F 12/0802 Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
• CPC Class Codes: G06F 12/1009 using page tables, e.g. page table structures
• CPC Class Codes: G06F 12/1045 associated with a data cache
• CPC Class Codes: G06F 13/1668 Details of memory controller
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0629 Configuration or reconfiguration of storage systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: H03M 13/09 Error detection only, e.g. using cyclic redundancy check [CRC] codes or single parity bit
• CPC Class Codes: H03M 13/15 Cyclic codes, i.e. cyclic shifts of codewords produce other codewords, e.g. codes defined by a generator polynomial, Bose-Chaudhuri-Hocquenghem [BCH] codes H03M13/17 takes precedence
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/23 using convolutional codes, e.g. unit memory codes
• CPC Class Codes: H04L 41/06 Management of faults, events, alarms or notifications
• CPC Class Codes: H04L 45/00 Routing or path finding of packets in data switching networks routing or path finding in wireless networks H04W40/00
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a dispersed storage processing module obtaining data for storage. The method continues with the dispersed storage processing module encoding the data in accordance with an error coding dispersal storage function to produce a plurality of sets of encoded data slices. The method continues with the dispersed storage processing module determining a proxy unit. The method continues with the dispersed storage processing module transmitting the plurality of sets of encoded data slices to the proxy unit, wherein the proxy unit disperses the plurality of sets of encoded data slices to a plurality of dispersed storage units.

Claims

1. A method for execution by a dispersed storage processing module, the method comprises:

obtaining data for storage;

encoding the data in accordance with an error coding dispersal storage function to produce a plurality of sets of encoded data slices;

determining a proxy unit; and

transmitting the plurality of sets of encoded data slices to the proxy unit, wherein the proxy unit disperses the plurality of sets of encoded data slices to a plurality of dispersed storage (DS) units.

2. The method of claim 1 further comprises:

transmitting metadata to the proxy unit, wherein the metadata includes a dispersal approach regarding how the proxy unit is to disperse the plurality of sets of encoded data slices.

3. The method of claim 2, wherein the dispersal approach further comprises at least one of:

disperse in a sequential fashion;

disperse in a daisy chain fashion;

disperse in a one-to-many fashion; and

forward to a second proxy unit, wherein the second proxy unit disperses the plurality of sets of encoded data slices to the plurality of dispersed storage units.

4. The method of claim 1 further comprises:

receiving dispersed storage information regarding dispersed storage of the plurality of sets of encoded data slices, wherein the dispersed storage information includes one or more of;

an encoded data slice storage confirmation identifier;

a DS unit identifier and at least one associated slice name;

a seed DS unit identifier;

an end DS unit identifier; and

storage path information.

5. The method of claim 1, wherein the determining the proxy unit further comprises:

selecting one of the plurality of dispersed storage units.

6. A method for execution by a dispersed storage (DS) unit, the method comprises:

receiving a plurality of sets of encoded data slices;

receiving metadata associated with the plurality of sets of encoded data slices;

interpreting the metadata to determine storage instructions regarding the plurality of encoded data slices; and

when the storage instructions indicate daisy chain storage;

locally storing first encoded data slices of each set of encoded data slices; and

forwarding other encoded data slices of the sets of encoded data slices to at least one other DS unit.

7. The method of claim 6 further comprises:

when the storage instructions indicate the daisy chain storage;

updating the metadata to produce updated metadata;

locally storing the updated metadata; and

forwarding the updated metadata to the at least one other DS unit.

8. The method of claim 6 further comprises:

when the storage instructions indicate the daisy chain storage;

updating the metadata to produce updated metadata;

determining whether the DS unit is an end DS unit; and

when the DS unit is the end DS unit, forwarding the updated metadata to a DS processing module.

9. The method of claim 6 further comprises:

when the storage instructions indicate one-to-many storage;

transmitting pillar numbered encoded data slices of the plurality of sets of encoded data slices to corresponding ones of a plurality of DS units, wherein the plurality of DS units includes the at least one other DS unit.

10. The method of claim 6 further comprises:

receiving a retrieval message from an initiator;

retrieving locally stored metadata in response to the retrieval message;

retrieving stored encoded data slices in accordance with the locally stored metadata;

sending the stored encoded data slices to the initiator; and

forwarding the retrieval message to the at least one other DS unit in accordance with the locally stored metadata.

11. A dispersed storage processing module comprises:

an interface; and

a processing module operable to;

obtain data for storage;

encode the data in accordance with an error coding dispersal storage function to produce a plurality of sets of encoded data slices;

determine a proxy unit; and

transmit, via the interface, the plurality of sets of encoded data slices to the proxy unit, wherein the proxy unit disperses the plurality of sets of encoded data slices to a plurality of dispersed storage (DS) units.

12. The dispersed storage processing module of claim 11, wherein the processing module further functions to:

transmit, via the interface, metadata to the proxy unit, wherein the metadata includes a dispersal approach regarding how the proxy unit is to disperse the plurality of sets of encoded data slices.

13. The dispersed storage processing module of claim 12, wherein the dispersal approach further comprises at least one of:

disperse in a sequential fashion;

disperse in a daisy chain fashion;

disperse in a one-to-many fashion; and

forward to a second proxy unit, wherein the second proxy unit disperses the plurality of sets of encoded data slices to the plurality of dispersed storage units.

14. The dispersed storage processing module of claim 11, wherein the processing module further functions to:

receive, via the interface, dispersed storage information regarding dispersed storage of the plurality of sets of encoded data slices, wherein the dispersed storage information includes one or more of;

an encoded data slice storage confirmation identifier;

a DS unit identifier and at least one associated slice name;

a seed DS unit identifier;

an end DS unit identifier; and

storage path information.

15. The dispersed storage processing module of claim 11, wherein the processing module further functions to determine the proxy unit by:

selecting one of the plurality of dispersed storage units.

16. A dispersed storage (DS) unit comprises:

an interface;

a memory; and

a processing module operable to;

receive, via the interface, a plurality of sets of encoded data slices;

receive, via the interface, metadata associated with the plurality of sets of encoded data slices;

interpret the metadata to determine storage instructions regarding the plurality of encoded data slices; and

when the storage instructions indicate daisy chain storage;

locally store, in the memory, first encoded data slices of each set of encoded data slices; and

forward, via the interface, other encoded data slices of the sets of encoded data slices to at least one other DS unit.

17. The dispersed storage unit of claim 16, wherein the processing module further functions to:

when the storage instructions indicate the daisy chain storage;

update the metadata to produce updated metadata;

locally store, in the memory, the updated metadata; and

forward, via the interface, the updated metadata to the at least one other DS unit.

18. The dispersed storage unit of claim 16, wherein the processing module further functions to:

when the storage instructions indicate the daisy chain storage;

update the metadata to produce updated metadata;

determine whether the DS unit is an end DS unit; and

forward, via the interface, the updated metadata to a DS processing module, when the DS unit is the end DS unit.

19. The dispersed storage unit of claim 16, wherein the processing module further functions to:

when the storage instructions indicate one-to-many storage;

transmit, via the interface, pillar numbered encoded data slices of the plurality of sets of encoded data slices to corresponding ones of a plurality of DS units,

wherein the plurality of DS units includes the at least one other DS unit.

20. The dispersed storage unit of claim 16, wherein the processing module further functions to:

receive, via the interface, a retrieval message from an initiator;

retrieve, from the memory, locally stored metadata in response to the retrieval message;

retrieve, from the memory, stored encoded data slices in accordance with the locally stored metadata;

send, via the interface, the stored encoded data slices to the initiator; and

forward, via the interface, the retrieval message to the at least one other DS unit in accordance with the locally stored metadata.