NESTED DISTRIBUTED STORAGE UNIT AND APPLICATIONS THEREOF

• US 20110055662A1
• Filed: 02/25/2010
• Published: 03/03/2011
• Est. Priority Date: 08/27/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: MARKISON, TIMOTHY W., GRUBE, GARY W.
• Granted Patent: US 9,047,217 B2
• Time in Patent Office: Days
• US Class Current: 714/763
• CPC Class Codes: G06F 11/1008 in individual solid state devices G06F11/1004 takes precedence
• CPC Class Codes: G06F 11/1088 Reconstruction on already foreseen single or plurality of spare disks
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 21/6272 by registering files or documents with a third party
• CPC Class Codes: H04L 63/126 the source of the received data

Abstract

A method for execution by a DS storage unit begins with the DS storage unit receiving an encoded slice of a plurality of encoded slices, wherein the plurality of encoded slices was generated from a data segment using an error encoding function. The method continues with the DS storage unit determining whether the encoded slice is to be sub-sliced using a sub-slicing encoding function. The method continues with the DS storage unit generating a plurality of encoded sub-slices from the encoded slices using the encoded sub-slicing encoding function when the encoded slice is to be sub-sliced. The method continues with the DS storage unit outputting the plurality of encoded sub-slices to a plurality of DS storage units.

Claims

1. A method for execution by a distributed storage (DS) storage unit, the method comprises:

receiving an encoded slice of a plurality of encoded slices, wherein the plurality of encoded slices was generated from a data segment using an error encoding function;

determining whether the encoded slice is to be sub-sliced using a sub-slicing encoding function;

when the encoded slice is to be sub-sliced, generating a plurality of encoded sub-slices from the encoded slices using the encoded sub-slicing encoding function; and

outputting the plurality of encoded sub-slices to a plurality of DS storage units.

2. The method of claim 1, wherein the outputting the plurality of encoded sub-slices further comprises:

outputting the plurality of encoded sub-slices to the plurality of DS storage units, wherein the plurality of DS storage units includes the DS storage unit.

3. The method of claim 1, wherein the determining whether the encoded slice is to be sub-sliced further comprises:

receiving a command to generate the plurality of encoded sub-slices.

4. The method of claim 1, wherein the determining whether the encoded slice is to be sub-sliced further comprises:

determining to sub-slice the encoded slice based on at least one of;

retrieval latency for each of the plurality of DS storage units;

priority of the selected encoded slice;

a data type of the data segment;

availability of each of the plurality of DS storage units; and

cost of storing an encoded sub-slice of the plurality of encoded sub-slices in each of the plurality of DS storage units.

5. The method of claim 1 further comprises:

selecting one of a plurality of sub-slicing encoding functions as the sub-slicing encoding function, wherein the plurality of sub-slicing encoding functions includes the error encoding function.

6. The method of claim 1 further comprises:

maintaining a list of the plurality of DS storage units storing the plurality of encoded sub-slices.

7. The method of claim 6 further comprises:

sending the list to a DS processing unit.

8. The method of claim 1 further comprises at least one of:

outputting a command with an encoded sub-slice of the plurality of encoded sub-slices to enable further sub-slicing of the encoded sub-slice; and

outputting a command with at least one of the plurality of encoded sub-slices to prohibit further sub-slicing of the at least one of the plurality of encoded sub-slices.

9. A distributed storage (DS) storage unit comprises:

a network interface;

memory;

a processing module; and

an encoding module, wherein;

the network interface receives an encoded slice of a plurality of encoded slices, wherein the plurality of encoded slices was generated from a data segment using an error encoding function;

the processing module determines whether the encoded slice is to be sub-sliced using a sub-slicing encoding function;

when the encoded slice is to be sub-sliced, the encoding module generates a plurality of encoded sub-slices from the encoded slices using the encoded sub-slicing encoding function;

when the encoded slice is not to be sub-sliced, the processing module coordinates storage of the encoded slice in the memory; and

the network interface outputs at least some of the plurality of encoded sub-slices to a plurality of DS storage units.

10. The DS storage unit of claim 9 further comprises:

the processing module identifying the plurality of DS storage units, which includes the DS storage unit;

the processing module selects one of the plurality of encoded sub-slices for storage within the DS storage unit to produce a selected encoded sub-slice;

the network interface outputs the plurality of encoded sub-slices, less the selected encoded sub-slice, to the plurality of DS storage units; and

the processing module coordinates storage of the selected encoded sub-slice in the memory.

11. The DS storage unit of claim 9, wherein the processing module further functions to determine whether the encoded slice is to be sub-sliced further by:

receiving a command via the network interface to generate the plurality of encoded sub-slices.

12. The DS storage unit of claim 9, wherein the processing module further functions to determine whether the encoded slice is to be sub-sliced further by:

determining to sub-slice the encoded slice based on at least one of;

retrieval latency for each of the plurality of DS storage units;

priority of the selected encoded slice;

a data type of the data segment;

availability of each of the plurality of DS storage units; and

cost of storing an encoded sub-slice of the plurality of encoded sub-slices in each of the plurality of DS storage units.

13. The DS storage unit of claim 9, wherein the processing module further functions to:

select one of a plurality of sub-slicing encoding functions as the sub-slicing encoding function, wherein the plurality of sub-slicing encoding functions includes the error encoding function.

14. The DS storage unit of claim 9, wherein the processing module further functions to:

maintain a list of the plurality of DS storage units storing the plurality of encoded sub-slices in the memory.

15. The DS storage unit of claim 14 further comprises:

the processing module sending the list to a DS processing unit via the network interface.

16. The DS storage unit of claim 9 further comprises at least one of:

the processing module generates a first command to enable further sub-slicing of the encoded sub-slice, wherein the processing module sends, via the network interface, the first command to a first set of the plurality of DS storage units;

the processing module generates a second command to prohibit further sub-slicing of the encoded sub-slice, wherein the processing module sends, via the network interface, the second command to a second set of the plurality of DS storage units.