CONCURRENT SET STORAGE IN DISTRIBUTED STORAGE NETWORK

• US 20110107026A1
• Filed: 06/16/2010
• Published: 05/05/2011
• Est. Priority Date: 10/30/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: KAZI, ASIMUDDIN, QUIGLEY, JOHN, LAL, AKSHAY
• Granted Patent: US 8,589,637 B2
• Time in Patent Office: Days
• US Class Current: 711/114
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
• CPC Class Codes: G06F 21/6272 by registering files or documents with a third party
• CPC Class Codes: G06F 2221/2107 File encryption
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

For each original data segment, a distributed storage processing unit generates encoded slices designed to prevent the original data segment from being reconstructed using fewer than a threshold number of encoded slices. Multiple encoded slices are generated for each of two different data segments, and the slices associated with the first and second data segment are stored substantially concurrently in different storage sets employing different distributed storage units. Encoded slices for even and odd data segments can be stored in different storage sets, or longer sequences of data segments can be stored in alternating storage sets. Storage sets can also be determined by the vault generation of a particular data segment.

Claims

1. A method for use in a distributed storage processing unit, the method comprising:

generating a first plurality of data slices including different encoded versions of a first data portion;

generating a second plurality of data slices including different encoded versions of a second data portion; and

sending the first plurality of data slices and the second plurality of data slices to be stored concurrently in different storage sets, each of the different storage sets including a plurality of distributed storage units different from the distributed storage units included in others of the different storage sets;

at least a read threshold number of data slices from a first storage set required to reconstruct the first data portion; and

at least a read threshold number of data slices from a second storage set required to reconstruct the second data portion.

2. The method of claim 1, wherein the first data portion and the second data portion are data segments, the method further comprising:

sending data slices generated from even data segments to be stored in the first storage set; and

sending data slices generated from odd data segments to be stored in the second storage set.

3. The method of claim 1, wherein the first data portion and the second data portion are data segments, the method further comprising:

sending data slices generated from a first sequence of data segments to be stored in the first storage set; and

storing data slices generated from a second sequence of data segments to be stored in the second storage set.

4. The method claim 1, wherein the first data portion and the second data portion are different vault generations of a same data object, the method further comprising:

sending data slices generated from different generations of each of a plurality of data objects to be stored in different storage sets.

5. The method of claim 4, further comprising:

dividing each of the plurality of data objects into a plurality of sub-files;

labeling the plurality of sub-files with a same file name, but with different vault generations; and

generating the first plurality of data slices and the second plurality of data slices based, at least in part on the different vault generations.

6. The method of claim 1, further comprising:

determining which of a plurality of predetermined parallel storage schemes to implement based on a system parameter.

7. The method of claim 6, wherein the system parameter is selected from the group consisting of data type, priority level, security level, preferred performance level, system loading indicator, and system configuration.

8. A distributed storage processing unit comprising:

a processor to generate a first plurality of data slices including different encoded versions of a first data portion and a second plurality of data slices including different encoded versions of a second data portion;

an interface to send the first plurality of data slices and the second plurality of data slices to be stored concurrently in different storage sets, each of the different storage sets including a plurality of distributed storage units different from the distributed storage units included in others of the different storage sets;

at least a read threshold number of data slices from a first storage set required to reconstruct the first data portion; and

at least a read threshold number of data slices from a second storage set required to reconstruct the second data portion.

9. The distributed storage processing unit of claim 8, wherein the first data portion and the second data portion are data segments, the distributed storage processing unit further comprising:

the interface further to send data slices generated from even data segments to be stored in the first storage set and to send data slices generated from odd data segments to be stored in the second storage set.

10. The distributed storage processing unit of claim 8, wherein the first data portion and the second data portion are data segments, the distributed storage processing unit further comprising:

the interface further to send data slices generated from a first sequence of data segments to be stored in the first storage set; and

the interface further to send data slices generated from a second sequence of data segments to be stored in the second storage set.

11. The distributed storage processing unit claim 8, wherein the first data portion and the second data portion are different vault generations of a same data object, the distributed storage processing unit further comprising:

the interface further to send data slices generated from different generations of each of a plurality of data objects to be stored in different storage sets.

12. The distributed storage processing unit of claim 11, further comprising:

the processor further to;

divide each of the plurality of data objects into a plurality of sub-files;

label the plurality of sub-files with a same file name, but with different vault generations; and

generate the first plurality of data slices and the second plurality of data slices based, at least in part on the different vault generations.

13. The distributed storage processing unit of claim 8, further comprising:

the processor further to determine which of a plurality of predetermined parallel storage schemes to implement based on a system parameter.

14. The distributed storage processing unit of claim 13, wherein the system parameter is selected from the group consisting of data type, priority level, security level, preferred performance level, system loading indicator, and system configuration.

15. A distributed storage processing unit comprising:

a grid module to generate a first plurality of different data slices based on a first data segment;

the grid module further to generate a second plurality of different data slices based on a second data segment;

a storage module to;

assign the first plurality of different data slices to a first storage set, the first storage set designated to be stored in a first plurality of distributed storage units;

assign the second plurality of different data slices to a second storage set, the second storage set designated to be stored in a second plurality of distributed storage units different from the first plurality of distributed storage units;

an interface to send the second plurality of different data slices to be stored substantially in parallel with sending the first plurality of different data slices to be stored.

16. The distributed storage processing unit of claim 15, further comprising:

the storage module to assign data slices generated from even data segments to the first storage set, and to assign data slices generated from odd data segments to the second storage set.

17. The distributed storage processing unit of claim 15, further comprising:

the storage module to assign data slices generated from a first sequence of data segments to the first storage set, and to assign data slices generated from a second sequence of data segments to the second storage set.

18. The distributed storage processing unit claim 15, wherein the first data segment and the second data segment are sub-segments of a same data segment, the distributed storage processing unit further comprising:

the grid module to generate slice names of the first data segment and the second data segment, the slice names including different vault generations; and

the storage module further to assign data slices having slice names that include different vault generations to different storage sets.

19. The distributed storage processing unit of claim 18, further comprising:

an access module to determine a vault generation associated with each of a plurality of sub-files having a same file name; and

the grid module to generate the first plurality of different data slices and the second plurality of different data slices based, at least in part on the vault generation.

20. The distributed storage processing unit of claim 15, further comprising:

the storage module further to determine which of a plurality of predetermined parallel storage schemes to implement based on a system parameter.

21. The distributed storage processing unit of claim 20, wherein the system parameter is selected from the group consisting of data type, priority level, security level, preferred performance level, system loading indicator, and system configuration.