Storing data in a dispersed storage network

• US 8,683,286 B2
• Filed: 09/12/2012
• Issued: 03/25/2014
• Est. Priority Date: 11/01/2011
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Gladwin, S. Christopher, Leggette, Wesley, Baptist, Andrew, Resch, Jason K.
• Primary Examiner(s): Decady, Albert
• Assistant Examiner(s): Alshack, Osman
• Application Number:
  US13/611,803
• Publication Number:
  US 20130111293A1
• Time in Patent Office: 559 Days
• Field of Search: 714/752, 714/770, 714/774, 714/746
• US Class Current: 714/752
• CPC Class Codes: G06F 11/073 in a memory management context, e.g. virtual memory or cache management memory management G06F12/00; testing of static memory units G11C29/00
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: H04L 1/0057 Block codes H04L1/0061, H04L1/0064 take precedence
• CPC Class Codes: H04L 1/0061 Error detection codes
• CPC Class Codes: H04L 1/0072 Error control for data other than payload data, e.g. control data
• CPC Class Codes: H04L 1/0083 Formatting with frames or packets; Protocol or part of protocol for error control
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins for a first group of data segments by a dispersed storage (DS) processing module selecting a first vault parameter set from a plurality of vault parameter sets and generating a first plurality of sets of slice names for a first plurality of sets of encoded slices in accordance with the first vault parameter set. The method continues for a second group of data segments with the DS processing module selecting a second vault parameter set from the plurality of vault parameter sets and generating a second plurality of sets of slice names for a second plurality of sets of encoded slices in accordance with the second vault parameter set.

Claims

1. A method for execution by a computing device, the method comprises:

receiving first data for storage in a vault of a dispersed storage network (DSN) memory, wherein the DSN memory includes a plurality of storage nodes, wherein the vault is a virtual memory within the DSN memory and includes multiple storage nodes;

selecting, for storing the first data in the vault, a first vault parameter set from a plurality of vault parameter sets associated with the vault, wherein the first vault parameter set includes a first per set decode threshold and a first total number of encoded data slices per set regarding dispersed error encoding data segments of the first data;

encoding the data segments of the first data in accordance with the first vault parameter set to produce a first plurality of sets of encoded slices;

outputting the first plurality of sets of encoded data slices for storage in the vault;

receiving second data for storage in the vault of the DSN memory;

selecting, for storing the second data, a second vault parameter set from the plurality of vault parameter sets associated with the vault, wherein the second vault parameter set includes a second per set decode threshold and a first total number of encoded data slices per set regarding dispersed error encoding data segments of the second data;

encoding the data segments of the second data in accordance with the second vault parameter set to produce a second plurality of sets of encoded slices, wherein a set of encoded data slices of the first plurality of sets of encoded data slices includes less encoded data slices than a set of encoded data slices of the second plurality of sets of encoded data slices; and

outputting the second plurality of sets of encoded data slices for storage in the vault.

2. The method of claim 1 further comprises:

selecting the first vault parameter set based on one or more of;

identifying a storage requirement for the first data; and

interpreting metadata associated with the first data; and

selecting the second vault parameter set based on one or more of;

identifying a storage requirement for the second data; and

interpreting metadata associated with the second data.

3. The method of claim 1 further comprises:

the first set of parameters including identity of a first dispersed storage error encoding algorithm and first error coding redundancy information; and

the second set of parameters including identity of a second dispersed storage error encoding algorithm and second error coding redundancy information.

4. The method of claim 1 further comprises:

generating a first plurality of sets of slice names for the first plurality of sets of encoded slices in accordance with the first vault parameter set, wherein a slice name of the first plurality of sets of slice names includes a vault identifier, a first vault parameter set identifier, and a common object name for the first data; and

generating a second plurality of sets of slice names for the second plurality of sets of encoded slices in accordance with the second vault parameter set, wherein a slice name of the second plurality of sets of slice names includes the vault identifier, a second vault parameter set identifier, and a common object name for the second data.

5. The method of claim 4 further comprises:

the slice name of the first plurality of sets of slice names further including a first slice index and a first segment number, wherein the first slice index corresponds to a slice number within a set of the first plurality of sets of encoded slices and the first segment number corresponds to a data segment number of the first group of data segments; and

the slice name of the second plurality of sets of slice names further including a second slice index and a second segment number, wherein the second slice index corresponds to a slice number within a set of the second plurality of sets of encoded slices and the second segment number corresponds to a data segment number of the second group of data segments.

6. The method of claim 1 further comprises:

obtaining first metadata regarding the first data; and

for the first metadata;

selecting a third vault parameter set from the plurality of vault parameter sets, wherein the third vault parameter set includes a third set of parameters regarding dispersed error encoding the first metadata;

encoding the first metadata in accordance with the third vault parameter set to produce a first set of metadata encoded slices; and

outputting the first set of metadata encoded slices for storage in a third set of the multiple storage nodes of the vault.

7. The method of claim 1 further comprises:

obtaining second metadata regarding the second data; and

for the second metadata;

selecting a third vault parameter set from the plurality of vault parameter sets, wherein the third vault parameter set includes a third set of parameters regarding dispersed error encoding the second metadata;

encoding the second metadata in accordance with the third vault parameter set to produce a second set of metadata encoded slices; and

outputting the second set of metadata encoded slices for storage in a third set of the multiple storage nodes of the vault.

8. The method of claim 1 further comprises:

the first data including one of a first file, a first data object, a first data block, a first data stream, and a first file directory object; and

the second data including one of a second file, a second data object, a second data block, a second data stream, and a second file directory object.

9. A dispersed storage (DS) module comprises:

a first module, when operable within a computing device, causes the computing device to;

receive first data for storage in a vault of a dispersed storage network (DSN) memory, wherein the DSN memory includes a plurality of storage nodes, wherein the vault is a virtual memory within the DSN memory and includes multiple storage nodes; and

receiving second data for storage in the vault of the DSN memory;

a second module, when operable within the computing device, causes the computing device to;

select, for storing the first data in the vault, a first vault parameter set from a plurality of vault parameter sets associated with the vault, wherein the first vault parameter set includes a first per set decode threshold and a first total number of encoded data slices per set regarding dispersed error encoding data segments of the first group of data segments; and

select, for storing the second data in the vault, a second vault parameter set from the plurality of vault parameter sets associated with the vault, wherein the second vault parameter set includes a second per set decode threshold and a second total number of encoded data slices per set regarding dispersed error encoding data segments of the second group of data segments;

a third module, when operable within the computing device, causes the computing device to;

encode the data segments of the first data in accordance with the first vault parameter set to produce a first plurality of sets of encoded slices; and

encode the data segments of the second data in accordance with the second vault parameter set to produce a second plurality of sets of encoded slices, wherein a set of encoded data slices of the first plurality of sets of encoded data slices includes less encoded data slices than a set of encoded data slices of the second plurality of sets of encoded data slices; and

a fourth module, when operable within the computing device, causes the computing device to;

output the first plurality of sets of encoded data slices for storage in the vault; and

output the second plurality of sets of encoded data slices for storage in the vault.

10. The DS module of claim 9 further comprises:

the second module further functions to;

select the first vault parameter set based on one or more of;

identifying a storage requirement for the first data; and

interpreting metadata associated with the first data; and

select the second vault parameter set based on one or more of;

identifying a storage requirement for the second data; and

interpreting metadata associated with the second data.

11. The DS module of claim 9 further comprises:

the first set of parameters including identity of a first dispersed storage error encoding algorithm and first error coding redundancy information; and

the second set of parameters including identity of a second dispersed storage error encoding algorithm and second error coding redundancy information.

12. The DS module of claim 9 further comprises:

a fifth module, when operable within the computing device, causes the computing device to;

generate a first plurality of sets of slice names for the first plurality of sets of encoded slices in accordance with the first vault parameter set, wherein a slice name of the first plurality of sets of slice names includes a vault identifier, a first vault parameter set identifier, and a common object name for the first data; and

generate a second plurality of sets of slice names for the second plurality of sets of encoded slices in accordance with the second vault parameter set, wherein a slice name of the second plurality of sets of slice names includes the vault identifier, a second vault parameter set identifier, and a common object name for the second data.

13. The DS module of claim 12 further comprises:

the slice name of the first plurality of sets of slice names further including a first slice index and a first segment number, wherein the first slice index corresponds to a slice number within a set of the first plurality of sets of encoded slices and the first segment number corresponds to a data segment number of the first group of data segments; and

the slice name of the second plurality of sets of slice names further including a second slice index and a second segment number, wherein the second slice index corresponds to a slice number within a set of the second plurality of sets of encoded slices and the second segment number corresponds to a data segment number of the second group of data segments.

14. The DS module of claim 9 further comprises:

the second module further functions to;

obtain first metadata regarding the first data; and

for the first metadata, select a third vault parameter set from the plurality of vault parameter sets, wherein the third vault parameter set includes a third set of parameters regarding dispersed error encoding the first metadata;

the third module further functions to;

encode the first metadata in accordance with the third vault parameter set to produce a first set of metadata encoded slices; and

the fourth module further functions to;

output the first set of metadata encoded slices for storage in a third set of the multiple storage nodes of the vault.

15. The DS module of claim 9 further comprises:

the second module further functions to;

obtain second metadata regarding the second data; and

for the second metadata, select a third vault parameter set from the plurality of vault parameter sets, wherein the third vault parameter set includes a third set of parameters regarding dispersed error encoding the second metadata;

the third module further functions to;

encode the second metadata in accordance with the third vault parameter set to produce a second set of metadata encoded slices; and

the fourth module further functions to;

output the second set of metadata encoded slices for storage in a third set of the multiple storage nodes of the vault.

16. The DS module of claim 9 further comprises:

the first data includes one of a first file, a first data object, a first data block, a first data stream, and a first file directory object; and

the second data includes one of a second file, a second data object, a second data block, a second data stream, and a second file directory object.