Storing encoded data slices in a dispersed storage unit

• US 8,555,130 B2
• Filed: 10/04/2011
• Issued: 10/08/2013
• Est. Priority Date: 10/04/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Baptist, Andrew, Volvovski, Ilya, Leggette, Wesley, Dhuse, Greg, Resch, Jason K.
• Primary Examiner(s): Tabone, Jr., John J
• Application Number:
  US13/252,340
• Publication Number:
  US 20130086442A1
• Time in Patent Office: 735 Days
• Field of Search: 714/702, 714/718, 714/751, 714/752, 714/763, 714/769, 714/770, 714/773, 714/799, 714/6.1, 714/6.2, 714/6.22, 714/6.24, 714/42, 711/111, 711/114, 711/170, 709/231, 707/822
• US Class Current: 714/752
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: H03M 13/09 Error detection only, e.g. using cyclic redundancy check [CRC] codes or single parity bit
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/23 using convolutional codes, e.g. unit memory codes
• CPC Class Codes: H03M 13/29 combining two or more codes or code structures, e.g. product codes, generalised product codes, concatenated codes, inner and outer codes

Abstract

A method begins by a processing module receiving a write request that includes a batch of encoded data slices and a corresponding batch of slice names, wherein the batch of encoded data slices includes encoded data slices that have slices names that have a common data object storage name, a common slice storage name, and a different data segment storage name. The method continues with the processing module determining whether a storage file exists based on the common data object storage name. The method continues with the processing module creating the storage file based on the common data object storage name when the storage file does not exist. The method continues with the processing module storing the batch of encoded data slices in the storage file based on the corresponding batch of slice names.

Claims

1. A method for storing a data object in a dispersed storage network (DSN) comprises:

generating a data object storage name for the data object;

dividing the data object into a plurality of data segments;

generating a data segment storage name for each of the plurality of data segments;

for each data segment of the plurality of data segments;

dispersed storage error encoding the data segment to produce a set of encoded data slices;

generating a slice storage name for each of the set of encoded data slices; and

generating a slice name for each of the set of encoded data slices to produce a set of slice names, wherein the slice name includes the data object storage name, the data segment storage name, and the slice storage name; and

outputting a set of write requests to a set of dispersed storage (DS) units of the DSN, wherein a write request of the set of write requests includes a batch of encoded data slices and a corresponding batch of slice names, wherein the batch of encoded data slices includes encoded data slices that have slices names that have the data object storage name, a common slice storage name, and a different data segment storage name such that a DS unit of the set of DS units stores the batch of encoded data slices in a storage file.

2. The method of claim 1, wherein the data object storage name comprises at least one of:

a source identification code;

a vault identification code;

a file name;

a data object name; and

a user identification code.

3. The method of claim 1, wherein the generating the data object storage name comprises:

determining whether the data object storage name exists;

when the data object storage name does not exist, utilizing the data object storage name for the data object;

when the data object storage name exists, determining that the data object corresponds to the data object storage name;

when the data object corresponds to the data object storage name, determining whether the storage file has available storage space; and

when the storage file has available storage space, utilizing the data object storage name for the data object.

4. The method of claim 3 further comprises:

when the data object corresponds to the data object storage name or the storage file does not have available storage space, generating a new data object storage name for the data object to use as the data object storage name.

5. A method comprises:

receiving a write request that includes a batch of encoded data slices and a corresponding batch of slice names, wherein the batch of encoded data slices includes encoded data slices that have slices names that have a common data object storage name, a common slice storage name, and a different data segment storage name;

determining whether a storage file exists based on the common data object storage name;

when the storage file does not exist, creating the storage file based on the common data object storage name; and

storing the batch of encoded data slices in the storage file based on the corresponding batch of slice names.

6. The method of claim 5 further comprises:

when the storage file exists;

determining whether the storage file has available storage space;

when the storage file has available storage space, determining a write position within the storage file;

storing the batch of encoded data slices based on the write position.

7. The method of claim 5 further comprises:

when the storage file exists;

determining whether the storage file has available storage space;

when the storage file does not have available storage space, modifying parameters of the storage file; and

storing the batch of encoded data slices in accordance with the modified parameters.

8. The method of claim 7, wherein the modifying the parameters of the storage file comprises:

modifying one or more of a revision number, a checksum, integrity information, the slice name, the source name, a user identifier, a file name, a slice length corresponding to the encoded data slice, and a storage file identifier.

9. The method of claim 5, wherein the creating the storage file comprises:

initializing the storage file; and

determining a write position within the storage file based on at least one of;

a sum of each slice length corresponding to each previously stored encoded data slice based on the data segment storage name; and

a sum of a last write position and a slice length of a last written encoded data slice.

10. A method comprises:

receiving a read request that includes a data object storage name;

identifying a storage file based on the data object storage name;

retrieving a batch of encoded data slices and a corresponding batch of slice names from the storage file, wherein the batch of encoded data slices includes encoded data slices that have slices names that have the data object storage name, a common slice storage name, and a different data segment storage name; and

outputting the batch of encoded data slices and the corresponding batch of slice names.

11. The method of claim 10, wherein the retrieving the batch of encoded data slices comprises:

determining a read position within the storage file; and

retrieving an encoded data slice of the batch of encoded data slices from the storage file based on the read position.

12. The method of claim 11, wherein the determining the read position comprises at least one of:

a sum of each slice length corresponding to each previously stored encoded data slice based on the different data segment storage name; and

a sum of a last read position and a slice length of a last read encoded data slice.

13. A computer comprises:

an interface;

a memory; and

a processing module operable to;

generate a data object storage name for a data object to be stored in a dispersed storage network (DSN);

divide the data object into a plurality of data segments;

generate a data segment storage name for each of the plurality of data segments;

for each data segment of the plurality of data segments;

dispersed storage error encode the data segment to produce a set of encoded data slices;

generate a slice storage name for each of the set of encoded data slices; and

generate a slice name for each of the set of encoded data slices to produce a set of slice names, wherein the slice name includes the data object storage name, the data segment storage name, and the slice storage name; and

output, via the interface, a set of write requests to a set of dispersed storage (DS) units of the DSN, wherein a write request of the set of write requests includes a batch of encoded data slices and a corresponding batch of slice names, wherein the batch of encoded data slices includes encoded data slices that have slices names that have the data object storage name, a common slice storage name, and a different data segment storage name such that a DS unit of the set of DS units stores the batch of encoded data slices in a storage file.

14. The computer of claim 13, wherein the data object storage name comprises at least one of:

a source identification code;

a vault identification code;

a file name;

a data object name; and

a user identification code.

15. The computer of claim 13, wherein the processing module functions to generate the data object storage name by:

determining whether the data object storage name exists;

when the data object storage name does not exist, utilizing the data object storage name for the data object;

when the data object storage name exists, determining that the data object corresponds to the data object storage name;

when the data object corresponds to the data object storage name, determining whether the storage file has available storage space; and

when the storage file has available storage space, utilizing the data object storage name for the data object.

16. The computer of claim 15, wherein the processing module further functions to:

when the data object corresponds to the data object storage name or the storage file does not have available storage space, generate a new data object storage name for the data object to use as the data object storage name.

17. A computer comprises:

an interface;

a memory; and

a processing module operable to;

receive, via the interface, a write request that includes a batch of encoded data slices and a corresponding batch of slice names, wherein the batch of encoded data slices includes encoded data slices that have slices names that have a common data object storage name, a common slice storage name, and a different data segment storage name;

determine whether a storage file exists, within the memory, based on the common data object storage name;

when the storage file does not exist, create the storage file based on the common data object storage name; and

store, via the interface, the batch of encoded data slices in the storage file based on the corresponding batch of slice names.

18. The computer of claim 17, wherein the processing module further functions to:

when the storage file exists;

determine whether the storage file has available storage space;

when the storage file has available storage space, determine a write position within the storage file;

store, via the interface, the batch of encoded data slices based on the write position.

19. The computer of claim 17, wherein the processing module further functions to:

when the storage file exists;

determine whether the storage file has available storage space;

when the storage file does not have available storage space, modify parameters of the storage file; and

store, via the interface, the batch of encoded data slices in accordance with the modified parameters.

20. The computer of claim 19, wherein the processing module functions to modify the parameters of the storage file by:

modifying one or more of a revision number, a checksum, integrity information, the slice name, the source name, a user identifier, a file name, a slice length corresponding to the encoded data slice, and a storage file identifier.

21. The computer of claim 17, wherein the processing module functions to create the storage file by:

initializing the storage file; and

determining a write position within the storage file based on at least one of;

a sum of each slice length corresponding to each previously stored encoded data slice based on the data segment storage name; and

a sum of a last write position and a slice length of a last written encoded data slice.

22. A computer comprises:

an interface;

a memory; and

a processing module operable to;

receive, via the interface, a read request that includes a data object storage name;

identify a storage file, within the memory, based on the data object storage name;

retrieve, via the interface, a batch of encoded data slices and a corresponding batch of slice names from the storage file, wherein the batch of encoded data slices includes encoded data slices that have slices names that have the data object storage name, a common slice storage name, and a different data segment storage name; and

output, via the interface, the batch of encoded data slices and the corresponding batch of slice names.

23. The computer of claim 22, wherein the processing module functions to retrieve the batch of encoded data slices by:

determining a read position within the storage file; and

retrieving, via the interface, an encoded data slice of the batch of encoded data slices from the storage file based on the read position.

24. The computer of claim 23, wherein the processing module functions to determine the read position by at least one of:

a sum of each slice length corresponding to each previously stored encoded data slice based on the different data segment storage name; and

a sum of a last read position and a slice length of a last read encoded data slice.