Creating a New File for a Dispersed Storage Network

• US 20130138706A1
• Filed: 11/21/2012
• Published: 05/30/2013
• Est. Priority Date: 11/28/2011
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Dhuse, Greg, Leggette, Wesley, Resch, Jason K.
• Granted Patent: US 9,584,326 B2
• Time in Patent Office: Days
• US Class Current: 707/827
• CPC Class Codes: G06F 11/108 Parity data distribution in semiconductor storages, e.g. in SSD
• CPC Class Codes: G06F 12/0607 Interleaved addressing
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 65/10 Architectures or entities
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 9/0825 using asymmetric-key encryption or public key infrastructure [PKI], e.g. key signature or public key certificates
• CPC Class Codes: H04L 9/3236 using cryptographic hash functions

Abstract

A method begins by a dispersed storage (DS) processing module receiving a file creation request to create a new file from portions of a set of existing files. The method continues with the DS processing module identifying data segments associated with the set of existing files corresponding to the portions of the set of existing files to produce identified data segments and identifying sets of encoded data slices of the identified data segments to produce identified sets of encoded data slices. The method continues with the DS processing module determining addressing information for the identified sets of encoded data slices from existing addressing information of the set of existing files and generating a segment allocation table for the new file based on the addressing information of the identified sets of encoded data slices such that the new file is created without duplication of the portions of the existing files.

Claims

1. A method comprises:

receiving a file creation request to create a new file from portions of a set of existing files, wherein an existing file of the set of existing files includes a plurality of data segments that is stored as a plurality of sets of encoded data slices in a dispersed storage network (DSN) memory;

identifying data segments from the plurality of data segments of the set of existing files corresponding to the portions of the set of existing files to produce identified data segments;

identifying sets of encoded data slices of the plurality of sets of encoded data slices of the identified data segments to produce identified sets of encoded data slices;

determining addressing information for the identified sets of encoded data slices from existing addressing information of the set of existing files; and

generating a segment allocation table for the new file based on the addressing information of the identified sets of encoded data slices such that the new file is created without duplication of the portions of the existing files.

2. The method of claim 1 further comprises:

the existing file of the set of existing files further includes a set of data regions, wherein a data region of the set of data regions includes a consecutive subset of the plurality of data segments;

identifying data regions from the set of data regions of the set of existing files corresponding to the portions of the set of existing files to produce identified data regions;

identifying the sets of encoded data slices of the plurality of sets of encoded data slices of the identified data regions to produce the identified sets of encoded data slices; and

determining the addressing information for the identified sets of encoded data slices from a set of segment allocation tables associated with the set of existing files, wherein the set of segment allocation tables includes the existing addressing information of the set of existing files at data region level.

3. The method of claim 2, wherein the identifying the sets of encoded data slices comprises:

retrieving a set of segment allocation tables for the set of existing files based on the file creation request, wherein a segment allocation table of the set of segment allocation tables includes a plurality of entries, and wherein an entry of the plurality of entries includes a DSN address regarding storage of a data region of the set of data segments of the set of existing files in the DSN memory and segmentation information regarding the dividing of the data region into the consecutive subset of the plurality of data segments.

4. The method of claim 2, wherein addressing information for an identified set of encoded data slices of the identified sets of encoded data slices comprises:

a start segment vault source name corresponding to a data region associated with the identified set of encoded data slices; and

segmentation information including one or more of;

a segment size of data segments of the data region;

a total length of the data region; and

a segmentation approach of the data region.

5. The method of claim 1, wherein the generating the segment allocation table for the new file comprises:

identifying consecutive data segments of the identified data segments;

grouping the consecutive data segments into a region; and

generating addressing information for the region, which is stored as an entry in the segment allocation table, wherein individual addressing information of the identified sets of encoded data slices is determinable from the addressing information of the region.

6. The method of claim 1, wherein the identifying data segments comprises at least one of:

receiving, within the file creation request, identification of the identified data segments; and

searching, based on a search criteria, the set of existing files, or addressing information thereof, to identify the data segments.

7. The method of claim 1, wherein the identifying the sets of encoded data slices comprises:

retrieving a set of segment allocation tables for the set of existing files based on the file creation request, wherein a segment allocation table of the set of segment allocation tables includes a plurality of entries, and wherein an entry of the plurality of entries includes a DSN address regarding storage of a data segment of the plurality of data segments in the DSN memory.

8. The method of claim 1 further comprises:

obtaining a segment allocation table vault source name for the segment allocation table;

encoding the segment allocation table using a dispersed storage error coding function to produce encoded table slices; and

outputting the encoded table slices to the DSN memory for storage therein utilizing the segment allocation table vault source name.

9. A dispersed storage (DS) module comprises:

a first module, when operable within a computing device, causes the computing device to;

receive a file creation request to create a new file from portions of a set of existing files, wherein an existing file of the set of existing files includes a plurality of data segments that is stored as a plurality of sets of encoded data slices in a dispersed storage network (DSN) memory;

a second module, when operable within the computing device, causes the computing device to;

identify data segments from the plurality of data segments of the set of existing files corresponding to the portions of the set of existing files to produce identified data segments; and

identify sets of encoded data slices of the plurality of sets of encoded data slices of the identified data segments to produce identified sets of encoded data slices;

a third module, when operable within the computing device, causes the computing device to;

determine addressing information for the identified sets of encoded data slices from existing addressing information of the set of existing files; and

a fourth module, when operable within the computing device, causes the computing device to;

generate a segment allocation table for the new file based on the addressing information of the identified sets of encoded data slices such that the new file is created without duplication of the portions of the existing files.

10. The DS module of claim 9 further comprises:

the existing file of the set of existing files further includes a set of data regions, wherein a data region of the set of data regions includes a consecutive subset of the plurality of data segments;

the second module further functions to;

identify data regions from the set of data regions of the set of existing files corresponding to the portions of the set of existing files to produce identified data regions; and

identify the sets of encoded data slices of the plurality of sets of encoded data slices of the identified data regions to produce the identified sets of encoded data slices; and

the third module further functions to determine the addressing information for the identified sets of encoded data slices from a set of segment allocation tables associated with the set of existing files, wherein the set of segment allocation tables includes the existing addressing information of the set of existing files at data region level.

11. The DS module of claim 10, wherein the second module functions to identify the sets of encoded data slices by:

retrieving a set of segment allocation tables for the set of existing files based on the file creation request, wherein a segment allocation table of the set of segment allocation tables includes a plurality of entries, and wherein an entry of the plurality of entries includes a DSN address regarding storage of a data region of the set of data segments of the set of existing files in the DSN memory and segmentation information regarding the dividing of the data region into the consecutive subset of the plurality of data segments.

12. The DS module of claim 10, wherein addressing information for an identified set of encoded data slices of the identified sets of encoded data slices comprises:

a start segment vault source name corresponding to a data region associated with the identified set of encoded data slices; and

segmentation information including one or more of;

a segment size of data segments of the data region;

a total length of the data region; and

a segmentation approach of the data region.

13. The DS module of claim 9, wherein the fourth module functions to generate the segment allocation table for the new file by:

identifying consecutive data segments of the identified data segments;

grouping the consecutive data segments into a region; and

generating addressing information for the region, which is stored as an entry in the segment allocation table, wherein individual addressing information of the identified sets of encoded data slices is determinable from the addressing information of the region.

14. The DS module of claim 9, wherein the second module functions to identify data segments by at least one of:

receiving, within the file creation request, identification of the identified data segments; and

searching, based on a search criteria, the set of existing files, or addressing information thereof, to identify the data segments.

15. The DS module of claim 9, wherein the second module functions to identify the sets of encoded data slices by:

retrieving a set of segment allocation tables for the set of existing files based on the file creation request, wherein a segment allocation table of the set of segment allocation tables includes a plurality of entries, and wherein an entry of the plurality of entries includes a DSN address regarding storage of a data segment of the plurality of data segments in the DSN memory.

16. The DS module of claim 9 further comprises:

the fourth module further functions to;

obtain a segment allocation table vault source name for the segment allocation table;

encode the segment allocation table using a dispersed storage error coding function to produce encoded table slices; and

output the encoded table slices to the DSN memory for storage therein utilizing the segment allocation table vault source name.