Directory file system in a dispersed storage network

• US 8,918,674 B2
• Filed: 11/09/2010
• Issued: 12/23/2014
• Est. Priority Date: 01/28/2010
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Resch, Jason K., Baptist, Andrew, Palthepu, Srinivas, Dhuse, Greg, Volvovski, Ilya, Leggette, Wesley, Motwani, Manish
• Primary Examiner(s): SCHELL, JOSEPH O
• Application Number:
  US12/942,992
• Publication Number:
  US 20110185253A1
• Time in Patent Office: 1,505 Days
• Field of Search: 714/6.2, 714/6.22, 714/6.23, 714/6.24, 714/6.31, 711/162
• US Class Current: 714/6.2
• CPC Class Codes: G06F 11/1008 in individual solid state devices G06F11/1004 takes precedence
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a processing module receiving a data storage request, wherein the data storage request includes data and a data identifier (ID). The method continues with the processing module dispersed storage error encoding the data to produce a set of encoded data slices and determining a data dispersed storage network (DSN) address. The method continues with the processing module sending the set of encoded data slices to a DSN memory for storage at the data DSN address and updating a directory file with path information corresponding to the data ID and the data DSN address to produce an updated directory file. The method continues with the processing module dispersed storage error encoding the updated directory file to produce a set of encoded updated directory slices and sending the set of encoded updated directory slices to the DSN memory for storage at a directory DSN address.

Claims

1. A method for execution by a computing device, the method comprises:

determining a root directory dispersed storage network (DSN) address of a root directory regarding a request to read data;

retrieving, based on the root directory DSN address, a set of encoded root directory slices from a dispersed storage network (DSN) memory, wherein the set of encoded root directory slices corresponds to a data identifier (ID) of the data;

dispersed storage error decoding the set of encoded root directory slices to reproduce a root directory;

when a filename corresponding to the data ID is not stored in the root directory, entering a processing sequence that includes;

extracting a current intermediate directory DSN address from a preceding directory;

retrieving a set of encoded current intermediate directory slices from the DSN memory based on the current intermediate directory DSN address;

dispersed storage error decoding the set of encoded current intermediate directory slices to reproduce a current intermediate directory;

when the filename is not stored in the current intermediate directory, repeating the processing sequence for a next current intermediate directory DSN address; and

when the filename is stored in the current intermediate directory, extracting a data DSN address and exiting the processing sequence;

retrieving a set of encoded data slices from the DSN memory based on a data DSN address, the set of encoded data slices including a plurality of subsets of encoded data slices, each of the plurality of subsets of encoded data slices corresponding to a data segment of a plurality of data segments of the data, a number of encoded data slices within each of the subsets of encoded data slices corresponding to a number of pillars for each of the plurality of data segments, the number of pillars for each of the plurality of data segments being constant for a vault associated with a user of the data; and

dispersed storage error decoding the set of encoded data slices to reproduce the data.

2. The method of claim 1 further comprises:

when the filename is stored in the root directory, extracting the data DSN address from the root directory.

3. The method of claim 1, wherein the data ID comprises at least one of:

the filename;

a directory name;

a path; and

a block number.

4. The method of claim 1, wherein the retrieving the set of encoded root directory slices comprises:

determining a root directory DSN address based on at least one of;

a predetermination;

a user ID;

a user vault lookup based on the user ID; and

the data ID; and

retrieving the set of encoded root directory slices from the DSN memory based on the root directory DSN address.

5. The method claim 1, wherein the current intermediate directory DSN address comprises at least one of:

a source name; and

at least one slice name.

6. The method claim 1, wherein each of the root directory and the current intermediate directory comprises at least one of:

a filename;

a path;

a size indicator; and

a DSN address.

7. A method for execution by a computing device, the method comprises:

receiving a data storage request, wherein the data storage request includes data and a data identifier (ID);

dispersed storage error encoding the data to produce a set of encoded data slices, the set of encoded data slices including a plurality of subsets of encoded data slices, each of the plurality of subsets of encoded data slices corresponding to a data segment of a plurality of data segments of the data, a number of encoded data slices within each of the subsets of encoded data slices corresponding to a number of pillars for each of the plurality of data segments, the number of pillars for each of the plurality of data segments being constant for a vault associated with a user of the data;

determining a data dispersed storage network (DSN) address based on the data ID;

sending the set of encoded data slices to a DSN memory for storage at the data DSN address;

updating a directory file with path information corresponding to the data ID and the data DSN address to produce an updated directory file;

dispersed storage error encoding the updated directory file to produce a set of encoded updated directory slices; and

sending the set of encoded updated directory slices to the DSN memory for storage at a directory DSN address.

8. The method of claim 7, wherein the updating the directory file further comprises:

determining whether the directory file is stored in the DSN memory;

when the directory file is stored in the DSN memory;

retrieving a set of encoded directory slices from the DSN memory based on the directory DSN address; and

dispersed storage error decoding the set of encoded directory slices to reproduce the directory file; and

when the directory file is not stored in the DSN memory;

creating the directory file to include at least one of a filename field, a path field, a size indicator field, and a DSN address field.

9. The method of claim 7, wherein the updating the directory file further comprises:

obtaining a parent directory;

updating the parent directory with the directory DSN address and at least a portion of the data ID to produce an updated parent directory;

dispersed storage error encoding the updated parent directory to produce a set of encoded updated parent directory slices; and

sending the set of encoded updated parent directory slices to the DSN memory for storage at a parent directory DSN address.

10. The method of claim 7, wherein the data DSN address comprises at least one of:

a source name; and

at least one slice name.

11. The method of claim 7, wherein the directory file comprises:

one or more directories, wherein a directory of the one or more directories includes at least one of a filename, a path, a size indicator, and a DSN address.

12. A computer comprises:

an interface; and

a processing module operable to;

determine a root directory dispersed storage network (DSN) address of a root directory regarding a request to read data;

retrieve, via the interface and based on the root directory DSN address, a set of encoded root directory slices from a dispersed storage network (DSN) memory, wherein the set of encoded root directory slices corresponds to a data identifier (ID) of the data;

dispersed storage error decode the set of encoded root directory slices to reproduce a root directory;

when a filename corresponding to the data ID is not stored in the root directory, enter a processing sequence that includes;

extracting a current intermediate directory DSN address from a preceding directory;

retrieving, via the interface, a set of encoded current intermediate directory slices from the DSN memory based on the current intermediate directory DSN address;

dispersed storage error decoding the set of encoded current intermediate directory slices to reproduce a current intermediate directory;

when the filename is not stored in the current intermediate directory, repeating the processing sequence for a next current intermediate directory DSN address; and

when the filename is stored in the current intermediate directory, extracting a data DSN address and exiting the processing sequence;

retrieve, via the interface, a set of encoded data slices from the DSN memory based on a data DSN address, the set of encoded data slices including a plurality of subsets of encoded data slices, each of the plurality of subsets of encoded data slices corresponding to a data segment of a plurality of data segments of the data, a number of encoded data slices within each of the subsets of encoded data slices corresponding to a number of pillars for each of the plurality of data segments, the number of pillars for each of the plurality of data segments being constant for a vault associated with a user of the data; and

dispersed storage error decoding the set of encoded data slices to reproduce the data.

13. The computer of claim 12, wherein the processing module further functions to:

extract the data DSN address from the root directory when the filename is stored in the root directory.

14. The computer of claim 12, wherein the data ID comprises at least one of:

the filename;

a directory name;

a path; and

a block number.

15. The computer of claim 12, wherein the processing module further functions to retrieve the set of encoded root directory slices by:

determining a root directory DSN address based on at least one of;

a predetermination;

a user ID;

a user vault lookup based on the user ID; and

the data ID; and

retrieving, via the interface, the set of encoded root directory slices from the DSN memory based on the root directory DSN address.

16. The computer claim 12, wherein the current intermediate directory DSN address comprises at least one of:

a source name; and

at least one slice name.

17. The computer claim 12, wherein each of the root directory and the current intermediate directory comprises at least one of:

a filename;

a path;

a size indicator; and

a DSN address.

18. A computer comprises:

an interface; and

a processing module operable to;

receive, via the interface, a data storage request, wherein the data storage request includes data and a data identifier (ID);

dispersed storage error encode the data to produce a set of encoded data slices, the set of encoded data slices including a plurality of subsets of encoded data slices, each of the plurality of subsets of encoded data slices corresponding to a data segment of a plurality of data segments of the data, a number of encoded data slices within each of the subsets of encoded data slices corresponding to a number of pillars for each of the plurality of data segments, the number of pillars for each of the plurality of data segments being constant for a vault associated with a user of the data;

determine a data dispersed storage network (DSN) address based on the data ID;

send, via the interface, the set of encoded data slices to a DSN memory for storage at the data DSN address;

update a directory file with path information corresponding to the data ID and the data DSN address to produce an updated directory file;

dispersed storage error encode the updated directory file to produce a set of encoded updated directory slices; and

send, via the interface, the set of encoded updated directory slices to the DSN memory for storage at a directory DSN address.

19. The computer of claim 18, wherein the processing module further functions to update the directory file by:

determining whether the directory file is stored in the DSN memory;

when the directory file is stored in the DSN memory;

retrieving, via the interface, a set of encoded directory slices from the DSN memory based on the directory DSN address; and

dispersed storage error decoding the set of encoded directory slices to reproduce the directory file; and

when the directory file is not stored in the DSN memory;

creating the directory file to include at least one of a filename field, a path field, a size indicator field, and a DSN address field.

20. The computer of claim 18, wherein the processing module further functions to update the directory file by:

obtaining a parent directory;

updating the parent directory with the directory DSN address and at least a portion of the data ID to produce an updated parent directory;

dispersed storage error encoding the updated parent directory to produce a set of encoded updated parent directory slices; and

sending, via the interface, the set of encoded updated parent directory slices to the DSN memory for storage at a parent directory DSN address.

21. The computer of claim 18, wherein the data DSN address comprises at least one of:

a source name; and

at least one slice name.

22. The computer of claim 18, wherein the directory file comprises:

one or more directories, wherein a directory of the one or more directories includes at least one of a filename, a path, a size indicator, and a DSN address.