Selecting a directory of a dispersed storage network

• US 9,658,911 B2
• Filed: 02/14/2012
• Issued: 05/23/2017
• Est. Priority Date: 03/02/2011
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Baptist, Andrew, Resch, Jason K., Tsvetkov, Andrew
• Primary Examiner(s): Young, Kevin L
• Assistant Examiner(s): Tamaru, Michael K
• Application Number:
  US13/372,715
• Publication Number:
  US 20120226714A1
• Time in Patent Office: 1,925 Days
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0757 by exceeding a time limit, i.e. time-out, e.g. watchdogs
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity

Abstract

A method begins by a processing module receiving a dispersed storage network (DSN) access request accessing DSN memory regarding a set of encoded data slices, selecting a local DSN directory or a global DSN directory to produce a selected DSN directory, identifying an entry of the selected DSN directory regarding the set of encoded data slices to produce an identified entry, and accessing the DSN memory regarding the set of encoded data slices. The method continues with the processing module determining whether to update one or more of the local DSN directory and the global DSN directory based on the accessing the DSN memory and when the one or more of the local DSN directory and the global DSN directory is to be updated, updating the one or more of the local DSN directory and the global DSN directory in accordance with the accessing the DSN memory.

Claims

1. A method for execution by a dispersed storage (DS) processing unit of a dispersed storage network (DSN) the method comprises:

obtaining a revision level of a global parent directory file of a global file structure, wherein the global parent directory file is represented by one or more data segments, in which respective data segments are error encoded to produce N number of encoded data slices for each data segment, and wherein the N number of encoded data slices for the respective data segments are dispersed separately in a distributed storage network, where only a read threshold number T of the particular N number of encoded data slices are needed to be retrieved from the distributed storage network to reconstruct each data segment to reconstruct the global parent directory file, where T is less than N, and wherein a local parent directory file also stores a local parent directory structure in encoded data slices in order for the local parent directory file and the global parent directory file to mirror each other, wherein the obtaining the revision level of the global parent directory file of the global file structure includes;

transmitting a read request to the distributed storage network; and

performing a slice-to-slice comparison of respective encoded data slices of the global parent directory file and the local parent directory file to identify a revision level indicator mismatch between respective encoded data slices of the global parent directory file and the local parent directory file;

comparing the revision level of the global parent directory file with a revision level of a local parent directory file by comparing a revision level indicator included with at least one encoded data slice pertaining to the global parent directory file with a revision level of the local parent directory file;

when the comparing of the revision level indicator included with the at least one encoded data slice pertaining to the global parent directory file with the revision level of the local parent directory file is unfavorable;

retrieving encoded data slices pertaining to the global parent directory file from the distributed storage network;

decoding the encoded data slices pertaining to the global parent directory file to reconstruct the global parent directory file;

updating the local parent directory file based on the global parent directory file;

accessing an entry of the global parent directory file that points to an address of a global child directory file, wherein the global child directory file includes an entry that points to an address of a data object;

obtaining a revision level of the global child directory file, recovering and decoding encoded data slices pertaining to the global child directory file from the distributed storage network and using the global child directory file to update a local child directory file; and

utilizing updated local parent and local child directory files to perform an access request regarding the data object.

2. The method of claim 1 further comprises:

the global file structure includes a plurality of directory files, wherein a particular directory file of the plurality of directory files includes a plurality of entries, wherein an entry of the plurality of entries that points to an address of another directory file of the plurality of directory files or to an address of a particular data object, wherein the plurality of directory files is error encoded to produce a plurality of sets of encoded directory file slices, wherein a set of the plurality of sets of encoded directory file slices corresponds to one of the plurality of directory files, wherein the plurality of sets of encoded directory file slices are stored in storage units of the distributed storage network; and

the local directory file structure includes a copy of a set of the plurality of directory files, wherein a child directory file of the set of the plurality of directory files includes an entry that points to an address of another data object, wherein the parent directory file of the set of the plurality of directory files includes the entry that points to an address of the child directory file.

3. The method of claim 1, wherein the obtaining the revision level of the parent directory file of the global file structure comprises:

transmitting a single read encoded slice request to a storage unit of the distributed storage network.

4. The method of claim 1 further comprises:

when the comparing of the revision level indicator included with the at least one encoded data slice pertaining to the global parent directory file with the revision level of the local parent directory file is favorable, utilizing the local parent and local child directory files to perform the access request regarding the data object.

5. The method of claim 1 further comprises:

when the entry of the global parent directory file points to an address of an intervening child directory file, obtaining the global child directory file via the intervening child directory.

6. The method of claim 5 further comprises:

accessing an entry of the intervening child directory file that points to the address of the global child directory file to obtain the global child directory file.

7. A dispersed storage (DS) module comprises:

a first module, when operable in a computing device, causes the computing device to obtain a revision level of a global parent directory file of a global file structure, wherein the global parent directory file is represented by one or more data segments, in which respective data segments are error encoded to produce N number of encoded data slices for each data segment, and wherein the N number of encoded data slices for the respective data segments are dispersed separately in a distributed storage network, where only a read threshold number T of the particular N number of encoded data slices are needed to be retrieved from the distributed storage network to reconstruct each data segment to reconstruct the global parent directory file, where T is less than N, and wherein a local parent directory file also stores a local parent directory structure in encoded data slices in order for the local parent directory file and the global parent directory file to mirror each other, wherein the obtaining the revision level of the global parent directory file of the global file structure includes;

transmitting a read request to the distributed storage network; and

performing a slice-to-slice comparison of respective encoded data slices of the global parent directory file and the local parent directory file to identify a revision level indicator mismatch between respective encoded data slices of the global parent directory file and the local parent directory file;

a second module, when operable in the computing device causes the computing device to compare the revision level of the global parent directory file with a revision level of a local parent directory file by comparing a revision level indicator included with at least one encoded data slice pertaining to the global parent directory file with a revision level of the local parent directory file;

a third module, when operable in the computing device causes the computing device to;

when comparing the revision level indicator included with the at least one encoded data slice pertaining to the global parent directory file with the revision level of the local parent directory file is unfavorable;

retrieve encoded data slices pertaining to the global parent directory file from the distributed storage network;

decode the encoded data slices pertaining to the global parent directory to reconstruct the global parent directory file;

update the local parent directory file based on the global parent directory file;

access an entry of the global parent directory file that points to an address of a global child directory file, wherein the global child directory file includes an entry that points to an address of a data object;

obtain a revision level of the global child directory file, recover and decode encoded data slices pertaining to the global child directory file from the distributed storage network and use the global child directory file to update a local child directory file; and

a fourth module, when operable in the computing device causes the computing device to utilize updated local parent and local child directory files to perform an access request regarding the data object.

8. The DS module of claim 7 further comprises:

the global file structure includes a plurality of directory files, wherein a particular directory file of the plurality of directory files includes a plurality of entries, wherein an entry of the plurality of entries that points to an address of another directory file of the plurality of directory files or to an address of a particular data object, wherein the plurality of directory files is error encoded to produce a plurality of sets of encoded directory file slices, wherein a set of the plurality of sets of encoded directory file slices corresponds to one of the plurality of directory files, wherein the plurality of sets of encoded directory file slices are stored in storage units of the distributed storage network; and

the local directory file structure includes a copy of a set of the plurality of directory files, wherein a child directory file of the set of the plurality of directory files includes an entry that points to an address of another data object, wherein the parent directory file of the set of the plurality of directory files includes the entry that points to an address of the child directory file.

9. The DS module of claim 7, wherein the first module, when operable in the computing device, causes the computing device to obtain the revision level of the global parent directory file of the global file structure by:

transmitting a single read encoded slice request to a storage unit of the distributed storage network.

10. The DS module of claim 7, wherein the fourth module, when operable in the computing device causes the computing device to:

when the comparing of the revision level indicator included with the at least one encoded data slice pertaining to the global parent directory file with the revision level of the local parent directory file is favorable, utilizing the local parent and local child directory files to perform the access request regarding the data object.

11. The DS module of claim 7, wherein the third module, when operable in the computing device, causes the computing device to:

when the entry of the global parent directory file points to an address of an intervening child directory file, obtain the global child directory file via the intervening child directory.

12. The DS module of claim 11, wherein the third module, when operable in the computing device, further causes the computing device to:

access an entry of the intervening child directory file that points to the address of the global child directory file to obtain the global child directory file.