Dispersed B-tree directory trees

• US 10,169,129 B2
• Filed: 07/27/2018
• Issued: 01/01/2019
• Est. Priority Date: 03/02/2011
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Resch, Jason K., Leggette, Wesley B., Baptist, Andrew D., Volvovski, Ilya, Dhuse, Greg R.
• Primary Examiner(s): Chan, Tracy C.
• Application Number:
  US16/047,637
• Publication Number:
  US 20180336083A1
• Time in Patent Office: 158 Days
• Field of Search: None
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0757 by exceeding a time limit, i.e. time-out, e.g. watchdogs
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity

Abstract

A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and processing circuitry operably coupled to the interface and to the memory. The processing circuitry is configured to execute the operational instructions to perform various operations and functions. The computing device obtains directory metrics associated with a directory structure that is associated with a directory file that is segmented into a plurality of data segments and based on a determination to reconfigure the directory structure based on the directory metrics, the computing device determines a number of layers for a reconfigured directory structure, a number of spans per layer of the number of layers for the reconfigured directory structure, and directory entry reassignments. The computing device reconfigures the directory structure based on the number of layers, the spans per layer, and the directory entry reassignments.

Claims

1. A computing device comprising:

an interface configured to interface and communicate with a dispersed or distributed storage network (DSN);

memory that stores operational instructions; and

processing circuitry operably coupled to the interface and to the memory, wherein the processing circuitry is configured to execute the operational instructions to;

obtain, via the DSN and via the interface, directory metrics associated with a directory structure that is associated with a directory file that is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of encoded directory slices that are stored in at least one DSN memory at least one DSN address corresponding to a source name of the directory file;

determine whether to reconfigure the directory structure based on the directory metrics; and

based on a determination to reconfigure the directory structure based on the directory metrics;

determine a number of layers for a reconfigured directory structure;

determine a number of spans per layer of the number of layers for the reconfigured directory structure;

determine directory entry reassignments; and

reconfigure the directory structure based on the number of layers, the spans per layer, and the directory entry reassignments to generate the reconfigured directory structure including at least one of to create one or more children directory files, facilitate movement within the DSN of one or more directory entries from a parent directory file to the one or more children directory files, or to add pointers associated with the one or more children directory files to the parent directory file.

2. The computing device of claim 1, wherein the directory metrics includes at least one of a directory size indicator, a number of entries indicator, an access contention indicator, a DSN access performance level indicator, or a vault identifier (ID).

3. The computing device of claim 1, wherein the processing circuitry is further configured to execute the operational instructions to:

obtain, via the DSN and via the interface, the directory metrics associated with the directory structure based on at least one of receiving the directory metrics, a query, or a lookup.

4. The computing device of claim 1, wherein the processing circuitry is further configured to execute the operational instructions to:

determine the number of layers for the reconfigured directory structure based on at least one of the directory metrics, an estimated performance level, a DSN access performance level goal, or a directory metrics goal.

5. The computing device of claim 1, wherein the processing circuitry is further configured to execute the operational instructions to:

determine the number of spans per layer of the number of layers for the reconfigured directory structure based on at least one of the directory metrics, the number of layers, a DSN network bandwidth capacity indicator, a DSN network bandwidth utilization indicator, or a DSN network bandwidth utilization goal.

6. The computing device of claim 1, wherein the processing circuitry is further configured to execute the operational instructions to:

determine the directory entry reassignments based on at least one of a number of entries, the directory metrics, the number of layers, the spans per layer, one or more file IDs, an estimated frequency of access associated with a file ID, or a data type indicator.

7. The computing device of claim 1, wherein:

a decode threshold number of encoded directory slices are needed to recover the data segment;

a read threshold number of encoded directory slices provides for reconstruction of the data segment;

a write threshold number of encoded directory slices provides for a successful transfer of the set of encoded directory slices from a first at least one location in the DSN to a second at least one location in the DSN;

the set of encoded directory slices is of pillar width and includes a pillar number of encoded directory slices;

each of the decode threshold number, the read threshold number, and the write threshold number is less than the pillar number; and

the write threshold number is greater than or equal to the read threshold number that is greater than or equal to the decode threshold number.

8. The computing device of claim 1, wherein the DSN includes at least one of a wireless communication system, a wire lined communication system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

9. A computing device comprising:

an interface configured to interface and communicate with a dispersed or distributed storage network (DSN);

memory that stores operational instructions; and

processing circuitry operably coupled to the interface and to the memory, wherein the processing circuitry is configured to execute the operational instructions to;

obtain, via the DSN and via the interface, directory metrics associated with a directory structure that is associated with a directory file that is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of encoded directory slices that are stored in at least one DSN memory at least one DSN address corresponding to a source name of the directory file, wherein the directory metrics includes at least one of a directory size indicator, a number of entries indicator, an access contention indicator, a DSN access performance level indicator, or a vault identifier (ID);

determine whether to reconfigure the directory structure based on the directory metrics; and

based on a determination to reconfigure the directory structure based on the directory metrics;

determine a number of layers for a reconfigured directory structure;

determine a number of spans per layer of the number of layers for the reconfigured directory structure;

determine directory entry reassignments based on at least one of a number of entries, the directory metrics, the number of layers, the spans per layer, one or more file IDs, an estimated frequency of access associated with a file ID, or a data type indicator; and

reconfigure the directory structure based on the number of layers, the spans per layer, and the directory entry reassignments to generate the reconfigured directory structure including at least one of to create one or more children directory files, facilitate movement within the DSN of one or more directory entries from a parent directory file to the one or more children directory files, or to add pointers associated with the one or more children directory files to the parent directory file.

10. The computing device of claim 9, wherein the processing circuitry is further configured to execute the operational instructions to perform at least one of:

obtain, via the DSN and via the interface, the directory metrics associated with the directory structure based on at least one of receiving the directory metrics, a query, or a lookup;

or

determine the number of layers for the reconfigured directory structure based on at least one of the directory metrics, an estimated performance level, a DSN access performance level goal, or a directory metrics goal.

11. The computing device of claim 9, wherein the processing circuitry is further configured to execute the operational instructions to:

determine the number of spans per layer of the number of layers for the reconfigured directory structure based on at least one of the directory metrics, the number of layers, a DSN network bandwidth capacity indicator, a DSN network bandwidth utilization indicator, or a DSN network bandwidth utilization goal.

12. The computing device of claim 9, wherein:

a decode threshold number of encoded directory slices are needed to recover the data segment;

a read threshold number of encoded directory slices provides for reconstruction of the data segment;

a write threshold number of encoded directory slices provides for a successful transfer of the set of encoded directory slices from a first at least one location in the DSN to a second at least one location in the DSN;

the set of encoded directory slices is of pillar width and includes a pillar number of encoded directory slices;

each of the decode threshold number, the read threshold number, and the write threshold number is less than the pillar number; and

the write threshold number is greater than or equal to the read threshold number that is greater than or equal to the decode threshold number.

13. The computing device of claim 9, wherein the DSN includes at least one of a wireless communication system, a wire lined communication system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

14. A method for execution by a computing device, the method comprising:

obtaining, via a dispersed or distributed storage network (DSN) and via an interface of the computing device that is configured to interface and communicate with the DSN, directory metrics associated with a directory structure that is associated with a directory file that is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of encoded directory slices that are stored in at least one DSN memory at least one DSN address corresponding to a source name of the directory file;

determining whether to reconfigure the directory structure based on the directory metrics; and

based on a determination to reconfigure the directory structure based on the directory metrics;

determining a number of layers for a reconfigured directory structure;

determining a number of spans per layer of the number of layers for the reconfigured directory structure;

determining directory entry reassignments; and

reconfiguring the directory structure based on the number of layers, the spans per layer, and the directory entry reassignments to generate the reconfigured directory structure including at least one of to create one or more children directory files, facilitate movement within the DSN of one or more directory entries from a parent directory file to the one or more children directory files, or to add pointers associated with the one or more children directory files to the parent directory file.

15. The method of claim 14, wherein the directory metrics includes at least one of a directory size indicator, a number of entries indicator, an access contention indicator, a DSN access performance level indicator, or a vault identifier (ID).

16. The method of claim 14 further comprising:

obtaining, via the DSN and via the interface, the directory metrics associated with the directory structure based on at least one of receiving the directory metrics, a query, or a lookup.

17. The method of claim 14 further comprising at least one of:

determining the number of layers for the reconfigured directory structure based on at least one of the directory metrics, an estimated performance level, a DSN access performance level goal, or a directory metrics goal;

or

determining the number of spans per layer of the number of layers for the reconfigured directory structure based on at least one of the directory metrics, the number of layers, a DSN network bandwidth capacity indicator, a DSN network bandwidth utilization indicator, or a DSN network bandwidth utilization goal.

18. The method of claim 14 further comprising:

determining the directory entry reassignments based on at least one of a number of entries, the directory metrics, the number of layers, the spans per layer, one or more file IDs, an estimated frequency of access associated with a file ID, or a data type indicator.

19. The method of claim 14, wherein:

a decode threshold number of encoded directory slices are needed to recover the data segment;

a read threshold number of encoded directory slices provides for reconstruction of the data segment;

a write threshold number of encoded directory slices provides for a successful transfer of the set of encoded directory slices from a first at least one location in the DSN to a second at least one location in the DSN;

the set of encoded directory slices is of pillar width and includes a pillar number of encoded directory slices;

each of the decode threshold number, the read threshold number, and the write threshold number is less than the pillar number; and

the write threshold number is greater than or equal to the read threshold number that is greater than or equal to the decode threshold number.

20. The method of claim 14, wherein the DSN includes at least one of a wireless communication system, a wire lined communication system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).