Configuring storage resources of a dispersed storage network

• US 10,346,250 B2
• Filed: 11/14/2016
• Issued: 07/09/2019
• Est. Priority Date: 01/06/2014
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Khadiwala, Ravi, Leggette, Wesley, Baptist, Andrew, Dhuse, Greg, Volvovski, Ilya, Resch, Jason K., Motwani, Manish
• Primary Examiner(s): Le, Dieu Minh T
• Application Number:
  US15/350,672
• Publication Number:
  US 20170063402A1
• Time in Patent Office: 967 Days
• Field of Search: 714 611, 714 612, 714 613, 714 62, 714 621, 714 623, 714 41, 714 411, 714 43, 714 412
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/353 Adaptation to the channel
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method includes determining an acceptable site failure number for storing a set of encoded data slices based on available sites and storage units within the available sites. The method further includes determining dispersed storage error encoding parameters based on the acceptable site failure number, wherein the parameters includes a decode threshold number and a total width number. The method further includes encoding, in accordance with the parameters, a data segment of data to produce the set of encoded data slices. The method further includes generating a slice mapping based on the acceptable site failure number, the available sites, and the storage units within the available storage units within the available sites. The method further includes sending, based on the slice mapping, the set of encoded data slices to at least some of the storage units within at least some of the available sites for storage therein.

Claims

1. A method for execution by a computing device of a dispersed storage network (DSN), the method comprises:

determining, by the computing device prior to storing a set of encoded data slices, an acceptable site failure number for storing the set of encoded data slices within the DSN based on available sites within the DSN and storage units within the available sites, wherein the DSN includes a plurality of sites coupled via a wide area network (WAN), wherein a site of the plurality of sites includes one or more storage units, and wherein the one or more storage units are coupled within the site via a local area network (LAN);

determining, by the computing device, dispersed storage error encoding parameters based on the acceptable site failure number, wherein the dispersed storage error encoding parameters includes a decode threshold number and a total width number;

encoding, by the computing device and in accordance with the dispersed storage error encoding parameters, a data segment of data to produce the set of encoded data slices, wherein the set of encoded data slices includes the total width number of encoded data slices and wherein a decode threshold number of encoded data slices of the set of encoded data slices is required to recover the data segment;

generating, by the computing device, a slice mapping based on the acceptable site failure number, the available sites, and the storage units within the available storage units within the available sites, wherein the slice mapping maps less than the decode threshold number of encoded data slices to any one site of the available sites; and

sending, by the computing device and based on the slice mapping, a first sub-decode threshold number of encoded data slices of the set of encoded data slices to at least some of the storage units within a first site of the available sites for storage therein; and

sending, by the computing device and based on the slice mapping, a second sub-decode threshold number of encoded data slices of the set of encoded data slices to at least some of the storage units within a second site of the available sites for storage therein.

2. The method of claim 1 further comprises one of:

determining the decode threshold number on a segment-by-segment basis for the data; and

ascertaining the decode threshold number for the data.

3. The method of claim 1, wherein the determining the acceptable site failure number comprises:

identifying the available sites;

identifying available storage units within the available sites;

selecting at least some of the available sites and at least some of the available storage units to produce selected sites and selected storage units within the selected sites; and

determining the acceptable site failure number based on the selected sites, the selected storage units within the sites and a DSN performance factor.

4. The method of claim 3 further comprises:

determining the total width number by;

generating an expansion factor based on the number of selected sites and the acceptable site failure number; and

performing a mathematical function on the decode threshold value and the expansion factor to generate the total width value.

5. The method of claim 1, wherein the generating the slice mapping comprises:

dividing the set of encoded data slices into sub-groups of encoded data slices based on a number of the at least some of the available sites and the acceptable site failure number such that, when the acceptable site failure number of sites within the at least some of the available sites, storage units in remaining sites of the at least some of the available sites stored at least the decode threshold number of encoded data slices; and

establishing the slice mapping based on the sub-groups of encoded data slices and the at least some of the available sites.

6. A computing device of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

determine, prior to storing a set of encoded data slices, an acceptable site failure number for storing the set of encoded data slices within the DSN based on available sites within the DSN and storage units within the available sites, wherein the DSN includes a plurality of sites coupled via a wide area network (WAN), wherein a site of the plurality of sites includes one or more storage units, wherein the one or more storage units are coupled within the site via a local area network (LAN);

determine dispersed storage error encoding parameters based on the acceptable site failure number, wherein the dispersed storage error encoding parameters includes a decode threshold number and a total width number;

encode, in accordance with the dispersed storage error encoding parameters, a data segment of data to produce the set of encoded data slices, wherein the set of encoded data slices includes the total width number of encoded data slices and wherein the decode threshold number of encoded data slices of the set of encoded data slices is required to recover the data segment;

generate a slice mapping based on the acceptable site failure number, the available sites, and the storage units within the available storage units within the available sites, wherein the slice mapping maps less than the decode threshold number of encoded data slices to any one site of the available sites; and

send, via the interface and based on the slice mapping, a first sub-decode threshold number of encoded data slices of the set of encoded data slices to at least some of the storage units within a first site of the available sites for storage therein; and

sending, via the interface and based on the slice mapping, a second sub-decode threshold number of encoded data slices of the set of encoded data slices to at least some of the storage units within a second site of the available sites for storage therein.

7. The computing device of claim 6, wherein the processing is further operable to perform one of:

determining the decode threshold number on a segment-by-segment basis for the data; and

ascertaining the decode threshold number for the data.

8. The computing device of claim 6, wherein the processing is further operable to determine the acceptable site failure number by:

identifying the available sites;

identifying available storage units within the available sites;

selecting at least some of the available sites and at least some of the available storage units to produce selected sites and selected storage units within the selected sites; and

determining the acceptable site failure number based on the selected sites, the selected storage units within the sites and a DSN performance factor.

9. The computing device of claim 8, wherein the processing is further operable to:

determining the total width number by;

generating an expansion factor based on the number of selected sites and the acceptable site failure number; and

performing a mathematical function on the decode threshold value and the expansion factor to generate the total width value.

10. The computing device of claim 6, wherein the processing is further operable to generate the slice mapping by:

dividing the set of encoded data slices into sub-groups of encoded data slices based on a number of the at least some of the available sites and the acceptable site failure number such that, when the acceptable site failure number of sites within the at least some of the available sites, storage units in remaining sites of the at least some of the available sites stored at least the decode threshold number of encoded data slices; and

establishing the slice mapping based on the sub-groups of encoded data slices and the at least some of the available sites.

11. A computer readable memory comprises:

a first memory section that stores operational instructions that, when executed by a computing device of a dispersed storage network (DSN), causes the computing device to;

determine, prior to storing a set of encoded data slices, an acceptable site failure number for storing the set of encoded data slices within the DSN based on available sites within the DSN and storage units within the available sites, wherein the DSN includes a plurality of sites coupled via a wide area network (WAN), wherein a site of the plurality of sites includes one or more storage units, and wherein the one or more storage units are coupled within the site via a local area network (LAN);

determine dispersed storage error encoding parameters based on the acceptable site failure number, wherein the dispersed storage error encoding parameters includes a decode threshold number and a total width number;

encode, in accordance with the dispersed storage error encoding parameters, a data segment of data to produce the set of encoded data slices, wherein the set of encoded data slices includes the total width number of encoded data slices and wherein a decode threshold number of encoded data slices of the set of encoded data slices is required to recover the data segment; and

a second memory section that stores operational instructions that, when executed by the computing device, causes the computing device to;

generate a slice mapping based on the acceptable site failure number, the available sites, and the storage units within the available storage units within the available sites, wherein the slice mapping maps less than the decode threshold number of encoded data slices to any one site of the available sites; and

send, based on the slice mapping, a first sub-decode threshold number of encoded data slices of the set of encoded data slices to at least some of the storage units within a first site of the available sites for storage therein; and

send, based on the slice mapping, a second sub-decode threshold number of encoded data slices of the set of encoded data slices to at least some of the storage units within a second site of the available sites for storage therein.

12. The computer readable memory of claim 11, wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to perform one of:

determining the decode threshold number on a segment-by-segment basis for the data; and

ascertaining the decode threshold number for the data.

13. The computer readable memory of claim 11, wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to determine the acceptable site failure number by:

identifying the available sites;

identifying available storage units within the available sites;

selecting at least some of the available sites and at least some of the available storage units to produce selected sites and selected storage units within the selected sites; and

determining the acceptable site failure number based on the selected sites, the selected storage units within the sites and a DSN performance factor.

14. The computer readable memory of claim 13, wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to:

determine the total width number by;

generating an expansion factor based on the number of selected sites and the acceptable site failure number; and

performing a mathematical function on the decode threshold value and the expansion factor to generate the total width value.

15. The computer readable memory of claim 13, wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to generate the slice mapping by:

dividing the set of encoded data slices into sub-groups of encoded data slices based on a number of the at least some of the available sites and the acceptable site failure number such that, when the acceptable site failure number of sites within the at least some of the available sites, storage units in remaining sites of the at least some of the available sites stored at least the decode threshold number of encoded data slices; and

establishing the slice mapping based on the sub-groups of encoded data slices and the at least some of the available sites.