Storing data in multiple dispersed storage networks

• US 8,898,513 B2
• Filed: 05/11/2011
• Issued: 11/25/2014
• Est. Priority Date: 05/19/2010
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Grube, Gary W., Markison, Timothy W.
• Primary Examiner(s): Wilson, Yolanda L
• Application Number:
  US13/105,074
• Publication Number:
  US 20110289358A1
• Time in Patent Office: 1,294 Days
• Field of Search: 714/6.2, 714/6.24, 714/6.3, 714/4.12, 711/154, 711/162
• US Class Current: 714/6.2
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal

Abstract

A method begins by a processing module identifying a plurality of dispersed storage networks (DSNs) for storing copies of dispersed storage encoded data based on global data retrieval accesses of the copies of the dispersed storage encoded data. The method continues with the processing module determining a set of error coding dispersal storage parameters for at least one of the plurality of DSNs based on local data retrieval accesses allocated to the at least one of the plurality of DSNs. The method continues with the processing module encoding data in accordance with the set of error coding dispersal storage parameters to produce a copy of the copies of the dispersed storage encoded data and outputting the copy of the copies of the dispersed storage encoded data to the at least one of the plurality of DSNs.

Claims

1. A method for execution by a processing module of a computing device, the method comprises:

identifying a plurality of dispersed storage networks (DSNs) for storing copies of dispersed storage encoded data based on global data retrieval accesses of the copies of the dispersed storage encoded data, wherein a DSN of the plurality of DSN includes a plurality of dispersed storage units, wherein each dispersed storage unit of the plurality of dispersed storage units includes a plurality of memory devices;

determining a set of local error coding dispersal storage parameters for a local DSN of the plurality of DSNs based on local data retrieval accesses allocated to the at least one of the plurality of DSNs;

determining a set of global error coding dispersal storage parameters for at least one other of the plurality of DSNs based on the global data retrieval accesses of the copies of the dispersed storage encoded data;

encoding data in accordance with the set of local error coding dispersal storage parameters to produce a local copy of the copies of the dispersed storage encoded data;

encoding the data in accordance with the set of global error coding dispersal storage parameters to produce a global copy of the copies of the dispersed storage encoded data;

outputting the local copy of the copies of the dispersed storage encoded data for storage within the local DSN of the plurality of DSNs; and

outputting the global copy of the copies of the dispersed storage encoded data for storage within the at least one other of the plurality of DSNs.

2. The method of claim 1 further comprises determining the global data retrieval accesses based on one or more of:

quantity of retrieval accesses;

a data type of the data;

a data size indicator of the data;

a user device geographic affiliation;

a dispersed storage processing unit geographic affiliation;

a geographic requirement;

a DSN geographic affiliation;

a retrieval latency requirement;

a DSN performance level;

an availability requirement;

a reliability requirement;

a predetermination; and

a user identifier (ID).

3. The method of claim 1 further comprises:

determining the local data retrieval accesses for each of the plurality of DSNs based on an allocation portion of the global data retrieval accesses.

4. The method of claim 1 further comprises:

determining a second set of error coding dispersal storage parameters for at least a second one of the plurality of DSNs based on second local data retrieval accesses allocated to the at least the second one of the plurality of DSNs.

5. The method of claim 4 further comprises:

determining a common decode threshold for each of the local set, the global set, and the second set of error coding dispersal storage parameters;

determining a first plurality of parameter subsets for the local set of error coding dispersal storage parameters, which includes a first pillar width, a first write threshold, and a first read threshold;

determining a second plurality of parameter subsets for the second set of error coding dispersal storage parameters, which includes a second pillar width, a second write threshold, and a second read threshold; and

determining a third plurality of parameter subsets for the global set of error coding dispersal storage parameters, which includes a third pillar width, a third write threshold, and a third read threshold.

6. The method of claim 1 further comprises:

determining that a set of the plurality of DSNs have substantially similar local data retrieval accesses;

determining the global set of error coding dispersal storage parameters for the set of the plurality of DSNs;

encoding, by one of the set of DSNs, the data in accordance with the global set of error coding dispersal storage parameters to produce a plurality of sets of encoded data slices;

storing, by the one of the set of DSNs, the plurality of sets of encoded data slices within the one of the set of DSNs; and

outputting, by the one of the set of DSNs, the plurality of sets of encoded data slices to other DSNs of the set of the plurality of DSNs for storage therein.

7. The method of claim 1 further comprises:

receiving, by a second one of the at least one of the plurality of DSNs, the copy of the copies of the dispersed storage encoded data from a first one of the at least one of the plurality of DSNs;

determining, by the second one of the at least one of the plurality of DSNs, a set of local dispersed storage (DS) units;

storing a plurality of sets of encoded data slices of the copy of the copies of the dispersed storage encoded data within the set of local DS units.

8. The method of claim 7 further comprises:

identifying, by the second one of the at least one of the plurality of DSNs, a third one of the at least one of the plurality of DSNs; and

outputting, by the second one of the at least one of the plurality of DSNs, the copy of the copies of the dispersed storage encoded data to the third one of the at least one of the plurality of DSNs.

9. The method of claim 1 further comprises at least one of:

facilitating storage of at least one of the global and local error coding dispersal storage parameters; and

facilitating storage of a plurality of DSN identifiers (IDs), wherein each DSN ID of the plurality of DSN IDs corresponds to a DSN of the plurality of DSNs.

10. A computer comprises:

an interface;

a memory; and

a processing module operable to;

identify a plurality of dispersed storage networks (DSNs) for storing copies of dispersed storage encoded data based on global data retrieval accesses of the copies of the dispersed storage encoded data, wherein a DSN of the plurality of DSN includes a plurality of dispersed storage units, wherein each dispersed storage unit of the plurality of dispersed storage units includes a plurality of memory devices;

determine a set of local error coding dispersal storage parameters for a local DSN of the plurality of DSNs based on local data retrieval accesses allocated to the at least one of the plurality of DSNs;

determine a set of global error coding dispersal storage parameters for at least one other of the plurality of DSNs based on the global data retrieval accesses of the copies of the dispersed storage encoded data;

encode data in accordance with the set of local error coding dispersal storage parameters to produce a local copy of the copies of the dispersed storage encoded data;

encode the data in accordance with the set of global error coding dispersal storage parameters to produce a global copy of the copies of the dispersed storage encoded data;

output, via the interface, the local copy of the copies of the dispersed storage encoded data for storage within the local DSN of the plurality of DSNs; and

output, via the interface, the global copy of the copies of the dispersed storage encoded data for storage within the at least one other of the plurality of DSNs.

11. The computer of claim 10, wherein the processing module is further operable to determine the global data retrieval accesses based on one or more of:

quantity of retrieval accesses;

a data type of the data;

a data size indicator of the data;

a user device geographic affiliation;

a dispersed storage processing unit geographic affiliation;

a geographic requirement;

a DSN geographic affiliation;

a retrieval latency requirement;

a DSN performance level;

an availability requirement;

a reliability requirement;

a predetermination; and

a user identifier (ID).

12. The computer of claim 10, wherein the processing module further functions to:

determine the local data retrieval accesses for each of the plurality of DSNs based on an allocation portion of the global data retrieval accesses.

13. The computer of claim 10, wherein the processing module further functions to:

determine a second set of error coding dispersal storage parameters for at least a second one of the plurality of DSNs based on second local data retrieval accesses allocated to the at least the second one of the plurality of DSNs.

14. The computer of claim 13, wherein the processing module further functions to:

determine a common decode threshold for each of the local set, the global set, and the second set of error coding dispersal storage parameters;

determine a first plurality of parameter subsets for the local set of error coding dispersal storage parameters, which includes a first pillar width, a first write threshold, and a first read threshold;

determine a second plurality of parameter subsets for the second set of error coding dispersal storage parameters, which includes a second pillar width, a second write threshold, and a second read threshold; and

determine a third plurality of parameter subsets for the global set of error coding dispersal storage parameters, which includes a third pillar width, a third write threshold, and a third read threshold.

15. The computer of claim 10, wherein the processing module further functions to:

determine that a set of the plurality of DSNs have substantially similar local data retrieval accesses;

determine the global set of error coding dispersal storage parameters for the set of the plurality of DSNs;

encode the data in accordance with the set of global error coding dispersal storage parameters to produce a plurality of sets of encoded data slices;

store, via the interface, the plurality of sets of encoded data slices within one of the set of DSNs; and

output, via the interface the plurality of sets of encoded data slices to other DSNs of the set of the plurality of DSNs for storage therein.

16. The computer of claim 10, wherein the processing module further functions to:

receive, via the interface, the copy of the copies of the dispersed storage encoded data from a first one of the at least one of the plurality of DSNs;

determine a set of local dispersed storage (DS) units;

store, via interface, a plurality of sets of encoded data slices of the copy of the copies of the dispersed storage encoded data within the set of local DS units.

17. The computer of claim 16, wherein the processing module further functions to:

identify a third one of the at least one of the plurality of DSNs; and

output, via the interface, the copy of the copies of the dispersed storage encoded data to the third one of the at least one of the plurality of DSNs.

18. The computer of claim 10, wherein the processing module further functions to:

facilitate storage of at least one of the global and local error coding dispersal storage parameters; and

facilitate storage of a plurality of DSN identifiers (IDs), wherein each DSN ID of the plurality of DSN IDs corresponds to a DSN of the plurality of DSNs.