Coordinating storage of data in dispersed storage networks

• US 9,727,275 B2
• Filed: 09/29/2015
• Issued: 08/08/2017
• Est. Priority Date: 12/02/2014
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Kazi, Asimuddin
• Primary Examiner(s): Knapp, Justin R
• Application Number:
  US14/869,240
• Publication Number:
  US 20160154698A1
• Time in Patent Office: 679 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A method begins by a first computing device of a dispersed storage network (DSN) of a plurality of DSNs receiving a data segment for storage, encoding the data segment using a dispersed storage error encoding function to produce a set of encoded data slices, where each DSN of the plurality of DSNs utilizes the dispersed storage error encoding function, and generating a set of DSN addresses for the set of encoded data slices using a deterministic function. The method continues with a set of storage units of the DSN storing the set of encoded data slices using the set of DSN addresses. The method continues with storage units of the set of storage units and of another set of storage units of another DSN coordinating to store copies of encoded data slices of the set of encoded data slices in corresponding storage units of both sets of storage units.

Claims

1. A method comprises:

receiving, by a first computing device of a dispersed storage network (DSN) of a plurality of DSNs, a data segment of data for storage;

encoding, by the first computing device, the data segment using a dispersed storage error encoding function to produce a set of encoded data slices, wherein each DSN of the plurality of DSNs utilizes the dispersed storage error encoding function;

generating, by the first computing device, a set of DSN addresses for the set of encoded data slices using a deterministic function, wherein computing devices of other DSNs of the plurality of DSNs use the deterministic function to generate the set of DSN addresses;

storing, by a first set of storage units of the DSN, the set of encoded data slices in accordance with the set of DSN addresses;

coordinating between a first storage unit of the first set of storage units and a first storage unit of a second set of storage units of a second DSN of the plurality of DSNs to store a copy of a first encoded data slice of the set of encoded data slices in the first storage units of both sets of storage units; and

coordinating between a second storage unit of the first set of storage units and a second storage unit of the second set of storage units to store a copy of a second encoded data slice of the set of encoded data slices in the second storage units of both sets of storage units.

2. The method of claim 1 further comprises:

coordinating between each of a subset of storage units of the set of storage units and a corresponding subset of storage units of the other set of storage units to store a copy of a subset of encoded data slices of the set of encoded data slices in the subset of storage units of both sets of storage units, wherein a number of storage units in the subset of storage units is equal to or greater than a decode threshold number and is less than a pillar width number, wherein the decode threshold number corresponds to a minimum number of encoded data slices of the set of encoded data slices required to recover the data segment and the pillar width number corresponds to a number of encoded data slices in the set of encoded data slices.

3. The method of claim 1 further comprises:

verifying, by the first and second storage units of the set of storage units, a common revision level of a first encoded data slice of another set of encoded data slices and of a second encoded data slice of the other set of encoded data slices, wherein the first computing device encoded another data segment to produce the other set of encoded data slices.

4. The method of claim 3 further comprises:

when the common revision level of the first encoded data slice of the other set of encoded data slices is less than the common revision level of the second encoded data slice of the other set of encoded data slices, initiating a rebuilding for the first encoded data slice of the other set of encoded data slices;

storing, by the first storage unit of the set of storage units, a rebuilt first encoded data slice of the other set of encoded data slices; and

coordinating between the first storage unit of the set of storage units and the first storage unit of the other set of storage units to store a copy of the rebuilt first encoded data slice of the other set of encoded data slices in the first storage units of both sets of storage units.

5. The method of claim 1, wherein the coordinating between the first storage unit of the set of storage units and the first storage unit of the other set of storage units comprises:

sending, by the first storage unit of the other set of storage units, a list request to the first storage unit of the set of storage units;

sending, by the first storage unit of the set of storage units, a listing of encoded data slices stored by the first storage unit of the set of storage units to the first storage unit of the other set of storage units;

determining, by the first storage unit of the other set of storage units, that the first storage unit of the other set of storage units is not currently storing the copy of the first encoded data slice of the set of encoded data slices; and

when the first storage unit of the other set of storage units is not currently storing the copy of the first encoded data slice of the set of encoded data slices, sending, by the first storage unit of the other set of storage units, a read request to the first storage unit of the set of storage units.

6. The method of claim 1, wherein the coordinating between the first storage unit of the set of storage units and the first storage unit of the other set of storage units comprises:

sending, by the first storage unit of the set of storage units, an indication of storing the first encoded data slice of the set of encoded data slices; and

in response to the indication, sending, by the first storage unit of the other set of storage units, a read request to the first storage unit of the set of storage units.

7. The method of claim 1 further comprises:

receiving, by a second computing device of the second DSN of the plurality of DSNs, a second data segment of second data for storage;

encoding, by the second computing device, the second data segment using the dispersed storage error encoding function to produce a second set of encoded data slices;

generating, by the second computing device, a second set of DSN addresses for the second set of encoded data slices using the deterministic function;

storing, by the second set of storage units, the second set of encoded data slices in accordance with the second set of DSN addresses;

coordinating between the first storage unit of the second set of storage units and the first storage unit of the first set of storage units to store a copy of a first encoded data slice of the second set of encoded data slices in the first storage units of both sets of storage units; and

coordinating between the second storage unit of the second set of storage units and the second storage unit of the first set of storage units to store a copy of a second encoded data slice of the second set of encoded data slices in the second storage units of both sets of storage units.

8. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN) of a plurality of DSNs, causes the one or more computing devices to;

receive, by a first computing device of a dispersed storage network (DSN) of the plurality of DSNs, a data segment of data for storage;

encode, by the first computing device, the data segment using a dispersed storage error encoding function to produce a set of encoded data slices, wherein each DSN of the plurality of DSNs utilizes the dispersed storage error encoding function;

generate, by the first computing device, a set of DSN addresses for the set of encoded data slices using a deterministic function, wherein computing devices of other DSNs of the plurality of DSNs use the deterministic function to generate the set of DSN addresses;

store, by a first set of storage units of the DSN, the set of encoded data slices in accordance with the set of DSN addresses;

coordinate between a first storage unit of the first set of storage units and a first storage unit of a second set of storage units of a second DSN of the plurality of DSNs to store a copy of a first encoded data slice of the set of encoded data slices in the first storage units of both sets of storage units; and

coordinate between a second storage unit of the first set of storage units and a second storage unit of the second set of storage units to store a copy of a second encoded data slice of the set of encoded data slices in the second storage units of both sets of storage units.

9. The non-transitory computer readable storage medium of claim 8 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices to;

coordinate between each of a subset of storage units of the set of storage units and a corresponding subset of storage units of the other set of storage units to store a copy of a subset of encoded data slices of the set of encoded data slices in the subset of storage units of both sets of storage units, wherein a number of storage units in the subset of storage units is equal to or greater than a decode threshold number and is less than a pillar width number, wherein the decode threshold number corresponds to a minimum number of encoded data slices of the set of encoded data slices required to recover the data segment and the pillar width number corresponds to a number of encoded data slices in the set of encoded data slices.

10. The non-transitory computer readable storage medium of claim 8 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices to;

verify, by the first and second storage units of the set of storage units, a common revision level of a first encoded data slice of another set of encoded data slices and of a second encoded data slice of the other set of encoded data slices, wherein the first computing device encoded another data segment to produce the other set of encoded data slices.

11. The non-transitory computer readable storage medium of claim 10 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices to;

when the common revision level of the first encoded data slice of the other set of encoded data slices is less than the common revision level of the second encoded data slice of the other set of encoded data slices, initiate a rebuilding for the first encoded data slice of the other set of encoded data slices;

store, by the first storage unit of the set of storage units, a rebuilt first encoded data slice of the other set of encoded data slices; and

coordinate between the first storage unit of the set of storage units and the first storage unit of the other set of storage units to store a copy of the rebuilt first encoded data slice of the other set of encoded data slices in the first storage units of both sets of storage units.

12. The non-transitory computer readable storage medium of claim 8, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices to coordinate between the first storage unit of the set of storage units and the first storage unit of the other set of storage units by:

sending, by the first storage unit of the other set of storage units, a list request to the first storage unit of the set of storage units;

sending, by the first storage unit of the set of storage units, a listing of encoded data slices stored by the first storage unit of the set of storage units to the first storage unit of the other set of storage units;

determining, by the first storage unit of the other set of storage units, that the first storage unit of the other set of storage units is not currently storing the copy of the first encoded data slice of the set of encoded data slices; and

when the first storage unit of the other set of storage units is not currently storing the copy of the first encoded data slice of the set of encoded data slices, sending, by the first storage unit of the other set of storage units, a read request to the first storage unit of the set of storage units.

13. The non-transitory computer readable storage medium of claim 8, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices to coordinate between the first storage unit of the set of storage units and the first storage unit of the other set of storage units by:

sending, by the first storage unit of the set of storage units, an indication of storing the first encoded data slice of the set of encoded data slices; and

in response to the indication, sending, by the first storage unit of the other set of storage units, a read request to the first storage unit of the set of storage units.

14. The non-transitory computer readable storage medium of claim 8 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices to;

receive, by a second computing device of the other DSN of the plurality of DSNs, a second data segment of second data for storage;

encode, by the second computing device, the second data segment using the dispersed storage error encoding function to produce a second set of encoded data slices;

generate, by the second computing device, a second set of DSN addresses for the second set of encoded data slices using the deterministic function;

store, by the second set of storage units, the second set of encoded data slices in accordance with the second set of DSN addresses;

coordinate between the first storage unit of the second set of storage units and the first storage unit of the first set of storage units to store a copy of a first encoded data slice of the second set of encoded data slices in the first storage units of both sets of storage units; and

coordinate between the second storage unit of the second set of storage units and the second storage unit of the first set of storage units to store a copy of a second encoded data slice of the second set of encoded data slices in the second storage units of both sets of storage units.