Synchronizing storage of data copies in a dispersed storage network

• US 9,727,427 B2
• Filed: 10/29/2015
• Issued: 08/08/2017
• Est. Priority Date: 12/31/2014
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Horan, Scott Michael, Borich, Franco Vincent, Scholl, Daniel, Dhuse, Greg, Gray, Adam Michael, Baptist, Andrew, Khadiwala, Ravi, Leggette, Wesley, Cilfone, Bart
• Primary Examiner(s): Giardino, Jr., Mark
• Application Number:
  US14/927,446
• Publication Number:
  US 20160188218A1
• Time in Patent Office: 649 Days
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/20 using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins with storage units receiving a request to transfer a copy of a set of encoded data slices from two or more source virtual storage vaults to a destination virtual storage vault. The method continues with each storage unit obtaining a slice transfer map and determining whether the storage unit supports one of the two or more source virtual storage vaults. For each supporting storage unit, the method continues with each supporting storage unit determining, based on the slice transfer map, a sub-set of encoded data slices of the set of encoded data slices that is stored within the one of the two or more source virtual storage vaults by the supporting storage unit. The method continues with the supporting storage unit sending the sub-set of encoded data slices to a corresponding storage unit that is supporting the destination virtual storage vault.

Claims

1. In a dispersed storage network (DSN) that includes a plurality of storage units, wherein the plurality of storage units support a plurality of virtual storage vaults, wherein a first virtual storage vault of the plurality of virtual storage vaults is supported by a first set of storage units of the plurality of storage units, and wherein the first set of storage units is a sub-set of the plurality of storage units, a method comprises:

receiving, by at least some of the plurality of storage units, a request to transfer a copy of a set of encoded data slices from two or more source virtual storage vaults of the plurality of virtual storage vaults to a destination virtual storage vault of the plurality of virtual storage vaults, wherein a data segment of data is dispersed storage error encoded to produce the set of encoded data slices;

obtaining, by each storage unit of the at least some of the plurality of storage units, a slice transfer map that, for the set of encoded data slices, identifies the two or more source virtual storage vaults;

determining, by each storage unit of the at least some of the plurality of storage units, whether the each storage unit supports one of the two or more source virtual storage vaults;

for each supporting storage unit that supports one of the two or more source virtual storage vaults, determining, by the each supporting storage unit and based on the slice transfer map, a sub-set of encoded data slices of the set of encoded data slices that is stored within the one of the two or more source virtual storage vaults by the each supporting storage unit; and

sending, by the each supporting storage unit, the sub-set of encoded data slices to a corresponding storage unit of the plurality of storage units that is supporting the destination virtual storage vault.

2. The method of claim 1 further comprises:

receiving the request to transfer in response to a determination that the destination virtual storage vault failed a write request regarding the copy of the set of encoded data slices, wherein the write request is an instruction to write the copy of the set of encoded data slices to each of the two or more source virtual storage vaults and to the destination virtual storage vault.

3. The method of claim 1 further comprises:

receiving the request to transfer as part of a write request to write the copy of the set of encoded data slices to each of the two or more source virtual storage vaults and to the destination virtual storage vault, wherein a first portion of the write request writes the respective sub-sets of encoded data slices to the storage units supporting the two or more source virtual storage vaults and a second portion to transfer the copy of the sets of encoded data slices to the storage units supporting the destination virtual storage vault.

4. The method of claim 1, wherein the obtaining the slice transfer map comprises one or more of:

receiving the slice transfer map;

retrieving the slice transfer map from memory; and

generating the slice transfer map based on the request to transfer.

5. The method of claim 1 further comprises:

changing, by each of the corresponding storage units of the plurality of storage units that is supporting the destination virtual storage vault, DSN addresses of the set of encoded data slices of the copy of the set of encoded data slices to reflect that the copy is stored in the destination virtual storage vault.

6. The method of claim 5 further comprises:

updating a slice storage table to include the DSN addresses of the set of encoded data slices stored in the destination virtual storage vault.

7. The method of claim 1, wherein the at least some of the plurality of storage units comprises:

the storage units associated with the two or more source virtual storage vaults; and

the storage units associated with the destination virtual storage vault.

8. The method of claim 1, wherein the sending the sub-set of encoded data slices comprises:

sending, by a first supporting storage unit, a first encoded data slice to a first corresponding storage unit of the plurality of storage units that is supporting the destination virtual storage vault;

sending, by a second supporting storage unit, a second encoded data slice to a second corresponding storage unit of the plurality of storage units that is supporting the destination virtual storage vault;

sending, by a third supporting storage unit, a third encoded data slice to a third corresponding storage unit of the plurality of storage units that is supporting the destination virtual storage vault; and

sending, by a fourth supporting storage unit, a fourth encoded data slice to a fourth corresponding storage unit of the plurality of storage units that is supporting the destination virtual storage vault.

9. The method of claim 1, wherein the sending the sub-set of encoded data slices comprises:

sending, by a first supporting storage unit, a first and a second encoded data slice to a first corresponding storage unit of the plurality of storage units that is supporting the destination virtual storage vault; and

sending, by a second supporting storage unit, a third encoded data slice to a second corresponding storage unit of the plurality of storage units that is supporting the destination virtual storage vault.

10. A non-transitory computer readable storage medium of a dispersed storage network (DSN) that includes a plurality of storage units, wherein the plurality of storage units support a plurality of virtual storage vaults, wherein a first virtual storage vault of the plurality of virtual storage vaults is supported by a first set of storage units of the plurality of storage units, and wherein the first set of storage units is a sub-set of the plurality of storage units, the non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of the DSN, causes the one or more computing devices to;

receive, by at least some of the plurality of storage units, a request to transfer a copy of a set of encoded data slices from two or more source virtual storage vaults of the plurality of virtual storage vaults to a destination virtual storage vault of the plurality of virtual storage vaults, wherein a data segment of data is dispersed storage error encoded to produce the set of encoded data slices;

obtain, by each storage unit of the at least some of the plurality of storage units, a slice transfer map that, for the set of encoded data slices, identifies the two or more source virtual storage vaults;

determine, by each storage unit of the at least some of the plurality of storage units, whether the each storage unit supports one of the two or more source virtual storage vaults;

for each supporting storage unit that supports one of the two or more source virtual storage vaults, determine, by the each supporting storage unit and based on the slice transfer map, a sub-set of encoded data slices of the set of encoded data slices that is stored within the one of the two or more source virtual storage vaults by the each supporting storage unit; and

send, by the each supporting storage unit, the sub-set of encoded data slices to a corresponding storage unit of the plurality of storage units that is supporting the destination virtual storage vault.

11. The non-transitory computer readable storage medium of claim 10 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

receive the request to transfer in response to a determination that the destination virtual storage vault failed a write request regarding the copy of the set of encoded data slices, wherein the write request is an instruction to write the copy of the set of encoded data slices to each of the two or more source virtual storage vaults and to the destination virtual storage vault.

12. The non-transitory computer readable storage medium of claim 10 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

receive the request to transfer as part of a write request to write the copy of the set of encoded data slices to each of the two or more source virtual storage vaults and to the destination virtual storage vault, wherein a first portion of the write request writes the respective sub-sets of encoded data slices to the storage units supporting the two or more source virtual storage vaults and a second portion to transfer the copy of the sets of encoded data slices to the storage units supporting the destination virtual storage vault.

13. The non-transitory computer readable storage medium of claim 10, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to obtain the slice transfer map comprises by one or more of:

receiving the slice transfer map;

retrieving the slice transfer map from memory; and

generating the slice transfer map based on the request to transfer.

14. The non-transitory computer readable storage medium of claim 10 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

change, by each of the corresponding storage units of the plurality of storage units that is supporting the destination virtual storage vault, DSN addresses of the set of encoded data slices of the copy of the set of encoded data slices to reflect that the copy is stored in the destination virtual storage vault.

15. The non-transitory computer readable storage medium of claim 14 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

update a slice storage table to include the DSN addresses of the set of encoded data slices stored in the destination virtual storage vault.

16. The non-transitory computer readable storage medium of claim 10, wherein the at least some of the plurality of storage units comprises:

the storage units associated with the two or more source virtual storage vaults; and

the storage units associated with the destination virtual storage vault.

17. The non-transitory computer readable storage medium of claim 10, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to send the sub-set of encoded data slices by:

sending, by a first supporting storage unit, a first encoded data slice to a first corresponding storage unit of the plurality of storage units that is supporting the destination virtual storage vault;

sending, by a second supporting storage unit, a second encoded data slice to a second corresponding storage unit of the plurality of storage units that is supporting the destination virtual storage vault;

sending, by a third supporting storage unit, a third encoded data slice to a third corresponding storage unit of the plurality of storage units that is supporting the destination virtual storage vault; and

sending, by a fourth supporting storage unit, a fourth encoded data slice to a fourth corresponding storage unit of the plurality of storage units that is supporting the destination virtual storage vault.

18. The non-transitory computer readable storage medium of claim 10, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to send the sub-set of encoded data slices by:

sending, by a first supporting storage unit, a first and a second encoded data slice to a first corresponding storage unit of the plurality of storage units that is supporting the destination virtual storage vault; and

sending, by a second supporting storage unit, a third encoded data slice to a second corresponding storage unit of the plurality of storage units that is supporting the destination virtual storage vault.