Writing encoded data slices in a dispersed storage network

• US 9,778,987 B2
• Filed: 12/15/2014
• Issued: 10/03/2017
• Est. Priority Date: 01/31/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Shirley, Jr., Thomas Franklin, Grube, Gary W., Cilfone, Bart, Khadiwala, Ravi, Dhuse, Greg, Cocagne, Thomas Darrel, Storm, Michael Colin, Vedpathak, Yogesh Ramesh, Leggette, Wesley, Resch, Jason K., Baptist, Andrew, Volvovski, Ilya
• Primary Examiner(s): Yu, Jae
• Application Number:
  US14/570,366
• Publication Number:
  US 20150220396A1
• Time in Patent Office: 1,023 Days
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method for writing a set of encoded data slices to memory of a dispersed storage network (DSN) begins by a processing module identifying an encoded data slice of the set of encoded data slices for a redundant write operation to produce an identified encoded data slice. The method continues with the processing module generating a set of first write requests regarding the set of encoded data slices less the identified encoded data slice and generating a set of second write requests regarding the identified encoded data slice. The method continues with the processing module sending the set of first write requests to storage units of the DSN and sending the set of second write requests to a set of storage units of the DSN, where each storage unit of the set of storage units is sent a corresponding one of the set of second write requests.

Claims

1. A method for writing a set of encoded data slices to memory of a dispersed storage network (DSN), wherein the method is executed by one or more processing modules of one or more computing devices of the DSN, the method comprises:

identifying an encoded data slice of the set of encoded data slices for a redundant write operation to produce an identified encoded data slice;

generating a set of first write requests regarding the set of encoded data slices less the identified encoded data slice;

generating a set of second write requests regarding the identified encoded data slice, the set of second write requests including the identified encoded data slice and replications of the identified encoded data slice;

sending the set of first write requests to storage units of the DSN; and

sending the set of second write requests to a set of storage units of the DSN, wherein each storage unit of the set of storage units is sent a corresponding one of the set of second write requests.

2. The method of claim 1 further comprises:

receiving a write response from a storage unit of the set of storage units; and

sending a delete write request to remaining storage units of the set of storage units.

3. The method of claim 1 further comprises:

receiving write responses from at least some storage units of a combined set of storage units that includes the storage units and the set of storage units; and

when a threshold number of write responses have been received, sending a write commit command to each storage unit of the at least some storage units of the combined set of storage units.

4. The method of claim 3 further comprises:

determining whether the at least some storage units of the combined set of storage units include a storage unit of the set of storage units; and

when the at least some storage units of the combined set of storage units includes the storage unit of the set of storage units;

sending the write commit command to the storage unit of the set of storage units; and

sending a rollback command to each remaining storage unit of the set of storage units.

5. The method of claim 3 further comprises:

determining whether the at least some storage units of the combined set of storage units include a storage unit of the set of storage units; and

when the at least some storage units of the combined set of storage units does not include the storage unit of the set of storage units, sending a rollback command to each storage unit of the set of storage units.

6. The method of claim 3 further comprises:

receiving commit responses from the at least some storage units of the combined set of storage units that includes the storage units and the set of storage units; and

when a commit threshold number of the commit responses have been received, sending a write finalize command to each storage unit of the at least some storage units of the combined set of storage units.

7. The method of claim 6 further comprises:

determining whether the at least some storage units of the combined set of storage units that provided the commit responses include a storage unit of the set of storage units; and

when the at least some storage units of the combined set of storage units that provided the commit responses includes the storage unit of the set of storage units;

sending a write finalize command to the storage unit of the set of storage units; and

sending an undo command to each remaining storage unit of the set of storage units.

8. The method of claim 6 further comprises:

determining whether the at least some storage units of the combined set of storage units that provided the commit responses include a storage unit of the set of storage units; and

when the at least some storage units of the combined set of storage units that provided the commit responses does not include the storage unit of the set of storage units, sending an undo command to the set of storage units.

9. The method of claim 1 further comprises:

identifying a second encoded data slice of the set of encoded data slices for the redundant write operation to produce a second identified encoded data slice;

generating the set of first write requests regarding the set of encoded data slices less the identified encoded data slice and the second identified encoded data slice;

generating a set of third write requests regarding the second identified encoded data slice, the set of third write requests including the second identified encoded data slice and replications of the second identified encoded data slice; and

sending the set of third write requests to a second set of storage units of the DSN, wherein each storage unit of the second set of storage units is sent a corresponding one of the set of third write requests.

10. A computer readable memory device comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

identify an encoded data slice of a set of encoded data slices for a redundant write operation to produce an identified encoded data slice;

generate a set of first write requests regarding the set of encoded data slices less the identified encoded data slice;

generate a set of second write requests regarding the identified encoded data slice, the set of second write requests including the identified encoded data slice and replications of the identified encoded data slice;

send the set of first write requests to storage units of the DSN; and

send the set of second write requests to a set of storage units of the DSN, wherein each storage unit of the set of storage units is sent a corresponding one of the set of second write requests.

11. The computer readable memory device of claim 10 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

receive a write response from a storage unit of the set of storage units; and

send a delete write request to remaining storage units of the set of storage units.

12. The computer readable memory device of claim 10 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

receive write responses from at least some storage units of a combined set of storage units that includes the storage units and the set of storage units; and

when a threshold number of write responses have been received, send a write commit command to each storage unit of the at least some storage units of the combined set of storage units.

13. The computer readable memory device of claim 12 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

determine whether the at least some storage units of the combined set of storage units include a storage unit of the set of storage units; and

when the at least some storage units of the combined set of storage units includes the storage unit of the set of storage units;

send the write commit command to the storage unit of the set of storage units; and

send a rollback command to each remaining storage unit of the set of storage units.

14. The computer readable memory device of claim 12 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

determine whether the at least some storage units of the combined set of storage units include a storage unit of the set of storage units; and

when the at least some storage units of the combined set of storage units does not include the storage unit of the set of storage units, send a rollback command to each storage unit of the set of storage units.

15. The computer readable memory device of claim 12 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

receive commit responses from the at least some storage units of the combined set of storage units that includes the storage units and the set of storage units; and

when a commit threshold number of the commit responses have been received, send a write finalize command to each storage unit of the at least some storage units of the combined set of storage units.

16. The computer readable memory device of claim 15 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

determine whether the at least some storage units of the combined set of storage units that provided the commit responses include a storage unit of the set of storage units; and

when the at least some storage units of the combined set of storage units that provided the commit responses includes the storage unit of the set of storage units;

send a write finalize command to the storage unit of the set of storage units; and

send an undo command to each remaining storage unit of the set of storage units.

17. The computer readable memory device of claim 15 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

determine whether the at least some storage units of the combined set of storage units that provided the commit responses include a storage unit of the set of storage units; and

when the at least some storage units of the combined set of storage units that provided the commit responses does not include the storage unit of the set of storage units, send an undo command to the set of storage units.

18. The computer readable memory device of claim 10 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

identify a second encoded data slice of the set of encoded data slices for the redundant write operation to produce a second identified encoded data slice;

generate the set of first write requests regarding the set of encoded data slices less the identified encoded data slice and the second identified encoded data slice;

generate a set of third write requests regarding the second identified encoded data slice, the set of third write requests including the second identified encoded data slice and replications of the second identified encoded data slice; and

send the set of third write requests to a second set of storage units of the DSN, wherein each storage unit of the second set of storage units is sent a corresponding one of the set of third write requests.

19. A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

identify an encoded data slice of a set of encoded data slices for a redundant write operation to produce an identified encoded data slice;

generate a set of first write requests regarding the set of encoded data slices less the identified encoded data slice;

generate a set of second write requests regarding the identified encoded data slice, the set of second write requests including the identified encoded data slice and replications of the identified encoded data slice;

send, via the interface, the set of first write requests to storage units of the DSN; and

send, via the interface, the set of second write requests to a set of storage units of the DSN, wherein each storage unit of the set of storage units is sent a corresponding one of the set of second write requests.

20. The computing device of claim 19, wherein the processing module further functions to:

receive, via the interface, a write response from a storage unit of the set of storage units; and

send, via the interface, a delete write request to remaining storage units of the set of storage units.

21. The computing device of claim 19, wherein the processing module further functions to:

receive, via the interface, write responses from at least some storage units of a combined set of storage units that includes the storage units and the set of storage units; and

when a threshold number of write responses have been received, send, via the interface, a write commit command to each storage unit of the at least some storage units of the combined set of storage units.

22. The computing device of claim 21, wherein the processing module further functions to:

determine whether the at least some storage units of the combined set of storage units include a storage unit of the set of storage units; and

when the at least some storage units of the combined set of storage units includes the storage unit of the set of storage units;

send, via the interface, the write commit command to the storage unit of the set of storage units; and

send, via the interface, a rollback command to each remaining storage unit of the set of storage units.

23. The computing device of claim 21, wherein the processing module further functions to:

determine whether the at least some storage units of the combined set of storage units include a storage unit of the set of storage units; and

when the at least some storage units of the combined set of storage units does not include the storage unit of the set of storage units, send, via the interface, a rollback command to each storage unit of the set of storage units.

24. The computing device of claim 21, wherein the processing module further functions to:

receive, via the interface, commit responses from the at least some storage units of the combined set of storage units that includes the storage units and the set of storage units; and

when a commit threshold number of the commit responses have been received, send, via the interface, a write finalize command to each storage unit of the at least some storage units of the combined set of storage units.

25. The computing device of claim 24, wherein the processing module further functions to:

determine whether the at least some storage units of the combined set of storage units that provided the commit responses include a storage unit of the set of storage units; and

when the at least some storage units of the combined set of storage units that provided the commit responses includes the storage unit of the set of storage units;

send, via the interface, a write finalize command to the storage unit of the set of storage units; and

send, via the interface, an undo command to each remaining storage unit of the set of storage units.

26. The computing device of claim 24, wherein the processing module further functions to:

determine whether the at least some storage units of the combined set of storage units that provided the commit responses include a storage unit of the set of storage units; and

when the at least some storage units of the combined set of storage units that provided the commit responses does not include the storage unit of the set of storage units, send, via the interface, an undo command to the set of storage units.

27. The computing device of claim 19, wherein the processing module further functions to:

identify a second encoded data slice of the set of encoded data slices for the redundant write operation to produce a second identified encoded data slice;

generate the set of first write requests regarding the set of encoded data slices less the identified encoded data slice and the second identified encoded data slice;

generate a set of third write requests regarding the second identified encoded data slice, the set of third write requests including the second identified encoded data slice and replications of the second identified encoded data slice; and

send, via the interface, the set of third write requests to a second set of storage units of the DSN, wherein each storage unit of the second set of storage units is sent a corresponding one of the set of third write requests.