USING BROADCAST FOR PARALLELIZED AND RAPID SLICE REPLICATION IN A DISPERSED STORAGE NETWORK

• US 20170006099A1
• Filed: 06/28/2016
• Published: 01/05/2017
• Est. Priority Date: 06/30/2015
• Status: Abandoned Application

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Kazi, Asimuddin, Resch, Jason K.
• Application Number:
  US15/194,781
• Publication Number:
  US 20170006099A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/33 Synchronisation based on error coding or decoding
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/51 Discovery or management thereof, e.g. service location protocol [SLP] or web services

Abstract

Methods for use in a dispersed storage network (DSN) to enable rapid replication of data slices. Following dispersed storage error encoding of a data segment by a device of the DSN, slice naming information is generated for the resulting set of data slices. The slice naming information includes slice names and indicates a naming convention utilized for uniquely identifying replicated data slices. A set of write slice requests, including the slice naming information and the set of data slices, is then generated and broadcast or multicast to multiple sets of storage units of the DSN. Upon receipt of a write slice request, the recipient storage unit generates an updated slice name according to the naming convention and stores both the data slice and updated slice name in local memory. The data slice may be subsequently read from the storage unit by identifying the updated slice name in a read request.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

dispersed storage error encoding a data segment to produce a set of encoded data slices for replicated storage in sets of storage units of the DSN, where a data object is segmented into a plurality of data segments that include the data segment;

generating slice naming information for encoded data slices of the set of encoded data slices, the slice naming information indicating a naming convention utilized by the sets of storage units of the DSN for uniquely identifying encoded data slices as replicated in two or more sets of storage units;

generating a set of write slice requests, the set of write slice requests including the slice naming information and the set of encoded data slices; and

sending the set of write slice requests to the two or more sets of storage units of the DSN, wherein an individual write slice request of the set of write slice requests functions to initiate replicated storage of an associated encoded data slice in the two or more sets of storage units of the DSN in accordance with the naming convention.

2. The method of claim 1, wherein sending the set of write slice requests to the two or more sets of storage units of the DSN includes broadcasting the set of write slice requests over at least one wireless communication link of the DSN.

3. The method of claim 1, wherein sending the set of write slice requests to the two or more sets of storage units of the DSN includes multicasting, in a substantially simultaneous manner, the set of write slice requests to the two or more sets of storage units.

4. The method of claim 3, the multicasting including an internet protocol multicast transmission occurring, at least in part, over a wired communication link of the DSN.

5. The method of claim 1 further comprises:

generating a read slice request for an encoded data slice of the set of encoded data slices, the read slice request identifying, in accordance with the naming convention, a target storage unit of the plurality of sets of storage units of the DSN; and

sending the read slice request for processing by the target storage unit.

6. The method of claim 1, wherein sending the set of write slice requests to the two or more sets of storage units of the DSN includes generating and sending a digital signature or message authentication code for individual ones of the set of write slice requests.

7. The method of claim 1, prior to sending the set of write slice requests to the two or more sets of storage units of the DSN, encrypting the set of write slice requests for authentication by the two or more sets of storage units.

8. The method of claim 1, the slice naming information further indicating particular sets of storage units for replicated storage of particular encoded data slices of the set of encoded data slices.

9. The method of claim 1, the method repeated for additional data segments of the plurality of data segments.

10. A method for execution by one or more storage units of a dispersed storage network (DSN), the method comprises:

determining a naming convention for uniquely identifying replicated encoded data slices;

receiving, by a storage unit of the DSN, a write slice request to store an encoded data slice of a set of encoded data slices of a data segment, where a data object is segmented into a plurality of data segments that include the data segment, the write slice request including a slice name and the encoded data slice, the write slice request having been broadcast for reception by one or more additional storage units of the DSN;

determining that the encoded data slice is to be stored in the storage unit in accordance with the naming convention;

generating an updated slice name in accordance with the naming convention, the updated slice name uniquely identifying the encoded data slice in relation to the encoded data slice as replicated in the one or more additional storage units of the DSN receiving the write slice request; and

storing the encoded data slice and updated slice name in local memory of the storage unit.

11. The method of claim 10, wherein determining that the encoded data slice is to be stored in the storage unit in accordance with the naming convention is based on information contained within the write slice request.

12. The method of claim 10, wherein generating an updated slice name includes appending storage unit identification information to the slice name, the storage unit identification information distinct from storage unit identification information utilized by the one or more additional storage units of the DSN.

13. The method of claim 10 further comprises:

receiving, from a computing device of the DSN, a read slice request identifying the encoded data slice in accordance with the naming convention;

retrieving the encoded data slice from the local memory of the storage unit; and

sending the encoded data slice to the computing device.

14. The method of claim 13, wherein retrieving the encoded data slice includes utilizing stored integrity information corresponding to the encoded data slice to determine if the stored data of the encoded data slice is valid.

15. A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module operates to;

dispersed storage error encode a data segment to produce a set of encoded data slices for replicated storage in sets of storage units of the DSN, where a data object is segmented into a plurality of data segments that include the data segment;

generate slice naming information for encoded data slices of the set of encoded data slices, the slice naming information indicating a naming convention utilized by the sets of storage units of the DSN for uniquely identifying encoded data slices as replicated in two or more sets of storage units;

generate a set of write slice requests, the set of write slice requests including the slice naming information and the set of encoded data slices; and

send, via the interface, the set of write slice requests for reception by the two or more sets of storage units of the DSN, wherein an individual write slice request of the set of write slice requests functions to initiate replicated storage of an associated encoded data slice in the two or more sets of storage units of the DSN in accordance with the naming convention.

16. The computing device of claim 15, wherein sending, via the interface, the set of write slice requests to the two or more sets of storage units of the DSN includes wirelessly broadcasting the set of write slice requests over at least one wireless communication link of the DSN.

17. The computing device of claim 15, wherein sending, via the interface, the set of write slice requests to the two or more sets of storage units of the DSN includes multicasting, in a substantially simultaneous manner, the set of write slice requests to the two or more sets of storage units.

18. The computing device of claim 15, wherein the processing module further functions to:

generate a read slice request for an encoded data slice of the set of encoded data slices, the read slice request identifying, in accordance with the naming convention, a set of storage units of the plurality of sets of storage units of the DSN; and

send, via the interface, the read slice request for processing by the identified set of storage units.

19. The computing device of claim 15, wherein generating the set of write slice requests includes generating a digital signature or message authentication code for individual ones of the set of write slice requests, and appending the digital signature or message authentication code thereto.

20. The computing device of claim 15, wherein generating the set of write slice requests includes encrypting the set of write slice requests for authentication by the two or more sets of storage units.