Accessing data when transferring the data between storage facilities

• US 10,303,546 B2
• Filed: 06/01/2016
• Issued: 05/28/2019
• Est. Priority Date: 06/30/2015
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Motwani, Manish, Baptist, Andrew D., Khadiwala, Ravi V., Resch, Jason K.
• Primary Examiner(s): Cox, Natisha D
• Application Number:
  US15/170,404
• Publication Number:
  US 20170006103A1
• Time in Patent Office: 1,091 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/33 Synchronisation based on error coding or decoding
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/51 Discovery or management thereof, e.g. service location protocol [SLP] or web services

Abstract

A processing module issues a distributed agreement protocol (DAP) information to DSN storage units. The method continues with the processing module initiating, by each storage unit of a first set of the plurality of storage units, transfer of locally stored data slices to another storage unit of another set of the plurality of storage units. The method continues with the processing module, while transferring the locally stored data slices, a storage unit receiving an access request for a data slice that has been transferred, obtains the data slice by proxy from the another storage unit. The method continues with the processing module, when completing the transfer of the locally stored data slices, issuing, by each of the storage units, a DAP status to the transition coordinator and, when receiving a conformation of completion of transferring data slices by all of the storage units issuing updated DAP information.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

issuing, by a transition coordinator, distributed agreement protocol (DAP) information to DSN resources, where the DSN resources include a plurality of storage units, and wherein the DAP information includes a current system configuration and a next system configuration;

initiating, by each storage unit of a first set of the plurality of storage units, transfer of locally stored data slices to another storage unit of another set of the plurality of storage units;

while transferring the locally stored data slices, a storage unit receiving an access request for a data slice that has been transferred, obtains the data slice by proxy from the another storage unit;

when completing the transfer of the locally stored data slices, issuing, by each of the storage units, a DAP status to the transition coordinator; and

when receiving a conformation of completion of transferring data slices by all of the storage units issuing, by the transition coordinator, updated DAP information.

2. The method of claim 1, wherein the current system configuration and next system configuration each include weighting factors for the DSN resources.

3. The method of claim 1, wherein, before issuing the updated DAP information, the data slices are associated with the storage unit of the first set in accordance with the current system configuration, but not the next configuration.

4. The method of claim 3, wherein, after issuing the updated DAP information, the data slices are associated with the storage unit of the another set in accordance the next system configuration, which is now a current system configuration.

5. The method of claim 1, wherein, before the transfer, a DAP function is performed for each data slice name of a data slice to identify the another storage unit.

6. The method of claim 1, wherein the obtaining includes issuing a proxied access request to the another storage unit, receiving a proxy access response and issuing an access response to a requesting entity.

7. The method of claim 1, wherein the DAP status includes a completion indicator.

8. A dispersed storage (DS) module comprises:

a first module, when operable within a computing device, causes the computing device to;

issue, by a transition coordinator, distributed agreement protocol (DAP) information to distributed storage network (DSN) resources, where the DSN resources include a plurality of storage units, and wherein the DAP information includes a current system configuration and a next system configuration;

a second module, when operable within the computing device, causes the computing device to;

initiate transfer, by each storage unit, of locally stored encoded data slices to another corresponding storage unit in accordance with the DAP information;

a third module, when operable within the computing device, causes the computing device to;

while transferring encoded data slices and a storage unit receiving an access request for an encoded data slice that has been transferred, obtain the encoded data slice by proxy from the another storage unit;

a fourth module, when operable within the computing device, causes the computing device to;

when each storage unit completes the encoded data slice transfer, issue a DAP status to the transition coordinator; and

a fifth module, when operable within the computing device, causes the computing device to;

when receiving confirmation of completion of the transferring of the encoded data slices by a threshold number of storage units of the plurality of storage units, issue, by the transition coordinator, updated DAP information.

9. The dispersed storage (DS) module of claim 8, wherein the current system configuration and next system configuration each include weighting factors for the DSN resources.

10. The dispersed storage (DS) module of claim 8, wherein the encoded data slices are associated with the storage unit in accordance with the current system configuration but not the next configuration.

11. The dispersed storage (DS) module of claim 8, wherein, before issuing the updated DAP information, the data slices are associated with the storage unit in accordance with the current system configuration, but not the next configuration.

12. The dispersed storage (DS) module of claim 8, wherein after issuing the updated DAP information, the data slices are associated with the another storage unit in accordance the next system configuration, which is now a current system configuration.

13. The dispersed storage (DS) module of claim 8, wherein, before the transfer, a DAP function is performed for each data slice name of a data slice to identify the another storage unit.

14. The dispersed storage (DS) module of claim 8, wherein the obtaining of the encoded data slice from the another storage unit includes issuing a proxied access request to the another storage unit, receiving a proxy access response and issuing an access response to a requesting entity.

15. The dispersed storage (DS) module of claim 8, wherein the DAP status includes a completion indicator.

16. A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

identify distributed storage network (DSN) resources that utilize distributed agreement protocol (DAP) information, generate a DAP information message and send the DAP information message to the identified DSN resources, the DAP information message identifying another storage unit to transfer encoded data slices to in accordance with a next system configuration information;

issue write slice requests to transfer the encoded data slices to the another storage unit, where the write slice request includes the encode data slices;

during the transfer of the encoded data slices, issue a proxied encoded data slice access request to the another storage unit for all requests to the transferring encoded data slices;

generate a DAP status to indicate completion of the transfer of the encoded data slices to conform with the next system configuration information;

generate updated DAP information to indicate that a next system configuration is now a current system configuration;

identify the DSN resources that utilize the updated DAP information; and

send the updated DAP information to the identified DSN resources.

17. The computing device of claim 16, wherein the processing module further functions to

publish as part of a system registry the updated DAP information.

18. The computing device of claim 16, wherein the processing module further functions to

issue separate updated DAP information messages.