Access slices during multiple migrations

• US 10,305,982 B2
• Filed: 01/16/2017
• Issued: 05/28/2019
• Est. Priority Date: 01/26/2016
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Baptist, Andrew D., Motwani, Manish
• Primary Examiner(s): Ndiaye, Cheikh T
• Application Number:
  US15/407,023
• Publication Number:
  US 20170214741A1
• Time in Patent Office: 862 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/0837 with software control, e.g. non-cacheable data
• CPC Class Codes: G06F 12/0871 Allocation or management of cache space
• CPC Class Codes: G06F 12/12 Replacement control
• CPC Class Codes: G06F 12/122 of the least frequently used [LFU] type, e.g. with individual count value
• CPC Class Codes: G06F 12/128 adapted to multidimensional cache systems, e.g. set-associative, multicache, multiset or multilevel
• CPC Class Codes: G06F 16/214 Database migration support
• CPC Class Codes: G06F 16/2246 Trees, e.g. B+trees
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 2212/604 Details relating to cache allocation
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0632 by initialisation or re-initialisation of storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/5083 : Techniques for rebalancing the load in a distributed system
• CPC Class Codes: G06N 3/002 : Biomolecular computers, i.e. using biomolecules, proteins, cells using DNA G06N3/123; using neurons G06N3/061
• CPC Class Codes: H03M 13/1515 : Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 : using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04H 60/27 : Arrangements for recording or accumulating broadcast information or broadcast-related information
• CPC Class Codes: H04L 67/1095 : Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/55 : Push-based network services
• CPC Class Codes: H04L 67/568 : Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A method begins by identifying, when a first migration of a plurality of sets of encoded data slices from an original source storage set to an intermediate destination storage set is active, a new destination storage set of a second migration for the plurality of sets of encoded data slices. The method continues by issuing migration requests to storage sets associated with current storage of the plurality of sets of encoded data slices in accordance with a first cursor identifying a particular DSN storage address of a corresponding set of encoded data slices and, when the second migration is active, facilitating processing of a data access request to produce a data access response utilizing the first cursor of the first migration and a second cursor identifying a particular DSN address of a corresponding set of encoded data slices that is next up for migration.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

identifying, when a first migration of a plurality of sets of encoded data slices from an original source storage set (first storage set) to an intermediate destination storage set (second storage set) is active, a new destination storage set (third storage set) of a second migration for the plurality of sets of encoded data slices;

issuing migration requests to one or more storage sets associated with current storage of the plurality of sets of encoded data slices in accordance with a first cursor of the first migration, the first cursor identifying a particular DSN storage address of a corresponding set of encoded data slices; and

when the second migration of the plurality of sets of encoded data slices is active, facilitating processing of a data access request to produce a data access response utilizing the first cursor of the first migration and a second cursor of the second migration, where the second cursor identifies a particular DSN address of a corresponding set of encoded data slices that is next up for migration.

2. The method of claim 1, wherein the facilitating includes applying a decentralized agreement protocol function to a range of slice names for a plurality of storage sets utilizing alternate weights for at least some of the plurality the storage sets to identify a new recipient storage set.

3. The method of claim 1, wherein the issuing migration requests includes any of:

sending first and second migration requests, for an address range including respective first and second encoded data slices, to the second storage set to send the respective first and second encoded data slices to the third storage set;

sending third and fourth migration requests, for an address range including respective third and fourth encoded data slices, to the first storage set to send the respective third and fourth encoded data slices to the third storage set, when the first cursor indicates that all of the respective first encoded data slices have already been moved to the second storage set, at least part of the of the encoded data slices have been moved to the second storage set, and none of the respective fourth encoded data slices have been moved from the first storage set.

4. The method of claim 1, wherein the facilitating includes:

identifying a storage set for the data access requests based on the first and second cursors.

5. The method of claim 4, wherein the identifying a storage set for the data access requests based on the first and second cursors includes:

the first storage set when an DSN storage address of the data access request is greater than the first cursor, the second storage set when the DSN storage address of the data access request is less than the first cursor and greater than the second cursor, and the third storage set when the DSN address of the data access request is less than the second cursor.

6. The method of claim 1, wherein the facilitating further includes:

issuing slice access requests to the DSN storage address at a corresponding storage set, receiving slice access responses, and generating the data access response based on the slice access responses.

7. The method of claim 3, wherein the plurality of sets of encoded data slices are associated with a range of DSN storage addresses that includes sub-ranges for the respective first, second and third encoded data slices.

8. A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

identify, when a first migration of a plurality of sets of encoded data slices from an original source storage set (first storage set) to an intermediate destination storage set (second storage set) is active, a new destination storage set (third storage set) of a second migration for the plurality of sets of encoded data slices;

issue migration requests to one or more storage sets associated with current storage of the plurality of sets of encoded data slices in accordance with a first cursor of the first migration, the first cursor identifying a particular DSN storage address of a corresponding set of encoded data slices; and

when the second migration of the plurality of sets of encoded data slices is active, facilitate processing of a data access request to produce a data access response utilizing the first cursor of the first migration and a second cursor of the second migration, where the second cursor identifies a particular DSN address of a corresponding set of encoded data slices that is next up for migration.

9. The computing device of claim 8, wherein the facilitating includes applying a decentralized agreement protocol function to a range of slice names for a plurality of storage sets utilizing alternate weights for at least some of the plurality the storage sets to identify a new recipient storage set.

10. The computing device of claim 8, wherein the issuing includes any of:

sending first and second migration requests, for an address range including respective first and second encoded data slices, to the second storage set to send the respective first and second encoded data slices to the third storage set;

sending third and fourth migration requests, for an address range including respective third and fourth encoded data slices, to the first storage set to send the respective third and fourth encoded data slices to the third storage set, when the first cursor indicates that all of the respective first encoded data slices have already been moved to the second storage set, at least part of the of the encoded data slices have been moved to the second storage set, and none of the respective fourth encoded data slices have been moved from the first storage set.

11. The computing device of claim 8, wherein the facilitating includes:

identifying a storage set for the data access requests based on the first and second cursors.

12. The computing device of claim 11, wherein the identifying a storage set for the data access requests based on the first and second cursors includes:

a first storage set when an DSN storage address of the data access request is greater than the first cursor, a second storage set when the DSN storage address of the data access request is less than the first cursor and greater than the second cursor, and a third storage set when the DSN storage address of the data access request is less than the second cursor.

13. The computing device of claim 8, wherein the facilitating further includes:

issuing slice access requests to the DSN storage address at a corresponding storage set, receiving slice access responses, and generating the data access response based on the slice access responses.

14. The computing device of claim 8, wherein the plurality of sets of encoded data slices are associated with a range of DSN storage addresses that includes sub-ranges for the respective first, second and third encoded data slices.

15. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

identifying, while facilitating a first migration of a plurality of sets of encoded data slices from an original source storage set (first storage set) to an intermediate destination storage set (second storage set), for a second migration of the plurality of sets of encoded data slices, a new destination storage set (third storage set) for the plurality of sets of encoded data slices, wherein the plurality of sets of encoded data slices are associated with a range of DSN addresses, and wherein a first cursor of the first migration identifies a particular DSN address of a corresponding set of encoded data slices that is next up for migration from the original source storage set to the intermediate destinations storage set;

issuing migration requests to storage sets associated with current storage of sets of encoded data slices of the range of DSN addresses in accordance with the first cursor; and

while the second migration of the plurality of sets of encoded data slices from the original source storage set and the intermediate destinations storage set to the new destination storage set is active, facilitating processing of a data access request to produce a data access response utilizing the first cursor and a second cursor, wherein the second cursor identifies a particular DSN address of a corresponding set of encoded data slices that is next up for migration from at least one of the original source storage set and the intermediate destinations storage set to the new destination storage set.

16. The method of claim 15, wherein the facilitating includes applying a decentralized agreement protocol function to a range of slice names for a plurality of storage sets utilizing alternate weights for at least some of the plurality the storage sets to identify a new recipient storage set.

17. The method of claim 15, sending first and second migration requests, for an address range including respective first and second encoded data slices, to the second storage set to send the respective first and second encoded data slices to the third storage set;

sending third and fourth migration requests, for an address range including respective third and fourth encoded data slices, to the first storage set to send the respective third and fourth encoded data slices to the third storage set, when the first cursor indicates that all of the respective first encoded data slices have already been moved to the second storage set, at least part of the of the encoded data slices have been moved to the second storage set, and none of the respective fourth encoded data slices have been moved from the first storage set.

18. The method of claim 15, wherein the facilitating includes:

identifying a storage set for the data access requests based on the first and second cursors.

19. The method of claim 18, wherein the identifying a storage set for the data access requests based on the first and second cursors includes:

the first storage set when an DSN address of the data access request is greater than the first cursor, the second storage set when the DSN address of the data access request is less than the first cursor and greater than the second cursor, and the third storage set when the DSN address of the data access request is less than the second cursor.

20. The method of claim 15, wherein the facilitating further includes:

issuing slice access requests to the DSN address at a corresponding storage set, receiving slice access responses, and generating the data access response based on the slice access responses.