Migrating data in response to adding incremental storage resources in a dispersed storage network

• US 10,387,070 B2
• Filed: 12/11/2017
• Issued: 08/20/2019
• Est. Priority Date: 03/31/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Motwani, Manish, Resch, Jason K.
• Primary Examiner(s): Harrell, Robert B
• Application Number:
  US15/837,705
• Publication Number:
  US 20180101325A1
• Time in Patent Office: 617 Days
• Field of Search: 707827, 709215
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method for execution by a computing device includes detecting that an incremental storage cohort has been added to a storage generation to produce an updated plurality of storage cohorts of an updated storage generation, where each storage cohort includes a set of storage units. For each storage cohort, a slice listing process is initiated to identify a plurality of DSN addresses associated with storage of data objects within the each storage cohort. For each DSN address, ranked scoring information is obtained for the each storage cohort of the updated plurality of storage cohorts. One storage cohort is identified based on the ranked scoring information. When the identified storage cohort is different than another storage cohort associated with current storage of encoded data slices associated with the DSN address of the identified storage cohort, a migration process is initiated to migrate the encoded data slices to the identified storage cohort.

Claims

1. A method for execution by a computing device that includes a processor, the method comprises:

detecting that an incremental storage cohort has been added to a storage generation to produce an updated plurality of storage cohorts of an updated storage generation, wherein each storage cohort includes a set of storage units;

for each storage cohort of the updated plurality of storage cohorts, initiating a slice listing process to identify a plurality of DSN addresses associated with storage of a plurality of data objects within the each storage cohort;

for each DSN address of the plurality of DSN addresses, obtaining ranked scoring information for the each storage cohort of the updated plurality of storage cohorts;

identifying one storage cohort of the updated plurality of storage cohorts based on the ranked scoring information; and

when the identified storage cohort is different than another storage cohort associated with current storage of encoded data slices associated with a DSN address of the identified storage cohort, initiating a migration process to migrate the encoded data slices from the another storage cohort to the identified storage cohort.

2. The method of claim 1, wherein the computing device detects that the incremental storage cohort has been added to the storage generation in response to determining to add the incremental storage cohort.

3. The method of claim 2, wherein the computing device determines to add the incremental storage cohort in response to determining that an available storage level compares unfavorably to a threshold storage availability level.

4. The method of claim 1, wherein the encoded data slices are associated with at least one data segment, and wherein the at least one data segment was dispersed storage error encoded to produce the encoded data slices for storage in the set of storage units.

5. The method of claim 1, wherein initiating the slice listing process includes:

issuing list slice requests for the each storage cohort;

receiving list slice responses from the each storage cohort; and

extracting the plurality of DSN addresses from the received list slice responses.

6. The method of claim 1, wherein obtaining the ranked scoring information includes performing a distributed agreement protocol function on the each DSN address utilizing an identifier of the each storage cohort and at least one weight of the each storage cohort to produce the ranked scoring information for the each storage cohort.

7. The method of claim 1, wherein the one storage cohort is identified in response to determining the one storage cohort is associated with a highest score of the ranked scoring information.

8. The method of claim 1, wherein initiating the migration process includes:

retrieving the encoded data slices associated with the DSN address of the identified storage cohort from the another storage cohort; and

facilitating storage of the retrieved encoded data slices in the identified storage cohort.

9. A processing system of a computing device comprises:

at least one processor;

a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to;

detect that an incremental storage cohort has been added to a storage generation to produce an updated plurality of storage cohorts of an updated storage generation, wherein each storage cohort includes a set of storage units;

for each storage cohort of the updated plurality of storage cohorts, initiate a slice listing process to identify a plurality of DSN addresses associated with storage of a plurality of data objects within the each storage cohort;

for each DSN address of the plurality of DSN addresses, obtain ranked scoring information for the each storage cohort of the updated plurality of storage cohorts;

identify one storage cohort of the updated plurality of storage cohorts based on the ranked scoring information; and

when the identified storage cohort is different than another storage cohort associated with current storage of encoded data slices associated with a DSN address of the identified storage cohort, initiate a migration process to migrate the encoded data slices from the another storage cohort to the identified storage cohort.

10. The processing system of claim 9, wherein the computing device detects that the incremental storage cohort has been added to the storage generation in response to determining to add the incremental storage cohort.

11. The processing system of claim 10, wherein the computing device determines to add the incremental storage cohort in response to determining that an available storage level compares unfavorably to a threshold storage availability level.

12. The processing system of claim 9, wherein the encoded data slices are associated with at least one data segment, and wherein the at least one data segment was dispersed storage error encoded to produce the encoded data slices for storage in the set of storage units.

13. The processing system of claim 9, wherein initiating the slice listing process includes:

issuing list slice requests for the each storage cohort;

receiving list slice responses from the each storage cohort; and

extracting the plurality of DSN addresses from the received list slice responses.

14. The processing system of claim 9, wherein obtaining the ranked scoring information includes performing a distributed agreement protocol function on the each DSN address utilizing an identifier of the each storage cohort and at least one weight of the each storage cohort to produce the ranked scoring information for the each storage cohort.

15. The processing system of claim 9, wherein the one storage cohort is identified in response to determining the one storage cohort is associated with a highest score of the ranked scoring information.

16. The processing system of claim 9, wherein initiating the migration process includes:

retrieving the encoded data slices associated with the DSN address of the identified storage cohort from the another storage cohort; and

facilitating storage of the retrieved encoded data slices in the identified storage cohort.

17. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by a processing system of a dispersed storage network (DSN) that includes a processor and a memory, causes the processing system to;

detect that an incremental storage cohort has been added to a storage generation to produce an updated plurality of storage cohorts of an updated storage generation, wherein each storage cohort includes a set of storage units;

for each storage cohort of the updated plurality of storage cohorts, initiate a slice listing process to identify a plurality of DSN addresses associated with storage of a plurality of data objects within the each storage cohort;

for each DSN address of the plurality of DSN addresses, obtain ranked scoring information for the each storage cohort of the updated plurality of storage cohorts;

identify one storage cohort of the updated plurality of storage cohorts based on the ranked scoring information; and

when the identified storage cohort is different than another storage cohort associated with current storage of encoded data slices associated with a DSN address of the identified storage cohort, initiate a migration process to migrate the encoded data slices from the another storage cohort to the identified storage cohort.

18. The non-transitory computer readable storage medium of claim 17, wherein the encoded data slices are associated with at least one data segment, and wherein the at least one data segment was dispersed storage error encoded to produce the encoded data slices for storage in the set of storage units.

19. The non-transitory computer readable storage medium of claim 17, wherein obtaining the ranked scoring information includes performing a distributed agreement protocol function on the each DSN address utilizing an identifier of the each storage cohort and at least one weight of the each storage cohort to produce the ranked scoring information for the each storage cohort.

20. The non-transitory computer readable storage medium of claim 17, wherein the one storage cohort is identified in response to determining the one storage cohort is associated with a highest score of the ranked scoring information.