Modifying allocation of storage resources in a dispersed storage network

• US 10,360,107 B2
• Filed: 09/04/2018
• Issued: 07/23/2019
• Est. Priority Date: 02/26/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Shirley, Jr., Thomas F., Grube, Gary W., Cilfone, Bart R., Khadiwala, Ravi V., Dhuse, Greg R., Cocagne, Thomas D., Storm, Michael Colin, Vedpathak, Yogesh R., Leggette, Wesley B., Resch, Jason K., Baptist, Andrew D., Volvovski, Ilya
• Primary Examiner(s): Chase, Shelly A
• Application Number:
  US16/121,068
• Publication Number:
  US 20180373595A1
• Time in Patent Office: 322 Days
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 21/64 Protecting data integrity, e.g. using checksums, certificates or signatures
• CPC Class Codes: G06F 21/645 using a third party
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 2212/254 Distributed memory
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 63/101 Access control lists [ACL]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/565 Conversion or adaptation of application format or content adding application control or application functional data H04L67/561

Abstract

A method for execution by a resource allocation module includes facilitating migration of a first set of encoded data slices stored at a storage unit for decommissioning to a newly commissioned storage unit, and facilitating migration of a remaining set of encoded data slices stored at the storage unit for decommissioning as foster encoded data slices to at least one other storage unit. For each foster encoded data slice, it is determined whether to facilitate migration of the foster encoded data slice to the newly commissioned storage unit. When determining to facilitate the migration of the foster encoded data slice, the migration of the foster encoded data slice to the newly commissioned storage unit is facilitated. An association of the newly commissioned storage unit and identity of the foster encoded data slice is updated in response to detecting successful migration of the foster encoded data slice.

Claims

1. A method for execution by a resource allocation module that includes a processor, the method comprises:

facilitating migration of a first set of a plurality of encoded data slices stored at a corresponding storage unit for decommissioning of a set of storage units to a newly commissioned storage unit;

facilitate migration of a remaining set of the plurality of encoded data slices stored at the corresponding storage unit for decommissioning to at least one other storage unit of the set of storage units for storage as a set of foster encoded data slices;

facilitating deletion of the plurality of encoded data slices from the corresponding storage unit for decommissioning in response to detecting successful migration of the first set of the plurality of encoded data slices;

for each foster encoded data slice;

determining whether to facilitate migration of the each foster encoded data slice to the newly commissioned storage unit;

when determining to facilitate the migration of the each foster encoded data slice, facilitating the migration of the each foster encoded data slice from a corresponding other storage unit to the newly commissioned storage unit as an encoded data slice associated with the newly commissioned storage unit;

updating an association of the newly commissioned storage unit and identity of the each foster encoded data slice in response to detecting successful migration of the each foster encoded data slice; and

facilitating deletion of the each foster encoded data slice from the corresponding other storage unit in response to detecting successful migration of the each foster encoded data slice to the newly commissioned storage unit.

2. The method of claim 1, wherein the resource allocation module is implemented within the newly commissioned storage unit.

3. The method of claim 1, wherein facilitating the migration of the first set of the plurality of encoded data slices includes selecting a subset of the plurality of encoded data slices to be included in the first set of the plurality of encoded data slices.

4. The method of claim 3, wherein the subset of the plurality of encoded data slices are selected based on an expected access frequency for the encoded data slices.

5. The method of claim 3, wherein selecting the subset of the plurality of encoded slices to be included in the first set includes determining a number of encoded slices to be included in the first set.

6. The method of claim 3, wherein half of the encoded data slices of the plurality of encoded data slices are selected to be included in the first set in response to determining a performance level of the newly commissioned storage unit compares favorably to a low performance threshold level and compares unfavorably to a high performance threshold level.

7. The method of claim 1, wherein facilitating the migration of the remaining set of the plurality of encoded data slices includes selecting the at least one other storage unit based on at a storage capacity of the at least one other storage unit.

8. The method of claim 1, wherein determining whether to facilitate the migration of the each foster encoded data slice is based on a performance level of the newly commissioned storage unit.

9. The method of claim 1, wherein determining whether to facilitate the migration of the each foster encoded data slice includes determining whether a system loading level compares favorably to a system loading threshold level.

10. The method of claim 1, wherein updating the association of the newly commissioned storage unit and the identity of the each foster encoded data slice includes updating a dispersed hierarchical index to indicate the association of a slice name of the each foster encoded data slice and the newly commissioned storage unit.

11. A processing system of a resource allocation module comprises:

at least one processor;

a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to;

facilitate migration of a first set of a plurality of encoded data slices stored at a corresponding storage unit for decommissioning of a set of storage units to a newly commissioned storage unit;

facilitate migration of a remaining set of the plurality of encoded data slices stored at the corresponding storage unit for decommissioning to at least one other storage unit of the set of storage units for storage as a set of foster encoded data slices;

facilitate deletion of the plurality of encoded data slices from the corresponding storage unit for decommissioning in response to detecting successful migration of the first set of the plurality of encoded data slices;

for each foster encoded data slice in the set of foster encoded data slices;

determine whether to facilitate migration of the each foster encoded data slice to the newly commissioned storage unit;

when determining to facilitate the migration of the each foster encoded data slice, facilitate the migration of the each foster encoded data slice from a corresponding other storage unit to the newly commissioned storage unit as an encoded data slice associated with the newly commissioned storage unit;

update an association of the newly commissioned storage unit and identity of the each foster encoded data slice in response to detecting successful migration of the each foster encoded data slice; and

facilitate deletion of the each foster encoded data slice from the corresponding other storage unit in response to detecting successful migration of the each foster encoded data slice to the newly commissioned storage unit.

12. The processing system of claim 11, wherein the resource allocation module is implemented within the newly commissioned storage unit.

13. The processing system of claim 11, wherein facilitating the migration of the first set of the plurality of encoded data slices includes selecting a subset of the plurality of encoded data slices to be included in the first set of the plurality of encoded data slices.

14. The processing system of claim 13, wherein the subset of the plurality of encoded data slices are selected based on an expected access frequency for the encoded data slices.

15. The processing system of claim 13, wherein selecting the subset of the plurality of encoded slices to be included in the first set includes determining a number of encoded slices to be included in the first set.

16. The processing system of claim 13, wherein half of the encoded data slices of the plurality of encoded data slices are selected to be included in the first set in response to determining a performance level of the newly commissioned storage unit compares favorably to a low performance threshold level and compares unfavorably to a high performance threshold level.

17. The processing system of claim 11, wherein facilitating the migration of the remaining set of the plurality of encoded data slices includes selecting the at least one other storage unit based on at a storage capacity of the at least one other storage unit.

18. The processing system of claim 11, wherein determining whether to facilitate the migration of the each foster encoded data slice includes determining whether a system loading level compares favorably to a system loading threshold level.

19. The processing system of claim 11, wherein updating the association of the newly commissioned storage unit and the identity of the each foster encoded data slice includes updating a dispersed hierarchical index to indicate the association of a slice name of the each foster encoded data slice and the newly commissioned storage unit.

20. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by a processing system of a dispersed storage network (DSN) that includes a processor and a memory, causes the processing system to;

facilitate migration of a first set of a plurality of encoded data slices stored at a corresponding storage unit for decommissioning of a set of storage units to a newly commissioned storage unit;

facilitate migration of a remaining set of the plurality of encoded data slices stored at the corresponding storage unit for decommissioning to at least one other storage unit of the set of storage units for storage as a set of foster encoded data slices;

facilitate deletion of the plurality of encoded data slices from the corresponding storage unit for decommissioning in response to detecting successful migration of the first set of the plurality of encoded data slices;

for each foster encoded data slice;

determine whether to facilitate migration of the each foster encoded data slice to the newly commissioned storage unit;

when determining to facilitate the migration of the each foster encoded data slice, facilitate the migration of the each foster encoded data slice from a corresponding other storage unit to the newly commissioned storage unit as an encoded data slice associated with the newly commissioned storage unit;

update an association of the newly commissioned storage unit and identity of the each foster encoded data slice in response to detecting successful migration of the each foster encoded data slice; and

facilitate deletion of the each foster encoded data slice from the corresponding other storage unit in response to detecting successful migration of the each foster encoded data slice to the newly commissioned storage unit.