Migrating data slices in a dispersed storage network

• US 10,409,679 B2
• Filed: 11/28/2017
• Issued: 09/10/2019
• Est. Priority Date: 08/31/2012
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Motwani, Manish, Dhuse, Greg R., Leggette, Wesley B., Baptist, Andrew D., Volvovski, Ilya, Resch, Jason K.
• Primary Examiner(s): Vallecillo, Kyle
• Application Number:
  US15/824,291
• Publication Number:
  US 20180081759A1
• Time in Patent Office: 651 Days
• CPC Class Codes: G06F 11/0712 in a virtual computing platform, e.g. logically partitioned systems
• CPC Class Codes: G06F 11/0784 Routing of error reports, e.g. with a specific transmission path or data flow
• CPC Class Codes: G06F 11/0787 Storage of error reports, e.g. persistent data storage, storage using memory protection
• CPC Class Codes: G06F 11/1004 to protect a block of data words, e.g. CRC or checksum G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00
• CPC Class Codes: G06F 11/1044 with specific ECC/EDC distribution
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/142 Reconfiguring to eliminate the error group management mechanisms in a peer-to-peer network H04L67/1044
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 21/10 Protecting distributed programs or content, e.g. vending or licensing of copyrighted material protection in video systems or pay television H04N7/16 ; Digital rights management [DRM]
• CPC Class Codes: G06F 21/60 Protecting data
• CPC Class Codes: G06F 21/6272 by registering files or documents with a third party
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/611 Specific encoding aspects, e.g. encoding by means of decoding
• CPC Class Codes: H03M 13/616 Matrix operations, especially for generator matrices or check matrices, e.g. column or row permutations
• CPC Class Codes: H04L 2209/34 Encoding or coding, e.g. Huffman coding or error correction
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 9/0861 Generation of secret information including derivation or calculation of cryptographic keys or passwords
• CPC Class Codes: H04L 9/0891 : Revocation or update of secret information, e.g. encryption key update or rekeying
• CPC Class Codes: H04L 9/0894 : Escrow, recovery or storing of secret information, e.g. secret key escrow or cryptographic key storage
• CPC Class Codes: H04L 9/3247 : involving digital signatures

Abstract

A method for execution by a computing device includes obtaining utilization information of a pair of adjacent dispersed storage and task (DST) execution units of a common pillar. The utilization information of the pair of adjacent DST execution units is verified based on companion utilization information. Slices to migrate are selected in response to verifying the utilization information, and migration of the slices to migrate is facilitated. An updated storage map is generated based on the slices to migrate. A storage map modification package that includes a previous storage map and the updated storage map is generated. Attaching a signature from each DST execution unit of the pair of adjacent DST execution units to the storage map modification package is facilitated to produce a fully signed storage map modification package. The fully signed storage map modification package is published.

Claims

1. A method for execution by a computing device that includes a processor, the method comprises:

obtaining utilization information of a pair of adjacent dispersed storage and task (DST) execution units of a common pillar;

verifying the utilization information of the pair of adjacent DST execution units based on companion utilization information;

selecting slices to migrate in response to verifying the utilization information;

facilitating migration of the slices to migrate;

generating an updated storage map based on the slices to migrate;

generating a storage map modification package that includes a previous storage map and the updated storage map;

facilitating attaching a signature from each DST execution unit of the pair of adjacent DST execution units to the storage map modification package to produce a fully signed storage map modification package; and

publishing the fully signed storage map modification package.

2. The method of claim 1, wherein the slices to migrate are encoded data slices associated with at least one data segment, and wherein the at least one data segment was dispersed storage error encoded to produce the encoded data slices for storage in a set of DST execution units that includes the pair of adjacent DST execution units.

3. The method of claim 1, wherein the pair of adjacent DST execution units include a pair of DST execution units assigned to adjacent address ranges.

4. The method of claim 1, wherein the utilization information includes a storage capacity level and an assigned address range.

5. The method of claim 1, wherein verifying the utilization information includes:

identifying a set of DST execution units that includes the pair of adjacent DST execution units;

identifying companion DST execution units associated with the companion utilization information as remaining DST execution units of the set of DST execution units;

comparing storage utilization levels of each DST execution unit of the companion DST execution units to each other DST execution unit of the set of DST execution units; and

indicating verified utilization information when a calculated difference between the storage utilization levels compares favorably to a difference threshold.

6. The method of claim 1, wherein selecting the slices to migrate includes:

selecting a destination DST execution unit based on an available storage level of the destination DST execution unit; and

determining a number of slices to migrate to the destination DST execution unit based on the available storage level.

7. The method of claim 6, wherein selecting the slices to migrate includes:

selecting the slices to migrate from a high-end of a slice range when the destination DST execution unit is assigned higher addresses; and

selecting the slices to migrate from a low-end of the slice range when the destination DST execution unit is assigned lower addresses.

8. The method of claim 6, wherein the number of slices to migrate is determined to provide an updated level of storage of the destination DST execution unit that is less than a high storage level threshold.

9. The method of claim 1, wherein generating the storage map modification package includes appending the previous storage map to the updated storage map to produce the storage map modification package.

10. The method of claim 1, further comprising:

receiving an access request that indicates at least one slice of the slices to migrate;

identifying storage location information corresponding to the at least one slice based on the fully signed storage map modification package; and

facilitating access of the slice from a DST execution unit corresponding to the storage location information.

11. A processing system of a computing device comprises:

at least one processor;

a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to;

obtain utilization information of a pair of adjacent dispersed storage and task (DST) execution units of a common pillar;

verify the utilization information of the pair of adjacent DST execution units based on companion utilization information;

select slices to migrate in response to verifying the utilization information;

facilitate migration of the slices to migrate;

generate an updated storage map based on the slices to migrate;

generate a storage map modification package that includes a previous storage map and the updated storage map;

facilitate attaching a signature from each DST execution unit of the pair of adjacent DST execution units to the storage map modification package to produce a fully signed storage map modification package; and

publish the fully signed storage map modification package.

12. The processing system of claim 11, wherein the slices to migrate are encoded data slices associated with at least one data segment, and wherein the at least one data segment was dispersed storage error encoded to produce the encoded data slices for storage in a set of DST execution units that includes the pair of adjacent DST execution units.

13. The processing system of claim 11, wherein the pair of adjacent DST execution units include a pair of DST execution units assigned to adjacent address ranges.

14. The processing system of claim 11, wherein the utilization information includes a storage capacity level and an assigned address range.

15. The processing system of claim 11, wherein verifying the utilization information includes:

identifying a set of DST execution units that includes the pair of adjacent DST execution units;

identifying companion DST execution units associated with the companion utilization information as remaining DST execution units of the set of DST execution units;

comparing storage utilization levels of each DST execution unit of the companion DST execution units to each other DST execution unit of the set of DST execution units; and

indicating verified utilization information when a calculated difference between the storage utilization levels compares favorably to a difference threshold.

16. The processing system of claim 11, wherein selecting the slices to migrate includes:

selecting a destination DST execution unit based on an available storage level of the destination DST execution unit; and

determining a number of slices to migrate to the destination DST execution unit based on the available storage level.

17. The processing system of claim 16, wherein selecting the slices to migrate includes:

selecting the slices to migrate from a high-end of a slice range when the destination DST execution unit is assigned higher addresses; and

selecting the slices to migrate from a low-end of the slice range when the destination DST execution unit is assigned lower addresses.

18. The processing system of claim 16, wherein the number of slices to migrate is determined to provide an updated level of storage of the destination DST execution unit that is less than a high storage level threshold.

19. The processing system of claim 11, wherein the operational instructions, when executed by the at least one processor, further cause the processing system to:

receiving an access request that indicates at least one slice of the slices to migrate;

identifying storage location information corresponding to the at least one slice based on the fully signed storage map modification package; and

facilitating access of the slice from a DST execution unit corresponding to the storage location information.

20. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by a processing system of a dispersed storage network (DSN) that includes a processor and a memory, causes the processing system to;

obtain utilization information of a pair of adjacent dispersed storage and task (DST) execution units of a common pillar;

verify the utilization information of the pair of adjacent DST execution units based on companion utilization information;

select slices to migrate in response to verifying the utilization information;

facilitate migration of the slices to migrate;

generate an updated storage map based on the slices to migrate;

generate a storage map modification package that includes a previous storage map and the updated storage map;

facilitate attaching a signature from each DST execution unit of the pair of adjacent DST execution units to the storage map modification package to produce a fully signed storage map modification package; and

publish the fully signed storage map modification package.