Slice rebuilding in a dispersed storage network

• US 10,241,863 B2
• Filed: 09/02/2016
• Issued: 03/26/2019
• Est. Priority Date: 08/31/2012
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Resch, Jason K.
• Primary Examiner(s): Gee, Jason K
• Assistant Examiner(s): Holmes, Angela R
• Application Number:
  US15/255,748
• Publication Number:
  US 20160371146A1
• Time in Patent Office: 935 Days
• Field of Search: 726 30
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/142 Reconfiguring to eliminate the error group management mechanisms in a peer-to-peer network H04L67/1044
• CPC Class Codes: G06F 21/6272 by registering files or documents with a third party
• CPC Class Codes: G06F 21/645 using a third party
• CPC Class Codes: G06F 21/805 using a security table for the storage sub-system
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/2909 Product codes
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 2209/34 Encoding or coding, e.g. Huffman coding or error correction
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 67/60 Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources admission control or resource allocation H04L47/70
• CPC Class Codes: H04L 9/085 Secret sharing or secret splitting, e.g. threshold schemes
• CPC Class Codes: H04L 9/0861 Generation of secret information including derivation or calculation of cryptographic keys or passwords
• CPC Class Codes: H04L 9/0894 : Escrow, recovery or storing of secret information, e.g. secret key escrow or cryptographic key storage

Abstract

A processing system associated with a dispersed storage network is configured to detect a plurality of encoded data slices to be rebuilt, wherein the plurality of encoded data slices are associated with the DST execution unit; identify a plurality of slice names associated with the plurality of encoded data slices; obtain a slice name to slice memory device mapping for the DST execution unit; determine a schedule of rebuilding the plurality of encoded data slices, based on the mapping; and facilitate rebuilding the plurality of encoded data slices in accordance with the schedule of rebuilding.

Claims

1. A method for execution by a processing system of distributed storage and task (DST) execution unit that includes a processor, the method comprises:

detecting, via the processing system, a plurality of slice errors, wherein detecting the plurality of slices errors includes detecting that a retrieved integrity value does not compare favorably to a calculated integrity value for at least one of a plurality of encoded data slices;

determining, via the processing system, at least one of the plurality of encoded data slices to be rebuilt based on the plurality of slice errors, wherein the plurality of encoded data slices are associated with the DST execution unit;

identifying, via the processing system, a plurality of slice names associated with the plurality of encoded data slices;

obtaining, via the processing system, a slice name to slice memory device mapping for the DST execution unit;

determining, via the processing system, a schedule of rebuilding the plurality of encoded data slices, based on the mapping; and

facilitating, via the processing system, rebuilding each of the plurality of encoded data slices in accordance with the schedule of rebuilding.

2. The method of claim 1 wherein the identifying includes at least one of:

comparing the plurality of slice names from two or more list responses, extracting from an error message, extracting from a read slice response, or receiving comparing the plurality of slice names.

3. The method of claim 1 wherein the obtaining includes at least one of receiving:

an address range assignment for the DST execution unit, estimating address range assignments for each slice memory of a plurality of slice memories associated with the DST execution unit, retrieving the slice name to slice memory device mapping, initiating a query for the slice name to slice memory device mapping, a look up of the slice name to slice memory device mapping, or receiving the slice name to slice memory device mapping.

4. The method of claim 1 wherein the determining includes selecting a rebuilding order or rebuilding timeframe of rebuilding steps to achieve a load leveling performance goal with regards to accessing each slice memory of a plurality of slice memories.

5. The method of claim 1 wherein the facilitating includes at least one of:

rebuilding in accordance with the slice name to slice memory device mapping or issuing a rebuild request to a rebuilding module such that the rebuilding request includes a portion of the schedule of rebuilding.

6. The method of claim 5 wherein the processing system outputs rebuilding information to another DST execution unit that includes the portion of the schedule of rebuilding.

7. A processing system associated with a distributed storage and task (DST) execution unit comprises:

at least one processor;

a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to;

detect, via the processing system, a plurality of slice errors, wherein detecting the plurality of slices errors includes detecting that a retrieved integrity value does not compare favorably to a calculated integrity value for at least one of a plurality of encoded data slices;

determine at least one of the plurality of encoded data slices to be rebuilt based on the plurality of slice errors, wherein the plurality of encoded data slices are associated with the DST execution unit;

identify a plurality of slice names associated with the plurality of encoded data slices;

obtain a slice name to slice memory device mapping for the DST execution unit;

determine a schedule of rebuilding the plurality of encoded data slices, based on the mapping; and

facilitate rebuilding each of the plurality of encoded data slices in accordance with the schedule of rebuilding.

8. The processing system of claim 7 wherein the identifying includes at least one of:

comparing the plurality of slice names from two or more list responses, extracting from an error message, extracting from a read slice response, or receiving comparing the plurality of slice names.

9. The processing system of claim 7 wherein the obtaining includes at least one of receiving:

an address range assignment for the DST execution unit, estimating address range assignments for each slice memory of a plurality of slice memories associated with the DST execution unit, retrieving the slice name to slice memory device mapping, initiating a query for the slice name to slice memory device mapping, a look up of the slice name to slice memory device mapping, or receiving the slice name to slice memory device mapping.

10. The processing system of claim 7 wherein the determining includes selecting a rebuilding order or rebuilding timeframe of rebuilding steps to achieve a load leveling performance goal with regards to accessing each slice memory of a plurality of slice memories.

11. The processing system of claim 7 wherein the facilitating includes at least one of:

rebuilding in accordance with the slice name to slice memory device mapping or issuing a rebuild request to a rebuilding module such that the rebuilding request includes a portion of the schedule of rebuilding.

12. The processing system of claim 11 wherein the processing system outputs rebuilding information to another DST execution unit that includes the portion of the schedule of rebuilding.

13. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by a processing system associated with a distributed storage and task (DST) execution unit that includes a processor, causes the processing system to;

detect, via the processing system, a plurality of slice errors, wherein detecting the plurality of slices errors includes detecting that a retrieved integrity value does not compare favorably to a calculated integrity value for at least one of a plurality of encoded data slices;

determine at least one of the plurality of encoded data slices to be rebuilt based on the plurality of slice errors, wherein the plurality of encoded data slices are associated with the DST execution unit;

identify a plurality of slice names associated with the plurality of encoded data slices;

obtain a slice name to slice memory device mapping for the DST execution unit;

determine a schedule of rebuilding the plurality of encoded data slices, based on the mapping; and

facilitate rebuilding each of the plurality of encoded data slices in accordance with the schedule of rebuilding.

14. The non-transitory computer readable storage medium of claim 13 wherein the identifying includes at least one of:

comparing the plurality of slice names from two or more list responses, extracting from an error message, extracting from a read slice response, or receiving comparing the plurality of slice names.

15. The non-transitory computer readable storage medium of claim 13 wherein the obtaining includes at least one of receiving:

an address range assignment for the DST execution unit, estimating address range assignments for each slice memory of a plurality of slice memories associated with the DST execution unit, retrieving the slice name to slice memory device mapping, initiating a query for the slice name to slice memory device mapping, a look up of the slice name to slice memory device mapping, or receiving the slice name to slice memory device mapping.

16. The non-transitory computer readable storage medium of claim 13 wherein the determining includes selecting a rebuilding order or rebuilding timeframe of rebuilding steps to achieve a load leveling performance goal with regards to accessing each slice memory of a plurality of slice memories.

17. The non-transitory computer readable storage medium of claim 13 wherein the facilitating includes at least one of:

rebuilding in accordance with the slice name to slice memory device mapping or issuing a rebuild request to a rebuilding module such that the rebuilding request includes a portion of the schedule of rebuilding.