Considering object health of a multi-region object

• US 10,013,207 B2
• Filed: 09/22/2016
• Issued: 07/03/2018
• Est. Priority Date: 09/24/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Leggette, Wesley B.
• Primary Examiner(s): BRITT, CYNTHIA H
• Application Number:
  US15/272,968
• Publication Number:
  US 20170091033A1
• Time in Patent Office: 649 Days
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0661 Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 8/65 Updates security arrangements therefor G06F21/57
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/154 Error and erasure correction, e.g. by using the error and erasure locator or Forney polynomial
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 63/061 for key exchange, e.g. in peer-to-peer networks cryptographic mechanisms or cryptographic arrangements for key agreement H04L9/0838
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416
• CPC Class Codes: H04L 67/60 Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources admission control or resource allocation H04L47/70

Abstract

A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN) begins by identifying an unrecoverable encoded data slice of a data segment stored in a set of DSN storage units, where a region of a data object includes a plurality of data segments, and where the plurality of data segments includes the data segment. The method continues by determining whether the data segment is recoverable. The method continues, when recoverable, by salvaging the region by indicating that the region has corruption, updating a directory and replacing the data segment with filler data, and when not recoverable, by not salvaging the region by indicating that the region has been eliminated.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

identifying an unrecoverable encoded data slice of a data segment stored in a set of DSN storage units, where a region of a data object includes a plurality of data segments, and where the plurality of data segments includes the data segment;

determining whether the data segment is recoverable; and

when recoverable, salvaging the region by indicating that the region has corruption, updating a directory and replacing the data segment with filler data; and

when not recoverable, not salvaging the region by indicating that the region has been eliminated.

2. The method of claim 1, wherein the identifying includes at least one of:

interpreting received integrity information, receiving a list, or interpreting an error message.

3. The method of claim 1, wherein the determining includes indicating that the data segment is unrecoverable when less than a decode threshold number of recoverable encoded data slices will ever be available.

4. The method of claim 1, wherein the determining is based on one or more of:

an association of data segments to the region, an association of the region to another region, a predetermination, a request, a region priority indicator, a region priority threshold level, a comparison of the region priority indicator to the region priority threshold level, a region type indicator, or interpreting system registry information.

5. The method of claim 1, wherein for each other unrecoverable encoded data slice that is associated with at least one other data segment of the region, sending a rebuilt encoded data slice to a corresponding storage unit of the set of DSN storage units.

6. The method of claim 5, wherein the sending includes:

obtaining a decode threshold number of recoverable encoded data slices of a set of encoded data slices to utilize;

generating the rebuilt encoded data slice; and

issuing a write slice request to the corresponding storage unit of the set of DSN storage units, where the write slice request includes the rebuilt encoded data slice.

7. The method of claim 5, further comprising determining that a set of all other segments that are related to an unrecoverable segment are unavailable and initiating one or more of:

deprioritizing rebuilding of these other segments;

ceasing any rebuilding activity related to these other segments;

initiating cleanup operations of these other segments;

continue rebuilding other segments of the data object but flag this data object as partially corrupted;

or creating new readable segments to replace data segment with filler data.

8. A processing unit within a dispersed storage network (DSN) comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

identify an unrecoverable encoded data slice of a data segment stored in a set of DSN storage units, where a region of a data object includes a plurality of data segments, and where the plurality of data segments includes the data segment;

determine whether the data segment is recoverable; and

when recoverable, salvage the region by indicating that the region has corruption, updating a directory and replacing the data segment with filler data; and

when not recoverable, not salvage the region by indicating that the region has been eliminated.

9. The processing unit of claim 8, wherein the identifying of an unrecoverable encoded data slice of a data segment stored in a set of DSN storage units includes at least one of:

interpreting received integrity information, receiving a list, or interpreting an error message.

10. The processing unit of claim 8, wherein the determining whether the data segment is recoverable includes indicating that the data segment is unrecoverable when less than a decode threshold number of recoverable encoded data slices will ever be available.

11. The processing unit of claim 8, wherein the determining whether the data segment is recoverable is based on one or more of:

an association of data segments to the region, an association of the region to another region, a predetermination, a request, a region priority indicator, a region priority threshold level, a comparison of the region priority indicator to the region priority threshold level, a region type indicator, or interpreting system registry information.

12. The processing unit of claim 8, wherein for each other unrecoverable encoded data slice that is associated with at least one other data segment of the region, sending a rebuilt encoded data slice to a corresponding storage unit of the set of DSN storage units.

13. The processing unit of claim 12, wherein the sending includes:

obtaining a decode threshold number of recoverable encoded data slices of a set of encoded data slices to utilize;

generating the rebuilt encoded data slice; and

issuing a write slice request to the corresponding storage unit of the set of DSN storage units, where the write slice request includes the rebuilt encoded data slice.

14. The processing unit of claim 12, further comprising determining that a set of all other segments that are related to an unrecoverable segment are unavailable and initiating one or more of:

deprioritizing rebuilding of these other segments;

ceasing any rebuilding activity related to these other segments;

initiating cleanup operations of these other segments;

continue rebuilding other segments of the data object but flag this data object as partially corrupted;

or creating new readable segments to replace data segment with filler data.

15. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

identify an unrecoverable encoded data slice of a data segment stored in a set of DSN storage units, where a region of a data object includes a plurality of data segments, and where the plurality of data segments includes the data segment;

determine whether the data segment is recoverable; and

when recoverable, salvage the region by indicating that the region has corruption, updating a directory and replacing the data segment with filler data; and

when not recoverable, not salvage the region by indicating that the region has been eliminated.

16. The non-transitory computer readable storage medium of claim 15, wherein the identifying an unrecoverable encoded data slice of a data segment stored in a set of DSN storage units includes at least one of:

interpreting received integrity information, receiving a list, or interpreting an error message.

17. The non-transitory computer readable storage medium of claim 15, wherein the determining whether the data segment is recoverable includes indicating that the data segment is unrecoverable when less than a decode threshold number of recoverable encoded data slices will ever be available.

18. The non-transitory computer readable storage medium of claim 15, wherein the determining whether the data segment is recoverable is based on one or more of:

an association of data segments to the region, an association of the region to another region, a predetermination, a request, a region priority indicator, a region priority threshold level, a comparison of the region priority indicator to the region priority threshold level, a region type indicator, or interpreting system registry information.

19. The non-transitory computer readable storage medium of claim 15, wherein for each other unrecoverable encoded data slice that is associated with at least one other data segment of the region, sending a rebuilt encoded data slice to a corresponding storage unit of the set of DSN storage units.

20. The non-transitory computer readable storage medium of claim 19, wherein the sending includes:

obtaining a decode threshold number of recoverable encoded data slices of a set of encoded data slices to utilize;

generating the rebuilt encoded data slice; and

issuing a write slice request to the corresponding storage unit of the set of DSN storage units, where the write slice request includes the rebuilt encoded data slice.