Detecting and responding to data loss events in a dispersed storage network

• US 10,114,698 B2
• Filed: 01/05/2017
• Issued: 10/30/2018
• Est. Priority Date: 01/05/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Dhuse, Greg R., Leggette, Wesley B., Resch, Jason K.
• Primary Examiner(s): Chaudry, Mujtaba M
• Application Number:
  US15/399,270
• Publication Number:
  US 20180189138A1
• Time in Patent Office: 663 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems

Abstract

A method for use in a dispersed storage network operates to identify missing, out-of-date or otherwise compromised encoded data slices in a dispersed storage network (DSN), and when a decode threshold of encoded data slices is not available to rebuild an associated data object, determine whether a data loss event has occurred. When a data loss event is determined to have occurred the method continues by initiating a process to recover all or some of the lost data and may include notification to DSN entities that a data loss event has occurred.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

scanning a plurality of distributed storage units to identify one or more compromised encoded data slices (EDSs) of a set of EDSs, wherein the set of EDSs represents a first data segment;

when one or more compromised EDSs of the set of EDSs is found, determining whether a decode threshold number of EDSs of the set of EDSs is available to recover the first data segment;

when a decode threshold number of EDSs of the set of EDSs is determined not to be available to recover the first data segment, determining whether the first data segment is involved in an indeterminate state of processing a storage function;

when the first data segment is involved in an indeterminate state of processing a storage function, waiting until the processing a storage function is complete; and

when the first data segment is not involved in an indeterminate state of processing a storage function, initiating a process to recover at least a portion of the first data segment.

2. The method of claim 1, wherein at least one of the one or more processing modules of one or more computing devices of the DSN is a rebuild module, and the scanning a plurality of distributed storage units to identify one or more compromised encoded data slices is executed by the rebuild module.

3. The method of claim 1 further comprises:

determining whether there is a data loss event notification request associated with the first data segment, wherein the data loss event notification request is adapted to notify a DSN user that a data loss event has occurred.

4. The method of claim 3, wherein the data loss event notification request is at least a portion of a metadata associated with the first data segment.

5. The method of claim 1, wherein the one or more compromised EDS are at least one of missing or invalid EDSs.

6. The method of claim 1, wherein the process to recover at least a portion of the first data segment includes a least one of:

determining what data object the one or more compromised EDSs are part of, determining what container the one or more compromised EDSs are part of, or determining what vault the one or more compromised EDSs are part of.

7. The method of claim 1, wherein first data segment is a portion of a data object and wherein the process to recover at least a portion of the first data segment includes one or more processing modules of one or more computing devices of the DSN executing at least one of initiating a cross-mirror rebuild operation, initiating a cross-service provider rebuild operation, initiating a clean-up operation on a second data segment associated with the data object, initiating repair of the data object using a fixed pattern of bits, and repairing the one or more compromised EDSs to produce partially corrupt data.

8. The method of claim 1, wherein the first data segment is a portion of a data object and wherein the process to recover at least a portion of the first data segment includes one or more processing modules of one or more computing devices of the DSN rebuilding a DSN index.

9. The method of claim 1, wherein the scanning a plurality of distributed storage units to identify one or more compromised EDSs of a set of EDSs includes comparing expected EDS slice names to the EDS slice names of the EDSs of a set of EDSs being scanned.

10. The method of claim 1, wherein the indeterminate state of processing a storage function includes at least one of an incomplete write request, an incomplete delete operation, an edit operation or a data migration operation.

11. A computer readable memory comprises:

a first memory element that stores operational instructions that, when executed by a computing device of a dispersed storage network (DSN), causes the computing device to;

scan a plurality of distributed storage units to identify one or more compromised encoded data slices (EDSs) of a set of EDSs, wherein the set of EDSs represents a first data segment;

when one or more compromised EDSs of the set of EDSs is found, determine whether a decode threshold number of EDSs of the set of EDSs is available to recover the first data segment;

when a decode threshold number of EDSs of the set of EDSs is determined not to be available to recover the first data segment, determine whether the first data segment is involved in an indeterminate state of processing a storage function;

when the first data segment is involved in an indeterminate state of processing a storage function, wait until the processing a storage function is complete; and

when the first data segment is not involved in an indeterminate state of processing a storage function, a second memory element that stores operational instructions that, when executed by the computing device, causes the computing device to;

initiate a process to recover at least a portion of the first data segment.

12. The computer readable memory of claim 11, wherein the computing device of the DSN includes a rebuild module, and wherein the first memory element, when executed by the rebuild module, causes the computing device to the scan a plurality of distributed storage units to identify one or more compromised encoded data slices.

13. The computer readable memory of claim 11 further comprising:

determine whether there is a data loss event notification request associated with the first data segment, wherein the data loss event notification request is adapted to notify a DSN user that a data loss event has occurred.

14. The computer readable memory of claim 11, wherein a data loss event notification request is at least a portion of a metadata associated with the first data segment.

15. The computer readable memory of claim 11, wherein the one or more compromised EDS are at least one of missing or invalid EDSs.

16. The computer readable memory of claim 11, wherein the process to recover at least a portion of the first data segment includes a least one of:

determining what data object the one or more compromised EDSs are part of, determining what container the one or more compromised EDSs are part of, or determining what vault the one or more compromised EDSs are part of.

17. The computer readable memory of claim 11, wherein first data segment is a portion of a data object and wherein the process to recover at least a portion of the first data segment includes one or more processing modules of one or more computing devices of the DSN executing at least one of initiating a cross-mirror rebuild operation, initiating a cross-service provider rebuild operation, initiating a clean-up operation on a second data segment associated with the data object, initiating repair of the data object using a fixed pattern of bits, and repairing the one or more compromised EDSs to produce partially corrupt data.

18. The computer readable memory of claim 11, wherein the first data segment is a portion of a data object and wherein the process to recover at least a portion of the first data segment includes one or more processing modules of one or more computing devices of the DSN rebuilding a DSN index.

19. The computer readable memory of claim 11, wherein the first memory element that stores operational instructions that, when executed by a computing device of a DSN, further causes the computing device to compare expected EDS slice names to the EDS slice names of the EDSs of a set of EDSs being scanned.

20. The computer readable memory of claim 11, wherein the indeterminate state of processing a storage function includes at least one of an incomplete write request, an incomplete delete operation, an edit operation or a data migration operation.