Recovering from conflicts that emerge from eventually consistent operations

• US 10,176,191 B2
• Filed: 12/11/2017
• Issued: 01/08/2019
• Est. Priority Date: 09/05/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Abhijeet, Kumar, Baptist, Andrew D., Leggette, Wesley B., Dhuse, Greg R., Resch, Jason K.
• Primary Examiner(s): Pham, Tuan A
• Application Number:
  US15/837,979
• Publication Number:
  US 20180137140A1
• Time in Patent Office: 393 Days
• Field of Search: None
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1873 Versioning file systems, temporal file systems, e.g. file system supporting different historic versions of files
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/2308 Concurrency control transaction processing G06F9/466
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by detecting an inconsistency between a first version of an object at a first storage target within the DSN and a second version of the object at second storage target of the DSN. The method continues by accessing an operation log of the DSN to identify relevant entries regarding the object. The method continues by selecting an inconsistency resolution approach from a list of inconsistency resolution approaches based on a type of inconsistency between the first and second versions of the object and based on the relevant entries. When the selected inconsistency resolution approach is a multiple version storage option, the method continues by sending a first storage request to the first storage target to store the first version of the object and a second storage request to the second storage target to store the second version of the object.

Claims

1. A method comprises:

detecting, by a computing device of a dispersed storage network (DSN), an inconsistency between a first version of an object at a first storage target within the DSN and a second version of the object at second storage target of the DSN;

accessing, by the computing device, an operation log of the DSN to identify relevant entries regarding the object, wherein the operation log records requested data access operations for execution within the DSN;

selecting, by the computing device, an inconsistency resolution approach from a list of inconsistency resolution approaches based on a type of inconsistency between the first and second versions of the object and based on the relevant entries, wherein the list of inconsistency resolution approaches includes a multiple version storage option, a most current version storage option, and a merged version storage option;

when the selected inconsistency resolution approach is the multiple version storage option;

sending, by the computing device, a first storage request to the first storage target to store the first version of the object; and

sending, by the computing device, a second storage request to the second storage target to store the second version of the object;

when the selected inconsistency resolution approach is the merged version storage option;

determining, by the computing device, a specific inconsistency between the first version of the object and the second version of the object;

determining, by the computing device, whether the specific inconsistency is compatible or incompatible;

when the specific inconsistency is compatible;

updating the first and second versions of the object to include the specific inconsistency; and

when the specific inconsistency is incompatible;

utilizing the most current version storage option to resolve the specific inconsistency; and

when the selected inconsistency resolution approach is the most current version storage option;

determining, by the computing device, whether the first version of the object or the second version of the object is more current based on the relevant entries of the operation log;

when the first version of the object is more current, determining, by the computing device, determining an updating approach of the second version of the object based on a type of the object, wherein determining the updating approach includes, when the type of the object is an encoded data slice of a set of encoded data slices, the first version of the object is a first copy of the encoded data slice having a first revision level, and the second version of the object is a second copy of the encoded data slice having a second revision level, determining whether the first revision level is a more current revision level of the encoded data slice than the second revision level; and

implementing, by the computing device, the updating approach of the second version of the object such that the updated second version of the object is consistent with the first version of the object.

2. The method of claim 1, wherein determining the updating further approach comprises:

when the first revision level is the more current revision level of the encoded data slice, determining the updating approach to include overwriting of the second version of the object with the first version of the object by the second storage target, wherein the first storage target is a first storage unit in a first set of storage units and the second storage target is a first storage unit in a second set of storage units.

3. The method of claim 1, wherein determining the updating approach comprises:

when the type of the object is two or more encoded data slices of a set of encoded data slices, wherein the first version of the object is a first encoded data slice of the two or more encoded data slices and the second version of the object is a second encoded data slice of the two or more encoded data slices, wherein the first encoded data slice has a first revision level and the second encoded data slice has a second revision level, determining whether the first revision level is a more current revision level for the set of encoded data slices than the second revision level; and

when the first revision level is the more current revision level of the set of encoded data slices, determining the updating approach to include rebuilding the second encoded data slice to have the first revision level, wherein the first storage target is a first storage unit in a set of storage units and the second storage target is a second storage unit in the set of storage units.

4. The method of claim 1, wherein determining the updating approach comprises:

when the type of the object is an encoded data slice of a set of encoded data slices, the first version of the object is a first copy of the encoded data slice having a first metadata, and the second version of the object is a second copy of the encoded data slice having a second metadata, determining whether the first metadata is more current metadata for the encoded data slice than the second metadata; and

when the first metadata is the more current metadata for the encoded data slice than the second metadata, determining the updating approach to include overwriting of the second metadata with the first metadata by the second storage target.

5. The method of claim 1, wherein determining the updating approach comprises:

when the type of the object is two or more encoded data slices of a set of encoded data slices, wherein the first version of the object is a first encoded data slice of the two or more encoded data slices and the second version of the object is a second encoded data slice of the two or more encoded data slices, wherein the first encoded data slice has first metadata and the second encoded data slice has second metadata, determining whether the first metadata is more current metadata for the set of encoded data slices than the second metadata; and

when the first metadata is the more current metadata for the set of encoded data slices than the second metadata, determining the updating approach to include updating the second metadata to be consistent with the first metadata.

6. A computing device of a dispersed storage network (DSN), wherein the computing device comprises:

memory;

an interface; and

a processing module operably coupled to the memory and the interface, wherein the processing module is operable to;

detect an inconsistency between a first version of an object at a first storage target within the DSN and a second version of the object at second storage target of the DSN;

access an operation log of the DSN to identify relevant entries regarding the object, wherein the operation log records requested data access operations for execution within the DSN;

select an inconsistency resolution approach from a list of inconsistency resolution approaches based on a type of inconsistency between the first and second versions of the object and based on the relevant entries, wherein the list of inconsistency resolution approaches includes a multiple version storage option, a most current version storage option, and a merged version storage option;

when the selected inconsistency resolution approach is the multiple version storage option;

send, via the interface, a first storage request to the first storage target to store the first version of the object; and

send, via the interface, a second storage request to the second storage target to store the second version of the object;

when the selected inconsistency resolution approach is the merged version storage option;

determining, by the computing device, a specific inconsistency between the first version of the object and the second version of the object;

determining, by the computing device, whether the specific inconsistency is compatible or incompatible;

when the specific inconsistency is compatible;

updating the first and second versions of the object to include the specific inconsistency; and

when the specific inconsistency is incompatible;

utilizing the most current version storage option to resolve the specific inconsistency; and

when the selected inconsistency resolution approach is the most current version storage option;

determining, by the computing device, whether the first version of the object or the second version of the object is more current based on the relevant entries of the operation log;

when the first version of the object is more current, determining, by the computing device, determining an updating approach of the second version of the object based on a type of the object, wherein determining the updating approach includes, when the type of the object is an encoded data slice of a set of encoded data slices, the first version of the object is a first copy of the encoded data slice having a first revision level, and the second version of the object is a second copy of the encoded data slice having a second revision level, determining whether the first revision level is a more current revision level of the encoded data slice than the second revision level; and

implementing, by the computing device, the updating approach of the second version of the object such that the updated second version of the object is consistent with the first version of the object.

7. The computing device of claim 6, wherein the processing module is further operable to determine the updating approach by:

when the first revision level is the more current revision level of the encoded data slice, determining the updating approach to include overwriting of the second version of the object with the first version of the object by the second storage target, wherein the first storage target is a first storage unit in a first set of storage units and the second storage target is a first storage unit in a second set of storage units.

8. The computing device of claim 6, wherein the processing module is further operable to determine the updating approach by:

when the type of the object is two or more encoded data slices of a set of encoded data slices, wherein the first version of the object is a first encoded data slice of the two or more encoded data slices and the second version of the object is a second encoded data slice of the two or more encoded data slices, wherein the first encoded data slice has a first revision level and the second encoded data slice has a second revision level, determining whether the first revision level is a more current revision level for the set of encoded data slices than the second revision level; and

when the first revision level is the more current revision level of the set of encoded data slices, determining the updating approach to include rebuilding the second encoded data slice to have the first revision level, wherein the first storage target is a first storage unit in a set of storage units and the second storage target is a second storage unit in the set of storage units.

9. The computing device of claim 6, wherein the processing module is further operable to determine the updating approach by:

when the type of the object is an encoded data slice of a set of encoded data slices, the first version of the object is a first copy of the encoded data slice having a first metadata, and the second version of the object is a second copy of the encoded data slice having a second metadata, determining whether the first metadata is more current metadata for the encoded data slice than the second metadata; and

when the first metadata is the more current metadata for the encoded data slice than the second metadata, determining the updating approach to include overwriting of the second metadata with the first metadata by the second storage target.

10. The computing device of claim 6, wherein the processing module is further operable to determine the updating approach by:

when the type of the object is two or more encoded data slices of a set of encoded data slices, wherein the first version of the object is a first encoded data slice of the two or more encoded data slices and the second version of the object is a second encoded data slice of the two or more encoded data slices, wherein the first encoded data slice has first metadata and the second encoded data slice has second metadata, determining whether the first metadata is more current metadata for the set of encoded data slices than the second metadata; and

when the first metadata is the more current metadata for the set of encoded data slices than the second metadata, determining the updating approach to include updating the second metadata to be consistent with the first metadata.