Ensuring metadata and index consistency using write intents

• US 10,379,961 B2
• Filed: 04/11/2017
• Issued: 08/13/2019
• Est. Priority Date: 04/11/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Scholl, Daniel J., Jarczyk, Jeremy S., Leggette, Wesley B., Resch, Jason K., Vedpathak, Yogesh R., Motwani, Manish, Reid, Tyler K.
• Primary Examiner(s): Manoskey, Joseph D
• Application Number:
  US15/484,417
• Publication Number:
  US 20180293266A1
• Time in Patent Office: 854 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/2058 using more than 2 mirrored copies
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method begins by, receiving a write data object request and writing and committing the data object as a set of encoded data slices into DSN memory. The method continues by writing and committing an index consistency write-intent to DSN memory. The method continues by writing metadata of the data object to DSN memory. The method continues by write and committing an index entry to DSN memory. The method continues, during a finalization of the index consistency write-intent, by executing the index consistency write-intent to ensure consistency between the metadata of the data object and metadata located in the index entry.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

receiving a write data object request;

writing and committing the data object as a set of encoded data slices into DSN memory;

writing and committing an index consistency write-intent to DSN memory, wherein the index consistency write-intent specifies an intention to check, at a later time, for a consistent index with matching metadata associated with the data object;

writing metadata of the data object to DSN memory;

writing and committing an index entry to DSN memory;

committing the metadata of the data object to DSN memory;

deleting the index consistency write-intent; and

notifying a write data object requestor of a successful data object write.

2. The method of claim 1 further comprises, if during processing of the index consistency write-intent, metadata different from the metadata of the data object was encountered, determining that a discrepancy has occurred.

3. The method of claim 1 further comprises, if during processing of the index consistency write-intent, metadata different from the metadata of the data object was encountered, rolling back the metadata.

4. The method of claim 1 further comprises, if during processing of the index consistency write-intent, an unexpected revision value was encountered, determining that a discrepancy has occurred.

5. The method of claim 1 further comprises, if during processing of the index consistency write-intent, an unexpected revision value was encountered, rolling back the metadata.

6. The method of claim 1, wherein during a finalization of the index consistency write-intent, the index consistency write-intent would be executed to ensure consistency between the metadata of the data object and metadata located in the index entry.

7. The method of claim 1 further comprises, when the writing and committing the data object as a set of encoded data slices into DSN memory is unsuccessful, cleanup processing including:

reading the metadata of the data object to generate an index entry to be written to DSN memory;

writing the metadata of the data object to the DSN memory to lock it;

writing and committing the index entry to an index;

rolling back the metadata; and

deleting the index consistency write-intent.

8. The method of claim 1 further comprises, when the writing and committing an index consistency write-intent to DSN memory is unsuccessful, cleanup processing including:

reading the metadata of the data object to generate an index entry to be written to DSN memory;

writing the metadata of the data object to the DSN memory to lock it;

writing and committing the index entry to an index;

rolling back the metadata; and

deleting the index consistency write-intent.

9. The method of claim 1 further comprises, when the writing the metadata of the data object to DSN memory is unsuccessful, cleanup processing including:

reading the metadata of the data object to generate an index entry to be written to DSN memory;

writing the metadata of the data object to the DSN memory to lock it;

writing and committing the index entry to an index;

rolling back the metadata; and

deleting the index consistency write-intent.

10. The method of claim 1 further comprises, when the writing and committing an index entry to DSN memory is unsuccessful, cleanup processing including:

reading the metadata of the data object to generate an index entry to be written to DSN memory;

writing the metadata of the data object to the DSN memory to lock it;

writing and committing the index entry to an index;

rolling back the metadata; and

deleting the index consistency write-intent.

11. The method of claim 10, wherein the index entry is deleted if the metadata of the data object was not located in the DSN memory.

12. The method of claim 1 further comprises, when the committing the metadata of the data object to DSN memory is unsuccessful, cleanup processing including:

reading the metadata of the data object to generate an index entry to be written to DSN memory;

writing the metadata of the data object to the DSN memory to lock it;

writing and committing the index entry to an index;

rolling back the metadata; and

deleting the index consistency write-intent.

13. A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

receive a write data object request;

write and committing the data object as a set of encoded data slices into DSN memory;

write and committing an index consistency write-intent to DSN memory, wherein the index consistency write-intent specifies an intention to check, at a later time, for a consistent index with matching metadata associated with the data object;

write metadata of the data object to DSN memory;

write and committing an index entry to DSN memory;

commit the metadata of the data object to DSN memory;

delete the index consistency write-intent; and

notify a write data object requestor of a successful data object write.

14. The computing device of claim 13 further comprises, if during processing of the index consistency write-intent, metadata different from the metadata of the data object was encountered, determining that a discrepancy has occurred.

15. The computing device of claim 13 further comprises, if during processing of the index consistency write-intent, metadata different from the metadata of the data object was encountered, rolling back the metadata.

16. The computing device of claim 13, wherein during a finalization of the index consistency write-intent, the index consistency write-intent would be executed to ensure consistency between the metadata of the data object and metadata located in the index entry.

17. The computing device of claim 13, wherein the index consistency write-intent is written in a single transaction with the metadata of the data object.

18. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

receiving a write data object request;

writing and committing the data object as a set of encoded data slices into DSN memory;

writing and committing an index consistency write-intent to DSN memory, wherein the index consistency write-intent specifies an intention to check, at a later time, for a consistent index with matching metadata associated with the data object;

writing metadata of the data object to DSN memory;

write and committing an index entry to DSN memory; and

wherein during a finalization of the index consistency write-intent, the index consistency write-intent would be executed to ensure consistency between the metadata of the data object and metadata located in the index entry.

19. The method of claim 18 further comprises, if during finalization of the index consistency write-intent, metadata different from the metadata of the data object was encountered, rolling back the metadata of the data object.

20. The method of claim 18, wherein the index consistency write-intent is written in a single transaction with the metadata of the data object.