Fast recovery using self-describing replica files in a distributed storage system

• US 10,127,243 B2
• Filed: 09/22/2015
• Issued: 11/13/2018
• Est. Priority Date: 09/22/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Saxena, Mohit
• Primary Examiner(s): Butler, Sarai E
• Application Number:
  US14/860,771
• Publication Number:
  US 20170083539A1
• Time in Patent Office: 1,148 Days
• Field of Search: 714 19
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/85 Active fault masking without idle spares active fault masking without idle spare hardware where processing functionality is redundant G06F11/2035

Abstract

The embodiments relate to performing a write operation in a filesystem. Replica files embedded with respective sets of data are stored are on persistent storage local to replica nodes of the filesystem. The storage includes atomically updating the replica files to the replica nodes. Each set of data has information for self-characterizing each replica file. A map stored at persistent storage local to a remote node is updated. The map maintains information for identifying each replica file stored at each replica node.

Claims

1. A method for performing a write operation, the method comprising:

storing a parent file on persistent storage in a filesystem;

atomically persisting one or more replica files of the stored file on persistent storage local to one or more respective replica nodes in the filesystem, wherein each replica file is embedded with a respective set of data, comprising;

chronology data and a metadata link to an inode of the parent file; and

atomically updating a map stored on persistent storage local to a metadata node, wherein the map maintains information for identifying, at a given replica node, each replica file having at least one block allocated at the given replica node.

2. The method of claim 1, wherein the metadata link includes an inode number and an offset of a corresponding data node.

3. The method of claim 2, wherein the chronology data is a sequence number.

4. The method of claim 2, further comprising recovering from a failure at a first replica node, wherein the recovery comprises:

identifying a first replica file stored at the first replica node from a first set of data, the first set of data comprising first chronology data and a first metadata link;

identifying a second replica file stored at a second replica node from a second set of data, the second set of data comprising second chronology data and a second metadata link;

discovering a data conflict between the first and second replica files; and

reconciling the data conflict.

5. The method of claim 4, wherein identifying the first replica file comprises performing a node-to-file lookup using the map and at least the first metadata link.

6. The method of claim 4, wherein identifying the second replica file comprises:

performing a file-to-node lookup using the map to locate one or more replica files corresponding to the parent file;

identifying one or more consistent replica files, including comparing an address in the inode of the parent file with each metadata link of the located replica files;

comparing the chronology data of each consistent replica file to determine chronology data associated with a most recent update; and

selecting the second replica file in response to determining that the second chronology data is associated with the most recent update.

7. The method of claim 4, wherein reconciling the data conflict comprises updating the first replica file based on the second replica file.

8. A computer program product comprising a computer readable storage device having program code embodied therewith, the program code executable by a processing unit to:

store a parent file on persistent storage in a filesystem;

atomically persist one or more replica files of the stored file on persistent storage local to one or more respective replica nodes in the filesystem, wherein each replica file is embedded with a respective set of data comprising;

chronology data and a metadata link to an inode of the parent file; and

atomically update a map stored on persistent storage local to a metadata node, wherein the map maintains information for identifying, at a given replica node, each replica file having at least one block allocated at the given replica node.

9. The computer program product of claim 8, wherein the metadata link includes an inode number and an offset of a corresponding data node.

10. The computer program product of claim 9, wherein the chronology data is a sequence number.

11. The computer program product of claim 9, further comprising program code to recover from a failure at a first replica node, wherein the recovery comprises program code to:

identify a first replica file stored at the first replica node from a first set of data, the first set of data comprising first chronology data and a first metadata link;

identify a second replica file stored at a second replica node from a second set of data, the second set of data comprising second chronology data and a second metadata link;

discover a data conflict between the first and second replica files; and

reconcile the data conflict.

12. The computer program product of claim 11, wherein identifying the first replica file comprises performing a node-to-file lookup using the map and at least the first metadata link.

13. The computer program product of claim 11, wherein the second set of data comprises second chronology data and a second metadata link, and wherein identifying the second replica file comprises program code to:

perform a file-to-node lookup using the map to locate one or more replica files corresponding to the parent file;

identify one or more consistent replica files, including comparing an address in the inode of the parent file with each metadata link of the located replica files;

compare the chronology data of each consistent replica file to determine chronology data associated with a most recent update; and

select the second replica file in response to determining that the second chronology data is associated with the most recent update.

14. The computer program product of claim 11, wherein reconciliation of the data conflict comprises updating the first replica file based on the second replica file.

15. A filesystem comprising:

a processing unit in communication with memory;

a metadata node in communication with the processing unit, the metadata node having local persistent storage;

one or more replica nodes in communication with the metadata node, each replica node having local persistent storage;

a write manager in communication with the processing unit, wherein the write manager performs a write operation comprising;

storage of a parent file local to the metadata node;

atomic persistence one or more replica files of the parent file at the one or more replica nodes, wherein each replica file is embedded with a respective set of data, comprising;

chronology data and a metadata link to an inode of the parent file; and

an atomic update of a map stored local to the metadata node, wherein the map maintains information identifying, at a given replica node, each replica file having at least one block allocated at the given replica node.

16. The filesystem of claim 15, wherein the metadata link includes an inode number and an offset of a corresponding data node.

17. The filesystem of claim 16, further comprising a recovery manager in communication with the processing unit, wherein the recovery manager is configured to recover from a failure at a first replica node, wherein the recovery comprises the recovery manager to:

identify a first replica file stored at the first replica node from a first set of data, the first set of data comprising first chronology data and a first metadata link;

identify a second replica file stored at a second replica node from a second set of data, the second set of data comprising second chronology data and a second metadata link;

discover a data conflict between the first and second replica files; and

reconcile the data conflict.

18. The filesystem of claim 17, wherein the identification of the first replica file comprises the recovery manager to perform a node-to-file lookup using the map and at least the first metadata link.

19. The filesystem of claim 17, wherein the identification of the second replica file comprises the recovery manager to:

perform a file-to-node lookup using the map to locate one or more replica files corresponding to the parent file;

identify one or more consistent replica files, including comparison of an address in the inode of the parent file with each metadata link of the located replica files;

compare the chronology data of each consistent replica file to determine chronology data associated with a most recent update; and

select the second replica file in response to determining that the second chronology data is associated with the most recent update.

20. The filesystem of claim 17, wherein the reconciliation of the data conflict comprises an update of the first replica file based on the second replica file.