Synchronizing updates across cluster filesystems

• US 9,235,594 B2
• Filed: 08/20/2012
• Issued: 01/12/2016
• Est. Priority Date: 11/29/2011
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gupta, Karan, Naik, Manoj P., Schmuck, Frank B., Shah, Mansi A., Tewari, Renu
• Primary Examiner(s): Truong, Dennis
• Application Number:
  US13/589,256
• Publication Number:
  US 20130138616A1
• Time in Patent Office: 1,240 Days
• Field of Search: 707/639, 707/624, 707/681, 707/679, 707/682, 707/649, 707/674, 707/680, 707/640, 707/683, 707/684, 707/685, 707/999.202, 707/999.204, 711/162
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing

Abstract

Embodiments of the invention relate to synchronization of data in a shared pool of configurable computer resources. An image of the filesystem changes, including data and metadata, is captured in the form of a consistency point. Sequential consistency points are created, with changes to data and metadata in the filesystem between sequential consistency captured and placed in a queue for communication to a target filesystem at a target site. The changes are communicated as a filesystem operation, with the communication limited to the changes captured and reflected in the consistency point.

Claims

1. A method comprising:

periodically creating a consistency point in a source filesystem, each consistency point representing filesystem data and metadata at a point-in-time to establish a recovery point, the periodic creation of the consistency point including creating a first consistency point at a first point-in-time and a second consistency point at a second point-in-time, wherein the first consistency point and the second consistency point are any two consistency points;

comparing the first consistency point with the second consistency point, including identifying a difference between the first and second consistency points, wherein the comparison detects presence of a new inode and a changed directory at the filesystem;

responsive to the detection of the new inode, creating a temporary new target inode at a target file system;

mapping at least one target inode, including the temporary inode, to one or more corresponding source inodes;

responsive to the detection of the changed directory, identifying a file level change, wherein a changed file is associated with an established directory at the target filesystem;

establishing a link from the source filesystem to the target filesystem; and

applying the identified difference between the first and second consistency points, including applying the identified difference using the temporary new inode, including replaying the identified difference as one or more filesystem operations, wherein the difference is differential, and including updating the established directory with data associated with the changed file.

2. The method of claim 1, wherein the step of applying the identified difference employs a mapping of one of the source filesystem inodes to a target filesystem inode.

3. The method of claim 1, wherein the step of applying the identified difference includes identifying filesystem operations on filesystem objects using file names.

4. The method of claim 1, wherein the step of applying the identified difference between the first and second consistency points includes producing an exact delta of any metadata change and any data change, and replaying only those changes reflected in the exact delta.

5. The method of claim 1, further comprising generating an output file for the identified difference and replaying the output file as a series of filesystem operations where some of the operations may be replayed concurrently to a target site.

6. The method of claim 1, further comprising replaying the output file after communication failure between the source filesystem and the target filesystem.

7. The method of claim 1, wherein the source and target filesystems are heterogeneous filesystems.

8. The method of claim 1, further comprising reverting changes at the target filesystem by moving to an older consistency point.

9. The method of claim 1, wherein the first and second consistency points are not limited to successive consistency points.

19. The method of claim 1, further comprising, using the first and second consistency points, ascertaining a level of change, including identifying any changes at a directory level, and creating a new directory at a target site responsive to a detected absence of a directory, and establishing a link from the new directory to a corresponding source site directory.

20. The method of claim 19, further comprising identifying any changes at a file level in response to a detected presence of the directory at the directory level, including updating at least one of the directories with the identified file level changes.

10. A method comprising:

creating a first consistency point in a source filesystem at a first point-in-time and creating a second consistency point in the source filesystem at a second point-in-time, the first consistency point representing filesystem data and metadata at a first point-in-time to establish a first recovery point and the second consistency point representing filesystem data and metadata at a second point-in-time to establish a second consistency point, wherein the first consistency point and the second consistency point are any two consistency points;

following a communication failure associated with the source filesystem, comparing the first consistency point with the second consistency point, including identifying a change between the first and second consistency points, wherein the comparison detects presence of a new inode and a changed directory at the filesystem;

responsive to the detection of the new inode, creating a temporary new target inode at a target filesystem;

mapping at least one target inode, including the temporary inode, to one or more corresponding source inodes;

responsive to the detection of the changed directory, identifying a file level change, wherein a changed file is associated with an established directory at the target filesystem;

establishing a link from the source filesystem to the target filesystem; and

applying the identified change between the first and second consistency points, including applying the identified difference using the temporary new inode, including replaying the identified change as a filesystem operation, wherein the change is differential, and including updating the established directory with data associated with the changed file.

11. The method of claim 10, further comprising creating a third consistency point in the source filesystem at a point-in-time, the third consistency point being sequential with the first and second consistency point, and removing the first consistency point.

12. A computer program product comprising a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising:

computer readable program code configured to process data at a source site in a shared pool of configurable resources, including periodic creation of a consistency point in a source filesystem of the source site, each consistency point to capture actual source filesystem data and metadata at a point-in-time to establish a recovery point, wherein a first consistency point and a second consistency point are any two consistency points;

computer readable program code configured to compare the first consistency point with the second consistency point, including identification of any differences between the first and second consistency points, wherein the comparison detects presence of a new inode and a changed directory at the filesystem;

computer readable program code configured to create a temporary new target inode in response to detection of the new inode;

computer readable program code configured to map at least one target inode, including the temporary inode, to one or more corresponding source inodes;

computer readable program code configured to identify a file level change in response to detection of the changed directory, wherein a changed file is associated with an established directory at the target filesystem;

computer readable program code configured to establish a link from the source filesystem to the target filesystem; and

computer readable program code configured to apply the identified differences between the first and second consistency points, including the code to apply the identified difference using the temporary new inode, including the code to replay the identified difference as one or more filesystem operations, wherein the difference is differential, and including updating the established directory with data associated with the changed file.

13. The computer program product of claim 12, wherein the program code to replay the identified differences includes program code to identify filesystem operations on filesystem objects using file names.

14. The computer program product of claim 12, wherein the code configured to replay the identified differences between the first and second consistency points produces an exact delta of any metadata change and any data change, and replays only those changes reflected in the exact delta.

15. The computer program product of claim 12, further comprising computer program code configured to generate an output file for the identified differences and to concurrently replay the identified differences between the first and second consistency points to support multiple concurrent filesystem operations.

16. The computer program product of claim 12, further comprising computer program code configured to replay the output file after communication failure between the source site and a target site.

17. The computer program product of claim 12, wherein the first and second consistency points are not limited to successive consistency points.

18. The computer program product of claim 12, further comprising computer program code configured to revert changes at the target filesystem by moving to an older consistency point.