APPARATUS METHOD AND SYSTEM FOR ASYNCHRONOUS REPLICATION OF A HIERARCHICALLY-INDEXED DATA STORE

• US 20080250086A1
• Filed: 05/28/2008
• Published: 10/09/2008
• Est. Priority Date: 07/30/2003
• Status: Active Grant

• Current Assignee: ServiceNow Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Kisley, Richard Victor
• Granted Patent: US 7,984,016 B2
• Time in Patent Office: Days
• US Class Current: 707/204
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 2201/855 Details of asynchronous mirroring using a journal to transfer not-yet-mirrored changes
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users

Abstract

Changes such as write operations on a replication source comprising a hierarchically-indexed file system and associated data store are tracked and aggregated over a point-in-time image replication interval and communicated to a replication target. The replication target receives the changes, conducts operations to make the changes on a target volume or portion thereof, and conducts a point-in-time image replication operation to synchronize the replication target with the replication source. In the present invention the advantages of replication techniques developed for log-structured file systems are provided to systems with conventional hierarchical file systems. The invention also facilitates maintaining point-in-time retrievable images of a volume or portion thereof on a target volume while reducing the bandwidth and processing burden associated with replication.

Claims

1. An apparatus for asynchronous file-based replication of a hierarchically-indexed data:

store, the apparatus comprising;

a hierarchically indexed data store configured to store data in a hierarchical structure corresponding to a file system;

an operating system configured to access the hierarchically indexed data store and initiate write operations to the hierarchically indexed data store;

a first in-memory block map for tracking changes to the hierarchically indexed data store, the first in-memory block map comprising a plurality of status indicators configured to indicate changes to blocks on the hierarchically indexed data store;

a second in-memory block map for tracking changes to the hierarchically indexed data store, the second in-memory block map comprising a plurality of status indicators configured to indicate changes to blocks on the hierarchically indexed data store;

a tracking module comprising a file system driver configured to monitor write operations initiated by the operating system and track file regions that have changed since a first point-in-time replication instance by updating the status indicators in the second in-memory block map in a written order;

a replication module configured to asynchronously communicate data contained within file regions that have changed to a replication target as indicated by the first in-memory block map;

the file system driver further configured to access the first in-memory block map instead of the second in-memory block map in response to a second point-in-time replication instance;

the replication module further configured to access the second in-memory block map instead of the first in-memory block map in response to the second point-in-time replication instance and asynchronously communicate data contained within file regions-that have changed to the replication target as indicated by the second in-memory block map, wherein the data is communicated in an order that is independent of the written order.

2. The apparatus of claim 1, further comprising a replication target configured to asynchronously receive the data contained within the file regions that have changed from the replication module, write the data within corresponding files regions on the replication target, and initiate a point-in-time image replication operation configured to synchronize the replication target with the hierarchically indexed data store as structured at the second point-in-time replication instance.

3. The apparatus of claim 1, wherein the replication module is further configured to clear the first in-memory block map subsequent to asynchronously communicating the data contained within the file regions indicated by the first in-memory block map.

4. The apparatus of claim 1, wherein the replication module is further configured to conduct replication operations as directed by policies related to replication.

5. The apparatus of claim 1, further comprising a storage management module configured to set policies related to replication.

6. The apparatus of claim 1, further comprising a point-in-time image replication module configured to provide point-in-time image replication services to the hierarchically-indexed data store.

7. The apparatus of claim 1, wherein the tracking module is further configured to save information regarding the file regions that have changed since the first point-in-time image replication instance.

8. The apparatus of claim 1, wherein the file system driver is an installable driver.

9. The apparatus of claim 1, wherein the point-in-time image replication comprises a snapshot of the root node.

10. An apparatus for asynchronous file-based replication of a hierarchically-indexed data store, the apparatus comprising:

a hierarchically-indexed data store operable on a CPU

means for asynchronously communicating data contained within file regions that have changed to a replication target as indicated by a first in-memory block map;

a file system driver configured to monitor writs operations to the hierarchically-indexed data store and track file regions that have changed since a first point-in-time replication instance by updating block status indicators in a second-memory block map;

means for clearing the first in-memory block;

the file system driver further configured to update block status indicators in the first in-memory block map instead of the second in-memory block map in response to a second point-in-time replication instance; and

means for asynchronously communicating data contained within file regions that have changed to the replication target as indicated by the second in-memory block map instead of the first in-memory block map in response to the second point-in-time replication instance, wherein changes to the file regions occur in a written order and asynchronously communicating data occurs in an order that is independent of the written order.

11. The apparatus of claim 10, further comprising means for receiving data contained within the file regions that have changed, means for writing the data within corresponding files regions on a replication target, and means for initiating a point-in-time image replication operation configured to synchronize the replication target with the replication source.

12. A system for asynchronous file-based replication of a hierarchically-indexed data store, the system comprising:

a replication target comprising a CPU and a first hierarchically-indexed data store_configured to store data in a hierarchical structure corresponding to a file system;

a replication source comprising a CPU and a second hierarchically-indexed data store configured to store data in a hierarchical structure corresponding to a file system;

the replication source further comprising an operating system configured to access the second hierarchically indexed data store and initiate write operations to second the hierarchically indexed data store;

the replication source further comprising a first in-memory block map for tracking changes to the second hierarchically indexed data store, the first in-memory block map comprising a plurality of status indicators configured to indicate changes to blocks on the second hierarchically indexed data store;

the replication source further comprising a second in-memory block map for tracking changes to the second hierarchically indexed data store, the second in-memory block map comprising a plurality of status indicators configured to indicate changes to blocks on the second hierarchically indexed data store;

the replication source further comprising a tracking module comprising a file system driver configured to monitor write operations initiated by the operating system and track file regions that have changed since a first point-in-time replication instance by updating the status indicators in the second in-memory block map in a written order;

the replication source further comprising a replication module configured to asynchronously communicate data contained within file regions that have changed to a replication target as indicated by the first in-memory block map;

the file system driver further configured to clear the first in-memory block and access the first in-memory block map instead of the second in-memory block map in response to a second point-in-time replication instance;

the replication module further configured to access the second in-memory block map instead of the first in-memory block map in response to the second point-in-time replication instance and asynchronously communicate data contained within file regions that have changed to the replication target as indicated by the second in-memory block map, wherein the data is communicated in an order that is independent of the written order.

13. The system of claim 12, wherein the replication target is further configured to receive the data within changed file regions from the replication source, write the data within corresponding files regions on the replication target and initiate a point-in-time image replication operation configured to synchronize the replication target with the replication source.

14. The system of claim 12, wherein the replication module is further configured to clear the first in-memory block map subsequent to asynchronously communicating the data contained within the file regions indicated by the first in-memory block map.

15. A computer readable storage medium storing computer readable program code for conducting a method for asynchronous file-based replication of a hierarchically-indexed data store, the method comprising:

asynchronously communicating data contained within file regions that have changed to a replication target as indicated by a first in-memory block map;

monitoring write operations to a hierarchically-indexed data store with a file system driver in order to track file regions that have changed since a first point-in-time replication instance wherein changes to the file regions occur in a written order;

updating block status indicators in a second in-memory block map in response to the changes to the file regions;

clearing the first in-memory block and updating block status indicators in the first in-memory block map instead of the second in-memory block map in response to a second point-in-time replication instance;

asynchronously communicating data contained within file regions that have changed to the replication target as indicated by the second in-memory block map instead of the first in-memory block map in response to the second point-in-time replication instance, wherein asynchronously communicating data occurs in an order that is independent of the written order.

16. The computer readable storage medium of claim 15, wherein the method further comprises receiving the data contained within the file regions that have changed, writing the data within corresponding files regions on a replication target, and initiating a point-in-time image replication operation configured to synchronize the replication target with the replication source.

17. The computer readable storage medium of claim 15, wherein the method further comprises clearing the first in-memory block map subsequent to asynchronously communicating the data contained within the file regions indicated by the first in-memory block map.

18. The computer readable storage medium of claim 15, wherein the method further comprises communicating is conducted as directed by policies related to replication.

19. The computer readable storage medium of claim 15, wherein the method further comprises invoking point-in-time image replication services.

20. The computer readable storage medium of claim 15, wherein the method further comprises saving information regarding the file regions that have changed since the first point-in-time image replication instance.

21. The computer readable storage medium of claim 15, wherein the method further comprises tracking file regions is conducted in response to write operations.

22. The computer readable storage medium of claim 15, wherein the method further comprises saving information regarding the storage regions that have changed since the first point-in-time image replication instance.

23. The computer readable storage medium of claim 15, wherein the point-in-time image comprises a snapshot.

24. A file system driver for use in conducting point-in-time replication operations, comprising:

a file system driver configured to monitor write operations initiated by an operating system and track file regions that have changed since a first point-in-time replication instance by updating the status indicators in a second in-memory block map in a written order;

the file system driver further configured to access the first in-memory block map instead of the second in-memory block map in response to a second point-in-time replication instance.