Unordered idempotent replication operations

• US 8,321,380 B1
• Filed: 04/30/2009
• Issued: 11/27/2012
• Est. Priority Date: 04/30/2009
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Leverett, Bruce
• Primary Examiner(s): Spieler, William
• Application Number:
  US12/433,871
• Time in Patent Office: 1,307 Days
• Field of Search: 707/658, 707/660
• US Class Current: 707/660
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/184 implemented as replicated file system

Abstract

At least certain embodiments disclose a method and apparatus for replicating a file system at the logical block level in an unordered stream of individual data units. In at least certain embodiments, the replication operations include creating a snapshot of a file system of a source storage system, identifying changes in the file system since a previous replication operation by comparing the snapshot with a previous snapshot created during the earlier replication operation, and asynchronously transferring the identified changes to a destination storage system in an unordered stream of individual data units.

Claims

1. A method of replicating data residing in a data container of a source storage server, the method comprising:

creating a snapshot of the data container;

identifying changes in the data container of the source storage server over a period of time by comparing the snapshot with a previous snapshot, the previous snapshot corresponding to an earlier replication operation; and

asynchronously transferring the changes in the data container of the source storage server to a destination storage server in a stream of individual data units without first determining a relationship between the data units and a directory structure of a file system associated with the data container to replicate the data container of the source storage server in the destination storage server at a logical block level without regard to differences in geometry of physical persistent storage of the source and destination storage servers, and without regard to differences in data container format of the source and destination storage servers.

2. The method of claim 1, wherein the identifying comprises sequentially scanning each inode entry in an inode file associated with the data container of the source storage server to detect changes in a data object of the data container since the previous snapshot corresponding to the earlier replication operation, wherein the inode file includes a data structure representing a master list of data objects and each inode entry identifies a particular data object within the data container.

3. The method of claim 2, wherein the transferring comprises transferring changes detected in the data object during a single pass through the inode entries of the inode file and the transferring occurs at the same time remaining inode entries in the inode file are being sequentially scanned.

4. The method of claim 3, further comprising one or more from the list consisting of:

transferring a file in the data container of the source storage server to the destination storage server before transferring a directory of the data container of the source storage server in which the file is located;

transferring data of a file in the data container of the source storage server to the destination storage server before transferring metadata describing the file; and

transferring individual data blocks of a file out-of-order.

5. A method of replicating data residing in a data container of a source storage server, the method comprising:

receiving, at a destination storage server from a source storage server, a stream of individual data units transferred without a relationship between the data units and a directory structure of a file system associated with the data container having been determined, the stream of individual data units including changes in the data container of the source storage server since a previous replication operation; and

replicating the changes in the data container of the source storage server at the destination storage server at a logical block level using the stream of individual data units without regard to differences in geometry of physical persistent storage of the source and destination storage servers and without regard to differences in data container format of the source and destination storage servers.

6. The method of claim 5, wherein the stream of individual data units includes:

data associated with a data object that has been changed in the data container of the source storage server; and

an inode number of an inode file of the source storage server assigned to the data object that has been changed, wherein the inode file of the source storage server includes a data structure representing a master list of data objects of the data container in the source storage server and each inode entry identifies a particular data object within the data container.

7. The method of claim 6, further comprising:

matching the inode number of the data object in the inode file of the source storage server with a corresponding inode number in a replicated inode file at the destination storage server; and

adding the data object that has been changed to an inode entry in the replicated inode file at the destination storage server so that the inode number of the inode file of the source storage server is preserved in the replicated inode file of the destination storage server.

8. The method of claim 7, further comprising:

storing the data associated with the data object that has been changed to an unused block of data in physical persistent storage of the destination storage server; and

updating a buffer tree representing a logical block structure of a data container at the destination storage server to reference the data associated with the data object that has been changed, wherein the buffer tree includes a hierarchical metadata structure containing pointers to logical blocks of data in the data container at the destination storage server.

9. The method of claim 8, wherein updating the buffer tree comprises:

updating an indirect block in the buffer tree to include a pointer to the data associated with the data object that has been changed;

storing the indirect block to a different unused block of data in physical persistent storage of the destination storage server; and

repeating the updating and storing until all pointers to the data associated with the data object that has been changed in the buffer tree have been updated.

10. The method of claim 8, further comprising assigning a file block number to a highest-level indirect block in the buffer tree referencing the data associated with the data object that has been changed.

11. The method of claim 8, further comprising:

checking directory entries in the inode file at the destination storage server to determine whether a directory entry already exists in the directory;

updating the inode file to include the directory entry when the directory entry does not already exist; and

updating the inode file to replace the directory entry when the directory entry already exists.

12. A source storage system comprising:

a processor;

a network interface through which the source storage system can communicate with a destination storage server;

a storage manager configured to perform asynchronous replication operations, the storage manager including;

a snapshot differential module configured to;

create a snapshot of a data container stored in a source storage server; and

identify changes in the data container of the source storage server over a period of time by comparing the snapshot with a previous snapshot, the previous snapshot corresponding to an earlier replication operation; and

a source replication engine coupled with the snapshot differential module, the source replication engine configured to transfer the changes in the data container of the source storage server to the destination storage server in a stream of individual data units without first determining a relationship between the data units and a directory structure of a file system associated with the data container to replicate the data container of the source storage server in the destination storage server at a logical block level without regard to either differences in geometry of physical persistent storage of the source and destination storage servers or differences in data container format of the source and destination storage servers.

13. The source storage system of claim 12, wherein the snapshot differential module is further configured to sequentially scan each inode entry in an inode file of the data container of the source storage server to detect changes in a data object of the data container, wherein the inode file includes a data structure representing a master list of data objects and each inode entry identifies a particular data object within the data container.

14. The source storage system of claim 13, wherein the source replication engine is further configured to transfer changes detected in the data object during a single pass through the inode entries of the inode file, wherein the transferring occurs at the same time remaining inode entries in the inode file are being sequentially scanned.

15. The source storage system of claim 13, wherein the source replication engine is further configured to transfer one or more from the list consisting of:

a file in the data container of the source storage server to the destination storage server before transferring a directory of the data container of the source storage server in which the file is located;

a file in the data container of the source storage server to the destination storage server before transferring metadata describing the file; and

individual data blocks of a file out-of-order.

16. A destination storage system comprising:

a processor;

a network interface through which the destination storage system can communicate with a source storage system;

a storage manager configured to perform asynchronous replication operations, the storage manager including;

a destination replication engine configured to receive from a source storage server a stream of individual data units transferred without a relationship between the data units and a directory structure of a file system associated with the source storage server having been determined, the stream of individual data units including changes in a data container of the source storage server since a previous replication operation; and

a write allocation module coupled with the destination replication engine, the write allocation module configured to replicate the changes in the data container of the source storage server at a destination storage server at a logical block level using the stream of individual data units without regard to differences in geometry of physical persistent storage of the source and destination storage servers, and without regard to differences in data container format of the source and destination storage servers.

17. The destination storage system of claim 16, wherein the stream of individual data units includes:

data of a data object that has been changed in the data container of the source storage server; and

an inode number of an inode file of the source storage server assigned to the data object that has been changed, wherein the inode file of the source storage server includes a data structure representing a master list of data objects of the data container in the source storage server and each inode entry identifies a particular data object within the data container.

18. The destination storage system of claim 17, wherein the destination replication module is further configured to:

match the inode number of the data object in the inode file of the source storage server with a corresponding inode number in a replicated inode file at the destination storage server; and

add the data object that has been changed to an inode entry in the replicated inode file at the destination storage server so that the inode number of the inode file of the source storage server is preserved in the replicated inode file of the destination storage server.

19. The destination storage system of claim 18, wherein the write allocation module is further configured to:

store the data of the data object that has been changed to an unused block of data in physical persistent storage of the destination storage server; and

update a buffer tree representing a logical block structure of a data container at the destination storage server to reference the data of the data object that has been changed, wherein the buffer tree includes a hierarchical metadata structure containing pointers to logical blocks of data in the data container at the destination storage server.

20. The destination storage system of claim 19, wherein the destination replication engine is further configured to assign a file block number to a highest-level indirect block in the buffer tree pointing to the data of the data object that has been changed.

21. The destination storage system of claim 19, wherein the destination replication engine is further configured to:

check directory entries in the inode file at the destination storage server to determine whether a directory entry already exists in the directory;

update the inode file to include the directory entry to be added when the directory entry does not already exist; and

update the inode file to replace the directory entry when the directory entry already exists.