File system replication

• US 7,882,064 B2
• Filed: 07/06/2006
• Issued: 02/01/2011
• Est. Priority Date: 07/06/2006
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Zhu, Ming Benjamin, Lee, Edward K., Maheshwari, Umesh, Patterson, R. Hugo
• Primary Examiner(s): Trujillo; James
• Assistant Examiner(s): Casanova; Jorge A
• Application Number:
  US11/483,131
• Publication Number:
  US 20080010322A1
• Time in Patent Office: 1,671 Days
• Field of Search: 707/201, 707/204, 707/206, 707/613, 707/625, 707/638
• US Class Current: 707/625
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/184 implemented as replicated file system

Abstract

File system replication includes determining whether one of a plurality of files included in an original file system has been updated since a previous replication, the file having a plurality of data segments, and in the event that the file has been updated, locating among the plurality of data segments a previously stored data segment that is newly referenced by the file, and that does not require replication.

Claims

1. A method of file system replication comprising:

determining using a processor whether one of a plurality of files included in an original file system has been updated since a previous replication, including comparing a current content handle associated with the file at current time and a previous content handle associated with the file at an earlier time, wherein;

each content handle is derived based at least in part on a multi-layered hierarchical representation of the file;

the multi-layered hierarchical representation includes a bottom layer that includes a plurality of bottom layer nodes that corresponds to a plurality of data segments included in the file, and a second layer that includes a plurality of second layer nodes each being derived based at least in part on some of the bottom layer nodes;

each content handle is derived based on one or more nodes in the hierarchical representation of the file that are included in a layer other than the bottom layer; and

in the event that the file has been updated, locating among the plurality of data segments a previously not stored data segment;

replicating the previously not stored data segment; and

locating among the plurality of data segments a previously stored data segment that is newly referenced by the file, wherein locating the previously stored data segment is based at least in part on at least some of the plurality of data segment references, and the previously stored data segment is not replicated.

2. A method as recited in claim 1, wherein in the event that the file has been updated, the method further comprises updating metadata associated with the file on the replicated file system.

3. A method as recited in claim 1, wherein in the event that the file has been updated, the method further comprises determining an updated portion of metadata associated with the file, and replicating the updated portion of metadata on the replicated file system.

4. A method as recited in claim 1, wherein in the event that the file has been updated, the method further comprises replicating a data segment reference associated with the file on the replicated file system.

5. A method as recited in claim 4, wherein the data segment reference is determined based at least in part on a content handle associated with the file.

6. A method as recited in claim 1, wherein in the event that the file has been updated, the method further comprises determining a data segment reference that has been updated, and replicating the updated data segment reference.

7. A method as recited in claim 1, wherein determining whether the file has been updated includes processing a log record.

8. A method as recited in claim 1, wherein at least one of the previous content handle and the current content handle is stored in a log record.

9. A method as recited in claim 1, wherein at least one of the previous content handle and the current content handle is stored in a file system snapshot.

10. A method as recited in claim 1, wherein at least one of the previous content handle and the current content handle is stored in the original file system.

11. A method as recited in claim 1, wherein at least one of the previous content handle and the current content handle is stored in the replicated file system.

12. A method as recited in claim 1, wherein determining whether the file has been updated includes performing a file system walk.

13. A method as recited in claim 1, wherein determining whether the file has been updated includes comparing a plurality of snapshots.

14. A method as recited in claim 1, wherein determining whether the file has been updated includes comparing a plurality of snapshots of the original file system.

15. A method as recited in claim 1, wherein determining whether the file has been updated includes comparing a plurality of snapshots of the file.

16. A method as recited in claim 1, wherein locating the previously stored data segment includes looking up at least some of the plurality of data segment references on the replicated file system.

17. A method as recited in claim 1, wherein locating the previously stored data segment includes looking up at least some of the plurality of data segment references on the original file system.

18. A method as recited in claim 1, wherein locating the previously not replicated data segment is at least in part based on the current content handle associated with the file.

19. A method as recited in claim 18, wherein locating the previously not stored data segment based at least in part on the current content handle includes dereferencing the content handle to obtain a plurality of data segment references, and determining whether each of the plurality of data segment references corresponds to a stored data segment that has been previously stored on the replicated file system.

20. A method as recited in claim 18, wherein locating the previously not stored data segment is further based at least in part on the previous content handle.

21. A method as recited in claim 20, wherein locating the previously not stored data segment further includes:

dereferencing the current content handle to obtain a plurality of recent data segment references associated with the recent version of the file; and

dereferencing the previous content handles to obtain a plurality of previous data segment references associated with the earlier version of the file; and

determining a difference between the plurality of current data segment references and the plurality of previous data segment references.

22. A method as recited in claim 21, wherein locating the previously not stored data segment includes determining whether the previous content handle has been stored on the replicated file system.

23. A method as recited in claim 22, further comprising, in the event that the previous content handle has not been stored on the replicated file system, dereferencing the current content handle to obtain a plurality of recent data segment references associated with the current version of the file.

24. A method as recited in claim 1, further comprising:

prior to replicating the previously not stored data segment, preventing the previously not stored data segment from being deleted during garbage collection before the previously not stored data segment is replicated on the replicated file system.

25. A method as recited in claim 24, further comprising maintaining a log record that maintains a reference to the previously not stored data segment.

26. A method as recited in claim 24, further comprising keeping a reference count of the previously stored data segment to be greater than zero at least until the previously not stored data segment is replicated.

27. A method as recited in claim 24, further comprising creating a place holder in the original file system to maintain a reference to the previously not stored data segment.

28. A method as recited in claim 1, further comprising initializing the replicated file system by copying the original file system.

29. A method as recited in claim 1, further comprising initializing the replicated file system by copying data segments that are included in the original file system.

30. A file system replicator comprising:

a processor configured to;

determine whether one of a plurality of files included in an original file system has been updated since a previous replication, including comparing a current content handle associated with the file at current time and a previous content handle associated with the file at an earlier time, wherein;

each content handle is derived based at least in part on a multi-layered hierarchical representation of the file;

the multi-layered hierarchical representation includes a bottom layer that includes a plurality of bottom layer nodes that corresponds to a plurality of data segments included in the file, and a second layer that includes a plurality of second layer nodes each being derived based at least in part on some of the bottom layer nodes;

each content handle is derived based on one or more nodes in the hierarchical representation of the file that are included in a layer other than the bottom layer; and

in the event that the file has been updated, locate among the plurality of data segments a previously not stored data segment;

replicate the previously not stored data segment; and

locate among the plurality of data segments a previously stored data segment that is newly referenced by the file, wherein locating the previously stored data segment is based at least in part on at least some of the plurality of data segment references, and the previously stored data segment is not replicated; and

a memory coupled to the processor, configured to provide the processor with instructions.

31. A computer program product for file system replication, the computer program product being stored and/or recorded in a computer readable storage medium and comprising computer instructions for:

determining using a processor whether one of a plurality of files included in an original file system has been updated since a previous replication, including comparing a current content handle associated with the file at current time and a previous content handle associated with the file at an earlier time, wherein;

each content handle is derived based at least in part on a multi-layered hierarchical representation of the file;

the multi-layered hierarchical representation includes a bottom layer that includes a plurality of bottom layer nodes that corresponds to a plurality of data segments included in the file, and a second layer that includes a plurality of second layer nodes each being derived based at least in part on some of the bottom layer nodes;

each content handle is derived based on one or more nodes in the hierarchical representation of the file that are included in a layer other than the bottom layer; and

in the event that the file has been updated, locating among the plurality of data segments a previously not stored data segment;

replicating the previously not stored data segment; and

locating among the plurality of data segments a previously stored data segment that is newly referenced by the file, wherein locating the previously stored data segment is based at least in part on at least some of the plurality of data segment references, and the previously stored data segment is not replicated.