Asynchronously migrating a file system

• US 9,304,997 B2
• Filed: 08/27/2013
• Issued: 04/05/2016
• Est. Priority Date: 08/27/2013
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Beard, Derek, Althaus, Gregory, Yammine, Ghassan, Becker, Gregory, Venteicher, Bryan
• Primary Examiner(s): Ly, Anh
• Application Number:
  US14/011,696
• Publication Number:
  US 20150066845A1
• Time in Patent Office: 952 Days
• Field of Search: 707/796, 707/737, 707/791, 707/795, 707/625, 707/657, 707/637, 707/639, 707/640, 707/649, 707/692, 707/827, 707/613, 707/661, 707/610, 707/822, 707/E17.01, 707/E17.002, 707/E17.032, 707/E17.005, 711/100, 711/103, 711/122, 711/123, 711/135, 711/141, 711/154, 711/156, 711/162, 711/163, 711/E12.001, 711/E12.093, 711/171, 711/E12.002, 709/203, 709/205, 709/213, 709/216, 709/217, 709/218, 709/219, 709/220, 709/226, 709/229, 709/230, 709/232, 709/239
• US Class Current: 1/1
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation

Abstract

A system for migrating data from a source file system to a destination file system, in a manner that is transparent and seamless to clients of the source file system.

Claims

1. A method for migrating data, the method being implemented by one or more processors and comprising:

identifying a collection of file system objects that are associated with a source file system that is actively in use by a plurality of clients;

intercepting individual file system operations that are intended to be handled by the source file system at a location that is in-line and external to the source file system, so as to not disrupt the plurality of clients requesting file system operations to be performed on the source file system;

for at least a duration of time during which the source file system is in active use by the plurality of clients,

replicating at least a portion of the source file system, including each file system object of the collection, at a destination file system;

determining when individual file system operations after the source file system; and

asynchronously implementing one or more of the individual file system operations that are determined to alter the source file system on the destination file system, wherein asynchronously implementing one or more of the individual file system operations includes (i) generating a replay log that sequences multiple file system operations corresponding to a create type operation, a write type operation or a modification type operation, but not a read type operation, and (ii) asynchronously implementing the multiple file system operations from the replay log.

2. The method of claim 1, further comprising:

determining when the source file system and the destination file system are equivalent; and

synchronously implementing the individual file system operations that are determined to alter the source file system on the destination file system.

3. The method of claim 1, wherein intercepting the individual file system operations includes communicating with individual clients from the location of the source file system.

4. The method of claim 1, further comprising:

forwarding the individual file system operations to the source file system;

receiving a response to each of the individual file system operations; and

determining, from the response of one or more of the individual file system operations, a set of attributes of a corresponding file system object.

5. The method of claim 4, further comprising storing the set of attributes in association with the corresponding file system object.

6. The method of claim 5, wherein the set of attributes includes one or more timing parameters.

7. The method of claim 1, wherein replicating at least the portion of the source file system includes replicating (i) each file system object of the collection, and (ii) an organizational structure of the file system objects that comprise the at least portion of the source file system.

8. The method of claim 1, wherein replicating at least the portion of the source file system is performed asynchronously.

9. The method of claim 1, wherein asynchronously implementing one or more of the individual file system operations is performed while some file system objects of the collection are or have yet to be replicated with the destination file system.

10. The method of claim 1, wherein asynchronously implementing one or more of the individual file system operations includes determining that a given file system object specified by a given one of the file system operations has been replicated, then asynchronously implementing the given one of the file system operations in response to determining that the given file system object has been replicated.

11. The method of claim 1, further comprising:

copying a portion of the source file system, including individual file system objects that comprise the source file system, into a source cache resource; and

using the source cache resource to respond to requests from clients for individual file system objects of the portion.

12. The method of claim 11, wherein copying the portion of the source file system into the source cache resource includes copying an organizational structure of the portion of the source file system, and attributes of individual file system objects that comprise the portion of the source file system, into the source cache resource.

13. The method of claim 11, further comprising selecting file system objects of the portion for the source cache resource based at least in part on an anticipated activity level of the file system object.

14. A non-transitory computer-readable medium that stores instructions for replicating an active file system, the instructions being executable by one or more processors to cause the one or more processors to perform operations that include:

implementing a series of file system operations in order to traverse a source file system and identify file system objects that comprise the source file system;

maintaining a data structure in which each identified file system object is associated with an entry and a current set of attributes for that file system object;

replicating at least a portion of the source file system, including file system objects identified from traversing the source file system, on a destination file system;

intercepting, at a location that is in-line and external to the source file system, individual file system operations that are generated from a plurality of clients for the source file system, so as to not disrupt the plurality of clients requesting file system operations to be performed on the source file system;

wherein replicating at least a portion of the source file system includes;

determining a corresponding file system object specified by each of the file system operations;

determining, from the data structure, whether the corresponding file system object has previously been identified;

if the corresponding file system object has not previously been identified, then (i) determining a set of attributes for the corresponding file system object, (ii) adding an entry for the corresponding file system object and its set of attributes on the data structure, and (iii) replicating the corresponding data object at the destination file system; and

if the corresponding file system object has previously been identified, then (i) determining the set of attributes for the corresponding file system object from the source file system, (ii) determining a set of attributes for a replication of the corresponding file system object for the destination file system, and (iii) comparing the set of attributes determined from the source file system to the set of attributes for the replication in order to determine whether the replication is up to date or out of date.

15. The non-transitory computer-readable medium of claim 14, further comprising instructions in which the set of attributes for the corresponding file system object are determined by issuing, from the external node, a request to obtain from the source file system the set of attributes for the corresponding file system object, then forwarding the file system operation of the corresponding file system object to the source file system once the entry for the corresponding file system object has been added to the data structure.

16. The non-transitory computer-readable medium of claim 14, further comprising instructions in which each identified file system object and the corresponding data object are replicated asynchronously.

17. The non-transitory computer-readable medium of claim 14, wherein the instructions provide for determining a unique identifier for each file system object that is not a handle for the file system object.

18. The non-transitory computer-readable medium of claim 14, wherein the instructions for determining the unique identifier include determining the unique identifier of each file system object from a combination of attributes of that file system object.

19. The non-transitory computer-readable medium of claim 14, wherein instructions for replicating at least the portion of the source file system includes instructions for replicating (i) each file system object of the collection, and (ii) an organizational structure of the file system objects that comprise the at least portion of the source file system.

20. The non-transitory computer-readable medium of claim 14, wherein instructions for replicating at least the portion of the source file system includes instructions for asynchronously replicating individual file system objects that comprise at least the portion of the source file system.

21. The non-transitory computer-readable medium of claim 14, further comprising instructions for performing operations that include:

re-replicating the corresponding data object at the destination file system if the set of attributes for the replication are out of date.

22. The non-transitory computer-readable medium of claim 21, wherein the operation of re-replicating the corresponding data object includes re-starting replication of the source file system after a pause.

23. A non-transitory computer-readable medium that stores instructions for replicating an active file system, the instructions being executable by one or more processors to cause the one or more processors to perform operations that include:

identifying a collection of file system objects that are associated with a source file system that is actively in use by a plurality of clients;

intercepting individual file system operations that are intended to be handled by the source file system at a location that is in-line and external to the source file system, so as to not disrupt the plurality of clients requesting file system operations to be performed on the source file system;

for at least a duration of time during which the source file system is in active use by the plurality of clients,

replicating at least a portion of the source file system, including each file system object of the collection, at a destination file system;

determining when individual file system operations after the source file system; and

asynchronously implementing one or more of the individual file system operations that are determined to alter the source file system on the destination file system, wherein asynchronously implementing one or more of the individual file system operations includes (i) generating a replay log that sequences multiple file system operations corresponding to a create type operation, a write type operation or a modification type operation, but not a read type operation, and (ii) asynchronously implementing the multiple file system operations from the replay log.