System and method for asynchronous mirroring of snapshots at a destination using a purgatory directory and inode mapping

• US 20030182325A1
• Filed: 03/19/2002
• Published: 09/25/2003
• Est. Priority Date: 03/19/2002
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Manley, Stephen L., Owara, Shane S.
• Granted Patent: US 7,225,204 B2
• Time in Patent Office: Days
• US Class Current: 707/204
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/2066 Optimisation of the communication load
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

A system and method for updating a replicated destination file system snapshot with changes in a source file system snapshot, facilitates construction of a new directory tree on the destination from source update information using a temporary or “purgatory” directory that allows any modified and deleted files on the destination active file system to be associated with (e.g. moved to) the purgatory directory if and until they are reused. In addition, an inode map is established on the destination that maps source inode numbers to destination inode numbers so as to facilitate building of the destination tree using inode/generation number tuples. The inode map allows resynchronization of the source file system to the destination. The inode map also allows association of two or more destination snapshots to each other based upon their respective maps with the source.

Claims

1. A system for receiving a data stream of changed data from a snapshot on the source file system and updating a replica file system on a destination file system associated with the source file system comprising:

a directory stage process that reads all source directory information, and that (a) maps destination inodes on the replica file system to source inodes received from the source and using an inode map of source inodes to destination inodes and (b) updates the destination inodes based upon the mapped source inodes; and

a file stage process that populates the replica file system with file data from the source based upon offsets in the inode map between the source inodes and the destination modes.

2. The system as set forth in claim 1 wherein the directory stage is adapted to remove deleted files and modified files from the replica file system prior to the file stage.

3. The system as set forth in claim 2 wherein the directory stage process is adapted to create new entries in the inode map and corresponding new inodes in the replica file system from respective source inodes based upon an absence of reference to the source inodes in the inode map.

4. The system as set forth in claim 3 wherein the directory stage is adapted to move inodes having entries in the inode map that correspond to source inodes, indicating deletion and modification of files represented by the inodes, to a temporary directory from the replica file system.

5. The system as set forth in claim 4 wherein the directory stage includes a tree building process adapted to link inodes in the temporary directory that are reused by the replicated file system to the replicated file system.

6. The system as set forth in claim 5 wherein the temporary directory is adapted to be deleted with all unlinked inodes after the file stage process is complete.

7. The system as set forth in claim 1 wherein the snapshot on the source file system relates to a qtree on the source file system.

8. A method for rolling back an updating replica file system to a first state from a second state comprising the steps of:

creating a first exported snapshot of the replica file system in the first state;

initiating a roll back procedure at a predetermined time after the creation of the first exported snapshot;

halting modification of the replica file system in the second state;

creating a second exported snapshot of the replica file system in the second state;

determining changes in data between the second snapshot and the first snapshot; and

modifying the replica file system, after the step of halting, using the changes to restore the replica file system to the first state.

9. In a system wherein an old source snapshot transmits incremental snapshot changes in inodes to an old destination snapshot for update of the old destination snapshot, and wherein the inodes of the old source snapshot are mapped to inodes of the old destination snapshot with an old inode map indexed according to the old source, a method for flipping the old inode map for use by the old source snapshot as a new destination to receive snapshot changes in inodes from the old destination snapshot as a new source, the method comprising the steps of:

negotiating receipt of the old inode map at the new destination and storing the old inode map;

creating a new inode map having an index of entries defined by an ascending order of new source inode numbers equal to a number of entries in the new source;

for each new source inode number, respectively, looking up, in the stored old inode map, an entry that contains an old destination inode number, if any, matching the new source inode number; and

writing a corresponding old source inode number from the looked up entry in the stored old inode map as a new destination inode in the new inode map at the entry indexed by the respective new source inode number.

10. The method as set forth in claim 9 wherein the step of writing includes, where an entry that contains the old destination number is absent from the stored old inode map, writing an empty value as a new destination inode in the new inode map at the entry indexed by the respective new source inode number.

11. The method as set forth in claim 10 further comprising looking up and writing in the entry indexed by the respective new source inode number an associated file generation number relative to the new source and the new destination.

12. In a system wherein an old source snapshot transmits incremental snapshot changes in inodes to a first old destination snapshot and a second old destination snapshot for update of the old destination snapshot, and wherein the inodes of the old source snapshot are mapped to inodes of each of the first old destination snapshot and the second old destination snapshot with a respective first old inode map and a second old inode map each indexed according to the old source, a method for mirroring the first old destination snapshot as a new source snapshot on the second old destination snapshot as a new destination snapshot, the method comprising the steps of:

transferring the first old inode map from the new source to the new destination;

walking through index entries to inodes in the old source in each of the first old inode map and the second old inode map concurrently and, for each inode from the old source extracting an old destination inode number/generation number”

from each of the first old inode map and the second old inode map; and

creating a new associated inode map indexed with each old destination inode number/generation number extracted from the first old source map and associated with the old destination inode number/generation number from the second old source map.

14. The method as set forth in claim 12 wherein the step of reading inodes related to deleted files and modified files includes reading entries in an inode map that each have respective source inode numbers and source generation numbers for respective files and corresponding destination inode numbers and destination generation numbers for the respective files.

13. A method for receiving a data stream of changed data from a snapshot on the source file system and updating a replica file system on a destination file associated with the source file system comprising:

reading, from the data stream, inodes related to deleted files and modified files on the replica file system and associating any read deleted files and modified files on the replica file system with a temporary directory separate from a directory tree of the replicated file system;

reading, from the data stream, information retaining predetermined of the deleted files and the modified files and, in response to the reading, linking the predetermined of the deleted files and the modified files to respective new entries in the replicated file system; and

when updating of the replica file system is complete, deleting the temporary directory, whereby the predetermined of the deleted files and modified files remain associated with an updated version of the replicated file system.

15. A method for receiving a data stream of changed data from a snapshot on the source file system and updating a replica file system on a destination file system associated with the source file system comprising:

performing a directory stage process that reads all source directory information, and that (a) maps destination inodes on the replica file system to source inodes received from the source and using an inode map of source inodes to destination inodes and (b) updates the destination inodes based upon the mapped source inodes; and

performing a file stage process that populates the replica file system with file data from the source based upon offsets in the inode map between the source inodes and the destination inodes.

16. The method as set forth in claim 15 wherein the step of performing the directory stage process includes removing deleted files and modified files from the replica file system prior to performing the file stage process.

17. The method as set forth in claim 16 wherein the step of performing the directory stage process includes creating new entries in the inode map and corresponding new inodes in the replica file system from respective source inodes based upon an absence of reference to the source inodes in the inode map.

18. The method as set forth in claim 17 wherein the step of performing the directory stage process includes moving modes having entries in the mode map that correspond to source modes, indicating deletion and modification of files represented by the inodes, to a temporary directory from the replica file system.

19. The method as set forth in claim 18 wherein the step of performing the directory stage includes performing a tree building process so as to link modes in the temporary directory that are reused by the replicated fi le system to the replicated file system.

20. The method as set forth in claim 19 further comprising deleting the temporary directory with all unlinked modes after the file stage process is complete.

21. The method as set forth in claim 15 wherein the snapshot on the source file system relates to a qtree on the source file system.

22. In a system for receiving a data stream of data source file system and updating a replica file system on a destination file system associated with the source file system, an inode map for mapping inodes received from the source file system an tree on the destination file system comprising:

an index of inode map entries based upon source inode numbers from the source file system received in the data stream;

source generation numbers, in each of entries, with respect to files associated with the source inode numbers; and

destination inode numbers and destination inode numbers, in each of the entries, mapped to respective of the source inode numbers.

23. The inode map as set forth in claim 22 further comprising added entries for each added file received from the source file system.

24. The inode map as set forth in claim 23 further comprising deleted entries for each deleted file from the source file system.

25. The inode map as set forth in claim 24 further comprising entries having a different source generation number with respect to the destination generation number for each modified file from the source system.

26. The inode map as set forth in claim 25 wherein the data stream comprises changed data with respect to a base snapshot and an incremental snapshot at the source file system.

27. The inode map as set forth in claim 26 wherein the base snapshot and the incremental snapshot each define a snapshot of a qtree on the source file system at discrete points in time.