System and method for determining changes in two snapshots and for transmitting changes to a destination snapshot

• US 7,818,299 B1
• Filed: 09/02/2009
• Issued: 10/19/2010
• Est. Priority Date: 03/19/2002
• Status: Expired due to Fees

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Owara, Shane S., Federwisch, Michael L., Manley, Stephen L., Kleiman, Steven R.
• Primary Examiner(s): Mizrahi; Diane
• Application Number:
  US12/552,346
• Time in Patent Office: 412 Days
• Field of Search: 707/620, 707/640, 707/679, 707655-667
• US Class Current: 707/649
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/2066 Optimisation of the communication load
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99954 Version management

Abstract

A method for backing up a computer file system is disclosed. A first snapshot of the file system is generated at a first time, the first snapshot having first pointers to data blocks of the file system at the first time. A network adapter receives storage operations to write new data to the file system. A second snapshot of the file system is generated at a second time, the second snapshot having second pointers to data blocks of the file system at the second time, the data blocks at the second time including the new data. The first pointers to the second pointers are compared. In response to the second pointers differing from the first pointers, data of the file system which have been changed between the first snapshot and the second snapshot are copied to a backup copy of the file system.

Claims

1. A method for backing up a computer file system, comprising:

generating a first snapshot of the file system at a first time, the first snapshot having first pointers to data blocks of the file system at the first time;

receiving, by a network adapter, storage operations to write new data to the file system;

generating a second snapshot of the file system at a second time, the second snapshot having second pointers to data blocks of the file system at the second time, the data blocks at the second time including the new data;

comparing the first pointers to the second pointers; and

copying, in response to the second pointers differing from the first pointers, data of the file system which have been changed between the first snapshot and the second snapshot to a backup copy of the file system.

2. The method as in claim 1, further comprising:

storing the backup copy in a persistent memory of a backup computer.

3. The method as in claim 1, further comprising:

scanning a logical file block index of each snapshot to detect changed pointers in order to determine a one or more blocks which have changed between the first snapshot and the second snapshot.

4. The method as in claim 3, further comprising:

adapting the scanner to walk down a hierarchy of pointers associated with each logical block file index and retrieve only blocks with volume block numbers that are different in the second snapshot from the volume block numbers in the first snapshot while bypassing blocks having unchanged volume block numbers, in order to retrieve only data blocks which have changed between the first snapshot and the second snapshot.

5. The method of claim 1, further comprising:

traversing trees of blocks associated with a file by walking down the tree to identify a change in a hierarchy of the tree.

6. The method of claim 1, further comprising:

transmitting a changed block to a snapshot.

7. The method of claim 1, further comprising:

bypassing unchanged pointers between the first snapshot and the second snapshot.

8. The method of claim 1, further comprising:

scanning an index of logical file blocks for the first snapshot and the second snapshot looking for changed volume block numbers.

9. The method of claim 1, further comprising:

detecting changed blocks by locating changed inodes.

10. The method of claim 1, further comprising:

overlooking unchanged blocks by locating unchanged inodes.

11. The method of claim 1, further comprising:

determining when inodes are the same between the first snapshot and second snapshot but the files have changed as indicated by their version numbers.

12. The method of claim 1, further comprising:

scanning the changed inodes from the first snapshot and the second snapshot to locate data blocks which have changed.

13. The method of claim 12, further comprising:

identifying changed blocks by a block pointer difference.

14. The apparatus of claim 12, further comprising:

a process to identify changed blocks by a block pointer difference.

15. The method of claim 1, further comprising:

scanning a file offset down a tree of an inode until a difference in an underlying block is identified.

16. The method of claim 1, further comprising:

transferring a change in the first snapshot to the backup copy of the file system by an asynchronous write (lazy write) procedure.

17. The method of claim 1, further comprising:

transferring a data stream of change data over a computer network using a set of standalone headers with unique identifiers.

18. A method for backing up a computer file system, comprising:

generating a first snapshot of the file system at a first time, the first snapshot having first pointers to data blocks of the file system at the first time;

receiving, by a network adapter, storage operations to write new data to the file system;

generating a second snapshot of the file system at a second time, the second snapshot having second pointers to data blocks of the file system at the second time, the data blocks at the second time including the new data;

scanning a logical file block index of each snapshot to detect changed pointers in order to determine a one or more blocks which have changed between the first snapshot and the second snapshot; and

copying, in response to the detecting changed pointers, data of the file system which have been changed between the first snapshot and the second snapshot to a backup copy of the file system.

19. A method for backing up a computer file system, comprising:

generating a first snapshot of the file system at a first time, the first snapshot having first pointers to data blocks of the file system at the first time;

receiving, by a network adapter, storage operations to write new data to the file system;

generating a second snapshot of the file system at a second time, the second snapshot having second pointers to data blocks of the file system at the second time, the data blocks at the second time including the new data;

scanning a logical file block index of each snapshot to detect changed pointers in order to determine a one or more blocks which have changed between the first snapshot and the second snapshot;

walking the scanner down a hierarchy of pointers associated with each logical block file index and retrieve only blocks with volume block numbers that are different in the second snapshot from the volume block numbers in the first snapshot while bypassing blocks having unchanged volume block numbers, in order to retrieve only data blocks which have changed between the first snapshot and the second snapshot; and

copying, in response to locating volume block numbers in the second snapshot that are different from volume block numbers in the first snapshot, data of the file system which have been changed between the first snapshot and the second snapshot to a backup copy of the file system.

20. A computer file system apparatus, comprising:

a first snapshot of the file system generated at a first time, the first snapshot having first pointers to data blocks of the file system at the first time;

a network adapter to receive storage operations to write new data to the file system;

a second snapshot of the file system generated at a second time, the second snapshot having second pointers to data blocks of the file system at the second time, the data blocks at the second time including the new data; and

a backup copy of the file system to receive copies of data of the file system which have been changed between the first snapshot and the second snapshot.

21. The apparatus as in claim 20, further comprising:

a persistent memory of a backup computer to store the backup copy of the file system.

22. The apparatus as in claim 20, further comprising:

a logical file block index of each snapshot scanned to detect changed pointers in order to determine a one or more blocks which have changed between the first snapshot and the second snapshot.

23. The apparatus as in claim 22, further comprising:

the scanner adapted to walk down a hierarchy of pointers associated with each logical block file index and retrieve only blocks with volume block numbers that are different in the second snapshot from the volume block numbers in the first snapshot while bypassing blocks having unchanged volume block numbers, in order to retrieve only data blocks which have changed between the first snapshot and the second snapshot.

24. The apparatus of claim 20, further comprising:

trees of blocks associated with a file traversed by walking down the tree to identify a change in a hierarchy of the tree.

25. The apparatus of claim 24, further comprising:

unchanged pointers between the first snapshot and the second snapshot bypassed during walking down the tree.

26. The apparatus of claim 20, further comprising:

a changed block transmitted to a snapshot.

27. The apparatus of claim 20, further comprising:

an index of logical file blocks for the first snapshot and the second snapshot scanned to look for changed volume block numbers.

28. The apparatus of claim 20, further comprising:

changed blocks detected by locating changed inodes.

29. The apparatus of claim 20, further comprising:

unchanged blocks overlooked by locating unchanged inodes.

30. The apparatus of claim 20, further comprising:

a process to determine when inodes are the same between the first snapshot and second snapshot but the files have changed as indicated by their version numbers.

31. The apparatus of claim 30, further comprising:

a process to scan the changed inodes from the first snapshot and the second snapshot to locate data blocks which have changed.

32. The apparatus of claim 20, further comprising:

a process to scan a file offset down a tree of an inode until a difference in an underlying block is identified.

33. The apparatus of claim 20, further comprising:

a process to transfer a change in the first snapshot to the backup copy of the file system by an asynchronous write (lazy write) procedure.

34. The apparatus of claim 20, further comprising:

a process to transfer a data stream of change data over a computer network using a set of standalone headers with unique identifiers.

35. An apparatus for backing up a computer file system, comprising:

a first snapshot of the file system generated at a first time, the first snapshot having first pointers to data blocks of the file system at the first time;

a network adapter to receive storage operations to write new data to the file system;

a second snapshot of the file system generated at a second time, the second snapshot having second pointers to data blocks of the file system at the second time, the data blocks at the second time including the new data;

a logical file block index of each snapshot scanned to detect changed pointers in order to determine a one or more blocks which have changed between the first snapshot and the second snapshot; and

a process to copy, in response to the detecting changed pointers, data of the file system which have been changed between the first snapshot and the second snapshot to a backup copy of the file system.

36. An apparatus for backing up a computer file system, comprising:

a first snapshot of the file system at a first time, the first snapshot having first pointers to data blocks of the file system at the first time;

a network adapter to receive storage operations to write new data to the file system;

a second snapshot of the file system generated at a second time, the second snapshot having second pointers to data blocks of the file system at the second time, the data blocks at the second time including the new data;

a logical file block index of each snapshot scanned to detect changed pointers in order to determine a one or more blocks which have changed between the first snapshot and the second snapshot;

a process to walk a scanner down a hierarchy of pointers associated with each logical block file index and retrieve only blocks with volume block numbers that are different in the second snapshot from the volume block numbers in the first snapshot while bypassing blocks having unchanged volume block numbers, in order to retrieve only data blocks which have changed between the first snapshot and the second snapshot; and

a process to copy, in response to locating volume block numbers in the second snapshot that are different from volume block numbers in the first snapshot, data of the file system which have been changed between the first snapshot and the second snapshot to a backup copy of the file system.

37. A computer readable storage media, comprising:

said computer readable storage media containing program instructions for execution on a processor for a method of backing up a computer file system, the program instructions for,

generating a first snapshot of the file system at a first time, the first snapshot having first pointers to data blocks of the file system at the first time;

receiving, by a network adapter, storage operations to write new data to the file system;

generating a second snapshot of the file system at a second time, the second snapshot having second pointers to data blocks of the file system at the second time, the data blocks at the second time including the new data;

comparing the first pointers to the second pointers; and

copying, in response to the second pointers differing from the first pointers, data of the file system which have been changed between the first snapshot and the second snapshot to a backup copy of the file system.