System and method for providing continuous data protection

• US 7,769,723 B2
• Filed: 04/28/2006
• Issued: 08/03/2010
• Est. Priority Date: 04/28/2006
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: English, Robert M., Manley, Stephen L., Zheng, Ling, Lewis, Blake H.
• Primary Examiner(s): Vital; Pierre M
• Assistant Examiner(s): Rostami; Mohammad S
• Application Number:
  US11/413,882
• Publication Number:
  US 20070276878A1
• Time in Patent Office: 1,558 Days
• Field of Search: None
• US Class Current: 707/682
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 16/125 characterised by the use of retention policies retention policies for HSM systems G06F16/185

Abstract

A system and method provides continuous data protection using checkpoints in a write anywhere file system. During a consistency point of a write anywhere file system, freed blocks are identified and are appended to a delete log for retention. A consistency point log is updated with a new entry associated with the consistency point. If the file system needs to retrieve its state at a particular point in time, the stored blocks of the delete log may be recovered.

Claims

1. A method for providing continuous data protection in a file system, comprising:

generating one or more persistent consistency point images (PCPIs) of the file system, wherein each PCPI shares one or more file system blocks with at least one other generated PCPI;

executing a consistency point during which data changes contained in memory are flushed to persistent storage and one or more data blocks on the persistent storage are freed;

performing write allocation of the freed blocks to a delete log to prevent the freed blocks from being reclaimed for reuse by the file system;

allowing any of the freed blocks that are subsequently freed from the delete log to be reclaimed by the file system for reuse during future write allocation;

updating a consistency point log to list blocks which have been freed during execution of the consistency point, wherein the consistency point log comprises a header and at least one entry for each consistency point logged in the consistency point log; and

appending a new entry to the consistency point log and updating the header of the consistency point log.

2. The method of claim 1 further comprising:

identifying the one or more freed blocks freed during the consistency point; and

transferring the one or more identified freed blocks to a data container.

3. The method of claim 1 wherein the header comprises a total blocks retained field.

4. The method of claim 1 wherein the persistent storage comprises a nonvolatile random access memory.

5. The method of claim 1 wherein performing write allocation further comprises modifying one or more pointers within the delete log to point to the freed blocks.

6. The method of claim 1 wherein the delete log comprises a data container of the freed blocks of the file system.

7. The method of claim 1 further comprising accessing the one or more freed blocks by specifying a timestamp in an access interface.

8. The method of claim 7 wherein the timestamp is associated with a pathname.

9. The method of claim 7 wherein the timestamp is encoded in a file handle to enable access to data stored in the one or more freed blocks.

10. The method of claim 1 further comprising restoring the file system to an earlier point in time by resetting a root block of an active file system to a root block of the earlier point in time, wherein the root block address is saved in the consistency point log.

11. The method of claim 1 further comprising converting the file system with continuous data protection to a file system without continuous data protection by deleting the consistency point log and the delete log.

12. The method of claim 1 further comprising:

detecting that a retention policy threshold has been exceeded;

identifying one or more blocks associated with one or more consistency points exceeding the retention policy threshold; and

freeing the identified block for reuse by the file system.

13. A computer data storage system configured to provide continuous data protection in a file system, comprising:

means for generating one or more persistent consistency point images (PCPIs) of the file system, wherein each PCPI shares one or more file system blocks with at least one other generated PCPI;

means for executing a consistency point during which data changes contained in memory are flushed to persistent storage and one or more data blocks on the persistent storage are freed;

means for performing write allocation of the freed blocks to a delete log to prevent the freed blocks from being reclaimed for reuse by the file system;

means for allowing any of the freed blocks that are subsequently freed from the delete log to be reclaimed by the file system for reuse during future write allocation;

means for updating a consistency point log to list blocks which have been freed during execution of the consistency point, wherein the consistency point log comprises a header and at least one entry for each consistency point logged in the consistency point log; and

means for appending a new entry to the consistency point log and means for updating the header of the consistency point log.

14. The system of claim 13 further comprising:

means for identifying the one or more freed blocks freed during the consistency point; and

means for transferring the one or more identified freed blocks to a data container.

15. The system of claim 13 wherein the header comprises a total blocks retained field.

16. The system of claim 13 wherein the persistent storage comprises a nonvolatile random access memory.

17. The system of claim 13 wherein the means for performing write allocation further comprises means for modifying one or more pointers within the delete log to point to the freed blocks.

18. The system of claim 13 wherein the delete log comprises a data container of the freed blocks of the file system.

19. The system of claim 13 further comprising:

means for detecting that a retention policy threshold has been exceeded;

means for identifying one or more blocks associated with one or more consistency points exceeding the retention policy threshold; and

means for freeing the identified block for reuse by the file system.

20. A computer readable storage medium containing executable program instructions executed by a processor, comprising:

program instructions that generate one or more persistent consistency point images (PCPIs) of a file system, wherein each PCPI shares one or more file system blocks with at least one other generated PCPI;

program instructions that execute a consistency point during which data changes contained in memory are flushed to persistent storage and one or more data blocks on the persistent storage are freed;

program instructions that perform write allocation of the freed blocks to a delete log to prevent the freed blocks from being reclaimed for reuse by the file system;

program instructions that allow any of the freed blocks that are subsequently freed from the delete log to be reclaimed by the file system for reuse during future write allocation;

program instructions that update a consistency point log to list blocks which have been freed during execution of the consistency point, wherein the consistency point log comprises a header and at least one entry for each consistency point logged in the consistency point log; and

program instructions that append a new entry to the consistency point log and program instructions that update the header of the consistency point log.

21. A computer data storage system configured to provide continuous data protection in a file system, comprising:

a consistency point log stored within the file system configured to list a set of blocks which have been freed during execution of a consistency point, wherein the consistency point log comprises a header and at least one entry for each consistency point logged in the consistency point log and further configured to append a new entry to the consistency point log and update the header of the consistency point log;

a delete log comprising the set of blocks freed within the file system; and

a file system module implementing the file system and configured to identify the set of blocks freed during the consistency point.

22. The system of claim 21 wherein the file system is further configured to update the consistency point log with an entry associated with the consistency point.

23. The system of claim 21 wherein the file system is further configured to perform write allocation of the set of blocks freed to the delete log, wherein the identified set of blocks freed are not reclaimed by the file system module for reuse.

24. The system of claim 21 further comprising one or more pointers within the delete log configured to point to one or more freed blocks to perform write allocation.

25. The system of claim 21 wherein the header comprises a total blocks retained field.

26. A method for providing continuous data protection in a storage system, comprising:

updating a consistency point log to list blocks which have been freed during execution of a consistency point, wherein the consistency point log comprises a header and at least one entry for each consistency point logged in the consistency point log;

appending a new entry to the consistency point log and updating the header of the consistency point log;

tracking the blocks freed within the storage system by performing write allocation of the freed blocks to a delete log to prevent the freed blocks from being reclaimed for reuse by the storage system; and

releasing the blocks for reuse by the storage system in accordance with a user retention policy by subsequently freeing the released blocks from the delete log.

27. The method of claim 26 wherein tracking the blocks freed within the storage system further comprises:

identifying one or more of the freed blocks freed during the consistency point; and

transferring the one or more identified freed blocks to a data container.

28. The method of claim 27 further comprising modifying one or more pointers within the delete log to point to one or more freed blocks.

29. The method of claim 26 further comprising:

detecting that a user retention policy threshold has been exceeded;

identifying one or more of the freed blocks associated with one or more consistency points is exceeding the retention policy threshold; and

freeing the identified one or more freed blocks for reuse by the storage system.

30. The method of claim 29 wherein freeing the identified one or more freed blocks for reuse occurs in a first in first out manner in a data container.

31. A method for providing continuous data protection in a storage system, comprising:

logging received data operations in a persistent memory;

replacing data in an operation log with pointers to data blocks written in a checkpoint;

storing, during the course of write allocation, the received write operations in a data operations log;

updating the data operations log to list blocks which have been freed during execution of a consistency point, wherein the data operations log comprises a header and at least one entry for each consistency point logged in the data operations log; and

appending a new entry to the data operations log and updating the header of the data operations log.

32. The method of claim 31 further comprising:

replaying, to access data at a time between two checkpoints, relevant logged operations against an earlier checkpoint to recreate a set of desired data.

33. The method of claim 31 further comprising replicating checkpoints from a local storage system to a remote storage system.

34. The method of claim 31 wherein the persistent memory comprises a nonvolatile random access memory.

35. A method for providing continuous data protection in a storage system, comprising:

retaining data written to the storage system;

permitting access, by a user, to earlier data written to the storage system;

performing write allocation of a freed block to a delete log to prevent the freed block from being reclaimed for reuse by the storage system;

updating a consistency point log to list blocks which have been freed during execution of a consistency point, wherein the consistency point log comprises a header and at least one entry for each consistency point logged in the consistency point log;

appending a new entry to the consistency point log and updating the header of the consistency point log; and

expiring earlier data based on a pre-determined retention policy.

36. A method for providing continuous data protection in a storage system, comprising:

generating one or more persistent consistency point images (PCPIs) of the storage system, wherein each PCPI shares one or more storage system blocks with at least one other generated PCPI;

maintaining, in memory, modifications made to storage system blocks;

executing a consistency point, whereby the modifications are flushed to persistent storage;

freeing, in response to the executed consistency point, one or more data blocks on the persistent storage;

adding, to a delete log, the one or more freed data blocks by inserting pointers of the freed blocks into an inode buffer tree of the delete log;

updating a consistency point log to list the freed blocks which have been freed during execution of the consistency point, wherein the consistency point log comprises a header and at least one entry for each consistency point logged in the consistency point log; and

appending a new entry to the consistency point log and updating the header of the consistency point log.

37. A method for providing continuous data protection in a storage system, comprising:

generating one or more persistent consistency point images (PCPIs) of the storage system, wherein each PCPI shares one or more storage system blocks with at least one other generated PCPI;

maintaining, in a first memory, modified storage system blocks;

executing a consistency point, whereby the modifications are flushed to persistent storage;

freeing, in response to the executed consistency point, one or more data blocks on the persistent storage;

retaining, in one or more blocks of a delete log, the one or more freed data blocks, wherein at least one block of the delete log corresponds to the one or more data blocks previously freed during the consistency point;

updating a consistency point log to list blocks which have been freed during execution of the consistency point, wherein the consistency point log comprises a header and at least one entry for each consistency point logged in the consistency point log; and

appending a new entry to the consistency point log and updating the header of the consistency point log.

38. A computer readable storage medium containing executable program instructions executed by a processor comprising:

program instructions that generate one or more persistent consistency point images (PCPIs) of a storage system, wherein each PCPI shares one or more storage system blocks with at least one other generated PCPI;

program instructions that maintain in a first memory, modifications made to storage system blocks;

program instructions that execute a consistency point, whereby the modifications in the first memory are flushed to a second memory;

program instructions that free in response to the executed consistency point, one or more data blocks on the persistent storage;

program instructions that add to a delete log, the one or more freed data blocks by inserting pointers of the freed blocks into an inode buffer tree of the delete log;

program instructions that update a consistency point log to list the one or more freed data blocks which have been freed during execution of the consistency point, wherein the consistency point log comprises a header and at least one entry for each consistency point logged in the consistency point log; and

program instructions that append a new entry to the consistency point log and program instructions that update the header of the consistency point log.

39. A method for providing continuous data protection in a file system, comprising:

generating one or more persistent consistency point images (PCPIs) of the file system, wherein each PCPI shares one or more file system blocks with at least one other generated PCPI;

maintaining, in non-volatile random access memory (NVRAM), modifications made to file system blocks;

executing a consistency point, whereby the modifications in the NVRAM are flushed to persistent storage other than NVRAM;

freeing, in response to the executed consistency point, one or more data blocks on the persistent storage;

adding, to a delete log, the one or more freed data blocks by inserting pointers of the one or more freed blocks into an inode buffer tree of the delete log;

updating a consistency point log to list the one or more freed blocks which have been freed during execution of the consistency point, wherein the consistency point log comprises a header and at least one entry for each consistency point logged in the consistency point log; and

appending a new entry to the consistency point log and updating the header of the consistency point log.