System and method for managing data deduplication of storage systems utilizing persistent consistency point images

• US 7,921,077 B2
• Filed: 06/29/2006
• Issued: 04/05/2011
• Est. Priority Date: 06/29/2006
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Ting, Daniel, DeStefano, John Frederick, Manley, Stephen L., Zheng, Ling
• Primary Examiner(s): Wu; Yicun
• Assistant Examiner(s): HASAN, SYED HAROON
• Application Number:
  US11/478,272
• Publication Number:
  US 20080005201A1
• Time in Patent Office: 1,741 Days
• Field of Search: None
• US Class Current: 707/610
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

A system and method for managing data deduplication of a storage system utilizing persistent consistency point images (PCPIs). Once a target PCPI of a data transfer is generated, a backup management module of the storage system alerts a data deduplication module to begin deduplication of the data contained within the target PCPI. Once the deduplication procedure has been completed, the active file system of the storage system has been deduplicated, however, the target PCPI remains un-deduplicated. In response, the backup management module generates and exports a revised target PCPI. The previous target PCPI may then be deleted, thereby transitioning the exported PCPI'"'"'s image of the state of the file system to a deduplicated state.

Claims

1. A method for managing data deduplication on a target storage system, comprising:

receiving a transfer of data from a source storage system to the target storage system;

generating a first persistent consistency point image of the transfer of data, the first persistent consistency point image comprising a first pointer referencing a first block of data and a second pointer referencing a second block of data in an active file system of the target storage system, wherein the first and the second blocks of data comprise identical data;

performing data deduplication on the active file system while the first persistent consistency point image comprises the first and the second pointers referencing the first and the second blocks of data in the active file system;

generating, in response to performing the data deduplication on the active file system, a second persistent consistency point image of the transfer of data, wherein the second persistent consistency point image comprises the second pointer referencing the first data block;

storing the second persistent consistency point image at the target storage system; and

deleting the first persistent consistency point image that comprises the second pointer referencing the second data block.

2. The method of claim 1 wherein receiving the transfer of data from the source storage system comprises receiving one or more data streams associated with a data collection to be backed up to the target storage system.

3. The method of claim 2 wherein the data collection comprises a volume.

4. The method of claim 2 wherein the data collection comprises a file system.

5. The method of claim 1 further comprising:

exporting the first persistent consistency point image to one or more clients.

6. The method of claim 1 wherein the second persistent consistency point image is exported to one or more clients so that data deduplication occurs transparently to the one or more clients.

7. The method of claim 1 wherein after the data deduplication of the transfer of data, each remaining block of the transferred data is unique.

8. A system configured to manage data deduplication, comprising:

a target storage system configured to execute a storage operating system, the storage operating system comprising a backup management module configured to receive a transfer of data from a source storage system, the target storage system further configured to generate a first persistent consistency point image of the transfer of data, the first persistent consistency point image comprising a pointer referencing duplicate data in an active file system of the target storage system;

a deduplication module configured to execute on the target storage system and further configured to perform data deduplication on the active file system while the first persistent consistency point image comprises the pointer referencing the duplicate data in the active file system;

the backup management module further configured to generate, in response to the deduplication module performing the data deduplication on the active file system, a second persistent consistency point image of the transfer of data;

the target storage system further configured to store the second persistent consistency point image; and

the backup management module further configured to delete the first persistent consistency point image that comprises the pointer referencing the duplicate data.

9. The system of claim 8 wherein the transferred data comprises a volume.

10. The system of claim 8 wherein the transferred data comprises a file system.

11. The system of claim 8 wherein the target storage system is further configured to export the second persistent consistency point image to the one or more clients.

12. The system of claim 8 wherein the backup management module is further configured to initiate a backup data transfer during the data deduplication.

13. The system of claim 12 wherein the deduplication module is further configured to yield to the backup management module.

14. The system of claim 8 wherein the backup management module is further configured to export the first persistent consistency point image to one or more clients.

15. A computer readable medium containing executable program instructions executed by a processor, comprising:

program instructions that receive a transfer of data from a source storage system to a target storage system;

program instructions that generate a first persistent consistency point image of the transfer of data, the first persistent consistency point image comprising a first pointer referencing a first block of data and a second pointer referencing a second block of data in an active file system of the target storage system,

wherein the first and the second blocks of data comprise identical data;

program instructions that perform data deduplication on the active file system while the first persistent consistency point image comprises the first and the second pointers referencing the first and the second data blocks in the active file system;

program instructions that generate, in response to performing the data deduplication on the active file system, a second persistent consistency point image of the transfer of data, wherein the second persistent consistency point image comprises the second pointer referencing the first data block;

program instructions that store the second persistent consistency point image at the target storage system; and

program instructions that delete the first persistent consistency point image that comprises the pointer referencing the second data block.

16. The computer readable medium of claim 15 further comprising program instructions that receive one or more data streams associated with a data collection to be backed up to the target storage system.

17. The computer readable medium of claim 16 wherein the data collection comprises a volume.

18. The computer readable medium of claim 16 wherein the data collection comprises a file system.

19. The computer-readable medium of claim 15 further comprising program instructions that export the second persistent consistency point image to one or more clients.

20. A system configured to manage data deduplication, comprising:

a target storage system configured to execute a storage operating system, the storage operating system comprising a backup management module configured to receive a transfer of data from a source storage system and further configured to generate a first persistent consistency point image of the transfer of data,

the first persistent consistency point image comprising a first pointer referencing a first block of data and a second pointer referencing a second block of data in an active file system of the target storage system, wherein the first and the second blocks of data comprise identical data;

a deduplication module configured to execute on the target storage system and further configured to perform data deduplication on the active file system while the first persistent consistency point image comprises the first and the second pointers referencing the first and the second blocks of data in the active file system;

the deduplication module further configured to determine whether another transfer of data has begun;

the backup management module further configured to generate, in response to determining that another transfer of data has not begun, a second persistent consistency point image of the transfer of data, wherein the second persistent consistency point image comprises the second pointer referencing the first data block; and

the backup management module further configured to delete the first persistent consistency point image that comprises the second pointer referencing the second data block.

21. The system of claim 20 wherein the transfer of data comprises a volume.

22. The system of claim 20 wherein the transfer of data comprises a file system.

23. A computer data storage system, comprising:

one or more source storage systems configured to transmit a first set of one or more data streams;

a backup storage system accessible to one or more clients, the backup storage system configured to receive the first set of one or more data streams; and

a processor configured to execute a storage operating system on the backup storage system, the storage operating system configured to;

generate a first persistent consistency point image of the backup storage system, the first persistent consistency point image comprising a first pointer referencing a first block of data and a second pointer referencing a second block of data in an active file system of the backup storage system, wherein the first and the second blocks of data comprise identical data;

deduplicate data of the first set of one or more data streams on the active file system while the first persistent consistency point image comprises the first and the second pointers referencing the first and the second blocks of data in the active file system;

receive a second set of one or more data streams prior to the completion of deduplication;

generate a second persistent consistency point image in response to completing the deduplication, wherein the second persistent consistency point image comprises the second pointer referencing the first data block; and

retain, in response to receiving the second set of one or more data streams prior to the completion of deduplication, the first persistent consistency point image.