Deduplicating snapshots associated with a backup operation

• US 10,162,555 B2
• Filed: 05/04/2018
• Issued: 12/25/2018
• Est. Priority Date: 06/13/2014
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Kandamuthan, Nirmala
• Primary Examiner(s): Thammavong, Prasith
• Assistant Examiner(s): Kwong, Edmund H
• Application Number:
  US15/971,675
• Publication Number:
  US 20180314454A1
• Time in Patent Office: 235 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0673 Single storage device

Abstract

Deduplicating snapshot associated with a backup operation is disclosed, including: performing a backup operation including by generating a plurality of snapshots; maintaining, at a source system, deduplication data corresponding to one or more data blocks that have already been written to backup media during the backup operation; and using the deduplication data to deduplicate backup data across the plurality of snapshots.

Claims

1. A system, comprising:

a processor configured to;

receive an indication to perform a backup operation on a plurality of storage areas of a source system;

in response to the indication, perform the backup operation including by generating a plurality of snapshots corresponding to respective ones of the plurality of storage areas associated with the backup operation;

maintain, at the source system, deduplication data corresponding to one or more data blocks that have already been written to backup media during the backup operation, wherein the deduplication data comprises a plurality of identifiers corresponding to respective ones of data blocks that have already been written to the backup media; and

use the deduplication data, at the source system, to deduplicate backup data across the plurality of snapshots associated with the backup operation, wherein to use the deduplication data comprises to compare, at the source system, an identifier associated with a data block to back up in a first snapshot included in the plurality of snapshots to the plurality of identifiers,

wherein in response to a determination that a matching identifier is not found in the plurality of identifiers;

determine, at the source system, that the data block has not already been written to the backup media;

send, from the source system, to a backup storage underlying data of the data block to be stored as an entry associated with the data block in the first snapshot at the backup media at the backup storage; and

send, from the source system, to the backup storage a metadata block corresponding to the data block, wherein the metadata block is to be stored in the first snapshot at the backup media at the backup storage, wherein the metadata block is configured to be used to determine to which file or directory, or both, the data block belongs; and

a memory coupled to the processor and configured to store the deduplication data.

2. The system of claim 1, wherein the plurality of snapshots is configured to be stored at the backup media.

3. The system of claim 1, wherein a storage area included in the plurality of storage areas comprises a volume of storage.

4. The system of claim 1, wherein the identifier associated with the data block comprises a disk block number.

5. The system of claim 1, wherein the backup operation comprises a full backup.

6. The system of claim 1, wherein the backup operation comprises an incremental backup.

7. The system of claim 1, wherein the deduplication data is configured to be deleted subsequent to completion of the backup operation.

8. The system of claim 1, wherein the processor is further configured to restore the first snapshot, including by reading the entry associated with the data block included in the first snapshot:

in response to a first determination that the entry includes underlying data of the data block;

restore stored data associated with the entry associated with the data block to the source system; and

use the stored metadata block corresponding to the data block to determine to which file, directory, or both the data block belongs; and

in response to a second determination that the entry associated with the data block includes a representation of the data block;

use the representation included in the entry to locate a location at the backup media to restore data stored at the location at the backup media to the source system; and

use the stored metadata block corresponding to the data block to determine to which file, directory, or both the data block belongs.

9. The system of claim 1, wherein the determination comprises a first determination and wherein, in response to a second determination that the matching identifier is included in the deduplication data, the processor is further configured to:

determine, at the source system, that the data block has already been written to the backup media; and

send, from the source system, to the backup storage a representation of the data block to be stored as the entry associated with the data block in the first snapshot on the backup media at the backup storage, wherein the representation of the data block comprises associating data to a location at the backup media to which the data block was previously written, wherein the representation of the data block is determined based at least in part on information stored in the deduplication data, wherein the data block was previously written to the location at the backup media for a second snapshot of the plurality of snapshots.

10. The system of claim 9, wherein the representation of the data block comprises at least one of a hard link or a soft link.

11. The system of claim 1, wherein the metadata block comprises an inode.

12. A method, comprising:

receiving an indication to perform a backup operation on a plurality of storage areas of a source system;

in response to the indication, performing the backup operation including by generating a plurality of snapshots corresponding to respective ones of the plurality of storage areas associated with the backup operation;

maintaining, at the source system, deduplication data corresponding to one or more data blocks that have already been written to backup media during the backup operation, wherein the deduplication data comprises a plurality of identifiers corresponding to respective ones of data blocks that have already been written to the backup media; and

using the deduplication data, at the source system, to deduplicate backup data across the plurality of snapshots associated with the backup operation, wherein to use the deduplication data comprises to compare, at the source system, an identifier associated with a data block to back up in a first snapshot included in the plurality of snapshots to the plurality of identifiers,

wherein in response to a determination that a matching identifier is not found in the plurality of identifiers;

determining, at the source system, that the data block has not already been written to the backup media;

sending, from the source system, to a backup storage underlying data of the data block to be stored as an entry associated with the data block in the first snapshot at the backup media at the backup storage; and

sending, from the source system, to the backup storage a metadata block corresponding to the data block, wherein the metadata block is to be stored in the first snapshot at the backup media at the backup storage, wherein the metadata block is configured to be used to determine to which file or directory, or both, the data block belongs.

13. The method of claim 12, wherein the identifier associated with the data block comprises a disk block number.

14. The method of claim 12, wherein the deduplication data is configured to be deleted subsequent to completion of the backup operation.

15. The method of claim 12, further comprising restoring the first snapshot, including by reading the entry associated with the data block included in the first snapshot:

in response to a first determination that the entry includes underlying data of the data block;

restoring stored data associated with the entry associated with the data block to the source system; and

using the stored metadata block corresponding to the data block to determine to which file, directory, or both the data block belongs; and

in response to a second determination that the entry associated with the data block includes a representation of the data block;

using the representation included in the entry to locate a location at the backup media to restore data stored at the location at the backup media to the source system; and

using the stored metadata block corresponding to the data block to determine to which file, directory, or both the data block belongs.

16. The method of claim 12, wherein the determination comprises a first determination and wherein, in response to a second determination that the matching identifier is included in the deduplication data, further comprising:

determining, at the source system, that the data block has already been written to the backup media; and

sending, from the source system, to the backup storage a representation of the data block to be stored as the entry associated with the data block in the first snapshot on the backup media at the backup storage, wherein the representation of the data block comprises associating data to a location at the backup media to which the data block was previously written, wherein the representation of the data block is determined based at least in part on information stored in the deduplication data, wherein the data block was previously written to the location at the backup media for a second snapshot of the plurality of snapshots.

17. The method of claim 16, wherein the representation of the data block comprises at least one of a hard link or a soft link.

18. The method of claim 12, wherein the metadata block comprises an inode.

19. A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving an indication to perform a backup operation on a plurality of storage areas of a source system;

in response to the indication, performing the backup operation including by generating a plurality of snapshots corresponding to respective ones of the plurality of storage areas associated with the backup operation;

maintaining, at the source system, deduplication data corresponding to one or more data blocks that have already been written to backup media during the backup operation, wherein the deduplication data comprises a plurality of identifiers corresponding to respective ones of data blocks that have already been written to the backup media; and

using the deduplication data, at the source system, to deduplicate backup data across the plurality of snapshots associated with the backup operation, wherein to use the deduplication data comprises to compare, at the source system, an identifier associated with a data block to back up in a first snapshot included in the plurality of snapshots to the plurality of identifiers,

wherein in response to a determination that a matching identifier is not found in the plurality of identifiers;

determining, at the source system, that the data block has not already been written to the backup media;

sending, from the source system, to a backup storage underlying data of the data block to be stored as an entry associated with the data block in the first snapshot at the backup media at the backup storage; and

sending, from the source system, to the backup storage a metadata block corresponding to the data block, wherein the metadata block is to be stored in the first snapshot at the backup media at the backup storage, wherein the metadata block is configured to be used to determine to which file or directory, or both, the data block belongs.

20. The computer program product of claim 19, further comprising computer instructions for:

in response to a first determination that the entry includes underlying data of the data block;

restoring stored data associated with the entry associated with the data block to the source system; and

using the stored metadata block corresponding to the data block to determine to which file, directory, or both the data block belongs; and

in response to a second determination that the entry associated with the data block includes a representation of the data block;

using the representation included in the entry to locate a location at the backup media to restore data stored at the location at the backup media to the source system; and

using the stored metadata block corresponding to the data block to determine to which file, directory, or both the data block belongs.