Systems and methods for retaining and using data block signatures in data protection operations

• US 10,126,973 B2
• Filed: 03/29/2017
• Issued: 11/13/2018
• Est. Priority Date: 09/30/2010
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Vijayan, Manoj Kumar, Attarde, Deepak Raghunath
• Primary Examiner(s): Krofcheck, Michael
• Application Number:
  US15/472,737
• Publication Number:
  US 20170199699A1
• Time in Patent Office: 594 Days
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1466 to make the backup process non-disruptive
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices

Abstract

A system according to certain embodiments associates a signature value corresponding to a data block with one or more data blocks and a reference to the data block to form a signature/data word corresponding to the data block. The system further logically organizes the signature/data words into a plurality of files each comprising at least one signature/data word such that the signature values are embedded in the respective file. The system according to certain embodiments reads a previously stored signature value corresponding to a respective data block for sending from a backup storage system having at least one memory device to a secondary storage system. Based on an indication as to whether the data block is already stored on the secondary storage system, the system reads the data block from the at least one memory device for sending to the secondary storage system if the data block does not exist on the secondary storage system, wherein the signature value and not the data block is read from the at least one memory device if the data block exists on the secondary storage system.

Claims

1. A method of performing a copy operation, the method comprising:

copying at least a first file comprising data blocks from a first storage device to a second storage device to create a secondary copy of the file comprising secondary copy data blocks,

embedding deduplication signatures for the secondary copy data blocks in the secondary copy of the first file, wherein the embedded deduplication signatures are physically or logically contiguously embedded in the secondary copy of the file with respect to the secondary copy data blocks; and

in response to a command to copy one or more of the secondary copy data blocks to a third storage device;

comparing the embedded deduplication signatures to deduplication signatures stored at the third storage device without re-generating the values of the embedded deduplication signatures;

if the comparison indicates that a copy of the one or more of the secondary copy data blocks is already stored on the third storage device, storing a reference to the copy of the one or more secondary copy data blocks stored on the third storage device instead of transmitting the one or more secondary copy data blocks to the third storage device; and

if the comparison indicates that the copy of the one or more of the secondary copy data blocks is not stored on the third storage device, transmitting the one or more secondary copy data blocks to the third storage device.

2. The method of claim 1, wherein the one or more secondary copy data blocks is not read from the second storage device if the embedded deduplication signatures indicates that the one or more secondary copy data blocks exist on the third storage device.

3. The method of claim 1, wherein the deduplication signatures are embedded into the secondary copy of the first file when copying the first file to the second storage device.

4. The method of claim 1, wherein the deduplication signatures are stored in physically contiguous memory space with the secondary copy data blocks.

5. The method of claim 1, wherein the deduplication signatures are logically associated with the secondary copy data blocks.

6. The method of claim 1, wherein the third storage device maintains a deduplication signature table associated with the copy of the one or more secondary copy data blocks that exist on the third storage device.

7. The method of claim 1, wherein copying the one or more secondary storage data blocks from the second storage device to the third storage device is part of an auxiliary copy operation.

8. The method of claim 1, wherein a copy of the embedded deduplication signatures are stored in a deduplication signature table.

9. The method of claim 1, wherein the secondary storage data blocks are at least 512 times larger than corresponding embedded deduplication signatures.

10. The method of claim 1, wherein the copy of the one or more of the secondary copy data blocks are stored in association with the embedded deduplication signatures on the third storage device.

11. A system that performs a copy operation, the system comprising:

a first storage device that stores at least a first file comprising a plurality of data blocks;

a second storage device that stores a secondary copy of the first file comprising secondary copy data blocks that are a copy of the data blocks on the first storage device;

a plurality of embedded deduplication signatures in the secondary copy of the first file, wherein the plurality of embedded deduplication signatures are physically or logically contiguously embedded in the secondary copy of the file with respect to the secondary copy data blocks; and

a third storage device that stores a copy of one or more of the secondary copy data blocks copied from the second storage device to the third storage device;

computer hardware implementing a copy manager that directs;

comparing the embedded deduplication signatures to deduplication signatures stored at the third storage device without re-generating the values of the embedded deduplication signatures;

if the comparison indicates that a copy of the one or more of the secondary copy data blocks is already stored on the third storage device, storing a reference to the copy of the one or more secondary copy data blocks stored on the third storage device instead of transmitting the one or more secondary copy data blocks to the third storage device; and

if the comparison indicates that the copy of the one or more of the secondary copy data blocks is not stored on the third storage device, transmitting the one or more secondary copy data blocks to the third storage device.

12. The system of claim 11, wherein the one or more secondary copy data blocks is not read from the second storage device if the embedded deduplication signatures sent to the third storage device indicates that the one or more secondary copy data blocks exist on the third storage device.

13. The system of claim 11, wherein the deduplication signatures are embedded into the secondary copy of the first file when copying the first file to the second storage device.

14. The system of claim 11, wherein the deduplication signatures are stored in physically contiguous memory space with the secondary copy data blocks.

15. The system of claim 11, wherein the deduplication signatures are logically associated with the secondary copy data blocks.

16. The system of claim 11, further comprising a deduplication signature table associated with the copy of the one or more secondary copy data blocks that exist on the third storage device.

17. The system of claim 11, wherein copying the one or more secondary storage data blocks from the second storage device to the third storage device is part of an auxiliary copy operation.

18. The system of claim 11, wherein a copy of the embedded deduplication signatures are stored in a deduplication signature table.

19. The system of claim 11, wherein the secondary storage data blocks are at least 512 times larger than corresponding embedded deduplication signatures.

20. The system of claim 11, wherein the copy of the one or more of the secondary copy data blocks are stored in association with the stored deduplication signatures on the third storage device.