Systems and methods for retaining and using data block signatures in data protection operations

• US 9,239,687 B2
• Filed: 09/27/2013
• Issued: 01/19/2016
• Est. Priority Date: 09/30/2010
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Vijayan, Manoj Kumar, Attarde, Deepak Raghunath
• Primary Examiner(s): KROFCHECK, MICHAEL C
• Application Number:
  US14/040,247
• Publication Number:
  US 20140032864A1
• Time in Patent Office: 844 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1466 to make the backup process non-disruptive
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: H05K 999/99 dummy group

Abstract

A system according to certain embodiments associates a signature value corresponding to a data block with one or more data blocks and a reference to the data block to form a signature/data word corresponding to the data block. The system further logically organizes the signature/data words into a plurality of files each comprising at least one signature/data word such that the signature values are embedded in the respective file. The system according to certain embodiments reads a previously stored signature value corresponding to a respective data block for sending from a backup storage system having at least one memory device to a secondary storage system. Based on an indication as to whether the data block is already stored on the secondary storage system, the system reads the data block from the at least one memory device for sending to the secondary storage system if the data block does not exist on the secondary storage system, wherein the signature value and not the data block is read from the at least one memory device if the data block exists on the secondary storage system.

Claims

1. A method of performing a copy operation, the method comprising:

accessing with computer hardware a deduplication signature table containing a plurality of signatures corresponding to data blocks stored in at least one first storage device;

using the deduplication signature table, performing, with computer hardware, a first deduplicated copy operation of a plurality of files from at least one second storage device to the first storage device thereby creating a deduplicated copy of the plurality of files in the first storage device; and

with computer hardware and as part of a second deduplicated copy operation in which at least a subset of the plurality of data blocks which form the deduplicated copy are copied from the first storage device to at least one third storage device, for each respective data block in the subset;

accessing from the first storage device a previously stored signature corresponding to the respective data block and which is stored separately from the deduplication signature table, the previously stored signature included in a first signature/data word of a plurality of signature/data words which is associated with the respective data block and is embedded in a file that includes others of the plurality of signature/data words associated with other data blocks in the subset, wherein first signature/data words of the plurality of signature/data words each include a respective signature and a respective actual data block copy stored physically or logically contiguously with respect to one another, and wherein second signature/data words of the plurality of signature/data words each include a respective signature and a respective reference to an actual data block copy stored physically or logically contiguously with respect to one another;

transmitting the previously stored signature to the third storage device, wherein the previously stored signature is transmitted to the third storage device without re-generating the value of the previously stored signature using the respective data block;

receiving a message indicating whether a copy of the respective data block is already stored on the third storage device; and

if the message indicates that the respective data block is not already stored on the third storage device;

accessing the respective data block from the first storage device; and

transmitting the respective data block to the third storage device.

2. The method of claim 1, wherein the respective data block itself is not read from the first storage device as part of the second deduplicated storage operation if the message indicates that the respective data already block exists on the third storage device.

3. The method of claim 1, wherein each of the signature/data words includes one or more of the data block associated with the signature/data word or a reference to the data block associated with the signature/data word.

4. The method of claim 3, further comprising logically organizing the signature/data words into files each including one or more signature/data words such that the signatures are embedded in the files.

5. The method of claim 1, wherein the second deduplicated copy operation is completed using previously stored signatures that were created and stored on the first storage device prior to the second deduplicated copy operation.

6. The method of claim 1, wherein the first deduplicated copy operation is from primary storage to secondary storage, and the second deduplicated copy operation is an auxiliary copy operation within secondary storage.

7. The method of claim 1, wherein the deduplication signature table resides in a secondary storage system which includes the first storage device.

8. The method of claim 1, wherein the size of the data blocks that form the deduplicated copy and that are stored on the first storage device is at least about 512 times larger than the size of corresponding signatures that are also stored on the first storage device.

9. The method of claim 1, wherein the plurality of files are originally generated by at least one client computing device.

10. A system for performing a copy operation, the system comprising:

a signature table containing a plurality of signatures corresponding to data blocks stored in at least one first storage device; and

computer hardware in communication with the first storage device and also in communication with at least one second storage device and at least one third storage device, the computer hardware implementing a copy manager configured to;

access the signature table and, using the signature table, perform a first deduplicated copy operation on a plurality of files from the second storage device to the first storage device, thereby creating a deduplicated copy of the plurality of files in the first storage device;

as part of a second deduplicated copy operation in which at least a subset of the plurality of data blocks which form the copy are copied from the first storage device to the third storage device, for each respective data block in the subset;

access from the first storage device a stored signature corresponding to the respective data block and which is stored separately from the deduplication signature table, the stored signature included in a first signature/data word of a plurality of signature/data words which is associated with the respective data block and is embedded in a file that includes others of the plurality of signature/data words associated with other data blocks in the subset, wherein first signature/data words of the plurality of signature/data words each include a respective signature and a respective actual data block copy stored physically or logically contiguously with respect to one another, and wherein second signature/data words of the plurality of signature/data words each include a respective signature and a respective reference to an actual data block copy stored physically or logically contiguously with respect to one another;

transmit the stored signature to the third storage device, wherein the stored signature is transmitted to the third storage device without re-generating the value of the stored signature using the respective data block;

receive a message indicating whether a copy of the respective data block is already stored on the third storage device; and

if the message indicates that the respective data block is not already stored on the third storage device;

access the respective data block from the first storage device, and

transmit the respective data block to the third storage device.

11. The system of claim 10, wherein the respective data block itself is not read from the first storage device as part of the second deduplicated storage operation if the message indicates that the respective data already block exists on the third storage device.

12. The system of claim 10, wherein each of the signature/data words includes one or more of the data block associated with the signature/data word or a reference to the respective data block associated with the signature/data word.

13. The system of claim 12, the copy manager further configured to logically organize the signature/data words into files each including one or more signature/data words such that the signatures are embedded in the files.

14. The system of claim 10, wherein the second deduplicated copy operation is completed using previously stored signatures that were created and stored on the first storage device prior to the second deduplicated copy operation.

15. The system of claim 10, wherein the first deduplicated copy operation is from primary storage to secondary storage, and the second deduplicated copy operation is an auxiliary copy operation within secondary storage.

16. The system of claim 10, wherein the deduplication signature table resides in a secondary storage system which includes the first storage device.

17. The system of claim 10, wherein the size of the data blocks that form the deduplicated copy and that are stored on the first storage device is at least about 512 times larger than the size of corresponding signatures that are also stored on the first storage device.

18. The system of claim 10, wherein the plurality of files are originally generated by at least one client computing device.

19. Non-transitory computer-readable storage comprising computer-readable instructions that, when executed cause computer hardware to perform operations defined by the computer-readable instructions, the operations comprising:

accessing with computer hardware a deduplication signature table containing a plurality of signatures corresponding to data blocks stored in at least one first storage device;

using the deduplication signature table, performing, with computer hardware, a first deduplicated copy operation of a plurality of files from at least one second storage device to the first storage device thereby creating a deduplicated copy of the plurality of files in the first storage device; and

with computer hardware and as part of a second deduplicated copy operation in which at least a subset of the plurality of data blocks which form the deduplicated copy are copied from the first storage device to at least one third storage device, for each respective data block in the subset;

accessing from the first storage device a previously stored signature corresponding to the respective data block and which is stored separately from the deduplication signature table, the previously stored signature included in a first signature/data word of a plurality of signature/data words which is associated with the respective data block and is embedded in a file that includes others of the plurality of signature/data words associated with other data blocks in the subset, wherein first signature/data words of the plurality of signature/data words each include a respective signature and a respective actual data block copy stored physically or logically contiguously with respect to one another, and wherein second signature/data words of the plurality of signature/data words each include a respective signature and a respective reference to an actual data block copy stored physically or logically contiguously with respect to one another;

transmitting the previously stored signature to the third storage device, wherein the previously stored signature is transmitted to the third storage device without re-generating the value of the previously stored signature using the respective data block;

receiving a message indicating whether a copy of the respective data block is already stored on the third storage device; and

if the message indicates that the respective data block is not already stored on the third storage device;

accessing the respective data block from the first storage device; and

transmitting the respective data block to the third storage device.