SYSTEMS AND METHODS FOR SELECTIVE DATA REPLICATION

• US 20110238621A1
• Filed: 03/29/2010
• Published: 09/29/2011
• Est. Priority Date: 03/29/2010
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Agrawal, Vijay H.
• Granted Patent: US 8,504,517 B2
• Time in Patent Office: Days
• US Class Current: 707/610
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation

Abstract

Systems and methods for performing data replication are disclosed. Determining whether to update replicated data typically involves comparison of readily obtainable attributes of a given source file and its corresponding replicated file. Such attributes can be obtained from, for example, metadata. In certain situations, an additional assessment of the source and replicated files can be beneficial. For example, if integrity of an existing replicated file'"'"'s content is maintained, one may not want to re-replicate the corresponding source file. For large source files, such a decision can provide substantial reductions in expenditures of available computing and network resources. In certain embodiments, a threshold for identifying such large files can be based on one or more operating parameters such as network type and available bandwidth. In certain embodiments, replication file'"'"'s integrity can be checked by calculating and comparing checksums for the replication file and its corresponding source file.

Claims

1. A method for performing data replication, the method comprising:

performing an assessment on first data stored on a first storage device and second data stored on a second storage device, at least a portion of the second data previously having been replicated from the first data, the assessment comprising,

comparing one or more attributes of files in the first data with those of corresponding files in the second data, and

identifying a file having at least one of the one or more attributes different in the first and second data;

comparing the size of the identified file with a selected threshold value;

if the size of the identified file is less than or equal to the selected threshold value, replicating the identified file from the first storage device to the second storage device; and

if the size of the identified file is greater than the selected threshold value;

obtaining checksums for the identified file in the first data and its corresponding file in the second data;

comparing the checksums;

if the checksums are different, replicating the identified file from the first storage device to the second storage device; and

if the checksums are the same, synchronizing the one or more different attributes of the identified file in the first data and the corresponding file in the second data, and not replicating the identified file.

2. The method of claim 1, wherein the one or more attributes comprise one or more attributes obtainable from metadata.

3. The method of claim 2, wherein the one or more attributes obtainable from metadata comprise at least one attribute selected among file size, file creation time, file modification time, or file access time.

4. The method of claim 1, wherein the selected threshold value is obtained based on one or more storage policies.

5. The method of claim 4, wherein the one or more storage policies comprise assignment of the selected threshold value based on one or more of type of communication network between the first and second systems, available network resource, or assigned priority.

6. The method of claim 1, wherein the size of the identified file is selected based on a size of a data block, one or more of the data blocks constituting the identified file.

7. The method of claim 1, wherein said obtaining of checksums comprises calculating checksums for each of one or more data blocks associated with the identified file and the corresponding file.

8. The method of claim 7, wherein said replicating the identified file comprises replicating only one or more data blocks of the identified file whose checksums are different from those of the corresponding file.

9. A data replication system, comprising:

a data storage system configured to store replication of at least a portion of data from a client system, the client system capable of communicating with the data storage system to facilitate transfer of data therebetween; and

a replication agent in communication with the client system and the data storage system and configured to obtain information about an identified file on the client system, the identified file having at least one metadata attribute that is different from that of an existing replicated copy of the identified file on the data storage system, the replication agent further configured to;

obtain a size of the identified file,

compare the size of the identified file with a threshold value,

if the size is less than or equal to the threshold value, replicate the identified file so as to replace or update the existing replicated copy of the identified file, and

if the size is greater than the threshold value, (1) obtaining and comparing checksums of the identified file and the replicated file, and (2) replicating the identified file so as to replace or update the existing replicated copy of the identified file if the checksums are different.

10. The system of claim 9, wherein the replication agent is further configured to reconcile the metadata difference between the identified and replicated files but not replicate the identified file if the checksums of the identified file and the replicated file are the same.

11. The system of claim 9, wherein the threshold value is obtained based on one or more storage policies, a type of communication network between the client system and the data storage system, one or more network resources associated with the communication network, or a priority assigned to the replication agent.

12. The system of claim 9, wherein the threshold value comprises 256 kilobytes.

13. The system of claim 9, further comprising a user interface configured to receive user input indicative of the threshold value.

14. The system of claim 9, wherein the replication agent comprises a software application executable on the client system.

15. A replication system comprising:

means for identifying a first file in a first system based on a comparison of one or more attributes of the first file and a second file on a second system, the second file representing an existing replicated copy of the first file;

means for comparing the size of the identified file with a threshold value; and

means for determining whether to replicate one or more blocks of the first file to the second file again based at least in part on the size comparison.

16. The replication system of claim 15, wherein said means for determining comprises:

means for obtaining and comparing an assessment of contents of the first and second files; and

means for selectively replicating data blocks of the first file to the second file based on the assessment of the contents.

17. The replication system of claim 16, wherein the assessment of contents of the first and second files comprises a calculation of checksums of the first and second file.