Systems and methods for performing discrete data replication

• US 9,495,382 B2
• Filed: 12/09/2009
• Issued: 11/15/2016
• Est. Priority Date: 12/10/2008
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Agrawal, Vijay H., Prahlad, Anand
• Primary Examiner(s): Trujillo, James
• Assistant Examiner(s): MINA, FATIMA P
• Application Number:
  US12/634,135
• Publication Number:
  US 20100179941A1
• Time in Patent Office: 2,533 Days
• Field of Search: 707/654, 707/646, 707/641, 707/643, 707/644, 707/645, 707/649, 707/650
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/1815 Journaling file systems
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Systems and methods for performing intelligent replication of production data on a source storage device. In certain examples, a full backup of the production data is first performed and stored on a destination system. A journal module on the storage device (e.g., a shared storage device), or on at least one source system, tracks files on the storage device that change during a particular period of time. Following this period, a snapshot is taken of the production data. Upon mounting the snapshot, the replication system uses the journal information to identify modified files, conduct a block-level comparison of only the modified files with corresponding files on the destination system, and identify the particular data blocks that changed. The replication system then replicates only the changed blocks to the destination system. Certain systems further store information indicating a source system time of a known good state of the production data.

Claims

1. A method for replicating data in a computer network from a source storage device to a destination storage device, the method comprising:

replicating at a first time source data by copying the source data to create replication data,

wherein the source data is stored on at least one source storage device associated with a source system, the source system comprising one or more source computer devices, the source data comprising a plurality of source files generated by one or more applications executing on the source system, and

wherein the replication data is stored on at least one destination storage device associated with a replication system comprising one or more replication computer devices, wherein the one or more replication computer devices are different than the one or more source computer devices, the replication data comprising a plurality of destination files corresponding to the plurality of source files;

following the replicating, identifying, by the source system, changed source files by tracking at a logical file level which of the plurality of source files changed between the first time and a second time after the first time, wherein said tracking comprises maintaining a journal file on the source storage device with entries identifying the changed source files at the logical file level;

providing from the source system to the replication system, the changed source files at the logical file level by mounting the changed source files on the replication system wherein the replication system uses the mounting to obtain the logical file level associated with the changed source files;

using the one or more replication computer devices associated with the replication system to identify, based on the changed source files at the logical file level, previously replicated destination files that are related to the changed source files;

performing with one or more of the source computer devices associated with the source system, a source snapshot of the source data to capture a point-in-time copy of the source data on a block-by-block basis at the second time;

accessing, with the one or more replication computer devices associated with the replication system, data blocks in the source snapshot with the mounting;

comparing, with the one or more replication computer devices associated with the replication system, the data blocks in the source snapshot with data blocks in the previously replicated destination files stored on the at least one destination storage device on a block-by-block basis to identify changed blocks in the changed source files in the source data;

replicating, with the one or more replication computer devices associated with the replication system, the changed blocks of the changed source files by determining with the one or more replication computer devices which copies of the changed data blocks to replicate from the source system to the at least one destination storage device; and

not replicating unchanged blocks associated with the changed source files.

2. The method of claim 1, additionally comprising, prior to performing the snapshot, quiescing one or more applications having access to the source data on the source storage device.

3. The method of claim 2, wherein said quiescing is performed at a known good state of the one or more applications.

4. The method of claim 2, additionally comprising instructing the one or more applications to assume a known good state.

5. The method of claim 1, wherein the source storage device comprises a shared storage device.

6. The method of claim 1, additionally comprising transmitting with the changed blocks time information indicative of the second time, wherein the time information is relative to a clock associated with the source storage device.

7. The method of claim 6, additionally comprising, following said replicating the changed blocks:

performing a storage operation on the replication data to generate copied data; and

logically associating the copied data with the time information indicative of the second time.

8. The method of claim 1, wherein the duration between the first time and the second time is based on a storage policy stored in a preference database.

9. A system for performing data replication in a computer network environment, the data replication from a first storage device to a replication storage device, the system comprising:

a source system comprising one or more source computer devices and at least a first storage device configured to store source data as a plurality of source files, the source system having one or more applications executing thereon and operative to write data to the plurality of source files stored on the first storage device;

a replication system comprising one or more replication computer devices, wherein the one or more replication computer devices are different than the one or more source computer devices, the replication system configured to store replication data that is a replication of the source data at a first time, the replication data stored on at least one replication storage device associated with the replication system, the replication data comprising a plurality of destination files corresponding to the plurality of source files

a journal module configured to identify changed source files at a logical file level by tracking changes at the logical file level to the plurality of source files on the first storage device between a first time and a second time after the first time, the journal module tracking the changes at least partly by maintaining a journal file with entries identifying changed source files at the logical file level; and

wherein the replication system receives a copy of the changed source files at the logical file level by mounting the changed source files on the replication system wherein the replication system uses the mounting to obtain the logical file level associated with the changed source files;

a data agent module configured to execute on one or more of the first plurality of computer devices, the data agent module further configured to;

instruct a creation of a source snapshot of the source data with the one or more source computer devcies to capture a point-in-time copy of the source data on a block-by-block basis at the second time;

wherein the one or more replication computer devices associated with the replication system identify, based on the changed source files at the logical file level, the replication system identifies previously replicated destination files that are related to the changed source files;

the one or more replication computers devices associated with the replication system accesses data blocks in the source snapshot with the mounting;

the one or more replication computer devices associated with the replication system compares the data blocks in the source snapshot with data blocks in the replicated destination files stored on the at least one destination storage device on a block-by-block basis to identify changed blocks in the changed source files in the source data; and

the one or more replication computer devices associated with the replication system replicates the changed blocks of the changed source files to the replication copy on the replication storage device by determining with the one or more replication computer devices which copies of the changed data blocks to replicate from the source system to the at least one destination storage device and which unchanged blocks associated with the changed source files are not replicated.

10. The system of claim 9, wherein the first storage device is a shared storage device.

11. The system of claim 10, wherein the journal module comprises a new technology file system (NTFS) change journal.

12. The system of claim 9, wherein the changed blocks are identified using a secure hash algorithm.

13. The system of claim 9, wherein the source data consists essentially of application-specific data.

14. The system of claim 9, wherein the replication system further comprises a replication module in communication with the replication storage device, the replication module being in network communication with the data agent.

15. The system of claim 14, wherein the replication module is further configured to direct a storage operation to be performed on at least a portion of the replication files to generate copied data.

16. The system of claim 15, wherein the replication module is further configured to logically associate the copied data with a time stamp indicative of the second time.

17. The system of claim 16, wherein the second time identifies a time at which the source data is in a known good state.