Systems and methods for performing data replication

• US 7,651,593 B2
• Filed: 12/18/2006
• Issued: 01/26/2010
• Est. Priority Date: 12/19/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Ngo, David, Prahlad, Anand
• Primary Examiner(s): To; Baoquoc N
• Application Number:
  US11/640,826
• Publication Number:
  US 20070185938A1
• Time in Patent Office: 1,135 Days
• Field of Search: 707/200, 707/202, 707/203
• US Class Current: 202/204
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/1794 Details of file format conversion
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0673 Single storage device
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

Performing data management operations on replicated data in a computer network. Log entries are generated for data management operations of an application executing on a source system. Consistency point entries are used to indicate a time of a known good, or recoverable, state of the application. A destination system is configured to process a copy of the log and consistency point entries to replicate data in a replication volume, the replicated data being a copy of the application data on the source system. When the replicated data represents a known good state of the application, as determined by the consistency point entries, the destination system(s) may perform a storage operation (e.g., snapshot, backup) to copy the replicated data and to logically associate the copied data with a time information (e.g., time stamp) indicative of the source system time when the application was in the known good state.

Claims

1. A method for performing data management operations in a computer network, the method comprising:

monitoring operations associated with an application, the operations operative to write data to a first storage device;

storing information about the operations in a first log file on the first storage device;

inserting in the first log file a marker comprising time information identifying a time of a known good state of the application;

copying at least a portion of the first log file to a second log file on a second storage device;

processing the second log file to create replicated data;

generating a snapshot of the replicated data at a time after the second log file has been copied to the second storage device; and

associating the snapshot of the replicated data with the known good state of the application such that the snapshot of the replicated data is associated with the data on the first storage device.

2. The method of claim 1, additionally comprising quiescing the application at the time of the known good state of the application.

3. The method of claim 1, wherein said copying comprises replicating data associated with only one application.

4. The method of claim 1, additionally comprising performing a backup operation on the snapshot and associating a backup copy of the snapshot with the time information identifying the time of the known good state of the application.

5. A system for performing data management operations in a computer network environment, the system comprising:

at least one computer application configured to execute on a source computer;

a first storage device coupled to the source computer to receive data write operations from the at least one computer application;

at least one module configured to monitor the data write operations and to generate first log entries based on the data write operations, at least one of the log entries having a time stamp indicative of a time of a known good state of the at least one computer application;

a second storage device comprising second log entries wherein the second log entries comprise copies of at least a portion of the first log entries; and

a replication module coupled to the second storage device and configured to;

process, the copied data write operations in the second log entries to create replicated data stored at a first location on the second storage device,

perform a storage operation on the replicated data at the first location on the second storage device to copy data to a second location different than said first location, and

logically associate the copied data at the second location with the time stamp indicative of the time of the known good state of the at least one computer application such that the copied data at the second location is associated with the data on the first storage device.

6. The system of claim 5, wherein the at least one module comprises a filter driver.

7. The system of claim 5, wherein the storage operation comprises a snapshot operation or a backup operation.

8. The system of claim 5, wherein the replication module further comprises a replication log file configured to receive a copy of the log entries.

9. The system of claim 8, wherein the replication module further comprises a plurality of threads configured to process the copy of the log entries in the replication log file and to replay the data write operations associated with the log entries on the second storage device.

10. The system of claim 9, wherein said plurality of threads is configured to sequentially process the second log entries.

11. The system of claim 5, wherein the at least one computer application is a single computer application and the driver module is associated with only the single computer application.

12. The system of claim 5, further comprising a third storage device, wherein the second location is on the third storage device.

13. The system of claim 5, wherein the second storage device further comprises a plurality of volumes.

14. The system of claim 13, wherein the second location is on a different volume of the second storage device than the first location.

15. The system of claim 13, wherein the replication module comprises at least two replication servers, each of the replication servers being associated with a different one of the plurality of volumes.

16. A method for performing data management operations in a computer network, the method comprising:

monitoring data operations associated with an application, the data operations operative to write data to a first storage device;

populating a first log file on the first storage device with data entries indicative of the data operations;

inserting in the first log file a marker indicative of a known good state of the application, the marker including a time information identifying a time of the known good state of the application;

copying at least a portion of the first log file to a second storage device to create a second log file on the second storage device;

replaying to the second storage device data operations in the second log file to create replicated data stored at a first location on the second storage device;

performing a storage operation on the replicated data to copy the replicated data from the first location to a second location; and

associating the copy of the replicated data at the second location with the time information identifying the time of the known good state of the application such that the copy of the replicated data at the second location is associated with the data on the first storage device.

17. The method of claim 16, additionally comprising copying the data entries and the marker from the first log file to the second log file.

18. The method of claim 17, wherein said replaying comprises instantiating a plurality of threads configured to process the data entries in the second log file and to replay the data operations associated therewith to the second storage device.

19. The method of claim 18, wherein each of the plurality of threads is configured to process a particular type of data operation.

20. The method of claim 16, wherein said replaying comprises replaying to the second storage device the data operations of a first portion of the data entries, the method further comprising:

replaying to a third storage device the data operations of a second portion of the data entries to replicate data to a third location;

performing a second storage operation to copy replicated data from the third location to a fourth location; and

associating the copy of the replicated data at the fourth location with the time information identifying the time of the known good state of the application.

21. The method of claim 20, additionally comprising merging the copy of the replicated data at the second location with the copy of the replicated data at the fourth location.

22. The method of claim 21, additionally comprising storing the merged data in a fifth location.

23. The method of claim 22, additionally comprising associating the merged data with the time information identifying the time of the known good state of the application.

24. The method of claim 16, additionally comprising providing a user with an indication of the copy of the replicated data.