Systems and methods for performing data replication

• US 8,793,221 B2
• Filed: 09/12/2012
• Issued: 07/29/2014
• Est. Priority Date: 12/19/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Prahlad, Anand, Ngo, David
• Primary Examiner(s): To, Baoquoc N
• Application Number:
  US13/612,469
• Publication Number:
  US 20130006938A1
• Time in Patent Office: 685 Days
• Field of Search: 707/672
• US Class Current: 707/672
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/1794 Details of file format conversion
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0673 Single storage device
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

Performing data management operations on replicated data in a computer network. Log entries are generated for data management operations of an application executing on a source system. Consistency point entries are used to indicate a time of a known good, or recoverable, state of the application. A destination system is configured to process a copy of the log and consistency point entries to replicate data in a replication volume, the replicated data being a copy of the application data on the source system. When the replicated data represents a known good state of the application, as determined by the consistency point entries, the destination system(s) may perform a storage operation (e.g., snapshot, backup) to copy the replicated data and to logically associate the copied data with a time information (e.g., time stamp) indicative of the source system time when the application was in the known good state.

Claims

1. A method for performing data management operations, the method comprising:

monitoring operations associated with a computer application, the operations operative to write data to a first location on a first storage device at a first time;

storing information about the operations in a first log file on the first storage device;

copying at least a portion of the first log file on the first storage device to a second log file on a second storage device;

processing the second log file to create replicated data at a second location;

generating a snapshot of the replicated data at a time after the second log file has been copied to the second storage device; and

associating the snapshot of the replicated data with the first location wherein after the associating, the snapshot of the replicated data appears to be a snapshot of the data existing at the first time on the first location.

2. The method of claim 1, wherein monitoring operations comprises processing the operations with a filter driver.

3. The method of claim 1 further comprising quiescing the computer application at a time of a known good state of the computer application.

4. The method of claim 1, wherein processing the second log file comprises instantiating a plurality of threads that process data entries in the second log file and replay the data operations associated therewith to the second location.

5. The method of claim 4, wherein each of the plurality of threads process a particular type of data operation.

6. The method of claim 1, wherein processing the second log file comprises replaying to the second location device a first portion of data operations associated with data entries in the second log file.

7. The method of claim 6, wherein processing the second log file comprises:

replaying to a third storage device data operations of a second portion of the data entries to replicate data to a third location;

performing a second storage operation to copy replicated data from the third location to a fourth location; and

associating the copy of the replicated data at the fourth location with a time of a known good state of the computer application.

8. The method of claim 7, additionally comprising merging the copy of the replicated data at the second location with the copy of the replicated data at the fourth location.

9. The method of claim 1, wherein the second storage device further comprises a plurality of volumes.

10. The method of claim 9, wherein processing the second log file comprises at least two replication servers, each of the replication servers being associated with a different one of the plurality of volumes.

11. A system for performing data management operations, the system comprising:

a first storage device that stores data write operations from at least one computer application at a first location at a first time;

at least one module configured to monitor the data write operations and to generate first log entries based on the data write operations;

a second storage device comprising second log entries wherein the second log entries comprise copies of at least a portion of the first log entries; and

a replication module in communication with the second storage device and configured to;

process the copied data write operations in the second log entries to create replicated data stored in a second location,

associate the replicated data stored in the second location with the at least one computer application at the first location, wherein after the associating, the replicated data appears to be data existing at the first time on the first location.

12. The system of claim 11 further comprising a filter driver that is configured to monitor operations associated with the computer application.

13. The system of claim 11 wherein the system is configured to quiesce the computer application at a time of a known good state of the computer application.

14. The system of claim 11, wherein the replication module is configured to instantiate a plurality of threads that process data entries in the second log file and replay the data operations associated therewith to the second location.

15. The system of claim 14, wherein each of the plurality of threads process a particular type of data operation.

16. The system of claim 11, wherein the replication module is configured to replay to the second location a first portion of data operations associated with data entries in the second log file.

17. The system of claim 16, wherein the replication module is configured to:

replay to a third storage device data operations of a second portion of the data entries to replicate data to a third location;

perform a second storage operation to copy replicated data from the third location to a fourth location; and

associate the copy of the replicated data at the fourth location with a time of a known good state of the computer application.

18. The system of claim 17, additionally comprising merging the copy of the replicated data at the second location with the copy of the replicated data at the fourth location.

19. The system of claim 11, wherein the second storage device further comprises a plurality of volumes.

20. The system of claim 19, wherein processing the second log file comprises at least two replication servers, each of the replication servers being associated with a different one of the plurality of volumes.