Systems and methods for performing data replication

• US 9,971,657 B2
• Filed: 03/28/2017
• Issued: 05/15/2018
• Est. Priority Date: 12/19/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Prahlad, Anand, Ngo, David
• Primary Examiner(s): To, Baoquoc N
• Application Number:
  US15/471,640
• Publication Number:
  US 20170293532A1
• Time in Patent Office: 413 Days
• Field of Search: 707645, 707644
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/1794 Details of file format conversion
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0673 Single storage device
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

Performing data management operations on replicated data in a computer network. Log entries are generated for data management operations of an application executing on a source system. Consistency point entries are used to indicate a time of a known good, or recoverable, state of the application. A destination system is configured to process a copy of the log and consistency point entries to replicate data in a replication volume, the replicated data being a copy of the application data on the source system. When the replicated data represents a known good state of the application, as determined by the consistency point entries, the destination system(s) may perform a storage operation (e.g., snapshot, backup) to copy the replicated data and to logically associate the copied data with a time information (e.g., time stamp) indicative of the source system time when the application was in the known good state.

Claims

1. A method for performing data management operations, the method comprising:

monitoring data operations occurring at a first time to first data stored on one or more first storage devices associated with a primary storage system, the first data stored in a native format;

creating at least one consistency point marker associated with the first time, wherein the consistency point marker comprises at least the first time;

replicating the data operations with computer hardware using multiple replication threads in parallel to create replicated data of the first data on one or more second storage devices, the replicated data stored in the native format, wherein each replication thread provides notification after reaching the consistency point marker;

based at least in part on encountering a consistency point marker while replicating the data operations, suspending the data operations in each of the multiple replication threads that reach the consistency point marker, and generating a backup copy of the replicated data at a second time that occurs after the first time, the backup copy stored in secondary storage system in a backup format wherein the backup format is different than the native format; and

associating the backup copy of the replicated data from the multiple replication threads in the backup format with the first data stored on the one or more first storage devices in the native format based on the first time in the consistency point marker even though the backup copy is a backup copy of the replicated data occurring at the second time after the first time, wherein after the associating, the backup copy of the replicated data appears to be a backup performed on the first data existing at the first time on the one or more first storage devices, while the first data stored in the native format continues to available for use on the one or more first storage devices.

2. The method of claim 1 further comprising monitoring the data operations with a filter driver.

3. The method of claim 1 wherein the backup format facilitates compression of the replication data.

4. The method of claim 1 further comprising identifying a third time that data operations occur to the first data and replicating the data operations on the one or more second storage devices.

5. The method of claim 4 further comprising generating at a fourth time, a differential backup of the replicated data on the one or more second storage devices.

6. The method of claim 5 further comprising associating the differential backup of the replicated data occurring at the fourth time with the first data existing on the one or more first storage devices at the third time.

7. The method of claim 1 further comprising quiescing a computer application associated with the data operations at the first time and the consistency point marker indicates of a known good state of the computer application.

8. The method of claim 1 wherein the multiple replication threads that replay the data operations to the one or more second storage devices.

9. The method of claim 8 wherein each of the multiple replication threads process a particular type of data operation.

10. The method of claim 1 wherein a frequency of creating the consistency point is adjusted to satisfy risk criteria.

11. A system for performing data management operations, the system comprising:

computer hardware comprising one or more computer processors;

a data agent module that monitors data operations occurring at a first time to first data stored one or more first storage devices, wherein the data agent creates at least one consistency marker that comprises at least the first time, the first data stored in a native format;

a replication module that executes on the one or more computer processors, the replication module replicates the data operations using multiple replication threads in parallel to create replicated data of the first data on one or more second storage devices, the replication data stored in the native format, wherein each replication thread notifies the replication module after reaching the consistency point marker, wherein the replication module suspends data operations in each of the multiple replication threads that reach the consistency point marker;

a backup module that executes on the one or more computer processors, the backup module generates a backup copy of the replicated data from the multiple replication threads at a second time that occurs after the first time based at least in part on the replication module encountering a consistency point marker while replicating the data operations, the backup copy stored in a backup format that is different than the native format; and

a storage module configured to associate the replicated data in the backup format with the first data stored on the one or more first storage devices in the native format based on the first time in the consistency point marker even though the backup copy is a backup copy of the replicated data occurring at the second time after the first time, wherein after the associating, the backup copy of the replicated data appears to be a backup performed on the first data existing at the first time on the one or more first storage devices, while the first data stored in native format continues to be available for use on the one or more first storage devices.

12. The system of claim 11 further comprising a filter driver that monitors the data operations.

13. The system of claim 11 wherein the backup format facilitates compression of the replication data.

14. The system of claim 11 wherein the replication module replicates data operations that occur to the first data at a third time on the one or more second storage devices.

15. The system of claim 14 wherein the backup module generates at a fourth time, a differential backup of the replicated data on the one or more second storage devices.

16. The system of claim 15 wherein the storage module associates the differential backup of the replicated data occurring at the fourth time with the first data existing on the one or more first storage devices at the third time.

17. The system of claim 11 wherein a computer application associated with the data operations is quiesced at the first time and the consistency point marker indicates a time of a known good state of the computer application.

18. The system of claim 11 wherein the multiple replication threads that replay the data operations to the one or more second storage devices.

19. The system of claim 18 wherein each of the multiple replication threads process a particular type of data operation.

20. The method of claim 11 wherein the data agent adjusts a frequency of creating the consistency point to satisfy risk criteria.