Systems and methods for continuous data replication

• US 9,003,374 B2
• Filed: 05/05/2014
• Issued: 04/07/2015
• Est. Priority Date: 07/27/2006
• Status: Expired due to Fees

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Ngo, David
• Primary Examiner(s): COYER, RYAN D
• Application Number:
  US14/269,506
• Publication Number:
  US 20140244586A1
• Time in Patent Office: 337 Days
• Field of Search: 717/127, 707648-649
• US Class Current: 717/127
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Systems and methods for intelligent data replication are provided, which may include the identification and use of consistency points in application data, such that one or more replication volumes may be created using an “application aware” replication process that allows the system to intelligently replicate data or other information (e.g., metadata) relating to a particular process, application or specified set of data.

Claims

1. A method of correlating copies of application data of an operating software application in a storage system, the method comprising:

populating a first data log with data associated with a plurality of software applications, where one or more entries in the first data log comprise at least an application type field;

generating at least one indicator and an associated first time stamp, and placing the at least one indicator and associated first time stamp in the first data log;

periodically copying portions of the first data log to a second data log;

processing the second data log with a plurality of replication replay threads until the at least one indicator is reached, the replication replay threads replicate the data generated by the plurality of software applications to create replicated data, wherein each of the replication replay threads are associated with the plurality of software applications based on the application type field;

associating a second time stamp with the replicated data; and

associating the first time stamp with the second time stamp such that the data stored in the first data log is correlated with the replication data stored on the one or more first storage devices.

2. The method of claim 1 further comprising periodically copying the replication data stored on the one or more first storage devices to one or more second storage devices.

3. The method of claim 2 wherein the replication data is copied to the one or more second storage devices when the at least one indicator is encountered during a copy operation.

4. The method of claim 2 wherein copying of the replication data to the one or more second storage devices comprises taking a snapshot of the replication data on the one or more first storage devices.

5. The method of claim 4 wherein the snapshot of the replication data is associated with the first time stamp.

6. The method of claim 4 further comprising:

assigning a third time stamp to the snapshot; and

associating the third time stamp with the first time stamp such that the data stored in the first data log is correlated with the snapshot.

7. The method of claim 2 further comprising associating a location of the replication data stored on the one or more second storage devices with the first time stamp such that the data stored on the first data log becomes synchronized with the replication data stored on the one or more second storage devices.

8. The method of claim 2 further comprising copying location information associated with the replication data stored on the one or more second storage devices to an index.

9. The method of claim 1 further comprising:

suspending copy operations from the second data log to the one or more first storage devices when the at least one indicator is encountered; and

copying replication data from the one or more first storage devices to one or more second storage devices while the copy operations from the second data log to the one or more first storage devices are suspended.

10. The method of claim 1 wherein the at least one indicator is indicative of known recoverable state of one or more of the plurality of software applications.

11. A system of correlating copies of application data of an operating software application in a storage system comprising:

computer hardware that populates a first data log with data associated with a plurality of software applications, where one or more entries in the first data log comprise at least an application type field;

a software module installed on a computing device that generates at least one indicator and an associated first time stamp, and places the at least one indicator and associated first time stamp in the first data log;

computer hardware that periodically copies portions of the first data log to a second data log;

a computer hardware that processes the second data log with a plurality of replication replay threads until the at least one indicator is reached, the replication replay threads replicate the data generated by the plurality of software applications to create replicated data, wherein each of the replication replay threads are associated with the plurality of software applications based on the application type field;

computer hardware that associates a second time stamp with the replicated data; and

computer hardware that associates the first time stamp with the second time stamp such that the data stored in the first data log is correlated with the replication data stored on the one or more first storage devices.

12. The system of claim 11 wherein computer hardware periodically copies the replication data stored on the one or more first storage devices to one or more second storage devices.

13. The system of claim 12 wherein the replication data is copied to the one or more second storage devices when the at least one indicator is encountered during a copy operation.

14. The system of claim 12 wherein copying of the replication data to the one or more second storage devices comprises taking a snapshot of the replication data on the one or more first storage devices.

15. The system of claim 14 wherein the snapshot of the replication data is associated with the first time stamp.

16. The system of claim 14 further comprising computer hardware that assigns a third time stamp to the snapshot, wherein the third time stamp is associated with the first time stamp such that the data stored in the first data log is correlated with the snapshot.

17. The system of claim 12 further wherein computer hardware associates a location of the replication data stored on the one or more second storage devices with the first time stamp such that the data stored on the first data log becomes synchronized with the replication data stored on the one or more second storage devices.

18. The system of claim 12 wherein computer hardware copies location information associated with the replication data stored on the one or more second storage devices to an index.

19. The system of claim 11 further comprising:

computer hardware that suspends copy operations from the second data log to the one or more first storage devices when the at least one indicator is encountered; and

computer hardware that copies replication data from the one or more first storage devices to one or more second storage devices while the copy operations from the second data log to the one or more first storage devices are suspended.

20. The system of claim 11 wherein the at least one indicator is indicative of known recoverable state of one or more of the plurality of software applications.