Memory configuration for data replication system including identification of a subsequent log entry by a destination computer

• US 8,656,218 B2
• Filed: 09/12/2012
• Issued: 02/18/2014
• Est. Priority Date: 12/19/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Erofeev, Andrei
• Primary Examiner(s): Schell, Joseph
• Application Number:
  US13/612,003
• Publication Number:
  US 20130006926A1
• Time in Patent Office: 524 Days
• Field of Search: 714/6.2, 714/6.23, 714/6.32, 714/12, 714/15, 710/53, 710/54, 710/56
• US Class Current: 714/15
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Performing data management operations on replicated data in a computer network. Log entries are generated for data management operations of an application executing on a source system. Consistency point entries are used to indicate a time of a known good, or recoverable, state of the application. A destination system is configured to process a copy of the log and consistency point entries to replicate data in a replication volume, the replicated data being a copy of the application data on the source system. When the replicated data represents a known good state of the application, as determined by the consistency point entries, the destination system(s) may perform a storage operation (e.g., snapshot, backup) to copy the replicated data and to logically associate the copied data with a time information (e.g., time stamp) indicative of the source system time when the application was in the known good state.

Claims

1. A method for performing data replication in a computer network, the method comprising:

storing, with computer hardware comprising memory and one or more computer processors, one or more log entries in the memory wherein each of the log entries represents a data management operation generated by an application on a source computer for data stored on a first storage device, and wherein each of the log entries has a unique identification associated therewith;

transmitting copies of log entries in the memory to at least one destination computer;

replaying the one or more copies of the log entries received by the at least one destination computer to replicate the data management operations on data on a second storage device to create replicated data that replicates the data stored on the first storage device;

storing additional copies of the log entries in the memory in response to suspension of the replaying of the one or more copies of the log entries by the at least one destination computer;

deleting in the memory, one or more of the log entries based on exceeding a storage threshold associated with the memory

using the unique identification of a processed log entry on the destination computer to locate a log entry in the memory that occurs after the processed log entry; and

performing a synchronization operation to synchronize the data stored on the source computer with the data stored on the destination computer when the memory no longer has a copy of the log entry that occurs after the processed log entry.

2. The method of claim 1, further comprising storing additional copies of the log entries in the memory in response to a network failure.

3. The method of claim 1, comprising resuming the replaying of the one or more copies of the log entries by the at least one destination computer.

4. The method of claim 1, wherein the memory is a rolling cache.

5. The method of claim 1, wherein the memory is first-in-first-out memory.

6. The method of claim 1, wherein the transmitting is performed without waiting for an acknowledgement from the at least one destination computer.

7. The method of claim 1, additionally comprising detecting a network failure.

8. The method of claim 7, additionally comprising, following the network failure, querying the at least one destination computer for the unique identification of a most recently processed log entry.

9. A system for performing data replication in a computer network, the system comprising:

a plurality of log entries representing data management operations generated by an application on a source computer for data stored on a first storage device, each of the plurality of log entries having a unique identifier associated therewith;

computer hardware comprising memory and one or more computer processors, the memory configured to store one or more of the log entries;

at least one destination computer that receives copies of the log entries stored in the memory, wherein the destination computer processes the plurality of log entries to replicate the data management operations on data on a second storage device to create replicated data that replicates the data stored on the first storage device;

wherein the one or more computer processors are configured to store additional copies of the log entries in the memory in response to suspension of the processing of the one or more copies of the log entries by the at least one destination computer;

wherein the one or more computer processors are configured to delete one or more of the log entries from the memory based on exceeding a storage threshold associated with the memory;

wherein the one or more processors are configured to use the unique identification of a processed log entry on the destination computer to locate a log entry in the memory that occurs after the processed log entry; and

wherein the one or more processors are configured to perform a synchronization operation to synchronize the data stored on the source computer with the data stored on the destination computer when the memory no longer has a copy of the log entry that occurs after the processed log entry.

10. The system of claim 9, wherein the one or more processors are further configured to store additional copies of the log entries in the memory in response to a network failure.

11. The system of claim 9, wherein the destination computer is configured to resume the replaying of the one or more copies of the log entries after a suspension.

12. The system of claim 9, wherein the unique identifier comprises a sequence number.

13. The system of claim 9, wherein the memory comprises a disk cache.

14. The system of claim 9, comprising at least one replication routine that comprises a plurality of replication threads.

15. The system of claim 14, wherein each of the plurality of replication threads communicates with one of a plurality of destination computers.

16. The system of claim 14, wherein the at least one replication routine does not store a copy of transmitted log entries.

17. The system of claim 9, wherein the memory is a rolling cache.

18. The system of claim 9, wherein the memory is first-in-first-out memory.