Rolling cache configuration for a data replication system

• US 20070226438A1
• Filed: 12/18/2006
• Published: 09/27/2007
• Est. Priority Date: 12/19/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Erofeev, Andrei
• Granted Patent: US 7,661,028 B2
• Time in Patent Office: Days
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Performing data management operations on replicated data in a computer network. Log entries are generated for data management operations of an application executing on a source system. Consistency point entries are used to indicate a time of a known good, or recoverable, state of the application. A destination system is configured to process a copy of the log and consistency point entries to replicate data in a replication volume, the replicated data being a copy of the application data on the source system. When the replicated data represents a known good state of the application, as determined by the consistency point entries, the destination system(s) may perform a storage operation (e.g., snapshot, backup) to copy the replicated data and to logically associate the copied data with a time information (e.g., time stamp) indicative of the source system time when the application was in the known good state.

Claims

1. A system for facilitating data synchronization following a network failure in a data replication environment, the system comprising:

a plurality of log entries representing data management operations generated by an application for data on a first storage device, each of the plurality of log entries including a unique identifier;

a cache memory configured to store a portion of the plurality of log entries such that when a size of the portion of log entries exceeds a storage threshold of the cache memory, the cache memory deletes one or more of the least recently received log entries until the size is less than the storage threshold;

at least one destination computer configured to process the plurality of log entries to replicate data to a second storage device, the at least one destination computer being further configured to record the unique identifier of a most recently processed log entry; and

at least one replication routine configured to sequentially retrieve each of the plurality of log entries from the cache memory and to forward each log entry to the at least one destination computer without waiting for an acknowledgement from the at least one destination computer.

2. The system of claim 1, wherein the unique identifier comprises a sequence number.

3. The system of claim 1, wherein the cache memory comprises a permanent disk cache.

4. The system of claim 1, wherein the cache memory has a storage threshold of between approximately five gigabytes and approximately twenty gigabytes.

5. The system of claim 1, wherein the at least one destination computer comprises a plurality of destination computers.

6. The system of claim 5, wherein the at least one replication routine comprises a plurality of replication threads.

7. The system of claim 6, wherein each of the plurality of replication threads communicates with one of the plurality of destination computers.

8. The system of claim 1, wherein the at least one replication routine does not store a copy of transmitted log entries.

9. The system of claim 1, wherein, following a network failure, the at least one replication routine is configured to:

receive from the at least one destination computer the unique identifier of the most recently processed log entry;

locate a copy of the most recently processed log entry in the cache memory; and

forward copies of log entries following the most recently processed log entry to the at least one destination computer.

10. A method for performing data replication in a computer network, the method comprising:

storing log entries in a first-in first-out memory, wherein each of the log entries includes a unique identification and represents a data management operation generated by an application for data on a first storage device;

maintaining in the first-in first-out memory a history of the log entries stored therein such that the storage amount occupied by the history of the log entries is less than a storage threshold;

transmitting a copy of each log entry from the first-in first-out memory to at least one destination computer;

replaying the copies of the log entries received by the at least one destination computer to replicate the data management operations on data on a second storage device; and

storing on the at least one destination computer the unique identification of the most recently replayed log entry.

11. The method of claim 10, wherein said transmitting is performed without waiting for an acknowledgement from the at least one destination computer.

12. The method of claim 10, additionally comprising detecting a network failure.

13. The method of claim 11, additionally comprising, following the network failure, querying the at least one destination computer for the unique identification of the most recently replayed log entry.

14. The method of claim 13, additionally comprising:

receiving from the at least one destination computer the unique identification of the most recently replayed log entry;

locating a copy of the most recently replayed log entry in the first-in first-out memory; and

forwarding copies of log entries following the most recently replayed log entry to the at least one destination computer.

15. The method of claim 10, wherein said maintaining comprises deleting from the first-in first-out memory one or more of the oldest log entries when receiving new log entries.

16. A system for facilitating data synchronization following a network failure in a data replication environment, the system comprising:

means for representing data management operations generated by an application for data on a first storage device, each of said means for representing including a unique identifier;

means for storing a portion of said means for representing such that when a size of the portion of said means for representing reaches a storage threshold of said means for storing, said means for storing deletes one or more of the oldest said means for representing until the size is less than the storage threshold;

means for processing said means for representing to replicate data to a second storage device, said means for processing being further configured to record the unique identifier of a most recently processed means for representing; and

means for sequentially retrieving log entries from said means for storing, said means for retrieving being further configured to forward each means for representing to said means for processing without waiting for an acknowledgement from said means for processing.

17. The system of claim 16, wherein the storage threshold of said means for storing is between approximately five gigabytes and approximately twenty gigabytes.

18. The system of claim 16, wherein said means for processing comprises a plurality of routines.

19. The system of claim 18, wherein said second storage device comprises a plurality of volumes.

20. The system of claim 19, wherein each of the plurality of routines communicates with one of the plurality of volumes.