ROLLING CACHE CONFIGURATION FOR A DATA REPLICATION SYSTEM

• US 20100100529A1
• Filed: 12/18/2009
• Published: 04/22/2010
• Est. Priority Date: 12/19/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Erofeev, Andrei
• Granted Patent: US 8,271,830 B2
• Time in Patent Office: Days
• US Class Current: 707/610
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Performing data management operations on replicated data in a computer network. Log entries are generated for data management operations of an application executing on a source system. Consistency point entries are used to indicate a time of a known good, or recoverable, state of the application. A destination system is configured to process a copy of the log and consistency point entries to replicate data in a replication volume, the replicated data being a copy of the application data on the source system. When the replicated data represents a known good state of the application, as determined by the consistency point entries, the destination system(s) may perform a storage operation (e.g., snapshot, backup) to copy the replicated data and to logically associate the copied data with a time information (e.g., time stamp) indicative of the source system time when the application was in the known good state.

Claims

1. A method for performing data replication in a computer network, the method comprising:

storing log entries in a first-in first-out memory wherein each of the log entries represents a data management operation generated by an application on a source computer for data stored on a first storage device, and wherein each of the log entries has a unique identification associated therewith;

deleting in the first-in first-out memory one or more least recently received log entries based on a storage threshold;

transmitting copies of log entries in the first-in first-out to at least one destination computer;

replaying the copies of the log entries received by the at least one destination computer to replicate the data management operations on data on a second storage device to create replicated data that replicates the data stored on the first storage device; and

using the unique identification of the most recently processed log entry on the destination computer to locate a log entry in the first-in first-out memory that occurs after the most recently processed log entry; and

performing a mirror operation to synchronize the data stored on the source computer with the data stored on the destination computer when the cache memory no longer has a copy of the log entry that occurs after the most recently processed log entry.

2. The method of claim 1, wherein said transmitting is performed without waiting for an acknowledgement from the at least one destination computer.

3. The method of claim 1, additionally comprising detecting a network failure.

4. The method of claim 3, additionally comprising, following the network failure, querying the at least one destination computer for the unique identification of the most recently replayed log entry.

5. The method of claim 4, additionally comprising:

receiving from the at least one destination computer the unique identification of the most recently replayed log entry;

locating a copy of the most recently replayed log entry in the first-in first-out memory; and

forwarding copies of log entries following the most recently replayed log entry to the at least one destination computer.

6. The method of claim 1, wherein said deleting comprises deleting from the first-in first-out memory one or more of the oldest log entries when receiving new log entries.

7. A system for facilitating data synchronization following a network failure in a data replication environment, the system comprising:

a plurality of log entries representing data management operations generated by an application on a source computer for data stored on a first storage device, each of the plurality of log entries having a unique identifier associated therewith;

a cache memory that stores the log entries, wherein one or more least recently received log entries are deleted from the cache memory based on a storage threshold;

at least one destination computer that receives the log entries stored in the cache memory, wherein the destination computer processes the plurality of log entries to replicate data to a second storage device; and

at least one replication routine configured to sequentially retrieve each of the plurality of log entries from the cache memory based on a most recently processed log entry by the destination computer and wherein the replication routine performs a mirror operation to synchronize the data stored on the source computer with the data stored on the destination computer when the cache memory no longer has a copy of the log entry that occurs after the most recently processed log entry.

8. The system of claim 7, wherein the unique identifier comprises a sequence number.

9. The system of claim 7, wherein the cache memory comprises a permanent disk cache.

10. The system of claim 7, wherein the at least one replication routine comprises a plurality of replication threads.

11. The system of claim 10, wherein each of the plurality of replication threads communicates with one of the plurality of destination computers.

12. The system of claim 7, wherein the at least one replication routine does not store a copy of transmitted log entries.

13. The system of claim 7, wherein, following a network failure, the at least one replication routine is configured to:

receive from the at least one destination computer the unique identifier of the most recently processed log entry;

locate a copy of the most recently processed log entry in the cache memory; and

forward copies of log entries following the most recently processed log entry to the at least one destination computer.

14. A system for facilitating data synchronization following a network failure in a data replication environment, the system comprising:

means for representing data management operations generated by an application on a source computer for data stored on a first storage device, each of said means for representing including a log entry with a unique identifier;

means for storing a portion of said means for representing such that when a size of the portion of said means for representing reaches a storage threshold of said means for storing, said means for storing deletes one or more of the oldest said means for representing until the size is less than the storage threshold;

means for processing said log entries to replicate data to a second storage device, wherein the means for processing the log entries creates replicated data that replicates the data stored on the first storage device; and

means for using the unique identifier of a most recently processed log entry on the second storage device, to locate a log entry in the means for storing that occurs after the most recently processed log entry; and

means for performing a mirror operation to synchronize the data stored on the first computer with the data stored on the second computer when the means for storing no longer has a copy of the log entry that occurs after the most recently processed log entry.

15. The system of claim 14, wherein said means for processing comprises a plurality of routines.

16. The system of claim 15, wherein said second storage device comprises a plurality of volumes.

17. The system of claim 16, wherein each of the plurality of routines communicates with one of the plurality of volumes.