Rolling cache configuration for a data replication system

• US 9,208,210 B2
• Filed: 12/23/2013
• Issued: 12/08/2015
• Est. Priority Date: 12/19/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Erofeev, Andrei
• Primary Examiner(s): SCHELL, JOSEPH O
• Application Number:
  US14/138,666
• Publication Number:
  US 20140181029A1
• Time in Patent Office: 715 Days
• Field of Search: 714/6.2, 714/6.23, 714/6.31, 714/6.32, 714/12, 710/53, 710/54, 710/56
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Performing data management operations on replicated data in a computer network. Log entries are generated for data management operations of an application executing on a source system. Consistency point entries are used to indicate a time of a known good, or recoverable, state of the application. A destination system is configured to process a copy of the log and consistency point entries to replicate data in a replication volume, the replicated data being a copy of the application data on the source system. When the replicated data represents a known good state of the application, as determined by the consistency point entries, the destination system(s) may perform a storage operation (e.g., snapshot, backup) to copy the replicated data and to logically associate the copied data with a time information (e.g., time stamp) indicative of the source system time when the application was in the known good state.

Claims

1. A method for performing data replication, the method comprising:

receiving, at one or more destination computers, copies of log entries stored in a memory of one or more source computers, wherein the log entries stored in the memory of the one or more source computer are associated with data management operations that modify data on one or more first storage devices, wherein the copies of the log entries on the one or more destination computers are located remotely from the one or more source computers, wherein each copy of the log entries has a unique identifier associated therewith;

replaying, on the one or more destination computers, the copies of the log entries to replicate the data stored on the one or more first storage devices to create replicated data on one or more second storage devices; and

in response to an interruption in replaying the copies of the log entries on the one or more destination computers, using a first unique identifier associated with a recently processed log entry on the one or more destination computers to identify a subsequent log entry in the memory of the one or more source computers, wherein the subsequent log entry has a second unique identifier that occurs after the first unique identifier of a most recently processed log entry; and

performing a mirror operation to synchronize the data stored on the one or more first storage devices with data stored on the one or more destination computers when the subsequent log entry is no longer stored in the memory of the one or more source computers.

2. The method of claim 1, further comprising storing additional copies of the log entries in the memory in response to a network failure.

3. The method of claim 1, comprising resuming the replaying of the one or more copies of the log entries by the one or more destination computes.

4. The method of claim 1, wherein the memory is a rolling cache.

5. The method of claim 1, wherein the memory is first-in-first-out memory.

6. The method of claim 1, wherein the receiving is performed without waiting for an acknowledgement from the one or more destination computers.

7. The method of claim 1, additionally comprising detecting a network failure.

8. The method of claim 7, additionally comprising, following the network failure, querying the one or more destination computers for the unique identifier of a most recently processed log entry.

9. The method of claim 1, wherein the subsequent log entry sequentially follows the most recently processed log entry.

10. The method of claim 1, wherein the subsequent log entry is removed from the memory after the log entries exceed a storage threshold.

11. A system for performing data replication, the system comprising:

one or more destination computers that receive copies of log entries stored in a memory of one or more source computers, wherein the log entries stored in the memory of the one or more source computer are associated with data management operations that modify data on one or more first storage devices, wherein copies of the log entries on the one or more destination computers are located remotely from the one or more source computers, wherein each copy of the log entries has a unique identifier associated therewith;

one or more second storage devices, wherein the one or more destination computers replay the copies of the log entries to replicate the data stored on the one or more first storage devices to create replicated data on one or more second storage devices;

wherein in response to an interruption in the replaying of the copies of the log entries, the one or more destination computers uses a first unique identifier associated with a recently processed log entry on the one or more destination computers to identify a subsequent log entry in the memory of the one or more source computers, wherein the subsequent log entry has a second unique identifier that occurs after the first unique identifier of a most recently processed log entry; and

a storage manager that performs a mirror operation to synchronize the data stored on the one or more first storage devices with data stored on the one or more destination computers when the subsequent log entry is no longer stored in the memory of the one or more source computers.

12. The system of claim 11, wherein the storage manager stores additional copies of the log entries in the memory in response to a network failure.

13. The system of claim 11, wherein the one or more destination computers resume the replaying of the one or more copies of the log entries.

14. The system of claim 11, wherein the memory is a rolling cache.

15. The system of claim 11, wherein the memory is first-in-first-out memory.

16. The system of claim 11, wherein the one or more destination computers do not acknowledge the receiving of the copies of the log entries.

17. The system of claim 11, wherein the one or more destination computers detect a network failure.

18. The system of claim 17, wherein the storage manager, following the network failure, queries the one or more destination computers for the unique identifier of a most recently processed log entry.

19. The system of claim 11, wherein the subsequent log entry sequentially follows the most recently processed log entry.

20. The system of claim 11, wherein the storage manager removes the subsequent log entry from the memory after the log entries exceed a storage threshold.