Systems and methods for managing replicated database data

• US 8,204,859 B2
• Filed: 04/15/2009
• Issued: 06/19/2012
• Est. Priority Date: 12/10/2008
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Ngo, David
• Primary Examiner(s): Lewis, Cheryl
• Assistant Examiner(s): Waldron, Scott A
• Application Number:
  US12/424,115
• Publication Number:
  US 20100145909A1
• Time in Patent Office: 1,161 Days
• Field of Search: None
• US Class Current: 707/639
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Systems and methods for replicating database data and generating read-only copies of the replicated data in a clean shutdown state. For example, systems can include a tracking module (e.g., a filter driver) that monitors transactions from a database application to a source storage device to generate log entries having at least one marker indicating a known good state of the application. The systems further include a computer coupled to a target storage device comprising a database and log files. The computer processes the transactions, based on the log entries, to replicate data to the target storage device; performs a first snapshot on data stored in the database and log files; replays into the database data stored in the log files; performs another snapshot on the database; and reverts the database back to a state in which the database existed at the time of the first snapshot.

Claims

1. A method for managing replicated data in a database system, the method comprising:

monitoring a plurality of data transactions associated with a database application, the plurality of data transactions operative to write data to at least one source storage device;

copying the data to at least one target storage device based at least in part on the plurality of data transactions, wherein said at least one target storage device comprises a target database and target transaction log files, wherein said monitoring and copying is performed without shutting down the database application;

generating a first snapshot of at least a portion of the data stored in the target database and the target transaction log files, the first snapshot being indicative of stored data from the database application in a dirty shutdown state that prevents the first snapshot from being read or otherwise accessed by applications other than the database application, a stored indication of the dirty shutdown state being associated with the first snapshot;

replaying into the target database, following the first snapshot, data stored in the target transaction log files as one or more transaction logs such that the data stored in the target database following said replaying becomes out of sync with the data stored on the source storage device;

generating, following said replaying, a second snapshot of at least a portion of the target database, the second snapshot being indicative of stored data from the database application in a recoverable and clean shutdown state; and

reverting, following said generating the second snapshot, the target database back to a state in which the target database existed at the time of said generating the first snapshot, wherein the data stored in the target database following said reverting is re-synchronized with the data stored on the source storage device, wherein the second snapshot can be read or otherwise accessed without the use of the database application.

2. The method of claim 1, wherein said generating the first snapshot is performed based on a marker in the copied data indicating a known good state of the database application.

3. The method of claim 2, further comprising:

quiescing the database application at a time of the known good state of the database application; and

inserting the marker in the copied data.

4. The method of claim 1, wherein said copying the data further comprises transmitting between the at least one source storage device and the at least one target storage device a plurality of log entries indicative of the plurality of data transactions associated with the database application.

5. The method of claim 1, further comprising:

moving to a first memory location data overwritten in the target database during said replaying;

moving to a second memory location data added to the target database during said replaying; and

wherein said reverting comprises moving the overwritten data back to the target database following said generating the second snapshot.

6. The method of claim 1, further comprising suspending said copying prior to said generating the first snapshot.

7. The method of claim 6, wherein said suspending is performed at a time at which the data stored on the target database is in a known good state.

8. The method of claim 1, further comprising suspending said copying according to one or more storage policies defined by a user.

9. The method of claim 1, further comprising synchronizing the source database and the target database following said reverting.

10. The method of claim 1, further comprising reading the second snapshot with one or more application programming interfaces while the database application is offline with respect to the data in the second snapshot.

11. A system for performing data management operations in a computer network environment, the system comprising:

a database application configured to execute on a source computer;

a first storage device coupled to the source computer to receive data transactions from the database application;

at least one module configured to monitor the data transactions and to generate a plurality of log entries based on the data transactions, at least one of the plurality of log entries having a marker indicative of a time of a known good state of the database application;

a second storage device comprising a target database and target transaction log files; and

a target computer coupled to the second storage device and configured to;

process, based on the plurality of log entries, the data transactions to replicate data to the second storage device,

perform a first snapshot operation on data stored in both the target database and the target transaction log files to create a first snapshot, the first snapshot being indicative of stored data from the database application in a dirty shutdown state that prevents the first snapshot from being read or otherwise accessed by applications other than the database application, a stored indication of the dirty shutdown state being associated with the first snapshot,

following the first snapshot operation, replay into the target database data stored in the target transaction log files such that the data stored in the target database following the replay becomes out of sync with the data stored on the first storage device,

following said replaying, perform a second snapshot operation on at least a portion of the target database to create a second snapshot, the second snapshot being indicative of stored data from the database application in a recoverable and clean shutdown state, and

revert, following the performance of the second snapshot, the target database back to a state in which the target database existed at the time of said performing the first snapshot wherein the data stored in the target database following said reverting is re-synchronized with the data stored on the first storage device, wherein the second snapshot can be read or otherwise accessed without the use of the database application.

12. The system of claim 11, wherein the target computer further comprises a replication module having a plurality of threads configured to replay the data transactions to replicate the data to the second storage device.

13. The system of claim 11, wherein the target computer comprises at least one application programming interface configured to commit to the target database data stored in the transaction log files.

14. The system of claim 11, wherein the at least one module comprises a filter driver.

15. The system of claim 11, wherein performing the second snapshot operation produces a read-only copy of the target database data, the read-only copy being indicative of database data that can be read without the use of the database application.

16. The system of claim 11, wherein the target computer is further configured to:

store data overwritten in the target database during the replaying of the data stored in the transaction log files into the target database; and

store data added to the target database during the replaying of the data stored in the transaction log files into the target database.