SYSTEMS AND METHODS FOR MANAGING REPLICATED DATABASE DATA

• US 20100145909A1
• Filed: 04/15/2009
• Published: 06/10/2010
• Est. Priority Date: 12/10/2008
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Ngo, David
• Granted Patent: US 8,204,859 B2
• Time in Patent Office: Days
• US Class Current: 707/611
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Systems and methods for replicating database data and generating read-only copies of the replicated data in a clean shutdown state. For example, systems can include a tracking module (e.g., a filter driver) that monitors transactions from a database application to a source storage device to generate log entries having at least one marker indicating a known good state of the application. The systems further include a computer coupled to a target storage device comprising a database and log files. The computer processes the transactions, based on the log entries, to replicate data to the target storage device; performs a first snapshot on data stored in the database and log files; replays into the database data stored in the log files; performs another snapshot on the database; and reverts the database back to a state in which the database existed at the time of the first snapshot.

Claims

1. A method for managing replicated data in a database system, the method comprising:

monitoring a plurality of data transactions associated with a database application, the plurality of data transactions operative to write data to at least one source storage device;

copying the data to at least one target storage device based at least in part on the plurality of data transactions, wherein said at least one target storage device comprises a target database and target transaction log files, wherein said monitoring and copying is performed without shutting down the database application;

generating a first snapshot of at least a portion of the data stored in the target database and the target transaction log files;

replaying into the target database, following the first snapshot, data stored in the target transaction log files as one or more transaction logs;

generating, following said replaying, a second snapshot of at least a portion of the target database, the second snapshot being indicative of stored data from the database application in a recoverable state; and

reverting the target database back to a state in which the target database existed at the time of said generating the first snapshot.

2. The method of claim 1, wherein the second snapshot is further indicative of stored data from the database application in a clean shutdown state.

3. The method of claim 1, wherein said generating the first snapshot is performed based on a marker in the copied data indicating a known good state of the database application.

4. The method of claim 3, further comprising:

quiescing the database application at a time of the known good state of the database application; and

inserting the marker in the copied data.

5. The method of claim 1, wherein said copying the data further comprises transmitting between the at least one source storage device and the at least one target storage device a plurality of log entries indicative of the plurality of data transactions associated with the database application.

6. A system for performing data management operations in a computer network environment, the system comprising:

a database application configured to execute on a source computer;

a first storage device coupled to the source computer to receive data transactions from the database application;

at least one module configured to monitor the data transactions and to generate a plurality of log entries based on the data transactions, at least one of the plurality of log entries having a marker indicative of a time of a known good state of the database application;

a second storage device comprising a target database and target transaction log files; and

a target computer coupled to the second storage device and configured to;

process, based on the plurality of log entries, the data transactions to replicate data to the second storage device,

perform a first snapshot operation on data stored in both the target database and the target transaction log files,

following the first snapshot operation, replay into the target database data stored in the target transaction log files,

following said replaying, perform a second snapshot operation on at least a portion of the target database, and

revert the target database back to a state in which the target database existed at the time of said performing the first snapshot.

7. The system of claim 6, wherein the target computer further comprises a replication module having a plurality of threads configured to replay the data transactions to replicate the data to the second storage device.

8. The system of claim 6, wherein the target computer comprises at least one application programming interface configured to commit to the target database data stored in the transaction log files.

9. The system of claim 6, wherein the at least one module comprises a filter driver.

10. The system of claim 6, wherein performing the second snapshot operation produces a read-only copy of the target database data, the read-only copy being indicative of database data that can be read without the use of the database application.

11. A method for copying data generated on a source database system in a computer network, the method comprising:

processing replication data indicative of data transactions generated by a database application executing on a source system, the database transactions being directed to operations on a source database of a source storage device;

replaying the replication data on a target storage device to copy the data transactions to the target storage device, the target storage device comprising a target database and target transaction log files;

creating a first read-only copy of at least a portion of data stored in the target database and the target transaction log files;

committing to the target database transaction logs stored in the target transaction log files, said committing following said creating the first read-only copy;

creating, following said committing, a second read-only copy of at least a portion of the target database, the second read-only copy being indicative of database data from the database application in a recoverable state; and

reverting the target database to a state in which the target database existed at the time of said creating the first read-only copy.

12. The method of claim 11, further comprising:

moving to a first memory location data overwritten in the target database during said committing;

moving to a second memory location data added to the target database during said committing; and

wherein said reverting comprises moving the overwritten data back to the target database following said creating the second read-only copy.

13. The method of claim 11, further comprising suspending said replaying prior to said creating the first read-only copy.

14. The method of claim 13, wherein said suspending is performed at a time at which the data stored on the target database is in a known good state.

15. The method of claim 11, further comprising suspending said replaying according to one or more storage policies defined by a user.

16. The method of claim 11, further comprising synchronizing the source database and the target database following said reverting.

17. The method of claim 16, wherein the second read-only copy of the target database maintains a copy of the data of the target database in a clean shutdown state.

18. The method of claim 17, further comprising reading the copy of the data maintained by the second read-only copy with one or more application programming interfaces while the database application is offline with respect to the copy of the data.

19. A system for performing data management operations in a computer network environment, the system comprising:

means for processing replication data indicative of database operations generated by a database application executing on a source computer, the database operations being directed to data on a source storage device;

means for replaying the replication data on a target storage device to copy data to the target storage device, the target storage device comprising a target database and transaction log files;

means for performing a first snapshot operation on data stored in both the target database and the target transaction log files;

means for replaying into the target database data stored in the transaction log files;

said means for performing being further configured to perform a second snapshot operation on at least a portion of the target database following replaying of the data stored in the transaction log files, the second snapshot being indicative of database data from the database application in a clean shutdown state; and

means for reverting the target database back to a state in which the target database existed at the time of performing the first snapshot.

20. The system of claim 19, further comprising:

means for storing data overwritten in the target database during said replaying into the target database; and

means for storing data added to the target database during said replaying into the target database.