System and method for reconciling transactions between a replication system and a recovered database

• US 20020049776A1
• Filed: 02/12/2001
• Published: 04/25/2002
• Est. Priority Date: 02/11/2000
• Status: Active Grant

• Current Assignee: Quest Software, Inc.
• Original Assignee: Quest Software, Inc.
• Inventors: Aronoff, Eyal M., Collings, James W.
• Granted Patent: US 7,065,538 B2
• Time in Patent Office: Days
• US Class Current: 707/200
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

The invention is a reconcile system for providing data replication between a source system and a target system. The reconcile system includes a reconcile process that determines a recovery position for a failed or otherwise desynchronized target system, and purges queues of the reconcile system of stale transactions occurring before the recovery position, thereby ensuring synchronization between the source and target systems.

Claims

1. A device for performing replication between a source system and a target system, the device comprising:

a source system having data files, and log files storing transactions corresponding to changes made to the data files;

a target system; and

a replication system performing replication of at least portions of the data files of the source system to the target system by reading the log files and posting the changes from the log files to the target system, the replication system comprising;

transaction-level poster queues, each poster queue storing statements corresponding to a particular transaction from the source system, and

a reconcile process which purges transactions from the poster queues when the transactions have already been applied to the target system during recovery of the target system.

2. A replication system for replicating at least portions of the data contained in a source database to a target database, the replication system comprising:

poster queues which store information corresponding to changes made to at least portions of a source system;

at least one poster process which reads the information stored in the poster queues and generates commands interpretable by a target system and designed to change the target system to reflect the changes made to the at least portions of the source system; and

a reconcile process which purges stale information stored in the poster queues, the stale information corresponding to changes made to the target system during the instantiation or recovery thereof.

3. The replication system of claim 2, wherein the information comprises transactions.

4. The replication system of claim 2, wherein the at least one poster process reads a completion indicator from the poster queues, wherein the completion indicator corresponds to one or more finalized changes made to the source system.

5. The replication system of claim 4, wherein the completion indicator corresponds to a COMMIT statement.

6. The replication system of claim 2, wherein the reconcile process employs placement indicators to determine which information stored in the poster queues is stale.

7. The replication system of claim 6, wherein one of the placement indicators corresponds to a recovery marker placed by the target system, wherein the recovery marker identifies how much of the information the target system already applied during recovery thereof.

8. The replication system of claim 6, wherein one of the placement indicators corresponds to a particular portion of the information.

9. The replication system of claim 6, wherein each placement indicator comprises a sequence number identifying a log file where a particular portion of the information originated.

10. The replication system of claim 6, wherein each placement indicator comprises a displacement number identifying the displacement within a log file where a particular portion of the information originated.

11. The replication system of claim 2, further comprising a reader process which reads the information from the source system.

12. The replication system of claim 2, further comprising a reader queue which stores information read from the source system.

13. The replication system of claim 2, wherein the replication includes mirroring at least portions of the source system on at least one target system.

14. The replication system of claim 2, wherein the replication includes load balancing functions based on one of software and hardware configurations of the source and target systems.

15. The replication system of claim 2, wherein the replication provides broadcast functions.

16. The replication system of claim 2, wherein the replication provides consolidation functions.

18. The method of claim 17, further comprising restarting replication.

19. The method of claim 18, wherein the restarting of the replication includes restarting at least one poster process.

20. The method of claim 18, wherein the restarting of the replication includes rolling forward at least some of the information rolled back during the recovery of the copy.

21. The method of claim 17, wherein the creation of the copy includes employing a hot backup mode of a database management system of the source database.

22. The method of claim 17, wherein the recovery of the copy includes employing a database management system associated with the copy.

23. The method of claim 17, wherein the recovery of the copy includes placing a recovery marker in the recovered copy, thereby identifying a recovery position therein.

24. The method of claim 23, wherein the placement of the recovery marker occurs substantially near the end of recovering the copy.

25. The method of claim 23, wherein the reconciling finds the recovery marker and the stale transactions of the information correspond to those transactions that were completed on the source system before the placement of the recovery marker.

27. The method of claim 26, wherein the transaction data is read from the source database.

28. The method of claim 26, wherein the placement indicator of the completed transaction comprises a sequence number of a log file of the source database.

29. The method of claim 28, wherein the sequence number uniquely identifies the log file.

30. The method of claim 26, wherein the placement indicator of the recovery flag comprises a sequence number of a log file of the recovered database.

31. The method of claim 30, wherein the sequence number uniquely identifies the log file.

32. The method of claim 26, wherein the placement indicator comprises a displacement within a log file.

33. The method of claim 26, wherein the recovery flag is placed by a database management system of the recovered database.

17. A method of recovering or instantiating a target database during replication from a source database to the target database, the method comprising:

creating a copy of data from a source database;

recovering the copy; and

reconciling information contained in a replication system with information stored in the recovered copy, thereby purging stale transactions from the replication system.

26. A method of reconciling transactional information stored in a replication system with a recovered database, the method comprising:

parsing a log file of a recovered database to determine a placement indicator of a recovery flag;

reading transaction data corresponding to changes made to a source database to determine placement indicators of completed transactions; and

purging the transactional data when the placement indicator corresponding to the completed transaction occurred before the placement indicator of the recovery flag.

34. A device comprising:

a source system having a source database management system (SDBMS) which governs the storage of data within the source system and creates a log file tracking changes made to the source system;

a target system having a target database management system (TDBMS) which governs the storage of data within the target system and creates a log file tracking the changes made to the target system; and

a replication system having queues and communicating with the log file of the TDBMS and the log file of the SDBMS, thereby purging from the queues transactions applied after the beginning, but before the completion, of the recovery or instantiation of the target system, wherein the transactions correspond to the changes made to the source system.