Log forwarding to avoid deadlocks during parallel log replay in asynchronous table replication

• US 9,965,359 B2
• Filed: 03/13/2015
• Issued: 05/08/2018
• Est. Priority Date: 11/25/2014
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Lee, Juchang, Park, Chang-Gyoo, Na, Hyoung-Jun, Kim, Deok-Hoe
• Primary Examiner(s): Burke, Jeffrey A
• Assistant Examiner(s): Vu, Thong
• Application Number:
  US14/657,854
• Publication Number:
  US 20160147858A1
• Time in Patent Office: 1,152 Days
• Field of Search: 707615, 707685, 707612, 707611, 707633, 707610, 707678, 707607, 707613
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1474 in transactions G06F16/20 takes precedence
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/835 Timestamp

Abstract

Disclosed herein are system, method, and computer program product embodiments for removing a deadlock during replication from distributed source tables to a replica node. An embodiment operates by detecting a deadlock at a parallel log replayer at a replica node. A first replication log entry from a queue at the parallel log replayer is then selected based on whether removing the first replication log entry from the queue removes the deadlock. The first replication log entry is then forwarded to a waiting queue. A second replication log entry is then replayed at the parallel log replayer. After replaying the second replication log entry, the first replication log entry is replayed at the parallel log replayer.

Claims

1. A computer implemented method for removing a deadlock during replication from distributed source tables to a replica node, comprising:

detecting, by at least one processor, a deadlock at a parallel log replayer at a replica node;

selecting, by the at least one processor, a first replication log entry from a queue of a plurality of queues of the parallel log replayer based on whether removing the first replication log entry from the queue removes the deadlock, wherein the first replication log entry comprises an operation type, a table identifier, and a transaction identifier for a changed row;

inserting, by the at least one processor, the first replication log entry into a waiting queue;

removing, by the at least one processor, the first replication log entry from the queue of the plurality of queues of the parallel log replayer;

replaying, by the at least one processor, a second replication log entry from the queue of the plurality of queues at the parallel log replayer after removing the first replication log entry from the queue of the plurality of queues of the parallel log replayer; and

replaying, by the at least one processor, the first replication log entry from the waiting queue at the parallel log replayer after replaying the second replication log entry;

wherein at least one of the detecting, selecting, inserting, removing, replaying the second replication log entry, and replaying the first replication log entry are performed by one or more computers.

2. The method of claim 1, wherein the detecting is performed based on whether the parallel log replayer is blocked.

3. The method of claim 1, the selecting the first replication log entry further comprising:

selecting the first replication log entry from the head of the queue of the plurality of queues of the parallel log replayer.

4. The method of claim 1, the detecting the deadlock further comprising:

determining whether a cyclic dependency relationship exists among replication log entries at the parallel log replayer and transaction commit log entries at a transaction commit log replayer.

5. The method of claim 1, the detecting the deadlock further comprising:

determining whether a cyclic dependency relationship exists among the replication log entries at the parallel log replayer and the transaction commit log entries at a transaction commit log replayer based on transaction IDs associated with the replication log entries and the transaction commit log entries.

6. The method of claim 1, wherein the selecting, inserting, and removing steps are repeated based on whether the deadlock is removed.

7. The method of claim 1, wherein the first replication log entry further comprises a new row image for the changed row.

8. The method of claim 1, wherein the first replication log entry further comprises a row-ID column value associated with the changed row.

9. The method of claim 1, wherein the first replication log entry is based on an insert operation, update operation, or delete operation.

10. A system, comprising:

a memory; and

at least one processor coupled to the memory and configured to;

detect a deadlock at a parallel log replayer at a replica node;

select first replication log entry from a queue of a plurality of queues of the parallel log replayer based on whether removing the first replication log entry from the queue removes the deadlock, wherein the first replication log entry comprises an operation type, a table identifier, and a transaction identifier for a changed row;

insert, the first replication log entry into a waiting queue;

remove the first replication log entry from the queue of the plurality of queues of the parallel log replayer;

replay a second replication log entry from the queue of the plurality of queues at the parallel log replayer after removing the first replication log entry from the queue of the plurality of queues of the parallel log replayer; and

replay the first replication log entry from the waiting queue at the parallel log replayer after replaying the second replication log entry.

11. The system of claim 10, wherein the detecting is performed based on whether the parallel log replayer is blocked.

12. The system of claim 10, the at least one processor further configured to:

select the first replication log entry from the head of the queue of the plurality of queues of the parallel log replayer.

13. The system of claim 10, the at least one processor further configured to:

determine whether a cyclic dependency relationship exists among replication log entries at the parallel log replayer and transaction commit log entries at a transaction commit log replayer.

14. The system of claim 10, the at least one processor further configured to:

determine whether a cyclic dependency relationship exists among the replication log entries at the parallel log replayer and the transaction commit log entries at a transaction commit log replayer based on transaction IDs associated with the replication log entries and the transaction commit log entries.

15. The system of claim 10, the at least one processor further configured to:

repeat the selecting, inserting, and removing based on whether the deadlock is removed.

16. A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:

detecting a deadlock at a parallel log replayer at a replica node;

selecting a first replication log entry from a queue of a plurality of queues of the parallel log replayer based on whether removing the first replication log entry from the queue removes the deadlock, wherein the first replication log entry comprises an operation type, a table identifier, and a transaction identifier for a changed row;

inserting the first replication log entry into a waiting queue;

removing the first replication log entry from the queue of the plurality of queues of the parallel log replayer;

replaying a second replication log entry from the queue of the plurality of queues at the parallel log replayer after removing the first replication log entry from the queue of the plurality of queues of the parallel log replayer; and

replaying the first replication log entry from the waiting queue at the parallel log replayer after replaying the second replication log entry.

17. The non-transitory computer-readable device of claim 16, wherein the detecting is performed based on whether the parallel log replayer is blocked.

18. The non-transitory computer-readable device of claim 16, the selecting comprising:

selecting the first replication log entry from the head of the queue of the plurality of queues of the parallel log replayer.

19. The non-transitory computer-readable device of claim 16, the detecting comprising:

determining whether a cyclic dependency relationship exists among replication log entries at the parallel log replayer and transaction commit log entries at a transaction commit log replayer.

20. The non-transitory computer-readable device of claim 16, the detecting comprising:

determining whether a cyclic dependency relationship exists among the replication log entries at the parallel log replayer and the transaction commit log entries at a transaction commit log replayer based on transaction IDs associated with the replication log entries and the transaction commit log entries.

21. The non-transitory computer-readable device of claim 16, wherein the selecting, inserting, and removing operations are repeated based on whether the deadlock is removed.