Distributed Database Log Recovery

• US 20130117237A1
• Filed: 11/07/2011
• Published: 05/09/2013
• Est. Priority Date: 11/07/2011
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP AG (SAP SE)
• Inventors: Thomsen, Dirk, Schreter, Ivan
• Granted Patent: US 9,058,371 B2
• Time in Patent Office: Days
• US Class Current: 707/683
• CPC Class Codes: G06F 11/1474 in transactions G06F16/20 takes precedence
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Log entries are recorded in a data storage application (such as an in-memory database, etc.) for a plurality of transactions among nodes in a node hierarchy. The node hierarchy comprises master node having a plurality of slave nodes. Thereafter, at least a portion of the master node log entries are replayed until a first replay position is reached. Next, for each slave node, at least a portion of its respective log entries are replayed until the first replay position is reached (or an error occurs). Subsequently, replay of at least a portion of the log entries of the master node that are subsequent to the first replay position is initiated by the master node in parallel to at least a portion of the replaying by the slave nodes. Related apparatus, systems, techniques and articles are also described.

Claims

1. A computer program product comprising a non-transitory machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:

recording, in a data storage application, log entries for a plurality of transactions among nodes in a node hierarchy, the node hierarchy comprising a master node having a plurality of slave nodes;

replaying at least a portion of the master node log entries until a first replay position is reached;

replaying, for each slave node, at least a portion of its respective log entries until the first replay position is reached; and

initiating, by the master node in parallel to at least a portion of the replaying by the slave nodes, replay of at least a portion of its log entries subsequent to the first replay position.

2. A computer program product as in claim 1, wherein the node hierarchy comprises at least two master nodes, each having corresponding slave nodes, and wherein the first replay position differs for at least two of the master nodes.

3. A computer program product as in claim 1, wherein the operations further comprise:

performing a snapshot of a local persistency for the master node after replay to the first replay position.

4. A computer program product as in claim 3, wherein the snapshot is performed using shadow-paging.

5. A computer program product as in claim 1, wherein the operations further comprise:

performing a snapshot of a local persistency for each slave node after replay to the first replay position.

6. A computer program product as in claim 5, wherein the snapshot of the local persistency is performed using shadow-paging.

7. A computer program product as in claim 1, wherein:

the slave nodes replay their log entries to reach a transactionally-consistent state with the master node;

the data storage application is implemented on a distributed database system that includes the master node and the plurality of slave nodes; and

the replaying on the master node and the plurality of slave nodes occurs on the distributed database system.

8. A computer program product as in claim 1, wherein the data storage application comprises an in-memory database.

9. A computer program product as in claim 1, wherein the operations further comprise:

stopping, by at least one of the slave nodes, replay prior to the first replay position if a problem occurs during replay.

10. A computer program product as in claim 9, wherein the problem comprises a corrupt or missing log.

11. A computer program product as in claim 9, wherein the operations further comprise:

reporting, by the stopped at least one of the slave nodes, a maximum recoverable replay position prior to the first replay position to the master node.

12. A computer program product as in claim 11, wherein the operations further comprise:

resetting the master node persistency to the maximum recoverable replay position.

13. A computer program product as in claim 12, wherein the operations further comprise:

instructing each slave node by the master node to reset its persistency to the maximum recoverable replay position.

14. A computer program product as in claim 13, wherein the operations further comprise:

adding a filter by the master node preventing replay by the slave nodes beyond the maximum recoverable replay position.

15. A computer program product as in claim 14, wherein the operations further comprise:

discarding log entries stored at the master node and the corresponding slave nodes beyond the maximum recoverable replay position.

16. A method comprising:

recording, in a data storage application, log entries for a plurality of transactions among nodes in a node hierarchy, the node hierarchy comprising at least one master node having a plurality of slave nodes;

replaying, for each master node, at least a portion of its log entries until a first replay position is reached;

replaying, for each slave node of a corresponding master node, at least a portion of its respective log entries until a maximum recoverable position prior to the first replay position is reached on one of the slave nodes, the maximum recoverable position representing a latest transactionally-consistent state; and

replaying the log entries on the at least one master node and the corresponding slave nodes to the maximum recoverable position.

17. A method as in claim 16 further comprising:

performing a snapshot of a local persistency for the master node after replay to the first replay position; and

performing a snapshot of a local persistency for each slave node after replay to the first replay position.

18. A method as in claim 17, wherein the snapshots are performed using shadow-paging.

19. A system comprising:

at least one data processor;

memory coupled to the at least one data processor, the memory storing instructions to cause the at least one data processor to perform operations comprising;

recording, in a data storage application, log entries for a plurality of transactions among nodes in a node hierarchy, the node hierarchy comprising at least one master node having a plurality of slave nodes;

replaying, for each master node, at least a portion of its log entries until a first replay position is reached;

replaying, for each slave node of a corresponding master node, at least a portion of its respective log entries until a maximum recoverable position prior to the first replay position is reached on one of the slave nodes, the maximum recoverable position representing a latest transactionally-consistent state; and

replaying the log entries on the at least one master node and the corresponding slave nodes to the maximum recoverable position.

20. A system as in claim 19, wherein the operations further comprise:

performing a snapshot of a local persistency for the master node after replay to the first replay position; and

performing a snapshot of a local persistency for each slave node after replay to the first replay position.