Distributed database log recovery
First Claim
1. A non-transitory computer program product storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
- recording, in a data storage application, log entries for a plurality of transactions among nodes in a node hierarchy, the node hierarchy comprising a master node comprising a transaction coordinator and having a plurality of slave nodes, each slave node having a separate log at which its respective log entries are stored, the transaction coordinator storing respective prepare commit positions of each slave node in a commit record;
replaying, prior to replay of log entries at the slave nodes, at least a portion of the master node log entries until a first replay position is reached, the first replay position comprising positions of respective commit log records at the slave nodes until which the slave nodes must replay their respective logs in order to come to a transactionally-consistent state with the master node;
replaying, for each slave node and in response to at least one trigger from the master node, at least a portion of its respective log entries until the corresponding slave node reaches an end of its respective log prior to the first replay position;
initiating, by the transaction coordinator of the master node in parallel to at least a portion of the replaying by the slave nodes prior to the first replay position, replay of at least a portion of its log entries subsequent to the first replay position; and
discarding all log entries stored beyond the first replay position.
2 Assignments
0 Petitions
Accused Products
Abstract
Log entries are recorded in a data storage application (such as an in-memory database, etc.) for a plurality of transactions among nodes in a node hierarchy. The node hierarchy comprises master node having a plurality of slave nodes. Thereafter, at least a portion of the master node log entries are replayed until a first replay position is reached. Next, for each slave node, at least a portion of its respective log entries are replayed until the first replay position is reached (or an error occurs). Subsequently, replay of at least a portion of the log entries of the master node that are subsequent to the first replay position is initiated by the master node in parallel to at least a portion of the replaying by the slave nodes. Related apparatus, systems, techniques and articles are also described.
18 Citations
39 Claims
-
1. A non-transitory computer program product storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
-
recording, in a data storage application, log entries for a plurality of transactions among nodes in a node hierarchy, the node hierarchy comprising a master node comprising a transaction coordinator and having a plurality of slave nodes, each slave node having a separate log at which its respective log entries are stored, the transaction coordinator storing respective prepare commit positions of each slave node in a commit record; replaying, prior to replay of log entries at the slave nodes, at least a portion of the master node log entries until a first replay position is reached, the first replay position comprising positions of respective commit log records at the slave nodes until which the slave nodes must replay their respective logs in order to come to a transactionally-consistent state with the master node; replaying, for each slave node and in response to at least one trigger from the master node, at least a portion of its respective log entries until the corresponding slave node reaches an end of its respective log prior to the first replay position; initiating, by the transaction coordinator of the master node in parallel to at least a portion of the replaying by the slave nodes prior to the first replay position, replay of at least a portion of its log entries subsequent to the first replay position; and discarding all log entries stored beyond the first replay position. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
-
16. A method comprising:
-
recording, in a data storage application, log entries for a plurality of transactions among nodes in a node hierarchy, the node hierarchy comprising at least one master node comprising a transaction coordinator and having a plurality of slave nodes, the transaction coordinator coordinating replay among the plurality of slave nodes, each slave node having a separate log at which its respective log entries are stored, the transaction coordinator storing respective prepare commit positions of each slave node in a commit record; replaying, for each master node and prior to replay of log entries at the slave nodes, at least a portion of its log entries until a first replay position is reached, the first replay position comprising positions of respective commit log records at the slave nodes until which the slave nodes must replay their respective logs in order to come to a transactionally-consistent state with the master node; performing, for each master node, a snapshot of a local persistency for the master node; replaying, for each slave node of a corresponding master node and in response to at least one trigger from the corresponding master node, at least a portion of its respective log entries until a maximum recoverable position prior to the first replay position is reached on one of the slave nodes, the maximum recoverable position representing a latest transactionally-consistent state and being specified by a last commit log record replayed by each corresponding master node; performing a snapshot of a local persistency for each slave node after replay to the maximum recoverable position; replaying the log entries on the at least one master node and the corresponding slave nodes to the maximum recoverable position, wherein the at least one master node replays log entries at a point in time subsequent to a point in time at which the corresponding slave nodes are replaying log entries; and discarding all log entries stored beyond the maximum recoverable position. - View Dependent Claims (17, 31, 32, 33, 34, 35, 36, 37, 38, 39)
-
-
18. A system comprising:
-
at least one data processor; and memory coupled to the at least one data processor, the memory storing instructions to cause the at least one data processor to perform operations comprising; recording, in a data storage application, log entries for a plurality of transactions among nodes in a node hierarchy, the node hierarchy comprising at least one master node comprising a transaction coordinator and having a plurality of slave nodes, the transaction coordinator coordinating replay among the plurality of slave nodes, each slave node having a separate log at which its respective log entries are stored, the transaction coordinator storing respective prepare commit positions of each slave node in a commit record; replaying, for each master node and prior to replay of log entries at the slave nodes, at least a portion of its log entries until a first replay position is reached, the first replay position comprising positions of respective commit log records at the slave nodes until which the slave nodes must replay their respective logs in order to come to a transactionally-consistent state with the master node; replaying, for each slave node of a corresponding master node and in response to at least one trigger from the corresponding master node, at least a portion of its respective log entries until a maximum recoverable position prior to the first replay position is reached on one of the slave nodes, the maximum recoverable position representing a latest transactionally-consistent state; replaying, in parallel, the log entries on the at least one master node and the corresponding slave nodes to the maximum recoverable position, wherein the at least one master node replays log entries at a point in time subsequent to a point in time at which the corresponding slave nodes are replaying log entries; and discarding all log entries stored beyond the maximum recoverable position. - View Dependent Claims (19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30)
-
Specification