Non-disruptive data movement and node rebalancing in extreme OLTP environments
First Claim
1. A method, comprising:
- moving a data partition containing a data page, in response to a request for moving the data partition, from a source node to a destination node while simultaneously allowing an application read and write access to the data partition on the source node, wherein the source node retains ownership of the data partition;
propagating data changes from the source node to the destination node, wherein the data changes were applied by the application to the data partition on the source node during the moving;
iteratively propagating additional data changes from the source node to the destination node until a number of remaining additional data changes is below a threshold value, wherein the additional data changes were applied by the application to the data partition on the source node during a previous propagation;
limiting the read and write access of the application to the source node containing the data partition at the conclusion of the iterative propagation;
instructing the application to perform future read and write operations on the destination node;
switching ownership of the data partition to the destination node; and
resuming the read and write access of the application to the data partition.
1 Assignment
0 Petitions
Accused Products
Abstract
Systems and methods for non-disruptive cache startup, shutdown, data partition movement, and node rebalancing for a data fabric within a data grid in a database environment. The data grid is a clustered in-memory database (IMDB) cache comprising one or more data fabrics including multiple IMDB cache nodes. The method takes a time tag and moves data from source to destination nodes while allowing updates to data on the source nodes. The method propagates updates occurring after the time tag to destination nodes from source nodes and switches applications and ownership to the destination nodes. Another method slices data for a partitioned fabric and stores slices on nodes. Another method starts a data fabric by moving data from a grid backend to the nodes. In an embodiment, another method rebalances nodes in a non-disruptive manner. When a fabric is shutdown, the method saves data from the nodes to the grid backend.
-
Citations
22 Claims
-
1. A method, comprising:
-
moving a data partition containing a data page, in response to a request for moving the data partition, from a source node to a destination node while simultaneously allowing an application read and write access to the data partition on the source node, wherein the source node retains ownership of the data partition; propagating data changes from the source node to the destination node, wherein the data changes were applied by the application to the data partition on the source node during the moving; iteratively propagating additional data changes from the source node to the destination node until a number of remaining additional data changes is below a threshold value, wherein the additional data changes were applied by the application to the data partition on the source node during a previous propagation; limiting the read and write access of the application to the source node containing the data partition at the conclusion of the iterative propagation; instructing the application to perform future read and write operations on the destination node; switching ownership of the data partition to the destination node; and resuming the read and write access of the application to the data partition. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A non-transitory computer readable storage medium having instructions stored thereon that, in response to execution by a computing device, cause the computing device to perform operations for performing a non-disruptive cache shut down in a database environment, the operations comprising:
-
receiving a command to shut down a data fabric within a data grid in the database environment; creating a time tag indicating the state of data at a point in time within an in-memory database (IMDB) system nodes; moving a data partition containing a data page associated with the time tag, in response to the receiving, from the IMDB system node to a disk resident database (DRDB), while simultaneously allowing an application read and write access to the data partition on the IMDB system nodes, wherein the IMDB system node retains ownership of the data partition; propagating data changes from the IMDB system node to the DRDB, wherein the data changes were applied by the application to the data partition on the IMDB system node during the moving; iteratively propagating additional data changes from the IMDB system node to the DRDB until a sufficiently small number of additional data changes remain, wherein the additional data changes were applied by the application to the data partition on the IMDB system node during a previous propagation iteration, and wherein the sufficiently small number is based on a threshold value; upon finishing the iterative propagation, limiting the read and write access of the application to the source node containing the data partition; and shutting down the data fabric.
-
-
11. A computer program product comprising a non-transitory computer readable storage medium having computer program logic stored thereon that, in response to execution by a processor, causes the processor to perform a non-disruptive cache startup in a database environment by performing operations comprising:
-
receiving a command to start up a data fabric within a data grid in the database environment; reading metadata within the data grid indicating a data partition associated with an IMDB system node; moving the data partition containing a data page from a disk resident database (DRDB) to the IMDB system node, in response to the receiving, while simultaneously allowing an application read and write access to the data partition on the DRDB, wherein the DRDB retains ownership of the data partition; propagating data changes from the DRDB to the IMDB system node, wherein the data changes were applied by the application to the data partition on the DRDB during the moving; iteratively propagating additional data changes from the source node to the destination node until a number of remaining additional data changes is below a threshold value, wherein the additional data changes were applied by the application to the data partition on the source node during a previous propagation; limiting the read and write access of the application to the DRDB containing the data partition on the conclusion of the iterative propagation; instructing the application to perform future read and write operations on the IMDB system node; switching ownership of the data page to IMDB system node; and resuming the read and write access of the application to the data page. - View Dependent Claims (12, 13, 14)
-
-
15. A system, comprising:
-
a data fabric comprising a first in-memory database (IMDB) system node and a second IMDB system node, the first IMDB system node and the second IMDB system node organized to store data according to a data granularity type; a load determining module configured to determine a first load value for the first IMDB system node and a second load value for the second IMDB system node, wherein the load determining module is further configured to identify, based on the first and second load values, a more-loaded IMDB system node and a less-loaded IMDB system node; a rebalancing module configured to, in response to a request to rebalance IMDB system nodes of the data fabric, move a data partition containing a data page from the more-loaded IMDB system node of the first IMDB system node and the second IMDB system node to the less-loaded IMDB system node to better distribute a data or processing load within the data fabric and retain data partition ownership with the more-loaded IMDB system node, wherein the rebalancing module is further configured to simultaneously allow an application read and write access to the data partition on the more loaded IMDB system node during the moving; an update catch-up module configured to; propagate data changes from the more-loaded IMDB system node to the less-loaded IMDB system node, wherein the data changes were applied by the application to the data partition on the more-loaded IMDB system node during the moving, and iteratively propagate additional data changes from the source node to the destination node until a number of remaining additional data changes is below a threshold value, wherein the additional data changes were applied by the application to the data partition on the source node during a previous propagation; and a switching module configured to; limit the read and write access of the application to the more-loaded IMDB system node containing the data partition at the conclusion of the iterative propagation instruct the applications to perform future read and write operations on the less-loaded IMDB system node; switch ownership of the data partition to the less-loaded IMDB system node; and resuming the read and write access of the application to the data partition. - View Dependent Claims (16, 17, 18, 19, 20, 21, 22)
-
Specification