System and method for providing partition persistent state consistency in a distributed data grid

• US 8,930,316 B2
• Filed: 11/07/2012
• Issued: 01/06/2015
• Est. Priority Date: 10/15/2012
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Lee, Robert H., Gleyzer, Gene
• Primary Examiner(s): To, Baoquoc N
• Application Number:
  US13/671,376
• Publication Number:
  US 20140108347A1
• Time in Patent Office: 790 Days
• Field of Search: 707/644, 707/655
• US Class Current: 707/644
• CPC Class Codes: G06F 11/16 Error detection or correction of the data by redundancy in hardware
• CPC Class Codes: G06F 11/2041 with more than one idle spare processing component
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: H04L 41/0668 by dynamic selection of recovery network elements, e.g. replacement by the most appropriate element after failure
• CPC Class Codes: H04L 43/0811 by checking connectivity
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

A system and method can provide partition persistent state consistency in a distributed data grid. The distributed data grid can provide a plurality of copies of a partition on a plurality of cluster nodes in the distributed data grid, wherein the plurality of cluster nodes includes a primary owner node and one or more backup nodes for the partition. The primary owner node can propagate one or more modifications of the partition from the primary owner node to the one or more backup nodes. The distributed data grid can ensure consistency among the plurality copies of the partition on the plurality of cluster nodes in the distributed data grid.

Claims

1. A method for providing partition persistent state consistency in a distributed data grid, comprising:

providing a primary copy of a partition on a first node in the distributed data grid, wherein the primary copy of the partion is associated with a primary copy version number;

providing a backup copy of the partition on a second node in the distributed data grid, wherein the backup copy of the partition is associated with a backup copy version number;

making a first modification to the primary copy of the partition;

propagating the first modification from the first node to the second node by transmitting a first message containing said first modification from the first node to the second node;

making a second modification to the primary copy of the partition;

propagating the second modification from the first node to the second node by transmitting a second message containing said second modification from the first node to the second node;

receiving the second message at the second node;

determining at the second node whether the first modification has been applied to the backup copy of the partition by checking the backup copy version number; and

deferring applying the second modification on the backup copy of the partition until the first modification is received and applied as indicated by checking the backup copy version number.

2. The method according to claim 1, further comprising:

receiving the first message at the second node;

applying the first modification on the backup copy of the partition; and

applying the second modification on the backup copy of the partition after the first modification has been received and applied as indicated by checking the backup copy version number.

3. The method according to claim 1, further comprising:

retransmitting the second message containing said second modification from the first node to the second node in response to a special message sent from the second node to the first node.

4. The method according to claim 1, further comprising:

sending a special message from the second node to the first node wherein the special message contains the last modification received by the second node;

receiving the special message at the first node;

retransmitting the second message containing said second modification from the first node to the second node in response to the special message.

5. The method according to claim 1, further comprising:

sending an empty message from the second node to the first node;

receiving the empty message at the first node;

retransmitting the second message containing said second modification from the first node to the second node in response to the empty message.

6. The method according to claim 4, further comprising:

providing another backup copy of the partition on a third node in the distributed data grid, wherein the another backup copy of the partition is associated with another backup copy version number;

propagating the first modification from the first node to the third node by transmitting another first message containing said first modification from the first node to the third node;

propagating the second modification from the first node to the third node by transmitting another second message containing said second modification from the first node to the third node;

receiving the another second message at the third node;

determining at the third node whether the first modification has been applied to the another backup copy of the partition by checking the another backup copy version number; and

deferring applying the second modification on the another backup copy of the partition until the first modification is received and applied as indicated by checking the another backup copy version number.

7. The method according to claim 1, further comprising:

providing a new backup copy of the partition on a third node in the distributed data grid, wherein the new backup copy of the partition is associated with a new backup copy version number;

sending a batch of modifications made prior to the second modification to the third node in order to update the new backup copy of the partition on the third node; and

propagating the second modification from the first node to the third node by transmitting another second message containing said second modification from the first node to the third node;

receiving the another second message at the third node;

determining at the third node whether the batch of modifications has been applied to the new backup copy of the partition by checking the new backup copy version number; and

deferring applying the second modification on the new backup copy of the partition until the batch of modifications is received and applied as indicated by checking the new backup copy version number.

8. The method according to claim 1, further comprising:

upon failure of the first node, replacing the first node with a new first node; and

repropagating at least one modification from the new first node to the second node.

9. The method according to claim 1, further comprising:

allowing the first node to manage a state of the partition.

10. The method according to claim 1, further comprising:

supporting a partition ownership change protocol, a partition backup protocol, and a partition transfer protocol in the distributed data grid.

11. A system which provides partition persistent state consistency in a distributed data grid, the system comprising:

one or more microprocessors;

a first node and a second node in the distributed data grid operating on said one or more microprocessors;

a primary copy of a partition on the first node in the distributed data grid, wherein the primary copy of the partion is associated with a primary copy version number;

a backup copy of the partition on the second node in the distributed data grid, wherein the backup copy of the partition is associated with a backup copy version number;

wherein the first node is configured to,

make a first modification to the primary copy of the partition,

propagate the first modification from the first node to the second node by transmitting a first message containing said first modification from the first node to the second node,

make a second modification to the primary copy of the partition, and

propagate the second modification from the first node to the second node by transmitting a second message containing said second modification from the first node to the second node; and

wherein the second node is configured to,

receive the second message,

determine whether the first modification has been applied to the backup copy of the partition by checking the backup copy version number, and

defer applying the second modification on the backup copy of the partition until the first modification is received and applied as indicated by checking the backup copy version number.

12. The system according to claim 11, wherein the second node is further configured to:

receive the first message at the second node;

apply the first modification on the backup copy of the partition; and

apply the second modification on the backup copy of the partition after the first modification has been received and applied as indicated by checking the backup copy version number.

13. The system according to claim 11, wherein the first node is futher configured to:

retransmit the second message containing said second modification from the first node to the second node in response to a special message sent from the second node to the first node.

14. The system according to claim 11, wherein:

the second node is configured to send a special message from the second node to the first node wherein the special message contains the last modification received by the second node; and

the first node is configured to receive the special message, and retransmit the second message containing said second modification from the first node to the second node in response to the special message.

15. The system according to claim 11, wherein:

the second node is configured to send an empty message from the second node to the first node; and

the first node is configured to receive the empty message, and retransmit the second message containing said second modification from the first node to the second node in response to the empty message.

16. The system according to claim 11, further comprising:

a third node in the distributed data grid;

another backup copy of the partition on the third node in the distributed data grid, wherein the another backup copy of the partition is associated with another backup copy version number;

wherein the first node is configured to,

propagate the first modification from the first node to the third node by transmitting a another first message containing said first modification from the first node to the third node,

propagate the second modification from the first node to the third node by transmitting another second message containing said second modification from the first node to the third; and

wherein the third node is configured to,

receive the another second message,

determine whether the first modification has been applied to the another backup copy of the partition by checking the another backup copy version number, and

defer applying the second modification on the another backup copy of the partition until the first modification is received and applied as indicated by checking the another backup copy version number.

17. The system according to claim 11, further comprising:

a third node in the distributed data grid;

a new backup copy of the partition on the third node in the distributed data grid, wherein the new backup copy of the partition is associated with a new backup copy version number;

wherein the first node is further configured to,

send a batch of modifications made prior to the second modification to the third node in order to update the new backup copy of the partition on the third node, and

propagate the second modification from the first node to the third node by transmitting another second message containing said second modification from the first node to the third node; and

wherein the third node is configured to,

receive the another second message,

determine whether the batch of modifications has been applied to the new backup copy of the partition by checking the new backup copy version number, and

defer applying the second modification on the new backup copy of the partition until the batch of modifications is received and applied as indicated by checking the new backup copy version number.

18. The system according to claim 11, further comprising a partition ownership change protocol, a partition backup protocol, and a partition transfer protocol in the distributed data grid.

19. The system according to claim 11, wherein the first node manages a state of the partition.

20. A non-transitory machine readable storage medium having instructions stored thereon for providing partition persistent state consistency in a distributed data grid, which instructions, when executed cause a system to perform steps comprising:

providing a primary copy of a partition on a first node in the distributed data grid, wherein the primary copy of the partion is associated with a primary copy version number;

providing a backup copy of the partition on a second node in the distributed data grid, wherein the backup copy of the partition is associated with a backup copy version number;

making a first modification to the primary copy of the partition;

propagating the first modification from the first node to the second node by transmitting a first message containing said first modification from the first node to the second node;

making a second modification to the primary copy of the partition;

propagating the second modification from the first node to the second node by transmitting a second message containing said second modification from the first node to the second node;

receiving the second message at the second node;

determining at the second node whether the first modification has been applied to the backup copy of the partition by checking the backup copy version number; and

deferring applying the second modification on the backup copy of the partition until the first modification is received and applied as indicated by checking the backup copy version number.