Fault tolerant listener registration in the presence of node crashes in a data grid

• US 9,965,364 B2
• Filed: 05/08/2017
• Issued: 05/08/2018
• Est. Priority Date: 10/31/2014
• Status: Active Grant

• Current Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Original Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Inventors: Markus, Mircea, Burns, William Rosenquist
• Primary Examiner(s): Guyton, Philip
• Application Number:
  US15/589,465
• Publication Number:
  US 20170242761A1
• Time in Patent Office: 365 Days
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0757 by exceeding a time limit, i.e. time-out, e.g. watchdogs
• CPC Class Codes: G06F 11/1658 Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2025 using centralised failover control functionality
• CPC Class Codes: G06F 11/2035 without idle spare hardware
• CPC Class Codes: G06F 11/2043 where the redundant components share a common memory address space
• CPC Class Codes: G06F 16/2255 Hash tables
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/2291 User-Defined Types; Storage management thereof
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency
• CPC Class Codes: G06Q 20/102 Bill distribution or payments
• CPC Class Codes: G06Q 20/4016 : involving fraud or risk level assessment in transaction processing
• CPC Class Codes: G06Q 40/02 : Banking, e.g. interest calculation or account maintenance credit or loans G06Q40/03

Abstract

A processing device to receive, from a second node in the data grid system, a first filter. The processing device may detect a failure of a third node of the data grid system. The processing device may determine that the backup data stored at the first node matches the first filter. The processing device may send, to the second node, a duplicate data notification indicating that the backup data comprises, at least in part, first data duplicative to second data communicated to the second node from the third node.

Claims

1. A non-transitory computer readable storage medium having instructions that, when executed by a processing device of a first node in a data grid system, cause the processing device to:

receive, from a second node in the data grid system, a first filter;

detect a failure of a third node of the data grid system;

determine that backup data stored at the first node matches the first filter; and

send, to the second node, a duplicate data notification indicating that the backup data comprises, at least in part, first data duplicative to second data communicated to the second node from the third node.

2. The non-transitory computer readable storage medium of claim 1, wherein the processing device is further to communicate, to the second node, a part of the first data that matches the first filter.

3. The non-transitory computer readable storage medium of claim 1, wherein the processing device is to receive, detect, determine, and send during an iteration period of a search query defined by search criteria of the first filter.

4. The non-transitory computer readable storage medium of claim 1, wherein the first node, the second node, and the third node are part of a node cluster in the data grid system.

5. The non-transitory computer readable storage medium of claim 1, wherein in response to the failure of the third node, the processing device is further to remove a second filter received from the third node.

6. The non-transitory computer readable storage medium of claim 1, wherein to detect the failure of the third node, the processing device is further to determine that a heartbeat message initiated by the third node has not been received at the first node within a threshold period of time.

7. The non-transitory computer readable storage medium of claim 1, wherein to detect the failure of the third node, the processing device is further to:

determine that a heartbeat message initiated by the third node has not been received at the first node within a first threshold period of time;

in response to not receiving the heartbeat message within the first threshold period of time, send a check node request to a fourth node in the data grid system; and

receive a failure notification from the fourth node.

8. The non-transitory computer readable storage medium of claim 1, the processing device further to:

rebuild a queue of one or more modifications to a second memory of the third node; and

communicate the queue to the first node.

9. The non-transitory computer readable storage medium of claim 8, wherein the one or more modifications comprise at least one of:

a cache write, a cache entry removal or deletion, or a cache modification.

10. A system comprising:

a first memory to store data; and

a processor coupled to the first memory, the processor to;

maintain a queue indicating one or more modifications to data in a cache of a first node in a data grid system, wherein the one or more modifications comprise at least one of a cache write, a cache entry removal or deletion, or a cache modification;

detect a failure of the first node in the data grid has crashed; and

send the queue to a second node.

11. The system of claim 10, wherein the processor is further to order the queue to indicate modifications that occurred during an iteration period.

12. The system of claim 10, the processor further to:

receive a filter from the first node;

determine the data stored at the first memory matches the filter; and

send at least a portion of the data that matches the filter to the second node.

13. The system of claim 10, the processor further to:

rebuild the queue to obtain a rebuilt queue of the one or more modifications to the data in the cache when the first node has crashed, wherein the processor is further to rebuild the queue during an iteration period of a search query to define a filter; and

send the rebuilt queue to the second node.

14. The system of claim 13, wherein the processor is further to record the one or more modifications during the iteration period.

20. The system of claim 10, wherein to detect the failure of the first node, the processor is further to:

determine that a heartbeat message initiated by the first node has not been received from the first node within a threshold period of time;

communicate a check node request to a third node;

receive a failure notification from the third node; and

determine that the first node has crashed in view of the failure notification.

15. A system comprising:

a first memory to store data; and

a processor coupled to the first memory, the processor to;

maintain a queue indicating one or more modifications to data in a second memory of a first node in a data grid system;

detect a failure of the first node in the data grid has crashed; and

send the queue to a second node, wherein to detect the failure of the first node, the processor is further to;

determine that a heartbeat message initiated by the first node has not been received from the first node within a threshold period of time;

communicate a check node request to a third node;

receive a failure notification from the third node; and

determine that the first node has crashed in view of the failure notification.

16. A method comprising:

sending, from a first node, a data filter to a second node;

receiving, at the first node, first data from a first memory of the second node that matches the data filter;

sending, from the first node, the data filter to a third node;

receiving, at the first node, backup data from a second memory of the third node that matches the data filter; and

adding at least a portion of the backup data to the first data, wherein the portion of the backup data that does not include second data that is duplicative of the first data.

17. The method of claim 16, further comprising receiving, at the first node, a duplicate data notification from the third node indicating that the backup data comprises third data duplicative of the second data received from the second node.

18. The method of claim 16, wherein the first node, the second node, and the third node are in a data grid system.

19. The method of claim 16, wherein:

the backup data is third data of the second node stored at the third node; and

the backup data is sent by the third node to the first node when the second node has crashed.