Data communication in a distributed data grid

• US 10,572,486 B2
• Filed: 02/26/2015
• Issued: 02/25/2020
• Est. Priority Date: 02/26/2015
• Status: Active Grant

• Current Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Original Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Inventors: Burns, William Rosenquist
• Primary Examiner(s): Kyle, Tamara T
• Assistant Examiner(s): Hoffler, Raheem
• Application Number:
  US14/633,016
• Publication Number:
  US 20160253391A1
• Time in Patent Office: 1,825 Days
• Field of Search: 707722
• CPC Class Codes: G06F 16/156 Query results presentation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24564 Applying rules; Deductive queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/438 Presentation of query results
• CPC Class Codes: G06F 16/638 Presentation of query results
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G06F 16/838 Presentation of query results
• CPC Class Codes: G06F 16/9038 Presentation of query results

Abstract

A method of communicating data in a data grid. The method includes receiving, from an iterator in a data grid, a search request to search for data matching search criteria in a first memory coupled to a node. The method further includes searching, by a processor of the node, for data in the memory matching the search criteria to obtain result data. The method further includes communicating, by the processor, the result data to the iterator. The method further includes pausing communicating the result data to the iterator in view of a pause condition pertaining to the memory capacity threshold of the second memory. The method further includes, in response to a resume condition, resuming communicating the result data to the iterator.

Claims

1. A first node, comprising:

a first memory to store data as one or more cache entries of a distributed cache distributed over a grid of nodes comprising at least the first node and an iterator node;

a first processor coupled to the first memory, the first processor to;

receive, over a network from a second processor of the iterator node, a search request to search the data in the first memory, wherein the search request corresponds to a search query of the grid received at the iterator node and wherein the search request comprises;

search criteria defined by the search query; and

a memory capacity threshold of a second memory coupled to the second processor of the iterator node;

identify result data comprising data in the first memory that matches the search criteria;

communicate, to the iterator node, the result data;

determine whether pause criteria of a pause condition is satisfied, the pause condition pertaining to the memory capacity of the second memory exceeding the memory capacity threshold during communication of the result data, wherein the memory capacity of the second memory is determined in view of an amount of result data communicated by the first node and by other nodes to the iterator node;

in response to satisfying the pause criteria of the pause condition, pause communicating the result data to the iterator node for at least a predetermined length of time, wherein the pause condition is to prevent an out of memory (OOM) condition from occurring at the iterator node, wherein prevention of the OOM condition improves communication efficiency and network bandwidth in the grid of nodes;

determine that the predetermined length of time is satisfied and that a resume criteria of a resume condition is satisfied responsive to receiving, from the iterator node, a resume message, wherein the resume message indicates to the first node to resume communicating result data to the iterator node, the resume condition pertaining to memory capacity of the second memory falling below the memory capacity threshold; and

in response to satisfying the resume criteria of the resume condition, resume communicating the result data to the iterator node.

2. The first node of claim 1, wherein the pause criteria of the pause condition comprises communication, by the first node, of a threshold amount of the result data to the iterator node prior to pausing the communicating, the threshold amount associated with the memory capacity threshold of the second memory.

3. The first node of claim 1, further comprising determining whether the pause criteria of the pause condition is satisfied by:

communicating, with other nodes in the grid, to determine the amount of the result data communicated by the first node and the other nodes to the iterator node; and

determining when the amount of result data communicated by the first node and the other nodes comprises a threshold amount of data that is associated with the memory capacity threshold of the second memory.

4. The first node of claim 1, wherein the pause criteria of the pause condition is satisfied responsive to receiving, from the iterator node, a pause message indicating to the first node to pause communicating result data to the iterator node in view of the memory capacity threshold.

5. The first node of claim 1, wherein the search request further comprises the predetermined length of time for the first node to pause communicating the result data when the pause condition occurs.

6. The first node of claim 5, wherein the resume condition comprises occurrence of an expiration of the predetermined length of time.

7. An iterator node, comprising:

a first memory coupled to the iterator node to store data, wherein the iterator node is coupled to a first node and a second node in a grid of nodes;

a processor coupled to the first memory, the processor to;

receive, from an application, a search query of the grid;

communicate, over a network to the first node in the grid, a search request corresponding to the search query, the search request to search for data in a second memory of the first node, wherein the search request comprises;

search criteria defined by the search query; and

a memory capacity threshold of the first memory;

receive, from the first node, result data comprising the data in the second memory that matches the search criteria;

determine whether pause criteria of a pause condition is satisfied, the pause condition pertaining to the memory capacity of the first memory exceeding the memory capacity threshold during communication of the result data, wherein the memory capacity of the second memory is determined in view of an amount of result data communicated by the first node and by other nodes to the iterator node;

in response to satisfying the pause criteria of the pause condition, pause receiving the result data from the second node for at least a predetermined length of time, wherein pause condition is to prevent an out of memory (OOM) condition from occurring at the iterator node, wherein prevention of the OOM condition improves communication efficiency and network bandwidth in the grid of nodes;

determine that the predetermined length of time is satisfied and that a resume criteria of a resume condition is satisfied, the resume condition pertaining to memory capacity of the first memory falling below the memory capacity threshold;

responsive to determining the resume criteria is satisfied, sending, from a resume message to the first node, the resume message indicating to the first node to resume communicating result data to the iterator node; and

resume, in view of the resume message, receiving the result data from the first node.

8. The iterator node of claim 7, wherein the processor is further to receive, from the application, an iteration request, wherein the iteration request comprises the search criteria of the search query.

9. The iterator node of claim 7, wherein the processor is further to communicate, to the second node in the grid, a search request to search for data in a memory coupled to the second node.

10. The iterator node of claim 7, wherein satisfying the pause criteria comprises receiving, from the first node, a threshold amount of result data, wherein the threshold amount of result data is a memory capacity of the first memory.

11. The iterator node of claim 7, wherein the processor is further to communicate, to the first node, a pause message indicating to the first node to pause communicating the result data to the iterator node in view of the memory capacity threshold.

12. The iterator node of claim 7, wherein the pause criteria is a threshold length of time in view of a size of the first memory.

13. The iterator node of claim 7, wherein the search request comprises the predetermined length of time for the first node to pause when the pause criteria occurs.

14. The iterator node of claim 7, wherein the processor is further to:

communicate, to a third node in the grid, a second search request to search for data in a third memory of the third node;

receive, from the third node, second result data comprising data in the third memory that matches the search criteria;

pause receiving the result data from the third node in view of the pause condition; and

resume receiving the result data from the third node in view of the resume condition.

15. A method, comprising:

receiving, over a network from an iterator node in a grid of nodes, a search request to search data in a first memory of a first node of the nodes in the grid, wherein the search request corresponds to a search query of the grid received at the iterator node and wherein the search request comprises;

search criteria defined by the search query; and

a memory capacity threshold of a second memory coupled to the iterator node;

identifying, by a processor of the first node, result data comprising data in the first memory that matches the search criteria;

communicating, by the processor, the result data to the iterator node;

determining whether pause criteria of a pause condition is satisfied, the pause condition pertaining to the memory capacity of the second memory exceeding the memory capacity threshold during communication of the result data, wherein the memory capacity of the second memory is determined in view of an amount of result data communicated by the first node and by other nodes to the iterator node;

in response to satisfying the pause criteria of the pause condition, pausing communicating the result data to the iterator node for at least a predetermined length of time, wherein the pause condition is to prevent an out of memory (OOM) condition from occurring at the iterator node, wherein prevention of the OOM condition improves communication efficiency and network bandwidth in the grid of nodes;

determining that the predetermined length of time is satisfied and that a resume criteria of a resume condition is satisfied responsive to receiving, from the iterator node, a resume message, wherein the resume message indicates to the first node to resume communicating result data to the iterator node, the resume condition pertaining to memory capacity of the second memory falling below the memory capacity threshold; and

in response to satisfying the resume criteria of the resume condition, resuming communicating the result data to the iterator node.

16. The method of claim 15, wherein satisfying the pause criteria comprises at least one of:

communicating, by the node, a threshold amount of result data to the iterator node and receiving, from the iterator node, a pause message indicating to the first node to pause communicating the result data to the iterator node;

or

communicating, with other nodes in the grid, to determine an amount of the result data communicated by the first node and the other nodes to the iterator node and determining when the amount of the result data communicated to the iterator node by the first node and the other nodes comprises a threshold amount of the result data;

wherein the threshold amount of data is associated with the memory capacity threshold.

17. The method of claim 15, wherein satisfying the resume criteria further comprises determining occurrence of an expiration of a predetermined length of time.

18. The method of claim 15, wherein the search request comprises the predetermined length of time for the first node to pause when the pause criteria occurs.