System and method for supporting out-of-order message processing in a distributed data grid

• US 9,083,614 B2
• Filed: 11/07/2012
• Issued: 07/14/2015
• Est. Priority Date: 10/15/2012
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Falco, Mark, Gleyzer, Gene
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): Misir, Dave
• Application Number:
  US13/671,481
• Publication Number:
  US 20140108533A1
• Time in Patent Office: 979 Days
• Field of Search: 709204-207, 719/318
• US Class Current: 1/1
• CPC Class Codes: G06F 11/16 Error detection or correction of the data by redundancy in hardware
• CPC Class Codes: G06F 11/2041 with more than one idle spare processing component
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: H04L 41/0668 by dynamic selection of recovery network elements, e.g. replacement by the most appropriate element after failure
• CPC Class Codes: H04L 43/0811 by checking connectivity
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

A system and method can support out-of-order message processing in a distributed data grid. A cluster node in the distributed data grid can receive a plurality of messages at a cluster node in the distributed data grid. The distributed data grid allows each message in the plurality of messages to determine whether it needs to be handled in a particular order on a first thread. The cluster node can dispatch one or more of said messages that do not need to be handled in the particular order on the first thread to one or more different threads.

Claims

1. A method for supporting out-of-order message processing in a distributed data grid comprising a plurality of cluster nodes operating on one or more microprocessors, the distributed data grid comprising a message transfer layer which guarantees in-order delivery of messages to said plurality of cluster nodes, the method comprising:

embedding a callback function in each of a plurality of messages directed to a cluster node of said plurality of cluster nodes, wherein said callback function is configured to inform whether said message can be pre-processed and/or whether said message requires in-order processing on a message processing service queue;

transmitting said plurality of messages to said cluster node over said message transfer layer;

receiving said plurality of messages at said cluster node;

preprocessing each message of said plurality of messages based on the callback function embedded in each message;

in response to said preprocessing, pushing a first subset of said plurality of messages onto the message processing service queue for in-order processing; and

in response to said preprocessing, dispatching a second subset of said plurality of messages to one or more different threads for out-of-order processing.

2. The method according to claim 1, wherein the message transfer layer which guarantees in-order delivery of messages to said plurality of cluster nodes utilizes the Transmission Control Protocol/Internet Protocol (TCP/IP).

3. The method according to claim 1, wherein the message transfer layer which guarantees in-order delivery of messages to said plurality of cluster nodes utilizes the Tangosol Cluster Management Protocol (TCMP).

4. The method according to claim 1, further comprising:

said embedding step comprises embedding a callback function in the form of application code into each of said plurality of messages directed to said cluster node.

5. The method according to claim 4, further comprising:

said embedding step comprises embedding a callback function in the form of a hard-coded flag into each of said plurality of messages directed to said cluster node.

6. The method according to claim 1, further comprising:

processing said first subset of said plurality of messages in-order from said message processing service queue on said cluster node.

7. The method according to claim 1, further comprising:

processing one or more of said plurality of messages on the one or more different threads in parallel.

8. A system for supporting out-of-order message processing in a distributed data grid, comprising:

one or more microprocessors;

a plurality of cluster nodes in the distributed data grid running on the one or more microprocessors;

a message transfer layer which guarantees in-order delivery of messages to said plurality of cluster nodes;

a message preprocessor in a cluster node of said plurality of cluster nodes;

a plurality of messages directed to said cluster node of said plurality of cluster nodes, wherein each of said plurality of messages includes an embedded callback function, wherein said callback function is configured to inform whether said message can be pre-processed and/or whether said message requires in-order processing on a message processing service queue;

wherein when said plurality of messages are received from said message trasfer layer at said cluster node, the message preprocessor performs steps comprising,

preprocessing each message of said plurality of messages based on the callback function embedded in each message,

in response to said preprocessing, pushing a first subset of said plurality of messages onto the message processing service queue for in-order processing, and

in response to said preprocessing, dispatching a second subset of said plurality of messages to one or more different threads for out-of-order processing.

9. The system according to claim 8, wherein the message transfer layer which guarantees in-order delivery of messages to said plurality of cluster nodes utilizes the Transmission Control Protocol/Internet Protocol (TCP/IP).

10. The system according to claim 9, wherein the message transfer layer which guarantees in-order delivery of messages to said plurality of cluster nodes utilizes the Tangosol Cluster Management Protocol (TCMP).

11. The system according to claim 8, wherein:

the embedded callback function in each of said plurality of messages comprises application code.

12. The system according to claim 11, wherein:

the embedded callback function in each of said plurality of messages comprises a hard-coded flag.

13. The system according to claim 8, wherein:

the cluster node operates to process said first subset of said plurality of messages in-order from the message processing service queue.

14. The system according to claim 8, wherein:

the cluster node operates to process said second subset of said plurality of messages on the one or more different threads in parallel.

15. A non-transitory machine readable storage medium having instructions stored thereon for supporting out-of-order message processing in a distributed data grid comprising a plurality of cluster nodes and a message transfer layer which guarantees in-order delivery of messages to said plurality of cluster nodes, which instructions, when executed, cause said distributed data grid to perform steps comprising:

embedding a callback function in each of a plurality of messages directed to a cluster node of said plurality of cluster nodes, wherein said callback function is configured to inform whether said message can be pre-processed and/or whether said message requires in-order processing on a message processing service queue;

transmitting said plurality of messages to said cluster node over said message transfer layer;

receiving said plurality of messages at said cluster node;

preprocessing each message of said plurality of messages based on the callback function embedded in each message;

in response to said preprocessing, pushing a first subset of said plurality of messages onto the message processing service queue for in-order processing; and

in response to said preprocessing, dispatching a second subset of said plurality of messages to one or more different threads for out-of-order processing.

16. The non-transitory machine readable storage medium according to claim 15, wherein the message transfer layer which guarantees in-order delivery of messages to said plurality of cluster nodes utilizes the Transmission Control Protocol/Internet Protocol (TCP/IP).

17. The non-transitory machine readable storage medium according to claim 15, wherein the message transfer layer which guarantees in-order delivery of messages to said plurality of cluster nodes utilizes the Tangosol Cluster Management Protocol (TCMP).

18. The non-transitory machine readable storage medium according to claim 15, wherein said embedding step comprises embedding a callback function in the form of application code into each of said plurality of messages directed to said cluster node.

19. The non-transitory machine readable storage medium according to claim 15, wherein said embedding step comprises embedding a callback function in the form of a hard-coded flag into each of said plurality of messages directed to said cluster node.

20. The non-transitory machine readable storage medium according to claim 15, wherein, said instructions further comprise:

processing said first subset of said plurality of messages in-order from said message processing service queue on said cluster node.