System and method for store and forward routing for distributed destinations

• US 8,094,669 B2
• Filed: 04/24/2008
• Issued: 01/10/2012
• Est. Priority Date: 04/24/2007
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Barnes, Thomas E., Gallagher, William John
• Primary Examiner(s): Yao, Kwang B
• Assistant Examiner(s): Huq, Obaidul
• Application Number:
  US12/109,078
• Publication Number:
  US 20080267209A1
• Time in Patent Office: 1,356 Days
• Field of Search: 370/351, 370/428, 370/389, 370/412, 370/419, 370/420, 370/421, 370/422, 370/425, 709/204, 709/206, 709/207, 709/223, 709/224, 709/225, 709/226
• US Class Current: 370/428
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering

Abstract

The present invention enables a store-and-forward service providing an efficient and reliable forwarding of messages stored in a message queue to a distributed destination. It provides customers with high availability and scalability for applications such as JMS queues and topics. It achieves load balancing and failover amongst all individual members of the distributed destination while ensuring that each message is delivered exactly once.

Claims

1. A system for storing and forwarding messages to a distributed destination, comprising:

one or more microprocessors;

a source operable to produce a plurality of messages, wherein each message is adapted to be forwarded to the distributed destination, wherein the distributed destination includes a plurality of member destinations, and each said member destination is operable to accept and process one or more messages, and wherein each said message is assigned with a unique sequence number that associates each said message with a particular member destination, wherein the unique sequence number indicates how to route said message to a particular member destination;

a source queue operable to store the plurality of messages; and

a forwarder running on the one or more microprocessors that is operable to;

consume one or more messages of the plurality of messages from the source queue;

determine, for each said message, a particular member destination from the plurality of member destinations in the distributed destination based on an evaluation of the unique sequence number assigned to the message;

deliver the one or more messages of the plurality of messages to the distributed destination exactly once;

provide ordered forwarding of sequences of multiple sub-orderings from a local destination to a remote distributed remote destination;

preserve quality of service even in an event of a subsequent restart after a system failure;

use an algorithm that yields the same persistent I/O count as algorithms that don'"'"'t preserve exactly-once or ordering;

require no significant additional implementation on the remote destinations other than a duplicate elimination algorithm; and

have no dependency on a XA transaction standard.

2. The system according to claim 1, wherein:

each said message is one of a text, an image, a voice, a telex, a fax, an e-mail, a paging, and an electronic data describing events, requests, and replies over a communication network.

3. The system according to claim 1, wherein:

the source is one of a producer, a front-end for a subsystem, a Web Service client for a Web services subsystem, and a sending agent.

4. The system according to claim 1, wherein:

each of the plurality of member destinations can be one of a queue, a topic, a remote server instance, a cluster, a JMS destination, a Web Services component, and a receiving agent.

5. The system according to claim 1, wherein:

the distributed destination locates on a cluster of one or more servers.

6. The system according to claim 1, wherein:

the forwarder uses a load balancing strategy that is round-robin load balancing.

7. The system according to claim 1, wherein:

the source queue is further operable to persist the one or more of the plurality of messages to disk before delivering.

8. The system according to claim 1, wherein:

the forwarder is further operable to;

examine the sequence number of one of the plurality of messages; and

perform a lookup in a mapping table to determine which member destination the message should be forwarded to.

9. The system according to claim 1, wherein:

the forwarder is further operable to;

deliver the one or more messages of the plurality of messages exactly once after network failure via at least one of;

a XA transaction;

resending the one or more messages of the plurality of messages to the particular member destination, which is operable to detect and discard duplicated messages resent; and

resending the one or more messages of the plurality of messages to a same physical member destination in the distributed destination.

10. The system according to claim 1, wherein:

when the distributed destination includes a fixed number of member destinations, the particular member destination for delivering the one or more messages is determined based on an evaluated value that equals to the unique sequence number assigned to the message modulo the fixed number of member destinations in the distributed destination.

11. The system according to claim 10, wherein:

the forwarder uses a table to determine a particular member destination in the distributed destination to receive the message based on the evaluated value, wherein the table includes an in-doubt message lists.

12. The system according to claim 1, wherein:

the forwarder is further operable to preserve a message order for a plurality of messages belonging to a well defined subset, wherein the forwarder waits until all previous message in the subset have been forwarded before the forwarder forwards a subsequent message.

13. The system according to claim 1, wherein:

the forwarder uses a message order number for a plurality of messages belonging to a subset to determine how to route a message in the plurality of messages, wherein the message is forwarded to a distributed destination that is associated with an evaluated value that equals to the message order number assigned to the message modulo a fixed number of member destinations in the distributed destination.

14. A method for storing and forwarding messages to a distributed destination, comprising:

producing, via a source, a plurality of messages, wherein each message is adapted to be forwarded to the distributed destination, wherein the distributed destination includes a plurality of member destinations, and each said member destination is operable to accept and process one or more said messages;

allowing each said message to be assigned with a unique sequence number that associated each said message with a particular member destination, wherein the unique sequence number indicates how to route said message to a particular member destination;

storing, via a source queue, the plurality of messages;

consuming one or more messages of the plurality of messages from the source queue;

determining, for each said message, a particular member destination from the plurality of member destinations in the distributed destination based on an evaluation of the unique sequence number assigned to the message;

delivering the one or more messages of the plurality of messages to the distributed destination exactly once;

providing ordered forwarding of sequences of multiple sub-orderings from a local destination to a remote distributed remote destination;

preserving quality of service even in an event of a subsequent restart after a system failure;

using an algorithm that yields the same persistent I/O count as algorithms that don'"'"'t preserve exactly-once or ordering;

requiring no significant additional implementation on the remote destinations other than a duplicate elimination algorithm; and

having no dependency on a XA transaction standard.

15. The method according to claim 14, further comprising:

persisting the one or more of the plurality of messages to disk before delivering.

16. The method according to claim 14, further comprising:

examining the sequence number of one of the plurality of messages; and

performing a lookup in a mapping table to determine which member destination the message should be forwarded to.

17. The method according to claim 14, further comprising:

resending the one or more messages of the plurality of messages to the member destination after network failure; and

detecting and discarding duplicated messages.

18. The method according to claim 17, further comprising:

resending the one or more of the plurality of messages are to the same physical member destination in the distributed destination.

19. A non-transitory machine readable medium having instructions stored thereon that when executed cause a system to:

produce, via a source, a plurality of messages, wherein each message is adapted to be forwarded to a distributed destination, wherein the distributed destination includes a plurality of member destinations, and each said member destination is operable to accept and process one or more said messages;

allow each said message to be assigned with a unique sequence number that associates each said message with a particular member destination, wherein the unique sequence number indicates how to route said message a particular member destination;

store, via a source queue, the plurality of messages;

consume one or more of the plurality of messages from the source queue;

determine, for each said message, a particular member destination from the plurality of member destinations in the distributed destination based on an evaluation of the unique sequence number assigned to the message;

deliver the one or more messages of the plurality of messages to the distributed destination exactly once;

provide ordered forwarding of sequences of multiple sub-orderings from a local destination to a remote distributed remote destination;

preserve quality of service even in an event of a subsequent restart after a system failure;

use an algorithm that yields the same persistent I/O count as algorithms that don'"'"'t preserve exactly-once or ordering;

require no significant additional implementation on the remote destinations other than a duplicate elimination algorithm; and

have no dependency on a XA transaction standard.