System and method for message processing and routing

• US 20080228792A1
• Filed: 04/04/2008
• Published: 09/18/2008
• Est. Priority Date: 05/01/2003
• Status: Active Grant

• Current Assignee: Goldman Sachs & Co. LLC (Goldman Sachs Group Incorporated)
• Original Assignee: Goldman Sachs & Company (Goldman Sachs Group Incorporated)
• Inventors: Marzo, Michael R., Kanayama, Tomozumi, Beguin, Julien George, Reed, Carl J., Krasheninnikov, Konstantin A.
• Granted Patent: US 8,775,667 B2
• Time in Patent Office: Days
• US Class Current: 707/100
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A message routing system that allows applications at either end of the system to run as-is without modification. The system functions in a multithreaded environment and is capable of handling complex routing rules and message transformation. It is also capable of learning and executing new routing rules and message transformations in formats previously unrecognized by the system. The system enables precise and reliable logging of messages throughout processing and supports publication of enterprise-wide broadcast messages. The system further preferably employs cooperating inbound and outbound transport processes for consuming, routing, processing, safely storing and publishing messages in batches of logical units of work to ensure that the logical units of work are not lost in system transactions. The system also preferably utilizes a replay server for preserving and replaying messages that might otherwise fail to reach their intended destinations.

Claims

1. A computerized message routing system comprising:

consumer process means for consuming messages from a publisher and for writing the messages to at least one file;

publisher process means for reading messages that have been written by said consumer process means to said at least one file and for publishing the messages to at least one subscriber; and

a file system in communication with said consumer process means and said publisher process means, said file system comprising;

said at least one file, wherein said at least one file stores messages written from said consumer process means in batches; and

means for maintaining write and read offsets for message batches that are written to said at least one file by said consumer process means and that are read from said at least one file by said publisher process means, whereby the write and read offsets enable data to be persisted in said at least one file such that duplicate messages are not written by said consumer process means to said at least one file or published by said publisher process means to the at least one subscriber in the event message recovery is required.

2. The system of claim 1 wherein the write and read offsets include:

an END offset written by said consumer process means to said means for maintaining offsets for a batch of messages stored in said at least one file; and

a START offset written by said publisher process means to said means for maintaining offsets for a batch of messages read from said at least one file.

3. The system of claim 2 wherein said START offset precedes said END offset for a batch of messages.

4. A method for preventing duplicate publication of data in a computerized message routing system comprising:

consuming messages from a publisher and writing the messages in batches to at least one file;

reading messages from the at least one file and publishing the messages to at least one subscriber; and

maintaining write and read offsets for message batches that are written to and read from the at least one file, whereby the write and read offsets enable data to be persisted in the at least one file such that duplicate messages cannot be written to the at least one file or published to the at least one subscriber in the event message recovery is required.

5. The method of claim 4 wherein the step of maintaining write and read offsets includes:

writing an END offset for a batch of messages stored in said at least one file; and

writing a START offset for a batch of messages read from said at least one file.

6. The method of claim 4 wherein the said START offset precedes said END offset for a batch of messages.