System and method for message processing and routing

• US 8,521,906 B2
• Filed: 08/20/2012
• Issued: 08/27/2013
• Est. Priority Date: 05/01/2003
• Status: Expired due to Term

• Current Assignee: Goldman Sachs & Co. LLC (Goldman Sachs Group Incorporated)
• Original Assignee: Goldman Sachs & Company (Goldman Sachs Group Incorporated)
• Inventors: Reed, Carl J., Marzo, Michael R., Kanayama, Tomozumi, Krasheninnikov, Konstantin A., Beguin, Julien George
• Primary Examiner(s): Walsh, John B.
• Application Number:
  US13/589,962
• Publication Number:
  US 20120311089A1
• Time in Patent Office: 372 Days
• Field of Search: None
• US Class Current: 709/238
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A message routing system that allows applications at either end of the system to run as-is without modification. The system functions in a multithreaded environment and is capable of handling complex routing rules and message transformation. It is also capable of learning and executing new routing rules and message transformations in formats previously unrecognized by the system. The system enables precise and reliable logging of messages throughout processing and supports publication of enterprise-wide broadcast messages. The system further preferably employs cooperating inbound and outbound transport processes for consuming, routing, processing, safely storing and publishing messages in batches of logical units of work to ensure that the logical units of work are not lost in system transactions. The system also preferably utilizes a replay server for preserving and replaying messages that might otherwise fail to reach their intended destinations.

Claims

1. A computerized message routing system comprising:

a computer processor;

a publisher capable of publishing messages each associated with at least one of a plurality of different message topics;

a router configured to;

run a different thread for each of the plurality of different message topics when executed by the computer processor, and

invoke an introspection module based on the plurality of different message topics to determine a destination for the messages;

wherein each of the different threads uses an inbound transport protocol and an outbound transport protocol and the router is communicably coupled to the publisher to consume and publish the messages including transforming the messages into different outbound transport protocols based on transformation requests; and

a consumer communicably coupled to the router and configured to receive a subset of the messages published by the router that are associated with one or more of the plurality of different message topics, wherein the consumer publishes a set of transformation procedures to the router in the transformation requests.

2. The computerized message routing system of claim 1, wherein the messages published by the router include an indication of the outbound transport protocol in a file name of each of the messages.

3. The computerized message routing system of claim 1, wherein the consumer runs a consumer thread for each of the plurality of different message topics.

4. The computerized message routing system of claim 1, wherein the router transforms the messages by passing the messages to a plug-in.

5. The computerized message routing system of claim 1, wherein the router performs pre-routing transformations and post-routing transformations.

6. The computerized message routing system of claim 1, wherein the messages include stock trading data.

7. The computerized message routing system of claim 1, further comprising a replay server to preserve messages and to replay messages at the request of the consumer.

8. A method for operating a router, the method comprising:

consuming a message received from a publisher, wherein consuming the message includes determining a destination for the message by invoking an introspection module based on a message topic;

receiving, from the destination, a set of transformation procedures;

transforming, using a processor, the message in accordance with the set of transformation procedures received from the destination; and

publishing the message to the destination.

9. The method of claim 8, further comprising:

storing, on a replay server, the message published by the router;

receiving a request from the destination to republish the message; and

republishing the message from the replay server to the destination.

10. The method of claim 8, wherein transforming the message includes passing the message to a plug-in.

11. The method of claim 8, wherein transforming the message includes pre-routing transformations and post-routing transformations.

12. The method of claim 8, wherein the messages include stock trading data.

13. The method of claim 8, wherein the message includes, as part of the file name, an indication of a transport being used.

14. The method of claim 8, wherein determining the destination for the message comprises extracting at least one routing key from the message based on the message topic.

15. A router comprising:

a processor;

a memory having executable instructions, wherein the memory is communicably coupled to the processor and configured to execute the instructions stored on the memory to;

run multiple threads to consume messages from publishers;

split the messages among the multiple threads based on message topics to optimize the number of threads being utilized;

invoke an introspection module based on at least one of the message topics to determine a destination subscriber for the messages based on the message topics;

receive, from the destination subscriber, a set of transformation procedures;

transform, using the processor, the message in accordance with the set of transformation procedures received from the destination subscriber; and

publish the message to the destination subscriber.

16. The router of claim 15, wherein the executable instructions further cause the processor to determine the message topics associated with the messages by examining the content of the messages and to further assign the messages to one of the multiple threads based on the message topics.

17. The router of claim 16, wherein the message topics include trade messages.

18. The router of claim 15, wherein the executable instructions further cause the processor to store all messages published by the router and republish any message as requested by the destination subscriber.