System and method for message processing and routing

• US 8,458,275 B2
• Filed: 08/27/2012
• Issued: 06/04/2013
• Est. Priority Date: 05/01/2003
• Status: Active Grant

• Current Assignee: Goldman Sachs & Co. LLC (Goldman Sachs Group Incorporated)
• Original Assignee: Goldman Sachs & Company (Goldman Sachs Group Incorporated)
• Inventors: Reed, Carl J., Marzo, Michael R., Kanayama, Tomozumi, Krasheninnikov, Konstantin, Beguin, Julien George
• Primary Examiner(s): Walsh, John B.
• Application Number:
  US13/595,455
• Publication Number:
  US 20120324030A1
• Time in Patent Office: 281 Days
• Field of Search: None
• US Class Current: 709/206
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A message routing system that allows applications at either end of the system to run as-is without modification. The system functions in a multithreaded environment and is capable of handling complex routing rules and message transformation. It is also capable of learning and executing new routing rules and message transformations in formats previously unrecognized by the system. The system enables precise and reliable logging of messages throughout processing and supports publication of enterprise-wide broadcast messages. The system further preferably employs cooperating inbound and outbound transport processes for consuming, routing, processing, safely storing and publishing messages in batches of logical units of work to ensure that the logical units of work are not lost in system transactions. The system also preferably utilizes a replay server for preserving and replaying messages that might otherwise fail to reach their intended destinations.

Claims

1. A computerized message routing system comprising:

a plurality of disparate messaging subsystems having different transaction management protocols, wherein the disparate messaging subsystems include a publisher to publish messages under general subjects using different transports;

a router using a processor to run a consumer process that includes multiple consumer threads, a publisher process that includes multiple publisher threads to publish messages from the router, and an application to execute multiple routing rules on inbound messages received from the publisher, wherein the application determines a set of specific subjects for each of the inbound messages on which outbound messages from the router will be published to consumers in accordance with a subscription based on the specific subjects and the general subjects; and

wherein additional consumer threads can be added to the multiple consumer threads based on performance of the router.

2. The computerized message routing system of claim 1, wherein each of the multiple consumer threads are assigned to receive messages relating to one of the general subjects.

3. The computerized message routing system of claim 1, further comprising an introspection module library having multiple introspection modules that can be dynamically loaded by the router.

4. The computerized message routing system of claim 3, wherein the application is further configured to determine a message type.

5. The computerized message routing system of claim 4, wherein the router dynamically loads a first introspection module that contains logic for handling the message type determined by the application.

6. The computerized message routing system of claim 1, further comprising a means for replaying the messages on demand.

7. The computerized message routing system of claim 1, further comprising a means for dynamically determining the multiple routing rules.

8. A method for operating a computerized message routing system comprising:

receiving, a message associated with a general subject initiated by a publisher;

consuming, via one of a plurality of consumer threads running on a processor, the message, wherein the plurality of consumer threads are dynamically adjusted based on performance of a router;

determining a set of specific subjects for the message; and

publishing the message to a set of consumers in accordance with a subscription based on the specific subjects and the general subject.

9. The method of claim 8, wherein publishing the message to the set of consumers includes transforming the message into a format previously unrecognized by the computerized message routing system.

10. The method of claim 8, further comprising receiving from the consumers a desired format.

11. The method of claim 10, wherein publishing the message to the set of consumers includes transforming the message into the desired format.

12. The method of claim 8, further comprising receiving subscriptions from the set of consumers.

13. The method of claim 8, wherein the message received from the publisher uses one of a plurality of transports.

14. A non-transitory computer-readable storage medium having a set of instructions stored thereon, wherein the set of instructions, when executed by one or more processors, cause the one or more processors to:

monitor a measure of thread management overhead at the router and create an additional router when the measure exceeds a threshold;

receive, from a publisher, by one of the router or the additional router, a message associated with a general subject;

consume, by one of the router or the additional router, via one of a plurality of consumer threads running on the one or more processors, the message from the publisher;

determine, by one of the router or the additional router, a set of specific subjects for the message; and

publish, by one of the router or the additional router, the message to a set of consumers in accordance with a subscription based on the specific subjects and the general subject.

15. The computer-readable storage medium of claim 14, wherein the set of instructions, when executed by the one or more processors, further cause the one or more processors to transform the message into a format previously unrecognized by the computerized message routing system.

16. The computer-readable storage medium of claim 14, wherein the set of instructions, when executed by the one or more processors, further cause the one or more processors to receive from the consumers a desired format and transform the message into the desired format.

17. The computer-readable storage medium of claim 14, wherein the set of instructions, when executed by the one or more processors, further cause the one or more processors to add or remove a consumer thread from the plurality of consumer threads based on the performance metrics.