End-to-end publish/subscribe middleware architecture

• US 7,970,918 B2
• Filed: 12/23/2005
• Issued: 06/28/2011
• Est. Priority Date: 01/06/2005
• Status: Expired due to Fees

• Current Assignee: Tervela, Inc. (Box Incorporated)
• Original Assignee: Tervela, Inc. (Box Incorporated)
• Inventors: Fraval, Pierre, Singh, Kul, Thompson, J. Barry
• Primary Examiner(s): NEURAUTER, GEORGE C
• Application Number:
  US11/316,778
• Publication Number:
  US 20060149840A1
• Time in Patent Office: 2,013 Days
• Field of Search: 709/230, 709/238, 719/313, 719/328
• US Class Current: 709/230
• CPC Class Codes: H04L 12/1859 adapted to provide push services, e.g. data channels
• CPC Class Codes: H04L 43/06 Generation of reports
• CPC Class Codes: H04L 43/0852 Delays
• CPC Class Codes: H04L 45/24 Multipath
• CPC Class Codes: H04L 47/11 Identifying congestion
• CPC Class Codes: H04L 47/15 in relation to multipoint traffic arrangements for broadcast or multicast in data networks H04L12/18
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 47/801 Real time traffic
• CPC Class Codes: H04L 47/806 Broadcast or multicast traffic
• CPC Class Codes: H04L 47/822 Collecting or measuring resource availability data
• CPC Class Codes: H04L 47/829 Topology based
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context

Abstract

Message publish/subscribe systems are required to process high message volumes with reduced latency and performance bottlenecks. The end-to-end middleware architecture proposed by the present invention is designed for high-volume, low-latency messaging by, among other things, reducing intermediary hops with neighbor-based routing, introducing efficient native-to-external and external-to-native protocol conversions, monitoring system performance, including latency, in real time, employing topic-based and channel-based message communications, and dynamically optimizing system interconnect configurations and message transmission protocols.

Claims

1. An end-to-end publish/subscription messaging system with a middleware architecture, comprising:

one or more messaging appliances configured to receive and route messages;

an interconnect utilizing channel-based messaging that routes messages over a first messaging layer based on one or more channels mapped to subscription topics, each of the one or more channels assigned to a communication pathway of a second messaging layer; and

a provisioning and management system linked via the interconnect to the one or more messaging appliances and configured to exchange administrative messages with each of the one or more messaging appliances,

wherein each of the one or more messaging appliances is further configured to execute the routing of messages by dynamically selecting, in real time, a message transmission protocol and a message routing path.

2. The system of claim 1, wherein the provisioning and management system is configured to perform functions associated with the administrative messages including system configuration, health and performance monitoring and reporting.

3. The system of claim 1, wherein the one or more messaging appliances is configured as an edge messaging appliance or a core messaging appliance.

4. The system of claim 3, wherein each edge messaging appliance is linked to a message transformation engine for transforming incoming messages from an external protocol to a native protocol and for transforming routed messages from the native protocol to the external protocol.

5. The system of claim 1, wherein the message transmission protocol is selected to be one of a unicast, multicast or broadcast protocol.

6. The system of claim 1, further including one or more application programming interfaces configured for interfacing between one or more applications and a respective one of the one or more messaging appliances.

7. The system of claim 6, in which a respective one of the one or more messaging appliances and a respective one of the one or more application programming interfaces are operative to communicate with each other by incorporating one or more messages in a single frame.

8. The system of claim 6, wherein each of the applications is configured to send requests, including registration and subscription requests, to a respective one of the one or more messaging appliances, wherein the provisioning and management system is further configured to handle digital rights management where each respective messaging appliance confirms and reports to the provisioning and management system whether an application attempting to register or subscribe with it is entitled to do so.

9. The system of claim 6, wherein each of the application programming interfaces is logically linked to a respective one of the one or more messaging appliances having been registered to it via a topic-based subscription.

10. The system of claim 6, wherein the one or more messaging appliances include one or more core messaging appliances to which the application programming interfaces register.

11. The system of claim 9, wherein the topic based subscriptions are each established via a subscription request, and wherein a single subscription request is capable of establishing subscriptions to a group of related topics.

12. The system of claim 1, in which the interconnect is an interconnect network over which the one or more messaging appliances and the provisioning and management systems are deployed, the network being configured with any number of routers, switches and subnets.

13. The system of claim 1, wherein the interconnect is a channel-based, fabric agnostic physical medium.

14. The system of claim 1, wherein the one or more messaging appliances, the provisioning and management systems, and the interconnects incorporate transport logic.

15. The system of claim 14, wherein the interconnect is configured for transport transparent channel-based messaging where messages are communicated in native protocol format independent of the transport logic.

16. The system of claim 1, wherein the messages are constructed such that they comprise one or more schema field and payload, and wherein one or more of the message appliances are configured to separate the schema and payload when the messages enter the system and combine the schema and payload when the messages leave the system.

17. The system of claim 1, wherein each of the one or more messaging appliances stores a routing table and wherein at least some of the one or more messaging appliances are configured to route messages via at least one channel to a neighbor application or device subscribed to all or a subset of messages transmitted via the at least one channel, and wherein each of the one or more messaging appliances is further configured to optimize the mapping of channels to subscriptions in the routing table based on network utilization, link latency and other efficiency metrics.

18. The system of claim 17, wherein channel and subscription data stored in the routing table are formatted in a tree structure or a dynamic map structure.

19. The system of claim 1, wherein the dynamic selection of the message transmission protocol and the message routing path is based on system topology or health and performance reports from the provisioning and management system, and wherein the dynamic selection of the message transmission protocol and the message routing path comprises dynamic resource allocation, dynamic channel creation, dynamic channel selection, or any combination thereof.

20. The system of claim 1, having boundaries that transcend regional, national or continental borders, with one or more subsystems in each region, country or continent, wherein the subsystems are linked via a networking infrastructure and each subsystem includes a provisioning and management system, an interconnect, and one or more messaging appliances.

21. The system of claim 1, wherein the provisioning and management system is integrated into one of the one or more messaging appliances or is implemented as a standalone appliance.

22. The system of claim 1, in which each of the one or more messaging appliances includes:

a network management stack linked to a physical interface management functional block;

a services block containing a system management services functional block and a time stamping service functional block, both linked via a network management internal communications logical bus to the network management stack; and

a native message layer in communication with a messaging transport layer, both of which link via a messaging internal communications logical bus to the services block.

23. The system of claim 22, wherein the native message layer includes an administrative message layer, a message routing engine, message transmit and message receive logic and a master protocol optimization service.

24. The system of claim 22, wherein the messaging transport layer includes channel management, and wherein the dynamic selection of a message routing path includes one or more of selection and creation of a channel.

25. The system of claim 24, wherein each channel is configured for network-based, node-based or memory-based transmission protocol and is associated with a physical interface to a physical medium which is fabric agnostic.

26. The system of claim 25, wherein the physical medium is configured as Ethernet, memory-based direct connect or Infiniband.

27. The system of claim 1, wherein the provisioning and management system includes message transport and native message layers linked to a configuration functional block and a monitoring functional block which is, in turn, connected via an inter-process communications bus to management blocks that encompass configuration management, real time monitoring, historical trending and application business reporting functional blocks.

28. The system of claim 27, wherein the provisioning and management system further includes one or both of:

a network management service connected on one side to the monitoring functional block and on the other side to the network stack of the operating system; and

a user interface connected to the management blocks.

29. The system of claim 25, wherein the interconnect includes a transport channel and the physical medium through which the one or more messaging appliances communicates with the provisioning and management system.

30. The system of claim 1, further comprising one or more caching engines each operatively connected to a respective messaging appliance for providing quality of service functionality including message data store and forward functionality.

31. The system of claim 30, wherein each caching engine includes a caching layer connected with a native message layer which is, in turn, connected to a message transport layer, wherein the caching layer includes storage, a storage service and an indexing service.

32. The system of claim 1 wherein one of the one or more messaging appliances is operatively connected to an application via an application programming interface that is registered to such messaging appliance and delivers messages between the application and the messaging appliance.

33. The system of claim 32, wherein each of the one or more messaging appliances includes a master protocol optimization service and each application programming interface includes a slave protocol optimization service responsive to its respective master protocol optimization service.

34. The system of claim 32, wherein each application programming interface includes, a communication engine and one or more application stubs linked thereto.

35. The system of claim 34, wherein the communication engine is a daemon.

36. The system of claim 32, wherein each application programming interface is deployed on top of an operating system in the client application host.

37. The system of claim 32, wherein each application programming interface includes:

an application delivery engine for transmitting messages to the application; and

an administrative message layer for handling administrative messages.

38. A system with a publish/subscribe middleware architecture, comprising:

one or more namespace domains; and

a physical domain interconnect medium for connecting between the namespace domains,

wherein each namespace domains includes;

one or more messaging appliances configured for receiving and routing messages,

an interconnect utilizing channel-based messaging that routes messages over a first messaging layer based on one or more channels mapped to subscription topics, each of the one or more channels assigned to a communication pathway of a second messaging layer; and

a provisioning and management system linked via the interconnect to the one or more messaging appliances and configured to exchange administrative messages with each of the one or more messaging appliances,

wherein each of the one or more messaging appliances is further configured to execute the routing of messages by dynamically selecting a message transmission protocol and a message routing path.

39. An enterprise system with a publish/subscribe middleware architecture, comprising:

a market data delivery infrastructure having one or more messaging appliances for receiving and routing market data messages;

a market order routing infrastructure having one or more messaging appliances to receive and route transaction order messages; and

an intermediate infrastructure in communication link with the market data delivery and market order routing infrastructures, respectively, wherein the intermediary infrastructure includes;

one or more messaging appliances configured for receiving and routing the market data and transaction order messages,

an interconnect utilizing channel-based messaging that routes messages over a first messaging layer based on one or more channels mapped to subscription topics, each of the one or more channels assigned to a communication pathway of a second messaging layer; and

a provisioning and management system linked via the interconnect to the one or more messaging appliances and configured for exchanging administrative messages with each of the one or more messaging appliances, including the messaging appliances in the market data delivery and market order routing infrastructures,

wherein each of the messaging appliances is further configured to execute the routing of messages it receives by dynamically selecting a message transmission protocol and a message routing path.

40. The enterprise system of claim 39, further comprising:

market data sources for publishing the market data messages; and

market data consumers for receiving the 10 market data messages and for publishing the transaction order messages, the market data consumers including one or more applications, wherein the intermediate infrastructure includes an application programming interface between each of the applications and one of the one or more messaging appliances in the intermediate infrastructure to which such application programming interface is registered, the application programming interfaces operative for delivering the market data messages to and transaction order messages from the applications.

41. The enterprise system of claim 39, further comprising protocol transformation engines and in which the one or more messaging appliances in the market data delivery and market order routing infrastructures are configured as edge messaging appliances and the one or more messaging appliances in the intermediate infrastructure are configured as core messaging appliances, wherein each edge messaging appliance employs its respective protocol transformation engine to transform messages between external protocol and native protocol, and wherein each core messaging appliance is configured for handling messages in the native protocol.

42. The enterprise system of claim 39, wherein the one or more messaging appliances are interconnected to provide network disintermediation.

43. The enterprise system of claim 39, wherein one or more of the one or more messaging appliances is an embedded component in a switching or routing device.