End-to-end publish/subscribe middleware architecture

• US 20060149840A1
• Filed: 12/23/2005
• Published: 07/06/2006
• Est. Priority Date: 01/06/2005
• Status: Active Grant

• Current Assignee: Tervela, Inc. (Box Incorporated)
• Original Assignee: Tervela, Inc. (Box Incorporated)
• Inventors: Fraval, Pierre, Singh, Kul, Thompson, J. Barry
• Granted Patent: US 7,970,918 B2
• Time in Patent Office: Days
• US Class Current: 709/224
• CPC Class Codes: H04L 12/1859 adapted to provide push services, e.g. data channels
• CPC Class Codes: H04L 43/06 Generation of reports
• CPC Class Codes: H04L 43/0852 Delays
• CPC Class Codes: H04L 45/24 Multipath
• CPC Class Codes: H04L 47/11 Identifying congestion
• CPC Class Codes: H04L 47/15 in relation to multipoint traffic arrangements for broadcast or multicast in data networks H04L12/18
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 47/801 Real time traffic
• CPC Class Codes: H04L 47/806 Broadcast or multicast traffic
• CPC Class Codes: H04L 47/822 Collecting or measuring resource availability data
• CPC Class Codes: H04L 47/829 Topology based
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context

Abstract

Message publish/subscribe systems are required to process high message volumes with reduced latency and performance bottlenecks. The end-to-end middleware architecture proposed by the present invention is designed for high-volume, low-latency messaging by, among other things, reducing intermediary hops with neighbor-based routing, introducing efficient native-to-external and external-to-native protocol conversions, monitoring system performance, including latency, in real time, employing topic-based and channel-based message communications, and dynamically optimizing system interconnect configurations and message transmission protocols.

Claims

1. A system with a middleware architecture, comprising:

one or more than one messaging appliance configured for receiving and routing messages;

an interconnect; and

a provisioning and management system linked via the interconnect and configured for exchanging administrative messages with each messaging appliance, wherein each messaging appliance is further configured to execute the routing of messages by dynamically selecting, in real time, a message transmission protocol and a message routing path.

2. A system as in claim 1, wherein the provisioning and management system is configured to perform functions associated with the administrative messages including system configuration, health and performance monitoring and reporting.

3. A system as in claim 1, wherein the provisioning appliance is configured for administering subscriptions, including subscription of consumers and external data destinations to one or more data message topics and subscription of messaging appliances to administrative message topics.

4. A system as in claim 1, wherein the messaging appliances include one or more of an edge messaging appliance and a core messaging appliance.

5. A system as in claim 1, wherein each edge messaging appliance is linked to a message transformation engine for transforming incoming messages from an external protocol to a native protocol and for transforming routed messages from the native protocol to the external protocol.

6. A system as in claim 1, wherein the message transmission protocol is selected to be one of a unicast, multicast or broadcast protocol.

9. A system as in claim 1, wherein each of the applications is configured to send requests, including registration and subscription requests, to a respective one of the messaging appliances, wherein the provisioning and management system is further configured to handle digital rights management where each respective messaging appliance confirms and reports to the provisioning and management system whether an application attempting to register or subscribe with it is entitled to do so.

10. A system as in claim 1, wherein each of the application programming interfaces is logically linked to a messaging appliance having been registered to it via a topic-based subscription.

11. A system as in claim 10, wherein the messaging appliances include one or more core messaging appliances to which the application programming interfaces register.

12. A system as in claim 1, wherein the subscriptions are topic-based each being established via a subscription request, and wherein a single subscription request is capable of establishing subscriptions to a group of related topics.

13. A system as in claim 1, in which the interconnect is an interconnect network over which the messaging appliances and provisioning and management systems are deployed, the network being configured with any number of routers, a switches and a subnets.

14. A system as in claim 1, wherein the interconnect is a channel-based, fabric agnostic physical medium.

15. A system as in claim 1, wherein the messaging appliances, provisioning and management systems and interconnect incorporate transport logic.

16. A system as in claim 15, configured for transport transparent channel-based messaging where messages are communicated in native protocol format independent of the transport logic.

17. A system as in claim 1, further comprising one or more external sources and external destinations, wherein the messaging appliances include one or more edge messaging appliances, each associated to a protocol translation engine and translating messages between external and native protocols, and one or more core messaging appliances, each conducting communications of messages with the native protocol, the external sources and destinations communicating with the edge messaging appliances which, in turn, communicate with the core messaging appliances using neighbor-based message routing.

18. A system as in claim 17, in which an edge MA is operative to route an ingress message simultaneously to both a native protocol consumer and an external protocol consumer.

19. A system as in claim 1, wherein the messages are constructed with schema and payload which are separated from each other when messages enter the system and which are combined when messages leave the system.

20. A system as in claim 1, wherein the provisioning and management system performs a namespace management function which includes digital rights management.

21. A system as in claim 1, wherein, with the namespace management, consumers or external destinations who are subscribed to topics associated with a particular namespace are entitled to publish and subscribe messages identified with such topics.

22. A system as in claim 1, wherein each messaging appliance has a routing table with routing of messages between messaging appliances being neighbor-based such that each messaging appliance is configured to route messages via one of its channels to a neighbor that has subscribed to all or a subset of messages transmitted via that channel, and wherein each messaging appliance is further configured to optimize the mapping between its subscriptions and channels in order to reduce the discard rate at neighbor that subscribe only to a subset of the messages.

23. A system as in claim 1, wherein the routing table has one of a plurality of structures two of which are a tree structure and a dynamic has map structure.

24. A system as in claim 1, wherein the messages and administrative messages have a topic-based format, each message having a header and a payload, the header including a topic field in addition to source and destination namespace identification fields.

25. A system as in claim 24, wherein the topic field includes a variable-length string or a key, the key being a unique value where, for keys, the provisioning and management system is configured with a database for maintaining a mapping between each such key and its respective topic the provisioning and management system being further configured for updating each of the messaging appliances about any changes in this mapping.

26. A system as in claim 24, wherein the messages include a subscription message with a topic field that has a variable-length string with any number of wild card characters for matching it with any topic substring provided that such topic and the subscription message have the same number of topic substrings.

27. A system as in claim 1, wherein the dynamic selection of transmission protocol and message routing path is based on system topology, health and performance reports from the provisioning and management system and it involves one or both of dynamic resource allocation and dynamic channel creation and/or selection.

28. A system as in claim 1, having boundaries that transcend regional, national or continental borders, with subsystems in each region, country or continent, wherein the subsystems are linked via a networking infrastructure and each subsystem includes a provisioning and management system, interconnect and one or more messaging appliances.

29. A system as in claim 1, wherein the provisioning and management system is integrated into one of the messaging appliances or is implemented as a standalone appliance.

30. A system as in claim 1, in which each messaging appliance includes:

a network management stack linked to a physical interface management functional block;

a services block containing a system management services functional block and a time stamping service functional block, both linked via a network management internal communications logical bus to the network management stack; and

a native message layer in communication with a messaging transport layer, both of which linked via a messaging internal communications logical bus to the services block.

31. A system as in claim 30, wherein the native message layer includes an administrative message layer, a message routing engine, message transmit and message receive logic and a master protocol optimization service.

32. A system as in claim 30, wherein the messaging transport layer include channel management, and wherein the dynamic selection of a message routing path includes selection and/or creation of a channel.

33. A system as in claim 32, wherein each channel is configured for network-based, node-based or memory-based transmission protocol and is associated with a physical interface to a physical medium which is fabric agnostic.

34. A system as in claim 33, wherein the physical medium is configured as Ethernet, memory-based direct connect or Infiniband.

35. A system as in claim 1, wherein the provisioning and management system includes message transport and native message layers linked to a configuration functional block and a monitoring functional block which is, in turn, connected via an inter-process communications bus to management blocks that encompass configuration management, real time monitoring, historical trending and application business reporting functional blocks.

36. A system as in claim 35, wherein the provisioning and management system further includes one or both of:

a network management service connected on one side to the monitoring functional block and on the other side to the network stack of the operating system; and

a user interface connected to the management blocks.

37. A system as in claim 32, wherein the interconnect includes the transport channel and physical medium through which the messaging appliance communicates with the provisioning and management system.

38. A system as in claim 1, further comprising one or more caching engines each operatively connected to a respective messaging appliance for providing quality of service functionality including message data store and forward functionality.

39. A system as in claim 38, wherein each caching engine includes a caching layer connected with a native message layer which is, in turn, connected to a message transport layer, wherein the caching layer includes storage, a storage service and an indexing service.

40. A system as in claim 1 wherein one or more of the messaging appliances is operatively connected to an application via an application programming interface that is registered to such messaging appliance and delivers messages between the application and the messaging appliance.

41. A system as in claim 40, wherein each messaging appliance includes a master protocol optimization service and each application programming interface includes a slave protocol optimization service responsive to its respective master protocol optimization service.

42. A system as in claim 40, wherein each application programming interface includes, a communication engine and one or more application stubs linked thereto.

43. A system as in claim 42, wherein the communication engine is a daemon.

44. A system as in claim 40, wherein each application programming interface is deployed on top of an operating system in the client application host.

45. A system as in claim 40, wherein each application programming interface includes:

an application delivery engine for transmitting messages to the application; and

an administrative message layer for handling administrative messages.

46. A system as in claim 1, wherein each of the messaging appliances and provisioning and management systems are configured for fault tolerance.

47. A system as in claim 46, in which the messaging appliances are arranged in fault tolerant pairs each pair including a primary and secondary messaging appliance, the secondary messaging appliance taking over a session from the primary messaging appliance upon failure of such session but without interfering with other active sessions on the primary messaging appliance.

7. A system as in 1, further including one or more application programming interfaces configured for interfacing between one or more applications and one of the messaging appliances.

8. A system as in claim 7, in which the messaging appliances and the one or more application programming interfaces are operative to communicate with each other by incorporating one or more messages in a single frame.

48. A system with a publish/subscribe middleware architecture, comprising:

one or more namespace domains; and

if more than one space domain exists, a domain interconnect medium for connecting between the namespace domains,

wherein each namespace domains includes;

one or more than one messaging appliance configured for receiving and routing messages,

an interconnect; and

a provisioning and management system linked via the interconnect and configured for exchanging administrative messages with each messaging appliance,

wherein each messaging appliance is further configured to execute the routing of messages by dynamically selecting a message transmission protocol and a message routing path.

49. An enterprise system with a publish/subscribe middleware architecture, comprising:

a market data delivery infrastructure having one or more messaging appliances for receiving and routing market data messages;

a market order routing infrastructure having one or more messaging appliances for receiving and routing transaction order messages; and

an intermediate infrastructure in communication link with the market data delivery and market order routing infrastructures, respectively, wherein the intermediary infrastructure includes;

one or more than one messaging appliance configured for receiving and routing the market data and transaction order messages,

an interconnect; and

a provisioning and management system linked via the interconnect and configured for exchanging administrative messages with each messaging appliance, including the messaging appliances in the market data delivery and market order routing infrastructures;

wherein each of the messaging appliances is further configured for executing the routing of messages it receives by dynamically selecting a message transmission protocol and a message routing path.

50. The enterprise system as in claim 49, further comprising:

market data sources for publishing the market data messages; and

market data consumers for receiving the market data messages and for publishing the transaction order messages, the market data consumers including one or more applications,

wherein the intermediate infrastructure includes an application programming interface between each of the applications and one of the messaging appliances in the intermediate infrastructure to which such application programming interface is registered, the application programming interfaces operative for delivering the market data messages to and transaction order messages form the applications.

51. The enterprise system of claim 49, further comprising protocol transformation engines and in which messaging appliances in the market data delivery and market order routing infrastructures are configured as edge messaging appliances and messaging appliances in the intermediate infrastructure are configured as core messaging appliances, wherein each edge messaging appliance employs its respective protocol transformation engine to transform messages between external protocol and native protocol, wherein each core messaging appliance is configured for handling messages in the native protocol.