Systems and methods for network virtualization

• US 8,321,578 B2
• Filed: 06/03/2011
• Issued: 11/27/2012
• Est. Priority Date: 01/06/2005
• Status: Active Grant

• Current Assignee: Netcharts-Gb LLC
• Original Assignee: Tervela, Inc. (Box Incorporated)
• Inventors: Thompson, J. Barry, Singh, Kul, Fraval, Pierre
• Primary Examiner(s): Neurauter, George C
• Application Number:
  US13/152,939
• Publication Number:
  US 20110231470A1
• Time in Patent Office: 543 Days
• Field of Search: 709/230, 709/238, 719/313, 719/328
• US Class Current: 709/230
• CPC Class Codes: H04L 12/1859 adapted to provide push services, e.g. data channels
• CPC Class Codes: H04L 43/06 Generation of reports
• CPC Class Codes: H04L 43/0852 Delays
• CPC Class Codes: H04L 45/24 Multipath
• CPC Class Codes: H04L 47/11 Identifying congestion
• CPC Class Codes: H04L 47/15 in relation to multipoint traffic arrangements for broadcast or multicast in data networks H04L12/18
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 47/801 Real time traffic
• CPC Class Codes: H04L 47/806 Broadcast or multicast traffic
• CPC Class Codes: H04L 47/822 Collecting or measuring resource availability data
• CPC Class Codes: H04L 47/829 Topology based
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context

Abstract

Message publish/subscribe systems are required to process high message volumes with reduced latency and performance bottlenecks. The end-to-end middleware architecture proposed by the present invention is designed for high-volume, low-latency messaging by, among other things, reducing intermediary hops with neighbor-based routing, introducing efficient native-to-external and external-to-native protocol conversions, monitoring system performance, including latency, in real time, employing topic-based and channel-based message communications, and dynamically optimizing system interconnect configurations and message transmission protocols.

Claims

1. An end-to-end publish/subscription messaging system with a middleware architecture, comprising:

at least one messaging appliance configured to receive and route messages; and

an interconnect utilizing channel-based messaging that routes messages over a first messaging layer based on at least one channel, each channel mapped to a subscription topic, each channel assigned to a communication pathway of a second messaging layer,

wherein each messaging appliance is further configured to execute the routing of messages by dynamically selecting, in real time, a message transmission protocol and a message routing path.

2. The system of claim 1, wherein the messaging appliances include one or more of an edge messaging appliance and a core messaging appliance.

3. The system of claim 1, wherein each edge messaging appliance is linked to a message transformation engine for transforming incoming messages from an external protocol to a native protocol and for transforming routed messages from the native protocol to the external protocol.

4. The system of claim 1, wherein the message transmission protocol is selected to be one of a unicast, multicast or broadcast protocol.

5. The system of claim 1, further including one or more application programming interfaces configured for interfacing between one or more applications and one of the messaging appliances.

6. The system of claim 5, in which the messaging appliances and the one or more application programming interfaces are operative to communicate with each other by incorporating one or more messages in a single frame.

7. The system of claim 1, wherein each of the applications is configured to send requests, including registration and subscription requests, to a respective one of the messaging appliances.

8. The system of claim 1, wherein each of the application programming interfaces is logically linked to a messaging appliance having been registered to it via a topic-based subscription.

9. The system of claim 8, wherein the messaging appliances include one or more core messaging appliances to which the application programming interfaces register.

10. The system of claim 1, wherein the subscriptions are topic-based each being established via a subscription request, and wherein a single subscription request is capable of establishing subscriptions to a group of related topics.

11. The system of claim 1, in which the interconnect is an interconnect network over which the messaging appliances is deployed, the network being configured with any number of routers, a switches and a subnets.

12. The system of claim 1, wherein the interconnect is a channel-based, fabric agnostic physical medium.

13. A system as in claim 1, wherein the messaging appliances and interconnect incorporate transport logic.

14. The system of claim 13, configured for transport transparent channel-based messaging where messages are communicated in native protocol format independent of the transport logic.

15. The system of claim 1, further comprising one or more external sources and external destinations, wherein the messaging appliances include one or more edge messaging appliances, each associated to a protocol translation engine and translating messages between external and native protocols, and one or more core messaging appliances, each conducting communications of messages with the native protocol, the external sources and destinations communicating with the edge messaging appliances which, in turn, communicate with the core messaging appliances using neighbor-based message routing.

16. The system of claim 15, in which an edge MA is operative to route an ingress message simultaneously to both a native protocol consumer and an external protocol consumer.

17. The system of claim 1, wherein the messages are constructed with schema and payload which are separated from each other when messages enter the system, and which are combined when messages leave the system.

18. The system of claim 1, wherein, with the namespace management, consumers or external destinations who are subscribed to topics associated with a particular namespace are entitled to publish and subscribe messages identified with such topics.

19. The system of claim 1, wherein each messaging appliance has a routing table with routing of messages between messaging appliances being neighbor-based such that each messaging appliance is configured to route messages via one of its channels to a neighbor that has subscribed to all or a subset of messages transmitted via that channel, and wherein each messaging appliance is further configured to optimize the mapping between its subscriptions and channels in order to reduce the discard rate at neighbor that subscribe only to a subset of the messages.

20. The system of claim 1, wherein the routing table has one of a plurality of structures two of which are a tree structure and a dynamic has map structure.

21. The system of claim 1, wherein the messages and administrative messages have a topic based format, each message having a header and a payload, the header including a topic field in addition to source and destination namespace identification fields.

22. The system of claim 21, wherein the topic field includes a variable-length string or a key, the key being a unique value.

23. The system of claim 21, wherein the messages include a subscription message with a topic field that has a variable-length string with any number of wild card characters for matching it with any topic substring provided that such topic and the subscription message have the same number of topic substrings.

24. The system of claim 1, wherein the dynamic selection of transmission protocol and message routing path is based on system topology, health and performance reports and it involves one or both of dynamic resource allocation and dynamic channel creation and/or selection.

25. The system of claim 1, having boundaries that transcend regional, national or continental borders, with subsystems in each region, country or continent, wherein the subsystems are linked via a networking infrastructure and each subsystem includes an interconnect and one or more messaging appliances.

26. The system of claim 1, in which each messaging appliance includes:

a network management stack linked to a physical interface management functional block;

a services block containing a system management services functional block and a time stamping service functional block, both linked via a network management internal communications logical bus to the network management stack; and

a native message layer in communication with a messaging transport layer, both of which linked via a messaging internal communications logical bus to the services block.

27. The system of claim 26, wherein the native message layer includes an administrative message layer, a message routing engine, message transmit and message receive logic and a master protocol optimization service.

28. The system of claim 26, wherein the messaging transport layer include channel management, and wherein the dynamic selection of a message routing path includes selection and/or creation of a channel.

29. The system of claim 28, wherein each channel is configured for network-based, node-based or memory-based transmission protocol and is associated with a physical interface to a physical medium which is fabric agnostic.

30. The system of claim 29, wherein the physical medium is configured as Ethernet, memory-based direct connect or Infiniband.

31. The system of claim 28, wherein the interconnect includes the transport channel and physical medium through which the messaging appliance communicates with the provisioning and management system.

32. The system of claim 1, further comprising one or more caching engines each operatively connected to a respective messaging appliance for providing quality of service functionality including message data store and forward functionality.

33. The system of claim 32, wherein each caching engine includes a caching layer connected with a native message layer which is, in turn, connected to a message transport layer, wherein the caching layer includes storage, a storage service and an indexing service.

34. The system of claim 1 wherein one or more of the messaging appliances is operatively connected to an application via an application programming interface that is registered to such messaging appliance and delivers messages between the application and the messaging appliance.

35. The system of claim 34, wherein each messaging appliance includes a master protocol optimization service and each application programming interface includes a slave protocol optimization service responsive to its respective master protocol optimization service.

36. The system of claim 34, wherein each application programming interface includes, a communication engine and one or more application stubs linked thereto.

37. The system of claim 36, wherein the communication engine is a daemon.

38. The system of claim 34, wherein each application programming interface is deployed on top of an operating system in the client application host.

39. The system of claim 34, wherein each application programming interface includes:

an application delivery engine for transmitting messages to the application; and

an administrative message layer for handling administrative messages.

40. The system of claim 1, wherein each of the messaging appliances are configured for fault tolerance.

41. The system of claim 40, in which the messaging appliances are arranged in fault tolerant pairs each pair including a primary and secondary messaging appliance, the secondary messaging appliance taking over a session from the primary messaging appliance upon failure of such session but without interfering with other active sessions on the primary messaging appliance.

42. A system with a publish/subscribe middleware architecture, comprising:

a plurality of namespace domains; and

a physical domain interconnect medium for connecting at least two of the namespace domains,

wherein each namespace domains includes;

at least one messaging appliance configured to receive and route messages; and

an interconnect utilizing channel-based messaging that routes messages over a first messaging layer based on at least one channel, each channel mapped to a subscription topic, each channel assigned to a communication pathway of a second messaging layer,

wherein each messaging appliance is further configured to execute the routing of messages by dynamically selecting a message transmission protocol and a message routing path.

43. An enterprise system with a publish/subscribe middleware architecture, comprising:

a market data delivery infrastructure having at least one messaging appliance for receiving and routing market data messages;

a market order routing infrastructure having at least one messaging appliance to receive and route transaction order messages; and

an intermediate infrastructure in communication with the market data delivery and market order routing infrastructures, respectively, wherein the intermediary infrastructure includes;

at least one messaging appliance configured for receiving and routing the market data and transaction order messages; and

an interconnect utilizing channel-based messaging that routes messages over a first messaging layer based on at least one channel, each channel mapped to a subscription topic, each channel assigned to a communication pathway of a second messaging layer,

wherein each of the messaging appliances is further configured to execute the routing of messages it receives by dynamically selecting a message transmission protocol and a message routing path.

44. The enterprise system of claim 43, further comprising:

market data sources for publishing the market data messages; and

market data consumers for receiving the market data messages and for publishing the transaction order messages, the market data consumers including at least one application,

wherein the intermediate infrastructure includes an application programming interface between each application and a messaging appliance in the intermediate infrastructure to which such application programming interface is registered, the application programming interface operative for delivering the market data messages to and transaction order messages from the application.

45. The enterprise system of claim 43, further comprising protocol transformation engines and in which each of the at least one messaging appliance in each of the market data delivery and market order routing infrastructures are configured as edge messaging appliances and each of the at least one messaging appliance in the intermediate infrastructure is configured as core messaging appliances, wherein each edge messaging appliance employs its respective protocol transformation engine to transform messages between external protocol and native protocol, wherein each core messaging appliance is configured for handling messages in the native protocol.

46. The enterprise system of claim 43, wherein the messaging appliances are interconnected to provide network disintermediation.

47. The enterprise system of claim 43, wherein at least one of messaging appliances is an embedded component in a switching or routing device.

48. A method for routing messages in a system with publish/subscribe middleware architecture, comprising:

providing a messaging fabric having at least one messaging appliance;

dynamically selecting in a messaging appliance, in real time, a message transmission protocol and a message routing path; and

routing at least one message from the dynamically selected messaging appliance to a destination.

49. A method as in claim 48, wherein the destination is selected from the group consisting of another messaging appliance, an application programming interface logically linked to the dynamically selected messaging appliance, or both.

50. A method as in claim 48, further comprising:

interconnecting each messaging appliance with a provisioning and management system; and

exchanging at least one administrative message between a provisioning and management system and the interconnected messaging appliance, the administrative message being associated with at least one of system configuration, health, performance monitoring, reporting, and namespace management.

51. A method as in claim 48, further comprising transforming the at least one routed message from a native protocol to an external protocol.

52. A method as in claim 48, further comprising transforming the at least one routed message from a native protocol to an external protocol.

53. A method as in claim 48, further comprising:

logically linking an application programming interface to a messaging appliance; and

registering, via a topic-based subscription, the application programming interface to the logically linked messaging appliance.

54. A method as in claim 48, further comprising storing the at least one message in a caching engine operatively connected to at least one messaging appliance for providing quality of service functionality.

55. A method as in claim 54, further comprising forwarding the stored message to an application programming interface logically linked to at least one messaging appliance.

56. An end-to-end messaging system with a middleware architecture, comprising:

at least one messaging appliance configured to receive and route messages; and

an interconnect utilizing channel-based messaging that routes messages over a first messaging layer based on at least one channel, each channel mapped to a topic, each channel assigned to a communication pathway of a second messaging layer,

wherein each messaging appliance is further configured to execute the routing of messages by dynamically selecting, in real time, a message transmission protocol and a message routing path.

57. A method for routing messages in an end-to-end messaging system with a middleware architecture, comprising:

routing, at a messaging appliance, a message over a first messaging layer based on at least one channel, each channel mapped to a topic, each channel assigned to a communication pathway of a second messaging layer, and

dynamically selecting, at the messaging appliance, in real time, a message transmission protocol and a message routing path.