Gapless delivery and durable subscriptions in a content-based publish/subscribe system

• US 20040003064A1
• Filed: 06/21/2002
• Published: 01/01/2004
• Est. Priority Date: 06/21/2002
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kaplan, Marc A., Strom, Robert E., Bhola, Sumeer K., Zhao, Yuanyuan, Astley, Mark C., Auerbach, Joshua S.
• Granted Patent: US 7,162,524 B2
• Time in Patent Office: Days
• US Class Current: 709/223
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A content-based publish/subscribe system for providing gapless message delivery from a publishing client to a subscribing client, the system includes a plurality of brokers operatively coupled to one another via a network. Each of the brokers include at least one processor and are configured as a publisher-hosting broker, a subscriber-hosting broker or an intermediate broker. The subscriber-hosting broker is operatively coupled to the publisher-hosting broker via the network through the intermediate broker. At least a portion of the plurality of brokers are configured so as to eliminate a need for persistent storage of messages at the intermediate broker and to substantially guarantee a gapless delivery of one or more messages transmitted by the publishing client to the subscribing client, even in the presence of a failure in the publish/subscribe system.

Claims

1. A content-based publish/subscribe system for providing gapless message delivery from a publishing client to a subscribing client, the system comprising:

a plurality of brokers operatively coupled to one another via a network, each of the brokers including at least one processor and being configured as one of a publisher-hosting broker, a subscriber-hosting broker and an intermediate broker, the subscriber-hosting broker being operatively coupled to the publisher-hosting broker via the network through the intermediate broker;

wherein at least a portion of the plurality of brokers are configured so as to eliminate a need for persistent storage of messages at the intermediate broker and to substantially guarantee a gapless delivery of one or more messages transmitted by the publishing client to the subscribing client, even in the presence of a failure in the publish/subscribe system.

2. The system of claim 1, wherein at least one of the plurality of brokers is selectively configurable as at least one of a publisher-hosting broker, a subscriber-hosting broker, and an intermediate broker.

3. The system of claim 1, wherein at least one of the plurality of brokers is configured to selectively filter content transmitted by the publishing client in response to criteria specified by the subscribing client.

4. The system of claim 1, wherein at least one subscriber-hosting broker includes persistent storage, the at least one subscriber-hosting broker being configured so as to provide a durable message stream to at least one durable subscribing client, the at least one durable subscribing client being capable of selectively disconnecting from and reconnecting to the publish/subscribe system.

5. The system of claim 4, wherein one or more messages deliverable to the at least one durable subscribing client are persistently stored in at least one publisher-hosting broker during a time interval within which the at least one durable subscribing client is disconnected from the system.

6. A content-based publish/subscribe system for providing gapless message delivery from a publishing client to a subscribing client, comprising:

a plurality of brokers operatively coupled together via a network, each of the brokers including at least one processor and memory coupled to the processor, each of the brokers being operative in at least one of a publisher-hosting broker mode, a subscriber-hosting broker mode and an intermediate broker mode, wherein;

in the publisher-hosting broker mode, the at least one processor is operative to;

(i) receive a message from a publishing client coupled to the publish/subscribe system;

(ii) store the received message and a stream state corresponding thereto; and

(iii) transmit the message to one or more downstream brokers;

in the intermediate broker mode, the at least one processor is operative to;

(i) receive a knowledge message from an upstream broker;

(ii) accumulate the received knowledge message with an existing information stream;

(iv) in response to a request for knowledge, transmit at least a portion of the accumulated knowledge messages satisfying at least a portion of the request for knowledge to one or more downstream brokers; and

(v) transmit curiosity messages corresponding the knowledge messages that cannot be satisfied to one or more upstream brokers;

in the subscriber-hosting broker mode, the at least one processor is operative to;

(i) receive a knowledge message from an upstream broker;

(ii) determine an original maximum time interval in which all messages therein are known;

(iii) accumulate one or more received knowledge messages in a subend knowledge stream;

(iv) determine a new maximum time interval in which all messages therein are known based on the accumulated knowledge messages; and

(v) when the new maximum time interval is greater than the original maximum time interval, transmit all knowledge messages between the new and original maximum time intervals to one or more subscriber clients;

whereby at least a portion of the plurality of brokers are configured so as to eliminate a need for persistent storage of messages at brokers operating in the intermediate broker mode and to substantially guarantee a gapless delivery of one or more messages transmitted by the publishing client to the subscribing client, even in the presence of a failure in the publish/subscribe system.

7. The system of claim 6, wherein in the publisher-hosting broker mode, the at least one processor, in response to a request for knowledge received from a downstream broker, is further operative to:

(iv) retrieve a stream state corresponding to a time interval designated in the request for knowledge;

(v) generate one or more knowledge messages corresponding to the time interval; and

(vi) transmitting the one or more knowledge messages to the downstream broker from which the request for knowledge was received.

8. The system of claim 6, wherein in the publisher-hosting broker mode, the at least one processor is further operative to store a timestamp corresponding to a time at which the message is received from the publishing client.

9. The system of claim 6, wherein in the intermediate broker mode, the at least one processor is further operative to selectively filter content transmitted by the publishing client in response to criteria specified by the subscribing client.

10. The system of claim 6, wherein in the intermediate broker mode, the at least one processor is further operative to selectively merge at least two information streams and generate a single information stream corresponding thereto.

11. In an intermediate broker associated with a content-based publish/subscribe system, a method for providing gapless message delivery from a publishing client to a subscribing client, comprising the steps of:

receiving a knowledge message from an upstream broker;

accumulating the received knowledge message with an existing information stream;

in response to a request for knowledge, transmitting at least a portion of the accumulated knowledge messages satisfying at least a portion of the request for knowledge to one or more downstream brokers; and

transmitting one or more curiosity messages corresponding to one or more knowledge messages that cannot be satisfied to one or more upstream brokers;

12. The method of claim 11, further comprising the step of selectively filtering content transmitted by the publishing client in response to criteria specified by the subscribing client.

13. In a subscriber-hosting broker associated with a content-based publish/subscribe system, a method for providing gapless message delivery from a publishing client to a subscribing client, comprising the steps of:

receiving a knowledge message from an upstream broker;

determining an original maximum time interval in which all messages therein are known;

accumulating one or more received knowledge messages in a subend knowledge stream;

determining a new maximum time interval in which all messages therein are known based on the accumulated knowledge messages; and

when the new maximum time interval is greater than the original maximum time interval, transmitting all knowledge messages between the new and original maximum time intervals to one or more subscriber clients;