Continuous feedback-controlled deployment of message transforms in a distributed messaging system
First Claim
1. A method for moving a message transform from a source broker to a target broker in a distributed messaging system, wherein the distributed messaging system includes a network of broker machines, wherein a plurality of message transforms are deployed in the network of broker machines, and wherein the plurality of message transforms take as input one or more sequences of messages and produce as output one or more sequences of messages, the method comprising:
- starting on the target broker a copy of a given message transform from the source broker;
duplicating all inputs into the given message transform on the source broker to additionally go to the copy of the given message transform on the target broker; and
duplicating all outputs of the given message transform on the source broker to also go from the copy of the given message transform on the target broker to appropriate children nodes within the distributed messaging system.
0 Assignments
0 Petitions
Accused Products
Abstract
The present invention solves the disadvantages of the prior art and provides a distributed messaging system supporting stateful subscriptions. A stateful publish-subscribe system extends the functionality of the content-based approach to include more general state-valued expressions. A plurality of broker machines is provided to deliver messages sent by publishing clients toward subscribing clients based upon the contents of the messages and stateful transformations requested by the subscribing clients. Subscription specifications are analyzed by a compiler and converted into a collection of message transforms and views. A deployment service component deploys tasks to optimize system performance. A monitoring services component detects a possible need to reconfigure. A performance optimization service component computes new assignment of transforms. A continuous deployment service implements a redeployment protocol that installs changes to transform placement while the existing publish-subscribe system continues to operate.
-
Citations
22 Claims
-
1. A method for moving a message transform from a source broker to a target broker in a distributed messaging system, wherein the distributed messaging system includes a network of broker machines, wherein a plurality of message transforms are deployed in the network of broker machines, and wherein the plurality of message transforms take as input one or more sequences of messages and produce as output one or more sequences of messages, the method comprising:
-
starting on the target broker a copy of a given message transform from the source broker; duplicating all inputs into the given message transform on the source broker to additionally go to the copy of the given message transform on the target broker; and duplicating all outputs of the given message transform on the source broker to also go from the copy of the given message transform on the target broker to appropriate children nodes within the distributed messaging system. - View Dependent Claims (2, 3, 4, 5)
-
-
6. A method for continuous deployment of message transforms in a distributed messaging system, wherein the distributed messaging system includes a network of broker machines, the method comprising:
-
deploying a plurality of message transforms in the network of broker machines, wherein the plurality of message transforms take as input one or more sequences of messages and produce as output one or more sequences of messages; receiving performance information from the network of broker machines; and redeploying at least a subset of the plurality of message transforms in the network of broker machines based on the performance information, wherein redeploying at least the subset of the plurality of message transforms includes moving a given message transform from a source broker to a target broker in the distributed messaging system. - View Dependent Claims (7, 8)
-
-
9. An apparatus for implementing a stateful subscription service in a distributed messaging system, the apparatus comprising:
-
an overlay network including a plurality of broker machines; a plurality of publishing clients that publish messages to published message streams; a performance optimization service that determines an assignment of message transforms to broker machines in the overlay network; and a continuous deployment service that deploys the assignment of message transforms to the plurality of broker machines, wherein the continuous deployment service redeploys at least a subset of the message transforms by moving a given message transform from a source broker to a target broker in the distributed messaging system. - View Dependent Claims (10, 11)
-
-
12. An apparatus for moving a message transform from a source broker to a target broker in a distributed messaging system, wherein the distributed messaging system includes a network of broker machines, wherein a plurality of message transforms are deployed in the network of broker machines, and wherein the plurality of message transforms take as input one or more sequences of messages and produce as output one or more sequences of messages, the apparatus comprising:
-
a bus; a communications unit connected to the bus; a memory connected to the bus, wherein the memory includes a set of computer usable program code; and a processor unit connected to the bus, wherein the processor unit executes the set of computer usable program code to; start on the target broker a copy of a given message transform from the source broker; duplicate all inputs into the given message transform on the source broker to additionally go to the copy of the given message transform on the target broker; and duplicate all outputs of the given message transform on the source broker to also go from the copy of the given message transform on the target broker to appropriate children nodes within the distributed messaging system. - View Dependent Claims (13, 14, 15, 16)
-
-
17. An apparatus for continuous deployment of message transforms in a distributed messaging system, wherein the distributed messaging system includes a network of broker machines, the apparatus comprising:
-
a bus; a communications unit connected to the bus; a memory connected to the bus, wherein the memory includes a set of computer usable program code; and a processor unit connected to the bus, wherein the processor unit executes the set of computer usable program code to; deploy a plurality of message transforms in the network of broker machines, wherein the plurality of message transforms take as input one or more sequences of messages and produce as output one or more sequences of messages; receive performance information from the network of broker machines; and redeploy at least a subset of the plurality of message transforms in the network of broker machines based on the performance information, wherein redeploying at least the subset of the plurality of message transforms includes moving a given message transform from a source broker to a target broker in the distributed messaging system. - View Dependent Claims (18, 19)
-
-
20. A computer program product, in a computer readable storage device, for moving a message transform from a source broker to a target broker in a distributed messaging system, wherein the distributed messaging system includes a network of broker machines, wherein a plurality of message transforms are deployed in the network of broker machines, and wherein the plurality of message transforms take as input one or more sequences of messages and produce as output one or more sequences of messages, the computer program product comprising:
-
instructions for starting on the target broker a copy of a given message transform from the source broker; instructions for duplicating all inputs into the given message transform on the source broker to additionally go to the copy of the given message transform on the target broker; and instructions for duplicating all outputs of the given message transform on the source broker to also go from the copy of the given message transform on the target broker to appropriate children nodes within the distributed messaging system.
-
-
21. A computer program product, in a computer readable storage device, for continuous deployment of message transforms in a distributed messaging system, wherein the distributed messaging system includes a network of broker machines, the computer program product comprising:
-
instructions for deploying a plurality of message transforms in the network of broker machines, wherein the plurality of message transforms take as input one or more sequences of messages and produce as output one or more sequences of messages; instructions for receiving performance information from the network of broker machines; and instructions for redeploying at least a subset of the plurality of message transforms in the network of broker machines based on the performance information.
-
-
22. A method for continuous deployment of message transforms in a distributed messaging system, wherein the distributed messaging system includes a network of broker machines, the method comprising:
-
copying a message transform from a source broker to a target broker, the target and source broker being included in the network of broker machines; duplicating input connections of the source broker on the target broker, the duplicating comprising; replicating the input connections of the source broker onto the target broker; initializing the replicated input connections to receive messages also destined for the source broker; copying a state of the message transform on the source broker to the target broker, the state comprising all information required to evaluate the message transform on the target broker as if the message transform resided on the source broker, the copying comprising; loading the state of the message transform and an associated sequence marker onto the target broker, the sequence marker indicating messages to be used in computing a next correct version of the message transform on the target broker; and duplicating output connections of the source broker onto the target broker, the duplicating comprising; connecting an output of the target broker to an output child of the source broker; initializing the target broker to route messages output from the target broker to the output child of the source broker; performing a self-diagnostic that checks an operation of the target broker to generate a diagnostic report; and responsive to the diagnostic report indicating a proper operation of the target broker, terminating the message transform of the source broker.
-
Specification