Dynamic stabilization for a stream processing system
First Claim
1. A method on a processing node comprising at least a current processing element for dynamically stabilizing a stream processing system, the method comprising:
- receiving at least one computing resource allocation target;
associating a plurality of downstream processing elements and an upstream processing element with at least one input buffer, wherein each of the downstream processing elements consumes data packets produced by the upstream processing element received on an output stream associated with the upstream processing element;
monitoring, for each downstream processing element in the plurality of downstream processing elements, an input rate of the downstream processing element;
identifying, based on the monitoring, a fastest input rate among each of the input rates monitored for the plurality of downstream processing elements; and
setting an output rate of the upstream processing element to the identified fastest input rate for the plurality of downstream processing elements.
0 Assignments
0 Petitions
Accused Products
Abstract
Disclosed are a method and a computer program storage product for dynamically stabilizing a stream processing system. The method includes receiving at least one computing resource allocation target. A plurality of downstream processing elements and an upstream processing element are associated with at least one input buffer. Each of the downstream processing elements consumes data packets produced by the upstream processing element received on an output stream associated with the upstream processing element. A fastest input rate among each downstream processing element in the plurality of downstream processing elements is identified. An output rate of the upstream processing element is set to the fastest input rate that has been determined for the plurality of downstream processing elements.
45 Citations
8 Claims
-
1. A method on a processing node comprising at least a current processing element for dynamically stabilizing a stream processing system, the method comprising:
-
receiving at least one computing resource allocation target; associating a plurality of downstream processing elements and an upstream processing element with at least one input buffer, wherein each of the downstream processing elements consumes data packets produced by the upstream processing element received on an output stream associated with the upstream processing element; monitoring, for each downstream processing element in the plurality of downstream processing elements, an input rate of the downstream processing element; identifying, based on the monitoring, a fastest input rate among each of the input rates monitored for the plurality of downstream processing elements; and setting an output rate of the upstream processing element to the identified fastest input rate for the plurality of downstream processing elements. - View Dependent Claims (2, 3, 4)
-
-
5. A computer program storage product for dynamically stabilizing a stream processing system, the computer program storage product comprising instructions configured to perform a method comprising:
-
receiving at least one computing resource allocation target; associating a plurality of downstream processing elements and an upstream processing element with at least one input buffer, wherein each of the downstream processing elements consumes data packets produced by the upstream processing element received on an output stream associated with the upstream processing element; monitoring, for each downstream processing element in the plurality of downstream processing elements, an input rate of the downstream processing element; identifying, based on the monitoring, a fastest input rate among each of the input rates monitored for the plurality of downstream processing elements; setting an output rate of the upstream processing element to the identified fastest input rate for the plurality of downstream processing elements; determining that the input buffer associated with the downstream processing element having the identified fastest input rate is able to accept at least one data packet; and forwarding, based on the determining, at least one data packet to two or more of the plurality of downstream processing elements according to the identified fasted input rate. - View Dependent Claims (6, 7, 8)
-
Specification