DYNAMIC STABILIZATION FOR A STREAM PROCESSING SYSTEM

• US 20130031282A1
• Filed: 07/23/2012
• Published: 01/31/2013
• Est. Priority Date: 06/13/2006
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: AMINI, Lisa D., SEHGAL, Anshul, SILBER, Jeremy I., VERSCHEURE, Olivier
• Granted Patent: US 8,601,178 B2
• Time in Patent Office: Days
• US Class Current: 710/29
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 47/30 in combination with information about buffer occupancy at either end or at transit nodes
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

Disclosed are a method and a computer program storage product for dynamically stabilizing a stream processing system. The method includes receiving at least one computing resource allocation target. A plurality of downstream processing elements and an upstream processing element are associated with at least one input buffer. Each of the downstream processing elements consumes data packets produced by the upstream processing element received on an output stream associated with the upstream processing element. A fastest input rate among each downstream processing element in the plurality of downstream processing elements is identified. An output rate of the upstream processing element is set to the fastest input rate that has been determined for the plurality of downstream processing elements.

Claims

1. A method on a processing node comprising at least a current processing element for dynamically stabilizing a stream processing system, the method comprising:

receiving at least one computing resource allocation target;

associating a plurality of downstream processing elements and an upstream processing element with at least one input buffer, wherein each of the downstream processing elements consumes data packets produced by the upstream processing element received on an output stream associated with the upstream processing element;

identifying a fastest input rate among each downstream processing element in the plurality of downstream processing elements; and

setting an output rate of the upstream processing element to the fastest input rate that has been determined for the plurality of downstream processing elements.

2. The method of claim 1, further comprising:

allocating the computing resource to the current processing element based on an input data flow rate of the upstream processing element varying.

3. The method of claim 2, wherein the current processing element is allocated a first set of computing resources if:

an input buffer occupancy of the current processing element is greater than zero and below a predefined threshold; and

the upstream-processing element has available tokens to expend on the first set of computing resources.

4. The method of claim 2, wherein allocating the computing resource further comprises:

allocating at least one token to the upstream processing element in proportion to a long-term CPU goal of the upstream processing element, wherein the upstream processing element expends the token for the computing resource thereby allowing the upstream processing element to process at least one data packet in accordance with a maximum input rate of the downstream processing element.

5. A computer program storage product for dynamically stabilizing a stream processing system, the computer program storage product comprising instructions configured to perform a method comprising:

receiving at least one computing resource allocation target;

associating a plurality of downstream processing elements and an upstream processing element with at least one input buffer, wherein each of the downstream processing elements consumes data packets produced by the upstream processing element received on an output stream associated with the upstream processing element;

identifying a fastest input rate among each downstream processing element in the plurality of downstream processing elements; and

setting an output rate of the upstream processing element to the fastest input rate that has been determined for the plurality of downstream processing elements.

6. The computer program storage product of claim 5, further comprising:

allocating the computing resource to a current processing element based on an input data flow rate of the upstream processing element varying.

7. The computer program storage product of claim 6, wherein the current processing element is allocated a first set of computing resources if:

an input buffer occupancy of the current processing element is greater than zero and below a predefined threshold; and

the upstream-processing element has available tokens to expend on the first set of computing resources.

8. The computer program storage product of claim 6, wherein allocating the computing resource further comprises:

allocating at least one token to the upstream processing element in proportion to a long-term CPU goal of the upstream processing element, wherein the upstream processing element expends the token for the computing resource thereby allowing the upstream processing element to process at least one data packet in accordance with a maximum input rate of the downstream processing element.