TRIGGERING WINDOW CONDITIONS BY STREAMING FEATURES OF AN OPERATOR GRAPH

• US 20130179585A1
• Filed: 01/11/2012
• Published: 07/11/2013
• Est. Priority Date: 01/11/2012
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Branson, Michael J., Santosuosso, John M., Schulz, Brandon W.
• Granted Patent: US 9,438,656 B2
• Time in Patent Office: Days
• US Class Current: 709/231
• CPC Class Codes: G01C 21/3415 Dynamic re-routing, e.g. recalculating the route when the user deviates from calculated route or after detecting real-time traffic data or accidents
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04W 28/10 Flow control between communication endpoints

Abstract

In a stream computing application, data may be transmitted between operators using tuples. However, the receiving operator may not evaluate these tuples as they arrive but instead wait to evaluate a group of tuples—i.e., a window. A window is typically triggered when a buffer associated with the receiving operator reaches a maximum window size or when a predetermined time period has expired. Additionally, a window may be triggered by a monitoring a tuple rate—i.e., the rate at which the operator receives the tuples. If the tuple rate exceeds or falls below a threshold, a window may be triggered. Further, the number of exceptions, or the rate at which an operator throws exceptions, may be monitored. If either of these parameters satisfies a threshold, a window may be triggered, thereby instructing an operator to evaluate the tuples contained within the window.

Claims

1-7. -7. (canceled)

8. A computer program product for processing data, the computer program product comprising:

a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code comprising computer-readable program code configured to;

receive streaming data tuples to be processed by a plurality of operators, the operators processing at least a portion of the received data tuples;

establish an operator graph of the plurality of operators, the operator graph defining at least one execution path in which a first operator of the plurality of operators is configured to receive data tuples from at least one upstream operator and transmit data tuples to at least one downstream operator;

measure a current rate at which data tuples pass from the upstream operator to the first operator during a period of time; and

trigger a data window in at least one operator of the plurality of operators based on the current rate, the window comprising a plurality of data tuples.

9. The computer program product of claim 8, wherein, after the data window is triggered, the at least one operator performs an operation using at least one of the plurality of data tuples in the data window.

10. The computer program product of claim 8, wherein the current rate is a ratio of a number of data tuples transmitted during a predefined period of time, and wherein triggering the data window is further based on whether the current rate exceeds or falls below a predefined threshold.

11. The computer program product of claim 8, further comprising computer-readable program code configured to:

determine a historical rate at which data tuples pass from the upstream operator to the first operator;

compare the historical rate to the current rate to determine whether a threshold is met; and

if the threshold is met, trigger the data window.

12. The computer program product of claim 8, wherein at least two upstream operators pass data tuples to the first operator, further comprising computer-readable program code configured to:

measure a plurality of rates at which the at least two upstream operators pass data to the first operator, wherein the current rate is one of the plurality of rates; and

trigger the data window in the at least one operator based on the plurality of rates.

13. The computer program product of claim 12, wherein triggering the data window is further based on whether the plurality of rates exceeds or falls below one or more thresholds, the thresholds are based on one of:

a predefined threshold, and

a difference between at least one historical rate and the plurality of rates.

14. The computer program product of claim 8, wherein the at least one operator on which the data window is triggered is the first operator.

15. A system for processing data, comprising:

at least two compute nodes, each compute node comprising at least one computer processor, wherein the at least two compute nodes are configured to host at least one of a plurality of operators that process streaming data tuples, the operators processing at least a portion of the received data tuples,

wherein the plurality of operators establish an operator graph, the operator graph defining at least one execution path in which a first operator of the plurality of operators is configured to receive data tuples from at least one upstream operator and transmit data tuples to at least one downstream operator; and

a window activator that is configured to measure a current rate at which data tuples pass from the upstream operator to the first operator during a period of time and trigger a data window in at least one operator of the plurality of operators based on the current rate, the window comprising a plurality of data tuples.

16. The system of claim 15, wherein, after the data window is triggered, the at least one operator performs an operation using at least one of the plurality of data tuples in the data window.

17. The system of claim 15, wherein the current rate is a ratio of a number of data tuples transmitted during a predefined period of time, and wherein triggering the data window is further based on whether the current rate exceeds or falls below a predefined threshold.

18. The system of claim 15, wherein the window activator is configured to:

determine a historical rate at which data tuples pass from the upstream operator to the first operator;

compare the historical rate to the current rate to determine whether a threshold is met; and

if the threshold is met, trigger the data window.

19. The system of claim 15, wherein at least two upstream operators pass data tuples to the first operator, wherein the window activator is configured to:

measure a plurality of rates at which the at least two upstream operators pass data to the first operator, wherein the current rate is one of the plurality of rates; and

trigger the data window in the at least one operator based on the plurality of rates.

20. The system of claim 19, wherein triggering the data window is further based on whether the plurality of rates exceeds or falls below one or more thresholds, the thresholds are based on one of:

a predefined threshold, and

a difference between at least one historical rate and the plurality of rates.