Triggering window conditions by streaming features of an operator graph

• US 9,531,781 B2
• Filed: 12/10/2012
• Issued: 12/27/2016
• Est. Priority Date: 01/11/2012
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Branson, Michael J., Santosuosso, John M., Schulz, Brandon W.
• Primary Examiner(s): Eskandarnia, Arvin
• Assistant Examiner(s): DESAI, MARGISHI V
• Application Number:
  US13/709,339
• Publication Number:
  US 20130179591A1
• Time in Patent Office: 1,478 Days
• Field of Search: 709/231
• US Class Current: 1/1
• CPC Class Codes: G01C 21/3415 Dynamic re-routing, e.g. recalculating the route when the user deviates from calculated route or after detecting real-time traffic data or accidents
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04W 28/10 Flow control between communication endpoints

Abstract

In a stream computing application, data may be transmitted between operators using tuples. However, the receiving operator may not evaluate these tuples as they arrive but instead wait to evaluate a group of tuples—i.e., a window. A window is typically triggered when a buffer associated with the receiving operator reaches a maximum window size or when a predetermined time period has expired. Additionally, a window may be triggered by a monitoring a tuple rate—i.e., the rate at which the operator receives the tuples. If the tuple rate exceeds or falls below a threshold, a window may be triggered. Further, the number of exceptions, or the rate at which an operator throws exceptions, may be monitored. If either of these parameters satisfies a threshold, a window may be triggered, thereby instructing an operator to evaluate the tuples contained within the window.

Claims

1. A method for processing data, comprising:

receiving streaming data tuples to be processed by a plurality of operators, the operators processing at least a portion of the received data tuples by operation of one or more computer processors;

establishing an operator graph of the plurality of operators, the operator graph defining at least one execution path in which a first operator of the plurality of operators is configured to receive data tuples from at least one upstream operator and transmit data tuples to at least one downstream operator;

measuring a current rate at which data tuples pass from the upstream operator to the first operator during a period of time; and

triggering a data window in at least one operator of the plurality of operators based on the current rate, the window comprising a set of data tuples to be processed by the at least one operator.

2. The method of claim 1, wherein, after the data window is triggered, the at least one operator performs an operation using the set data tuples in the data window.

3. The method of claim 1, wherein the current rate is a ratio of a number of data tuples transmitted during a predefined period of time, and wherein triggering the data window is further based on whether the current rate exceeds or falls below a predefined threshold.

4. The method of claim 1, further comprising:

determining a historical rate at which data tuples pass from the upstream operator to the first operator;

comparing the historical rate to the current rate to determine whether a threshold is met; and

if the threshold is met, triggering the data window.

5. The method of claim 1, wherein at least two upstream operators pass data tuples to the first operator, further comprising:

measuring a plurality of rates at which the at least two upstream operators pass data to the first operator, wherein the current rate is one of the plurality of rates; and

triggering the data window in the at least one operator based on the plurality of rates.

6. The method of claim 5, wherein triggering the data window is further based on whether the plurality of rates exceeds or falls below one or more thresholds, the thresholds are based on one of:

a predefined threshold, and

a difference between at least one historical rate and the plurality of rates.

7. The method of claim 1, wherein the at least one operator on which the data window is triggered is the first operator.