Triggering window conditions using exception handling

• US 9,430,117 B2
• Filed: 01/11/2012
• Issued: 08/30/2016
• Est. Priority Date: 01/11/2012
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Branson, Michael J., Santosuosso, John M., Schulz, Brandon W.
• Primary Examiner(s): Zele, Krista
• Assistant Examiner(s): DESAI, MARGISHI V
• Application Number:
  US13/347,974
• Publication Number:
  US 20130179586A1
• Time in Patent Office: 1,693 Days
• Field of Search: 717/156, 709/231
• US Class Current: 1/1
• CPC Class Codes: G06F 15/163 Interprocessor communication
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 5/06 for changing the speed of data flow, i.e. speed regularising or timing, e.g. delay lines, FIFO buffers; over- or underrun control therefor G06F7/78 takes precedence
• CPC Class Codes: G06F 9/44 Arrangements for executing specific programs
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream

Abstract

In a stream computing application, data may be transmitted between operators using tuples. However, the receiving operator may not evaluate these tuples as they arrive but instead wait to evaluate a group of tuples—i.e., a window. A window is typically triggered when a buffer associated with the receiving operator reaches a maximum window size or when a predetermined time period has expired. Additionally, a window may be triggered by a monitoring a tuple rate—i.e., the rate at which the operator receives the tuples. If the tuple rate exceeds or falls below a threshold, a window may be triggered. Further, the number of exceptions, or the rate at which an operator throws exceptions, may be monitored. If either of these parameters satisfies a threshold, a window may be triggered, thereby instructing an operator to evaluate the tuples contained within the window.

Claims

1. A computer-implemented method for processing data, comprising:

receiving streaming data tuples to be processed by a plurality of operators, the operators processing at least a portion of the received data tuples by operation of one or more computer processors;

establishing an operator graph of the plurality of operators, the operator graph defining at least one execution path in which a first operator of the plurality of operators is configured to receive data tuples from at least one upstream operator and transmit data tuples to at least one downstream operator;

determining a current number of exceptions occurring while the upstream operator performs an operation based on at least one of the streaming data tuples; and

triggering a data window in at least one operator of the plurality of operators different from the upstream operator when the current number of exceptions satisfies a threshold, the data window comprising a plurality of data tuples, and wherein triggering the data window causes the at least one operator to evaluate the plurality of data tuples in the data window.

2. The method of claim 1 wherein the threshold is a predefined threshold.

3. The method of claim 1, further comprising:

determining a historical number of exceptions that are thrown from the upstream operator;

comparing the historical number of exceptions to the current number of exceptions to determine whether the threshold is met; and

if the threshold is met, triggering the data window.

4. The method of claim 1, wherein at least two upstream operators throw exceptions, further comprising:

measuring the current number of exceptions for each of the at least two upstream operators; and

triggering the data window in the at least one operator based on the current number of exceptions for each of the at least two upstream operators.

5. The method of claim 4, wherein triggering the data window is further based on whether the current number of exceptions for each of the at least two upstream operators exceed or fall below one or more thresholds, the one or more thresholds are based on one of:

a predefined threshold, and

a difference between at least one historical current number of exceptions and the current number of exceptions for each of the at least two upstream operators.

6. The method of claim 1, wherein triggering the data window is based on an exception rate, the exception rate is a ratio of number of exceptions thrown by the upstream operator during a predefined period of time.

7. A computer program product for processing data, the computer program product comprising:

a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code comprising computer-readable program code configured to;

receive streaming data tuples to be processed by a plurality of operators, the operators processing at least a portion of the received data tuples;

establish an operator graph of the plurality of operators, the operator graph defining at least one execution path in which a first operator of the plurality of operators is configured to receive data tuples from at least one upstream operator and transmit data tuples to at least one downstream operator;

determine a current number of exceptions occurring while the upstream operator performs an operation based on at least one of the streaming data tuples; and

trigger a data window in at least one operator of the plurality of operators different from the upstream operator when the current number of exceptions satisfies a threshold, the data window comprising a plurality of data tuples, and wherein triggering the data window causes the at least one operator to evaluate the plurality of data tuples in the data window.

8. The computer program product of claim 7, wherein the threshold is a predefined threshold.

9. The computer program product of claim 7, further comprising computer-readable program code configured to:

determine a historical number of exceptions that are thrown from the upstream operator;

compare the historical number of exceptions to the current number of exceptions to determine whether the threshold is met; and

if the threshold is met, trigger the data window.

10. The computer program product of claim 7, wherein at least two upstream operators throw exceptions, further comprising computer-readable program code configured to:

measure the current number of exceptions for each of the at least two upstream operators; and

trigger the data window in the at least one operator based on the current number of exceptions for each of the at least two upstream operators.

11. The computer program product of claim 10, wherein triggering the data window is further based on whether the current number of exceptions for each of the at least two upstream operators exceed or fall below one or more thresholds, the one or more thresholds are based on one of:

a predefined threshold, and

a difference between at least one historical current number of exceptions and the current number of exceptions for each of the at least two upstream operators.

12. The computer program product of claim 7, wherein triggering the data window is based on an exception rate, the exception rate is a ratio of number of exceptions thrown by the upstream operator during a predefined period of time.

13. A system for processing data, comprising:

at least two compute nodes, each compute node comprising at least one computer processor, wherein the at least two compute nodes are configured to host at least one of a plurality of operators that process streaming data tuples, the operators processing at least a portion of the received data tuples,

wherein the plurality of operators establish an operator graph, the operator graph defining at least one execution path in which a first operator of the plurality of operators is configured to receive data tuples from at least one upstream operator and transmit data tuples to at least one downstream operator; and

a window activator that is configured to determine a current number of exceptions occurring while the upstream operator performs an operation based on at least one of the streaming data tuples and trigger a data window in at least one operator of the plurality of operators different from the upstream operator when the current number of exceptions satisfies a threshold, wherein the data window comprising a plurality of data tuples, and wherein triggering the data window causes the at least one operator to evaluate the plurality of data tuples in the data window.

14. The system of claim 13, wherein the threshold is a predefined threshold.

15. The system of claim 13, wherein the window activator is configured to:

determine a historical number of exceptions that are thrown from the upstream operator;

compare the historical number of exceptions to the current number of exceptions to determine whether the threshold is met; and

if the threshold is met, trigger the data window.

16. The system of claim 13, wherein at least two upstream operators throw exceptions, wherein the window activator is configured to:

measure the current number of exceptions for each of the at least two upstream operators; and

trigger the data window in the at least one operator based on the current number of exceptions for each of the at least two upstream operators.

17. The system of claim 16, wherein triggering the data window is further based on whether the current number of exceptions for each of the at least two upstream operators exceed or fall below one or more thresholds, the one or more thresholds are based on one of:

a predefined threshold, and

a difference between at least one historical current number of exceptions and the current number of exceptions for each of the at least two upstream operators.