Processing control in a streaming application

• US 9,087,082 B2
• Filed: 03/07/2013
• Issued: 07/21/2015
• Est. Priority Date: 03/07/2013
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Branson, Michael J., Santosuosso, John M.
• Primary Examiner(s): Ehichioya, Fred I
• Assistant Examiner(s): MACKES, KRIS E
• Application Number:
  US13/789,638
• Publication Number:
  US 20140258290A1
• Time in Patent Office: 866 Days
• Field of Search: 707/737
• US Class Current: 1/1
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries

Abstract

A method, system, and computer program product for processing a stream of tuples are disclosed. The method, system, and computer program product may include receiving a stream of tuples to be processed by a plurality of processing elements. Each tuple may have an associated processing history. The stream of tuples may be segmented into a plurality of partitions, each representing a subset of the stream of tuples. The method, system, and computer program product may include estimating the contribution each partition will have on a particular processing result and processing a partition if it substantially contributes to the particular processing result.

Claims

1. A method for processing a stream of tuples, comprising:

receiving the stream of tuples for processing by a plurality of processing elements to produce a processing result;

segmenting the stream of tuples into a plurality of partitions, each of the plurality of partitions including a subset of the stream of tuples;

maintaining a processing history for each of the plurality of partitions, the processing history including input values for tuples in the partition and further including output values for the tuples in the partition;

determining that a performance metric falls outside a performance threshold, the performance metric associated with the processing by the plurality of processing elements to produce the processing result;

determining, based on the processing history of a first partition and further based on the determining that the performance metric falls outside the performance threshold, that tuples in the first partition substantially contribute to the processing result;

determining, based on the processing history of a second partition and further based on the determining that the performance metric falls outside the performance threshold, that tuples in the second partition do not substantially contribute to the processing result;

continuing the processing of the tuples in the first partition by the plurality of processing elements; and

discontinuing the processing of the tuples in the second partition by the plurality of processing elements.

2. The method of claim 1, wherein the determining that the tuples in the first partition substantially contribute to the processing result and the determining that the tuples in the second partition do not substantially contribute to the processing result is enabled within a window.

3. The method of claim 1, wherein the determining that the tuples in the second partition do not substantially contribute to the processing result is associated with a confidence level, the method further comprising:

determining, after the discontinuing the processing of the tuples in the second partition, that the confidence level falls outside a modification threshold; and

resuming the processing of the tuples in the second partition based on the determining that the confidence level falls outside the modification threshold.

4. The method of claim 1, further comprising resuming the processing of the tuples in the second partition at a time interval.

5. The method of claim 1, wherein the determining that the tuples in the first partition substantially contribute to the processing result comprises:

determining that a confidence level falls outside a modification threshold.