Processing control in a streaming application

• US 9,087,083 B2
• Filed: 03/12/2013
• Issued: 07/21/2015
• Est. Priority Date: 03/07/2013
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Branson, Michael J., Santosuosso, John M.
• Primary Examiner(s): Ehichioya, Fred I
• Assistant Examiner(s): MACKES, KRIS E
• Application Number:
  US13/795,625
• Publication Number:
  US 20140258291A1
• Time in Patent Office: 861 Days
• Field of Search: 707/737
• US Class Current: 1/1
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries

Abstract

A method, system, and computer program product for processing a stream of tuples are disclosed. The method, system, and computer program product may include receiving a stream of tuples to be processed by a plurality of processing elements. Each tuple may have an associated processing history. The stream of tuples may be segmented into a plurality of partitions, each representing a subset of the stream of tuples. The method, system, and computer program product may include estimating the contribution each partition will have on a particular processing result and processing a partition if it substantially contributes to the particular processing result.

Claims

1. A computer program product for processing a stream of tuples, the computer program product comprising a non-transitory computer readable storage medium having program code embodied therewith, the program code comprising computer readable program code configured to:

receive the stream of tuples for processing by a plurality of processing elements to produce a processing result;

segment the stream of tuples into a plurality of partitions, each of the plurality of partitions including a subset of the stream of tuples;

maintain a processing history for each of the plurality of partitions, the processing history including input values for tuples in the partition and further including output values for the tuples in the partition;

determine that a performance metric falls outside a performance threshold, the performance metric associated with the processing by the plurality of processing elements to produce the processing result;

determine, based on the processing history of a first partition and further based on the determining that the performance metric falls outside the performance threshold, that tuples in the first partition substantially contribute to the processing result;

determine, based on the processing history of a second partition and further based on the determining that the performance metric falls outside the performance threshold, that tuples in the second partition do not substantially contribute to the processing result;

continue the processing of the tuples in the first partition by the plurality of processing elements; and

discontinue the processing of the tuples in the second partition by the plurality of processing elements.

2. The computer program product of claim 1, wherein the determining that the tuples in the first partition substantially contribute to the processing result and the determining that the tuples in the second partition do not substantially contribute to the processing result is enabled within a window.

3. The computer program product of claim 1, wherein the determining that the tuples in the second partition do not substantially contribute to the processing result is associated with a confidence level, the computer readable program code further configured to:

determine, after the discontinuing the processing of the tuples in the second partition, that the confidence level falls outside a modification threshold; and

resume the processing of the tuples in the second partition based on the determining that the confidence level falls outside the modification threshold.

4. The computer program product of claim 1, wherein the determining that the tuples in the first partition substantially contribute to the processing result comprises:

determining that a confidence level falls outside a modification threshold.

5. The computer program product of claim 1, the computer readable program code further configured to resume the processing of the tuples in the second partition at a time interval.

6. A computing system for processing a stream of tuples, the computing system comprising:

a memory; and

a processor in communication with the memory, wherein the computing system is configured to;

receive the stream of tuples for processing by a plurality of processing elements to produce a processing result;

segment the stream of tuples into a plurality of partitions, each of the plurality of partitions including a subset of the stream of tuples;

maintain a processing history for each of the plurality of partitions, the processing history including input values for tuples in the partition and further including output values for the tuples in the partition;

determine that a performance metric falls outside a performance threshold, the performance metric associated with the processing by the plurality of processing elements to produce the processing result;

determine, based on the processing history of a first partition and further based on the determining that the performance metric falls outside the performance threshold, that tuples in the first partition substantially contribute to the processing result;

determine, based on the processing history of a second partition and further based on the determining that the performance metric falls outside the performance threshold, that tuples in the second partition do not substantially contribute to the processing result;

continue the processing of the tuples in the first partition by the plurality of processing elements; and

discontinue the processing of the tuples in the second partition by the plurality of processing elements.

7. The computing system of claim 6, wherein the determining that the tuples in the second partition do not substantially contribute to the processing result is associated with a confidence level, the computing system further configured to:

determine, after the discontinuing the processing of the tuples in the second partition, that the confidence level falls outside a modification threshold; and

resume the processing of the tuples in the second partition based on the determining that the confidence level falls outside the modification threshold.

8. The computing system of claim 6, wherein the determining that the tuples in the first partition substantially contribute to the processing result comprises:

determining that a confidence level falls outside a modification threshold.

9. The computing system of claim 6, wherein the determining that the tuples in the first partition substantially contribute to the processing result and the determining that the tuples in the second partition do not substantially contribute to the processing result is enabled within a window.

10. The computing system of claim 6, the computing system further configured to resume the processing of the tuples in the second partition at a time interval.