PROCESSING CONTROL IN A STREAMING APPLICATION

• US 20140258291A1
• Filed: 03/12/2013
• Published: 09/11/2014
• Est. Priority Date: 03/07/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Branson, Michael J., Santosuosso, John M.
• Granted Patent: US 9,087,083 B2
• Time in Patent Office: Days
• US Class Current: 707/736
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries

Abstract

A method, system, and computer program product for processing a stream of tuples are disclosed. The method, system, and computer program product may include receiving a stream of tuples to be processed by a plurality of processing elements. Each tuple may have an associated processing history. The stream of tuples may be segmented into a plurality of partitions, each representing a subset of the stream of tuples. The method, system, and computer program product may include estimating the contribution each partition will have on a particular processing result and processing a partition if it substantially contributes to the particular processing result.

Claims

1. A computer program product for processing a stream of tuples, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code comprising computer readable program code configured to:

receive a stream of tuples, each tuple having an associated processing history, to be processed by a plurality of processing elements;

segment the stream of tuples into a plurality of partitions, wherein a partition represents a subset of the stream of tuples;

estimate a contribution each partition will have on a particular processing result based on the partition and the associated processing history; and

process a partition if it substantially contributes to the particular processing result.

2. The computer program product of claim 1, wherein the estimating is enabled when a performance metric falls outside a performance threshold.

3. The computer program product of claim 1, wherein the estimating is enabled within a window.

4. The computer program product of claim 1, further comprising resuming processing of an omitted partition when a confidence level falls outside a modification threshold.

5. The computer program product of claim 1, further comprising:

generating a combined partition processing history from the associated processing history of the tuples in the partition.

6. The computer program product of claim 1, wherein a partition substantially contributes to the processing result if a confidence level falls outside a modification threshold.

7. The computer program product of claim 1, wherein a partition substantially contributes to a processing result when the processing history indicates a tuple attribute value for a first partition is outside a deviation threshold from a tuple attribute value for a second partition.

8. A system for processing a stream of tuples, comprising:

a plurality of processing elements to receive a stream of tuples, each tuple having an associated processing history, the processing elements operating on one or more computer processors, each processing element having one or more stream operators; and

a first stream operator, wherein the first stream operator is configured to;

receive a stream of tuples, each tuple having an associated processing history, to be processed by a plurality of processing elements;

segment the stream of tuples into a plurality of partitions, wherein a partition represents a subset of the stream of tuples;

estimate a contribution each partition will have on a particular processing result based on the partition and the associated processing history; and

process a partition if it substantially contributes to the particular processing result.

9. The system of claim 8, further comprising resuming processing of an omitted partition when a confidence level falls outside a modification threshold.

10. The system of claim 8, the first stream operator further comprising:

generating a combined partition processing history from the associated processing history of the tuples in the partition.

11. The system of claim 8, wherein a partition substantially contributes to the processing result if a confidence level falls outside a modification threshold.

12. The system of claim 8, wherein a partition substantially contributes to a processing result when the processing history indicates a tuple attribute value for a first partition is outside a deviation threshold from a tuple attribute value for a second partition.