Management of stream operators with dynamic connections

• US 9,183,107 B2
• Filed: 03/05/2013
• Issued: 11/10/2015
• Est. Priority Date: 12/12/2012
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Branson, Michael J., Cobb, Bradford L., Santosuosso, John M.
• Primary Examiner(s): Whipple, Brian P
• Application Number:
  US13/784,995
• Publication Number:
  US 20140164601A1
• Time in Patent Office: 980 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3041 where the computing system component is an input/output interface interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units G06F13/00
• CPC Class Codes: G06F 11/3048 where the topology of the computing system or computing system component explicitly influences the monitoring activity, e.g. serial, hierarchical systems
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 9/4494 data driven
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio

Abstract

One embodiment is directed to a method for processing a stream of tuples in a stream-based application. A stream operator may receive a stream of tuples. A stream manager may determine whether a dynamic connection exists at a first stream operator. The dynamic connection may connect the first stream operator to a second stream operator. The stream manager may poll the first stream operator and the second stream operator for a presence of the dynamic connection. The stream manager may modify processing of one or more upstream stream operators in response to a change in use of the dynamic connection.

Claims

1. A system for processing a stream of tuples, comprising:

a plurality of processing elements to receive a stream of tuples, the processing elements operating on one or more computer processors, each processing element having one or more stream operators;

a monitor, wherein the monitor determines a presence of a dynamic connection between a first stream operator and a second stream operator, wherein the dynamic connection allows introduction of one or more additional stream operators without modifying an operator graph of the one or more stream operators, wherein the first stream operator is from a first application instance and the second stream operator is from a second application instance; and

a stream manager, wherein the stream manager is configured to;

determine, in response to the monitor determining an absence of the dynamic connection between the first stream operator and the second stream operator, whether an upstream stream operator from the dynamic connection transmits to another active stream operator;

deactivate the upstream stream operator in response to the absence of the dynamic connection.

2. The system of claim 1, wherein the stream manager is configured to deactivate the upstream stream operator by:

determining whether an attribute from the stream of tuples is processed by another active stream operator;

stopping the processing for the attribute in response to the attribute not being processed by another active stream operator.

3. The system of claim 2, wherein the stream manager is configured to stop the processing by:

adding a runtime flag to another active stream operator; and

discarding processing of the attribute in response to the presence of the runtime flag.

4. The system of claim 1, wherein the stream manager is configured to:

connect the upstream stream operator to the first stream operator;

determine whether the first stream operator transmits a tuple from the stream of tuples to the second stream operator; and

activate the upstream stream operator, the first stream operator, and the second stream operator, in response to the first stream operator transmitting the tuple.

5. The system of claim 1, wherein the stream manager is configured to determine whether the upstream stream operator from the dynamic connection transmits to another active stream operator by receiving a push connection status from the another active stream operator via continuous polling of dynamic connections by the another active stream operator.

6. A computer program product for processing a stream of tuples, the computer program product comprising a non-transitory computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

determine a presence of a dynamic connection between a first stream operator and a second stream operator, wherein the dynamic connection allows introduction of one or more additional stream operators without modifying an operator graph of one or more stream operators, wherein the first stream operator is from a first application instance and the second stream operator is from a second application instance;

determine, in response to the monitor determining an absence of the dynamic connection between the first stream operator and the second stream operator, whether an upstream stream operator from the dynamic connection transmits to another active stream operator; and

deactivate the upstream stream operator in response to the absence of the dynamic connection.

7. The computer program product of claim 6, wherein the computer readable program causes the computing device to deactivate the upstream stream operator by:

determining whether an attribute from the stream of tuples is processed by another active stream operator;

stopping processing for the attribute in response to the attribute not being processed by another active stream operator.

8. The computer program product of claim 7, wherein the computer readable program causes the computing device to stop the processing by:

adding a runtime flag to another active stream operator; and

discard processing of the attribute in response to the presence of the runtime flag.

9. The computer program product of claim 6, wherein the computer readable program causes the computing device to:

connect the upstream stream operator to the first stream operator;

determine whether the first stream operator transmits a tuple from the stream of tuples to the second stream operator; and

activate the upstream stream operator, the first stream operator, and the second stream operator, in response to the first stream operator transmitting the tuple.