UNFUSING A FAILING PART OF AN OPERATOR GRAPH

• US 20130166948A1
• Filed: 12/05/2012
• Published: 06/27/2013
• Est. Priority Date: 12/22/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: SANTOSUOSSO, John M., BRANSON, Michael J., SCHULTZ, Brandon W.
• Granted Patent: US 8,990,628 B2
• Time in Patent Office: Days
• US Class Current: 714/15
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0754 by exceeding limits
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/2025 using centralised failover control functionality
• CPC Class Codes: G06F 11/2035 without idle spare hardware
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029

Abstract

Techniques for managing a fused processing element are described. Embodiments receive streaming data to be processed by a plurality of processing elements. Additionally, an operator graph of the plurality of processing elements is established. The operator graph defines at least one execution path and wherein at least one of the processing elements of the operator graph is configured to receive data from at least one upstream processing element and transmit data to at least one downstream processing element. Embodiments detect an error condition has been satisfied at a first one of the plurality of processing elements, wherein the first processing element contains a plurality of fused operators. At least one of the plurality of fused operators is selected for removal from the first processing element. Embodiments then remove the selected at least one fused operator from the first processing element.

Claims

1. A method of managing a processing element, comprising:

receiving streaming data to be processed by a plurality of processing elements, the processing elements processing at least a portion of the received data by operation of one or more computer processors;

establishing an operator graph of the plurality of processing elements, the operator graph defining at least one execution path and wherein at least one of the processing elements of the operator graph is configured to receive data from at least one upstream processing element and transmit data to at least one downstream processing element;

detecting an error condition has been satisfied at a first one of the plurality of processing elements, wherein the first processing element contains a plurality of fused operators;

selecting at least one of the plurality of fused operators for removal from the first processing element; and

removing the selected at least one fused operator from the first processing element.

2. The method of claim 1, wherein selecting at least one of the plurality of fused operators further comprises:

determining priority information specifying a respective priority for each of the plurality of fused operators; and

selecting a fused operator having the lowest priority for removal from the first processing element, based on the determined priority information.

3. The method of claim 1, wherein the selected at least one fused operator is removed from the first processing element without interrupting the processing of the other fused operators in the plurality of fused operators.

4. The method of claim 1, wherein the error condition is at least one of (i) an occurrence of one or more exceptions, (ii) an occurrence of one or more error messages, and (iii) one or more system workload conditions.

5. The method of claim 1, further comprising:

creating a second processing element containing the removed at least one fused operator;

initiating execution of the second processing element; and

updating the operator graph to reference the created second processing element.

6. The method of claim 5, wherein the first processing element is hosted on a first compute node, and further comprising:

selecting a second compute node suitable to host the second processing element, wherein the second processing element is executed on the selected second compute node, and wherein the operator graph is updated to reference the second processing element on the selected second compute node.

7. The method of claim 1, wherein selecting at least one of the plurality of fused operators further comprises:

identifying one or more fused operators from the plurality of fused operators causing the error condition; and

selecting the identified one or more fused operators for removal.