Dynamic runtime choosing of processing communication methods
First Claim
1. A computer program product for processing data, the computer program product comprising:
- a computer-readable memory having computer-readable program code embodied therewith, the computer-readable program code comprising computer-readable program code configured to;
establish an operator graph of a plurality of processing elements, the operator graph defining at least one execution path, wherein a first processing element of the plurality of processing elements is located on a first compute node and configured to transmit data to at least one downstream processing element, and wherein the first processing element is capable of transmitting data to the at least one downstream processing element by selecting either or both of a first and a second communication protocol, whereby the first processing element is capable of dynamically switching from the first communication protocol to the second communication protocol, and vice versa;
transmit data from the first processing element to the at least one downstream processing element using a selected one of the first and the second communication protocols;
determine, while the selected one of the first and second communication protocols is selected, whether to switch to the other of the first and second communication protocols; and
upon determining to switch, determine whether to move the first processing element from the first compute node onto a second compute node before transmitting data from the first processing element to the at least one downstream processing element using the other of the communication protocols.
1 Assignment
0 Petitions
Accused Products
Abstract
Techniques are described for assigning and changing communication protocols for a pair of processing elements. The communication protocol determines how the pair of processing elements transmits data in a stream application. The pair may be assigned a communication protocol (e.g., TCP/IP or a protocol that uses a relational database, shared file system, or shared memory) before the operator graph begins to stream data. This assignment may be based on a priority of the processing elements and/or a priority of the communication protocols. After the operator graph begins to stream data, the pair of processing elements may switch to a different communication protocol. The decision to switch the communication protocol may be based on whether the pair of processing elements or assigned communication protocol is meeting established performance standards for the stream application.
74 Citations
21 Claims
-
1. A computer program product for processing data, the computer program product comprising:
a computer-readable memory having computer-readable program code embodied therewith, the computer-readable program code comprising computer-readable program code configured to; establish an operator graph of a plurality of processing elements, the operator graph defining at least one execution path, wherein a first processing element of the plurality of processing elements is located on a first compute node and configured to transmit data to at least one downstream processing element, and wherein the first processing element is capable of transmitting data to the at least one downstream processing element by selecting either or both of a first and a second communication protocol, whereby the first processing element is capable of dynamically switching from the first communication protocol to the second communication protocol, and vice versa; transmit data from the first processing element to the at least one downstream processing element using a selected one of the first and the second communication protocols; determine, while the selected one of the first and second communication protocols is selected, whether to switch to the other of the first and second communication protocols; and upon determining to switch, determine whether to move the first processing element from the first compute node onto a second compute node before transmitting data from the first processing element to the at least one downstream processing element using the other of the communication protocols. - View Dependent Claims (2, 3, 4, 5, 6, 18, 19)
-
7. A system for processing data, comprising:
-
a computer processor; and a memory containing a program that, when executed on the computer processor, performs an operation for processing data, comprising; establishing an operator graph of a plurality of processing elements, the operator graph defining at least one execution path, wherein a first processing element of the plurality of processing elements is located on a first compute node and configured to transmit data to at least one downstream processing element, and wherein the first processing element is capable of transmitting data to the at least one downstream processing element by selecting either or both of a first and a second communication protocol, whereby the first processing element is capable of dynamically switching from the first communication protocol to the second communication protocol, and vice versa; transmitting data from the first processing element to the at least one downstream processing element using a selected one of the first and second communication protocols; determining, while the selected one of the first and second communication protocols is selected, whether to switch to the other of the first and second communication protocols; and upon determining to switch, determining whether to move the first processing element from the first compute node onto a second compute node before transmitting data from the first processing element to the at least one downstream processing element using the other of the communication protocols. - View Dependent Claims (8, 9, 10, 11, 20, 21)
-
-
12. A computer program product for processing data, the computer program product comprising:
a computer-readable memory having computer-readable program code embodied therewith, the computer-readable program code comprising computer-readable program code configured to; establish an operator graph of a plurality of processing elements, the operator graph defining at least one execution path, wherein a first processing element of the plurality of processing elements is located on a first compute node and configured to transmit data to at least one downstream processing element, and wherein the first processing element is capable of transmitting data to the at least one downstream processing element by selecting one or more communications protocols from a plurality of available communications protocols; transmit, by one or more computer processors, data from the first processing element to the at least one downstream processing element using the selected communication protocols; and determine, based on one or more measures of performance for transmitting data using the selected communication protocols, whether moving the first processing element onto a second compute node will improve at least one of the measures of performance using one of the plurality of available communication protocols. - View Dependent Claims (13, 14, 15, 16, 17)
Specification