Joining operator graph elements via whole program optimization
First Claim
Patent Images
1. A computer program product to optimize distributed applications by joining processing elements, the computer program product comprising:
- a non-transitory computer-readable medium having computer-readable program code embodied therewith, the computer-readable program code executable by operation of one or more processors to perform an operation comprising;
analyzing respective instruction code of each of a plurality of processing elements of a distributed application, wherein the plurality of processing elements are organized in an operator graph, wherein the plurality of processing elements include at least a first processing element and a second processing element, which are of distinct processing element types and which are not in succession relative to one another in the operator graph;
precompiling the operator graph in order to save compiler artifacts corresponding to the first processing element and the second processing element, the compiler artifacts including instruction code and symbols, wherein the compiler artifacts are evaluated based on one or more specified rules;
determining, based on at least one of the analysis and the evaluation, that the instruction code of the first processing element and the instruction code of the second processing element satisfy at least one optimization rule;
upon determining that the at least one optimization rule is satisfied, joining the instruction code of the first processing element with the instruction code of the second processing element based on the at least one optimization rule and in order to generate a third processing element which includes at least some of the instruction code from both the first processing element and the second processing element; and
deploying the distributed application, which includes the third processing element and excludes both the first processing element and the second processing element, thereby reducing a processing overhead incurred by the distributed application relative to inclusion of both the first processing element and the second processing element.
1 Assignment
0 Petitions
Accused Products
Abstract
Techniques disclosed herein describe joining processing elements (e.g., of a streams processing environment) using optimization techniques, such as by whole program optimization. An optimization tool evaluates multiple processing elements based on one or more optimization rules. The optimization tool identifies, based on the evaluation, processing elements that can be merged. The optimization tool joins the identified processing elements together.
-
Citations
20 Claims
-
1. A computer program product to optimize distributed applications by joining processing elements, the computer program product comprising:
a non-transitory computer-readable medium having computer-readable program code embodied therewith, the computer-readable program code executable by operation of one or more processors to perform an operation comprising; analyzing respective instruction code of each of a plurality of processing elements of a distributed application, wherein the plurality of processing elements are organized in an operator graph, wherein the plurality of processing elements include at least a first processing element and a second processing element, which are of distinct processing element types and which are not in succession relative to one another in the operator graph; precompiling the operator graph in order to save compiler artifacts corresponding to the first processing element and the second processing element, the compiler artifacts including instruction code and symbols, wherein the compiler artifacts are evaluated based on one or more specified rules; determining, based on at least one of the analysis and the evaluation, that the instruction code of the first processing element and the instruction code of the second processing element satisfy at least one optimization rule; upon determining that the at least one optimization rule is satisfied, joining the instruction code of the first processing element with the instruction code of the second processing element based on the at least one optimization rule and in order to generate a third processing element which includes at least some of the instruction code from both the first processing element and the second processing element; and deploying the distributed application, which includes the third processing element and excludes both the first processing element and the second processing element, thereby reducing a processing overhead incurred by the distributed application relative to inclusion of both the first processing element and the second processing element. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
14. A system, the system comprising:
-
one or more computer processors; and a memory storing a program, which, when executed on the one or more computer processors, performs an operation comprising; analyzing respective instruction code of each of a plurality of processing elements of a distributed application, wherein the plurality of processing elements are organized in an operator graph, wherein the plurality of processing elements include at least a first processing element and a second processing element, which are of distinct processing element types and which are not in succession relative to one another in the operator graph; precompiling the operator graph in order to save compiler artifacts corresponding to the first processing element and the second processing element, the compiler artifacts including instruction code and symbols, wherein the compiler artifacts are evaluated based on one or more specified rules; determining, based on at least one of the analysis and the evaluation, that the instruction code of the first processing element and the instruction code of the second processing element satisfy at least one optimization rule; upon determining that the at least one optimization rule is satisfied, joining the instruction code of the first processing element with the instruction code of the second processing element based on the at least one optimization rule and in order to generate a third processing element which includes at least some of the instruction code from both the first processing element and the second processing element; and deploying the distributed application, which includes the third processing element and excludes both the first processing element and the second processing element, thereby reducing a processing overhead incurred by the distributed application relative to inclusion of both the first processing element and the second processing element. - View Dependent Claims (15, 16, 17, 18, 19, 20)
-
Specification