Automatically optimize performance of package execution
First Claim
1. A method for automatically optimizing scheduling of tasks in an extract-transform-load (ETL) package by a control flow application comprising the steps of:
- the control flow application running an automated process for the ETL package to optimize scheduling of a plurality of tasks included in the ETL package, wherein each of the plurality of tasks is an ETL task operable to be executed sequentially or in parallel with the other tasks, the automated process comprising;
the control flow application running a profiling phase to execute each of the plurality of tasks in the ETL package at least one time and measure a plurality of performance metrics; and
the control flow application running an optimization phase to optimize subsequent executions of the plurality of tasks in the ETL package, wherein, while running the optimization phase, the control flow application uses at least a portion of the performance metrics to make at least one adjustment to the ETL package by adjusting execution of at least one of the plurality of tasks.
2 Assignments
0 Petitions
Accused Products
Abstract
Various technologies and techniques are disclosed that automatically optimize package execution performance. A profiling phase executes each task in a control flow package and measures performance metrics, such as task execution length, task memory usage, task correlation to CPU versus input/output operations, network bandwidth, and running applications. An optimization phase optimizes subsequent executions of the package by using the performance metrics to make appropriate adjustments to the package. The profiling phase and optimizing phase are repeated over a period of time as appropriate to improve the package'"'"'s performance without requiring a user to make adjustments to the tasks in the package.
60 Citations
20 Claims
-
1. A method for automatically optimizing scheduling of tasks in an extract-transform-load (ETL) package by a control flow application comprising the steps of:
-
the control flow application running an automated process for the ETL package to optimize scheduling of a plurality of tasks included in the ETL package, wherein each of the plurality of tasks is an ETL task operable to be executed sequentially or in parallel with the other tasks, the automated process comprising; the control flow application running a profiling phase to execute each of the plurality of tasks in the ETL package at least one time and measure a plurality of performance metrics; and the control flow application running an optimization phase to optimize subsequent executions of the plurality of tasks in the ETL package, wherein, while running the optimization phase, the control flow application uses at least a portion of the performance metrics to make at least one adjustment to the ETL package by adjusting execution of at least one of the plurality of tasks. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A computer-readable storage medium having computer-executable instructions for causing a computer to perform steps comprising:
-
providing a control flow application operable to execute an extract-transform-load (ETL) control flow package; providing a package creation feature that allows a user to create an ETL control flow package with a plurality of tasks; providing an automatic performance improvement process that initiates a profiling phase and an optimization phase to optimize scheduling of one or more of the plurality of tasks in the ETL control flow package, wherein during the optimization phase the control flow application adjusts execution of at least one of the plurality of tasks of the ETL control flow package; and providing a monitoring feature to monitor the performance of execution of the plurality of tasks in the ETL control flow package over a period of time and adjust an execution plan of the plurality of tasks in the ETL control flow package according to a set of data gathered using the monitoring feature. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17)
-
-
18. A method for automatically optimizing package execution performance comprising the steps of:
-
a control flow application running a profiling phase to execute each of a plurality of tasks in an extract-transform-load (ETL) package at least one time and measure a plurality of performance metrics, wherein at least one of the plurality of performance metrics is selected from the group consisting of task execution length, task memory usage, task correlation to CPU versus input/output operations, network bandwidth, and running applications; the control flow application running an optimization phase to optimize subsequent executions of the ETL package, wherein, while running the optimization phase, the control flow application uses at least a portion of the performance metrics to make at least one adjustment to the ETL package by adjusting execution of at least one of the plurality of tasks; and wherein the profiling phase and optimizing phase are repeated over a period of time to improve a performance of the ETL package without requiring a user to make adjustments to execution of the tasks in the ETL package. - View Dependent Claims (19, 20)
-
Specification