Generating and implementing data integration job execution design recommendations

• US 9,501,377 B2
• Filed: 03/18/2014
• Issued: 11/22/2016
• Est. Priority Date: 03/18/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Greene, Lawrence A., Jacobson, Eric A., Li, Yong, Pu, Xiaoyan
• Primary Examiner(s): WU, BENJAMIN C
• Application Number:
  US14/217,567
• Publication Number:
  US 20150268990A1
• Time in Patent Office: 980 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/30 Monitoring
• CPC Class Codes: G06F 11/323 Visualisation of programs or trace data
• CPC Class Codes: G06F 11/34 Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 9/4843 by program, e.g. task dispatcher, supervisor, operating system
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]

Abstract

The method of managing performance of data integration are described. A performance analyzer may receive data about a data integration job execution. The performance analyzer may determine whether there is a performance issue of the data integration job execution. The performance analyzer analyzes the data about the data integration job execution when there is a performance issue. The performance analyzer generates a job execution design recommendation based on the analysis of the data and a set of predefined recommendation rules. The performance analyzer then displays the data about the data integration job execution and when there is a generated job execution design recommendation, displaying the job execution design recommendation.

Claims

1. A computer-implemented method of managing performance of data integration comprising:

receiving data about the data integration job execution, wherein the data integration job is for an extract, transform, and load (ETL) job;

determining whether there is a performance issue of the data integration job execution;

analyzing the data about the data integration job execution when there is a performance issue;

generating and implementing a job execution design recommendation based on analyzing the data and a set of predefined recommendation rules, wherein the set of predefined recommendation rules comprises an operator combining recommendation, wherein the operator combining recommendation specifies that the processes of two or more operators are to be carried out by a single operator; and

displaying the data about the data integration job execution and when there is a generated job execution design recommendation, displaying the job execution design recommendation, wherein the displaying the data of the data integration job includes displaying an execution plan of a designed job execution and a process view of one or more processes of an actual job execution.

2. The computer-implemented method of claim 1, wherein the data received is performance data of the data integration job execution and resource utilization data of an environment in which the data integration job execution is occurring.

3. The computer-implemented method of claim 1, wherein the receiving the data about the data integration job is performed in real-time.

4. The computer-implemented method of claim 1, wherein the receiving the data about the data integration job is retrieved from a memory.

5. A performance analyzer, comprising:

a processor; and

a memory communicatively coupled to the processor, wherein the memory is encoded with instructions and wherein the instructions when executed by the processor include;

receive data about the data integration job execution, wherein the data integration job is for an extract, transform, and load (ETL) job;

determine whether there is a performance issue of the data integration job execution;

analyze the data about the data integration job execution when there is a performance issue;

generate and implement a job execution design recommendation based on analyzing the data and a set of predefined recommendation rules, wherein the set of predefined recommendation rules comprises an operator combining recommendation, wherein the operator combining recommendation specifies that the processes of two or more operators are to be carried out by a single operator; and

display the data about the data integration job execution and when there is a generated job execution design recommendation, displaying the job execution design recommendation, wherein the displaying the data of the data integration job includes displaying an execution plan of a designed job execution and a process view of one or more processes of an actual job execution.

6. The performance analyzer of claim 5, wherein the data received is performance data of the data integration job execution and resource utilization data of an environment in which the data integration job execution is occurring.

7. The performance analyzer of claim 5, wherein the receiving the data about the data integration job is performed in real-time.

8. The performance analyzer of claim 5, wherein receiving data about the data integration job is retrieved from a memory.

9. A computer program product of managing performance of data integration, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code executable by a computer system to:

receive data about the data integration job execution, wherein the data integration job is for an extract, transform, and load (ETL) job;

determine whether there is a performance issue of the data integration job execution;

analyze the data about the data integration job execution when there is a performance issue;

generate and implement a job execution design recommendation based on analyzing the data and a set of predefined recommendation rules, wherein the set of predefined recommendation rules comprises an operator combining recommendation, wherein the operator combining recommendation specifies that the processes of two or more operators are to be carried out by a single operator; and

display the data about the data integration job execution and when there is a generated job execution design recommendation, displaying the job execution design recommendation, wherein the displaying the data of the data integration job includes displaying an execution plan of a designed job execution and a process view of one or more processes of an actual job execution.

10. The computer program product of claim 9, wherein the data received is performance data of the data integration job execution and resource utilization data of an environment in which the data integration job execution is occurring.

11. The computer program product of claim 9, wherein the receiving the data about the data integration job is performed in real-time.

12. The computer program product of claim 9, wherein receiving data about the data integration job is retrieved from a memory.