DEBUGGING FRAMEWORK FOR DISTRIBUTED ETL PROCESS WITH MULTI-LANGUAGE SUPPORT

• US 20140344625A1
• Filed: 09/30/2013
• Published: 11/20/2014
• Est. Priority Date: 05/17/2013
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Thatte, Sachin, Koleth, Sebu Thomas, Varma, Vikas
• Granted Patent: US 9,659,012 B2
• Time in Patent Office: Days
• US Class Current: 714/38.1
• CPC Class Codes: G06F 11/3636 by tracing the execution of the program
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses

Abstract

In various embodiments, a data integration system is disclosed which enables users to debug distributed data integration scenarios which are platform and technology independent. A debugger client can connect to a plurality of local and/or remote hosts executing portions of a distributed data integration scenario. The debugger client can additionally enable line-by-line debugging of the portions of the distributed data integration scenario using a plurality of language-specific interfaces. The language-specific interfaces can further enable the user to dynamically update and debug changes to the code during debugging, reducing the time and resources required by multiple recompilations of the code.

Claims

1. A method for debugging data integration code, comprising:

receiving data integration code at a debugger client which includes a plurality of language interfaces corresponding to a plurality of programming languages, wherein the data integration code corresponds to a distributed data integration scenario executing across one or more hosts in a user network and wherein the distributed data integration scenario includes a plurality of portions of data integration code written in one or more of the plurality of programming languages;

connecting to a first host based on the data integration code;

executing a first portion of the data integration code in a debugging mode at the first host, wherein the first portion of the data integration code is written in a first programming language;

sending debugging messages to the first host from the debugger client; and

wherein the first host can invoke one or more subsequent hosts to each execute a subsequent portion of the data integration code.

2. The method of claim 1, wherein connecting to a host based on the data integration code comprises:

identifying a communication protocol associated with the host based on metadata associated with the host; and

opening a connection with the host using the communication protocol.

3. The method of claim 1, wherein the debugging messages include one or more of:

setting breakpoints;

pausing execution;

resuming execution; and

requesting variable values.

4. The method of claim 1 wherein the first host invokes a subsequent host and identifies the subsequent host to the debugger client.

5. The method of claim 4 wherein the debugger client connects to the subsequent host using metadata associated with the subsequent host.

6. The method of claim 1, wherein during execution of the data integration code in debugging mode:

replacing a portion of the data integration code with a new portion of code;

executing the new portion of code in the debugging mode; and

updating source code corresponding to the data integration code based on the new portion of code.

7. The method of claim 6 further comprising recompiling the source code after it has been updated to reflect the new portion of code.

18. The system of claim 1 wherein the first host invokes a subsequent host and identifies the subsequent host to the debugger client, and wherein the debugger client connects to the subsequent host using metadata associated with the subsequent host.

8. A non-transitory computer readable storage medium including instructions stored thereon which, when executed by a processor cause the processor to perform the steps of:

receiving data integration code at a debugger client which includes a plurality of language interfaces corresponding to a plurality of programming languages, wherein the data integration code corresponds to a distributed data integration scenario executing across one or more hosts in a user network and wherein the distributed data integration scenario includes a plurality of portions of data integration code written in one or more of the plurality of programming languages;

connecting to a first host based on the data integration code;

executing a first portion of the data integration code in a debugging mode at the first host, wherein the first portion of the data integration code is written in a first programming language;

sending debugging messages to the first host from the debugger client; and

wherein the first host can invoke one or more subsequent hosts to each execute a subsequent portion of the data integration code.

9. The non-transitory computer readable storage medium of claim 8, wherein connecting to a host based on the data integration code comprises:

identifying a communication protocol associated with the host based on metadata associated with the host; and

opening a connection with the host using the communication protocol.

10. The non-transitory computer readable storage medium of claim 8, wherein the debugging messages include one or more of:

setting breakpoints;

pausing execution;

resuming execution; and

requesting variable values.

11. The non-transitory computer readable storage medium of claim 8 wherein the first host invokes a subsequent host and identifies the subsequent host to the debugger client.

12. The non-transitory computer readable storage medium of claim 11 wherein the debugger client connects to the subsequent host using metadata associated with the subsequent host.

13. The non-transitory computer readable storage medium of claim 8, wherein during execution of the data integration code in debugging mode:

replacing a portion of the data integration code with a new portion of code;

executing the new portion of code in the debugging mode; and

updating source code corresponding to the data integration code based on the new portion of code.

14. The non-transitory computer readable storage medium of claim 13 further comprising recompiling the source code after it has been updated to reflect the new portion of code.

15. A system of generating code for data integration based on a logical design, comprising:

a data integration system comprising one or more computing devices each including a computer readable medium and processor; and

a debugger client executing on the data integration system, wherein the debugger client includes a plurality of language interfaces corresponding to a plurality of programming languages, and is configured to

receive data integration code which wherein the data integration code corresponds to a distributed data integration scenario executing across one or more hosts in the data integration system and wherein the distributed data integration scenario includes a plurality of portions of data integration code written in one or more of the plurality of programming languages;

connect to a first host based on the data integration code;

execute a first portion of the data integration code in a debugging mode at the first host, wherein the first portion of the data integration code is written in a first programming language;

send debugging messages to the first host from the debugger client; and

wherein the first host can invoke one or more subsequent hosts to each execute a subsequent portion of the data integration code.

16. The system of claim 15, wherein connecting to a host based on the data integration code comprises:

identifying a communication protocol associated with the host based on metadata associated with the host; and

opening a connection with the host using the communication protocol.

17. The system of claim 15, wherein the debugging messages include one or more of:

setting breakpoints;

pausing execution;

resuming execution; and

requesting variable values.

19. The system of claim 15, wherein during execution of the data integration code in debugging mode, the debugger client is configured to:

replace a portion of the data integration code with a new portion of code;

execute the new portion of code in the debugging mode; and

update source code corresponding to the data integration code based on the new portion of code.

20. The system of claim 19 wherein the debugger client is further configured to recompile the source code after it has been updated to reflect the new portion of code.