SYSTEM AND METHOD FOR CODE GENERATION FROM A DIRECTED ACYCLIC GRAPH USING KNOWLEDGE MODULES

• US 20140344778A1
• Filed: 11/11/2013
• Published: 11/20/2014
• Est. Priority Date: 05/17/2013
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Lau, Thomas, Allan, David
• Granted Patent: US 10,073,867 B2
• Time in Patent Office: Days
• US Class Current: 717/106
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 8/20 Software design
• CPC Class Codes: G06F 8/34 Graphical or visual programming
• CPC Class Codes: G06F 8/35 model driven

Abstract

In various embodiments, a data integration system is disclosed which enables users to create a logical design which is platform and technology independent. The user can create a logical design that defines, at a high level, how a user wants data to flow between sources and targets. The tool can analyze the logical design, in view of the user'"'"'s infrastructure, and create a physical design. The logical design can include a plurality of components corresponding to each source and target in the design, as well as operations such as joins or filters, and access points. Each component when transferred to the physical design generates code to perform operations on the data. Depending on the underlying technology (e.g., SQL Server, Oracle, Hadoop, etc.) and the language used (SQL, pig, etc.) the code generated by each component may be different.

Claims

1. A method of generating code for data integration based on a logical design, comprising:

generating, using a computer including a computer readable medium and processor, a physical design of a data integration process based on a logical design of the data integration process, wherein the physical design includes a plurality of execution units;

assigning a knowledge module to each of a plurality of components in the plurality of execution units, wherein each knowledge module is specific to a type of component and a deployment'"'"'s language and technology type, and wherein each knowledge module is customizable by the user;

generating code for each of the plurality of execution units in the physical design, based on the assigned knowledge modules; and

executing the generated code in an order based on the physical design.

2. The method of claim 1 wherein each knowledge module includes one or more configurable settings.

3. The method of claim 2 wherein each knowledge module is a code template specific to a component type, technology type, and language type.

4. The method of claim 3 wherein each knowledge module includes one or more user configurable component, technology, or language options.

5. The method of claim 1 wherein the physical design is a directed, acyclic graph which indicates the order of execution of the generated code.

6. The method of claim 5 wherein a data structure representing code generated for each execution unit is created and passed to a next execution unit in the physical design.

7. The method of claim 1 wherein executing the generated code comprises:

simulating code generation; and

displaying the generated code.

16. The system of claim 1 wherein each knowledge module includes one or more configurable settings, and wherein each knowledge module is a code template specific to a component type, technology type, and language type.

17. The system of claim 16 wherein each knowledge module includes one or more user configurable component, technology, or language options.

8. A non-transitory computer readable storage medium including instructions stored thereon which, when executed by a processor cause the processor to perform the steps of:

generating, using a computer including a computer readable medium and processor, a physical design of a data integration process based on a logical design of the data integration process, wherein the physical design includes a plurality of execution units;

assigning a knowledge module to each of a plurality of components in the plurality of execution units, wherein each knowledge module is specific to a type of component and a deployment'"'"'s language and technology type, and wherein each knowledge module is customizable by the user;

generating code for each of the plurality of execution units in the physical design, based on the assigned knowledge modules; and

executing the generated code in an order based on the physical design.

9. The non-transitory computer readable storage medium of claim 8 wherein each knowledge module includes one or more configurable settings.

10. The non-transitory computer readable storage medium of claim 9 wherein each knowledge module is a code template specific to a component type, technology type, and language type.

11. The non-transitory computer readable storage medium of claim 10 wherein each knowledge module includes one or more user configurable component, technology, or language options.

12. The non-transitory computer readable storage medium of claim 8 wherein the physical design is a directed, acyclic graph which indicates the order of execution of the generated code.

13. The non-transitory computer readable storage medium of claim 12 wherein a data structure representing code generated for each execution unit is created and passed to a next execution unit in the physical design.

14. The non-transitory computer readable storage medium of claim 8 wherein executing the generated code comprises:

simulating code generation; and

displaying the simulated code.

15. A system of generating code for data integration based on a logical design, comprising:

a data integration system comprising one or more computing devices each including a computer readable medium and processor, wherein the data integration system is configured to

generating a physical design of a data integration process based on a logical design of the data integration process, wherein the physical design includes a plurality of execution units;

assigning a knowledge module to each of a plurality of components in the plurality of execution units, wherein each knowledge module is specific to a type of component and a deployment'"'"'s language and technology type, and wherein each knowledge module is customizable by the user;

generating code for each of the plurality of components in the physical design, based on the assigned knowledge modules; and

executing the generated code in an order based on the physical design.

18. The system of claim 15 wherein the physical design is a directed, acyclic graph which indicates the order of execution of the generated code.

19. The system of claim 18 wherein a data structure representing code generated for each execution unit is created and passed to a next execution unit in the physical design.

20. The method of claim 15 wherein executing the generated code comprises:

simulating code generation; and

displaying the simulated code.