SYSTEM AND METHOD FOR DECOMPOSITION OF CODE GENERATION INTO SEPARATE PHYSICAL UNITS THOUGH EXECUTION UNITS

• US 20140344310A1
• Filed: 11/11/2013
• Published: 11/20/2014
• Est. Priority Date: 05/17/2013
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Allan, David, Leigh, John, Lau, Thomas, Seetharaman, Ganesh, Klein, Joseph F.
• Granted Patent: US 9,633,052 B2
• Time in Patent Office: Days
• US Class Current: 707/803
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 8/20 Software design
• CPC Class Codes: G06F 8/34 Graphical or visual programming
• CPC Class Codes: G06F 8/35 model driven

Abstract

In various embodiments, a data integration system is disclosed which enables users to create a logical design which is platform and technology independent. The user can create a logical design that defines, at a high level, how a user wants data to flow between sources and targets. The tool can analyze the logical design, in view of the user'"'"'s infrastructure, and create a physical design. The logical design can include a plurality of components corresponding to each source and target in the design, as well as operations such as joins or filters, and access points. Each component when transferred to the physical design generates code to perform operations on the data. Depending on the underlying technology (e.g., SQL Server, Oracle, Hadoop, etc.) and the language used (SQL, pig, etc.) the code generated by each component may be different.

Claims

1. A method of generating a physical design from a logical design for data integration, comprising:

receiving a logical design of a data integration mapping;

applying a plurality of rules to the logical design;

generating a default physical design comprising a plurality of execution units based on the logical design; and

grouping the plurality of execution units into one or more execution unit groups, wherein each execution unit group includes independent execution units which can execute in parallel.

2. The method of claim 1 wherein the plurality of rules are based on one or more user preferences included in the logical design and an implementation environment, and wherein the plurality of rules identify which execution units support operations represented by logical components specified in the data integration mapping.

3. The method of claim 2 wherein the plurality of rules include at least one of:

a rule that adds components to source execution units by default;

a rule that adds components without dependencies to parallel execution units;

a rule that prevents reordering of operations; and

a rule that applies the one of more user preferences that do not conflict with other rules.

4. The method of claim 1 wherein each execution unit includes one or more components and wherein native code is generated for each component in each execution unit and executed according to the default physical design.

5. The method of claim 1, further comprising:

receiving changes to the default physical design, wherein the changes redefine one or more of the plurality of execution units;

dynamically generating a second physical design based on the changes to the default physical design; and

associating the second physical design with the logical design, wherein the logical design can be associated with a plurality of different physical designs.

6. The method of claim 5 wherein the changes include selecting and moving at least one component from a first execution unit to a second execution unit.

7. The method of claim 1, wherein the default physical design is comprised of a plurality of generic components, and wherein during installation each of the plurality of generic components can be bound to physical components in the implementation environment.

12. The non-transitory computer readable storage medium of claim 1, further comprising instructions which, when executed by a processor, cause the processor to perform the steps of:

receiving changes to the default physical design, wherein the changes redefine one or more of the plurality of execution units;

dynamically generating a second physical design based on the changes to the default physical design; and

associating the second physical design with the logical design, wherein the logical design can be associated with a plurality of different physical designs.

13. The non-transitory computer readable storage medium of claim 12 wherein the changes include selecting and moving at least one component from a first execution unit to a second execution unit.

8. A non-transitory computer readable storage medium including instructions stored thereon which, when executed by a processor, cause the processor to perform the steps of:

receiving a logical design of a data integration mapping;

applying a plurality of rules to the logical design;

generating a default physical design comprising a plurality of execution units based on the logical design; and

grouping the plurality of execution units into one or more execution unit groups, wherein each execution unit group includes independent execution units which can execute in parallel.

9. The non-transitory computer readable storage medium of claim 8 wherein the plurality of rules are based on one or more user preferences included in the logical design and an implementation environment, and wherein the plurality of rules identify which execution units support operations specified in the data integration mapping.

10. The non-transitory computer readable storage medium of claim 9 wherein wherein the plurality of rules include at least one of:

a rule that adds components to source execution units by default;

a rule that adds components without dependencies to parallel execution units;

a rule that prevents reordering of operations; and

a rule that applies the one of more user preferences that do not conflict with other rules.

11. The non-transitory computer readable storage medium of claim 8 wherein each execution unit includes one or more components and wherein native code is generated for each component in each execution unit and executed according to the default physical design.

14. The non-transitory computer readable storage medium of claim 8, wherein the default physical design is comprised of a plurality of generic components, and wherein during installation each of the plurality of generic components can be bound to physical components in the implementation environment.

15. A system of generating a physical design from a logical design for data integration, comprising:

a data integration system comprising one or more computing devices each including a computer readable medium and processor, wherein the data integration system is configured to

receive a logical design of a data integration mapping;

apply a plurality of rules to the logical design;

generate a default physical design comprising a plurality of execution units based on the logical design; and

group the plurality of execution units into one or more execution unit groups, wherein each execution unit group includes independent execution units which can execute in parallel.

16. The system of claim 15 wherein the plurality of rules are based on one or more user preferences included in the logical design and an implementation environment, wherein the plurality of rules identify which execution units support operations specified in the data integration mapping, and wherein the plurality of rules include at least one of:

a rule that adds components to source execution units by default;

a rule that adds components without dependencies to parallel execution units;

a rule that prevents reordering of operations; and

a rule that applies the one of more user preferences that do not conflict with other rules.

17. The system of claim 15 wherein each execution unit includes one or more components and wherein native code is generated for each component in each execution unit and executed according to the default physical design.

18. The system of claim 15, wherein the data integration system is further configured to:

receive changes to the default physical design, wherein the changes redefine one or more of the plurality of execution units;

dynamically generate a second physical design based on the changes to the default physical design; and

associate the second physical design with the logical design, wherein the logical design can be associated with a plurality of different physical designs.

19. The system of claim 18 further comprising:

a graphical user interface configured to receive the changes to the default physical design wherein the changes include selecting and moving at least one component from a first execution unit to a second execution unit.

20. The system of claim 15, wherein the default physical design is comprised of a plurality of generic components, and wherein during installation each of the plurality of generic components can be bound to physical components in the implementation environment.