Automated software code generation from a metadata-based repository

• US 6,604,110 B1
• Filed: 10/31/2000
• Issued: 08/05/2003
• Est. Priority Date: 08/31/2000
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: Ascential Software Corporation (International Business Machines Corporation)
• Inventors: Leman, Gregory D., Geiger, Robert J. Jr., Cowan, William F., Savage, William H.
• Primary Examiner(s): Mizrahi, Diane D.
• Assistant Examiner(s): MOFIZ, APU M
• Application Number:
  US09/703,161
• Time in Patent Office: 1,008 Days
• Field of Search: 707/1, 707/2, 707/10, 707/101, 707/102, 707/103.R, 707/104.1, 707/200, 705/10
• US Class Current: 707/602
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

Software code for execution in an EDM application in presenting data from one or more data sources to an EDM application user, is generated from the source metadata obtained by analysis of the data sources in accordance with the structural entities of a generic data repository.

Claims

1. A method of providing executable software program code for use in an enterprise data management (EDM) application of the type which presents data from one or more data sources to an EDM application database, for use therein in response to user commands, and in a manner defined by the EDM application, comprising:

establishing a data repository having an entity structure with a plurality of entities which collectively define the metadata characteristics of a generic model data source, and configuring, within said repository, one or more individual constructs of said plurality of entities, each in accordance with the construct of an associated one of the data sources;

analyzing each data source in response to the repository configured entities of their individual constructs to obtain the source metadata for each data source; and

defining the executable software program code to present data from each data source in the manner defined by the EDM application, and with a data content determined by the source metadata obtained for each data source.

2. The method of claim 1, wherein said entity structure of said data repository is independent of the construct of the data source.

3. The method of claim 1, wherein said entity structure of said data repository comprises plural entities, each of said plural entities corresponding to a different aspect of a generic model data source, and each having atomic elements corresponding to the metadata attributes of a generic entity, whereby said plural entities collectively define the metadata characteristics of a generic model data source.

4. The method of claim 3, wherein said step of analyzing includes:

obtaining, from the data source, those elements of source data which correspond to the metadata attributes of the configured data repository entities; and

recording the obtained elements of source data in said data repository, each in association with their corresponding metadata attribute.

5. The method of claim 4, wherein said step of analyzing further includes:

inferring selected aspects of the data source structure on the basis of said recorded elements of source data, and recording the inferred data source aspects in said data repository;

presenting said inferred data source aspects for review by an operator; and

modifying said inferred data source aspects in response to command of the operator.

6. The method of claim 4, wherein said data repository entities include at least one each of a database entity, a table entity, and a column entity, each thereof having plural metadata attributes, and each thereof having an associated child property entity, each said child property entity being capable of receiving therein additional metadata attributes to be associated with its parent entity, as provided by an operator, thereby permitting modification of the parent entity attributes without altering said data repository entity structure.

7. The method of claim 6, wherein:

said step of obtaining includes retrieving, as available from the data source, those elements of source data corresponding to the attributes of each of said database entity, said table entity, and said column entity.

8. The method of claim 7, wherein said step of obtaining further includes:

recording, in a first area of said data repository, those elements of source data received in said step of retrieving, and inferring therefrom the data source schema.

9. The method of claim 3 wherein:

said data repository entities include at least one each of a database analysis entity, a table analysis entity, and a column analysis entity; and

wherein said step of analyzing includes,

acquiring, by elemental examination of the data source schema, those elements of source data corresponding to the metadata attributes of one or all of said database analysis entity, said table analysis entity, and said column analysis entity;

recording, in said data repository, those elements of source data received in said step of acquiring, and inferring therefrom the data source schema.

10. The method of claim 9, wherein said defining comprises:

creating, within said repository, a migration specification utility comprising a plurality of mapping expressions, each of said mapping expressions defining the functional relationships between the attributes of a generic data source and those of a generic EDM application database;

entering into the migration specification utility the attributes of each data source as determined by its associated source metadata, and the attributes of the user defined EDM application database, to define an EDM application transformation expression which identifies the steps necessary to present data from each data source to the EDM application database; and

generating the executable software program code for the EDM application in accordance with the EDM application transformation expression.

11. The method of claim 10, wherein said generating includes:

using an extract, transform, and load (ETL) utility to generate the executable software program code directly from said EDM application transformation expression.

12. The method of claim 11, wherein:

said plurality of entities of said data repository further include entities which define the metadata characteristic of a generic ETL utility.

13. The method 12, wherein generating includes:

providing the EDM application transformation expression in the protocol of the ETL utility.

14. Apparatus for providing executable software program code for use in an enterprise data management (EDM) application of the type which presents data from one or more data sources to an EDM application database, for use therein in response to user commands, and in a manner defined by the EDM application, comprising:

a graphical user interface (GUI), responsive to commands entered by a user; and

an analysis signal processor, responsive to said GUI, and adapted for connectivity to each data source, said analysis signal processor having a memory medium for storing signals, including program signals;

said analysis signal processor, in response to said program signals;

creating, within said memory medium, a data repository having an entity structure with a plurality of entities which collectively define the metadata characteristics of a generic model data source, and configuring, within the repository, one or more individual constructs of the entities, each in accordance with the construct of an associated one of the data sources;

analyzing each data source in response to the repository configured entities of their individual constructs to obtain the source metadata for each data source; and

defining the executable software program code to present data from each data source in the manner defined by the EDM application, and with a data content determined by the source metadata obtained for each data source.

15. The apparatus of claim 14, wherein said entity structure of said data repository is independent of the construct of the data source.

16. The apparatus of claim 14, wherein said entity structure of said data repository comprises plural entities, each of said plural entities corresponding to a different aspect of a generic model data source, and each having atomic elements corresponding to the metadata attributes of a generic entity, whereby said plural entities collectively define the metadata characteristics of a generic model data source.

17. The apparatus of claim 16, wherein said step of analyzing includes:

obtaining, from the data source, those elements of source data which correspond to the metadata attributes of the configured data repository entities; and

recording the obtained elements of source data in said data repository, each in association with their corresponding metadata attribute.

18. The apparatus of claim 17, wherein said data repository entities include at least one each of a database entity, a table entity, and a column entity, each thereof having plural metadata attributes, and each thereof having an associated child property entity, each said child property entity being capable of receiving therein additional metadata attributes to be associated with its parent entity, as provided by an operator, thereby permitting modification of the parent entity attributes without altering said data repository entity structure.

19. The apparatus of claim 18, wherein said defining comprises:

creating, within said repository, a migration specification utility comprising a plurality of mapping expressions, each of said mapping expressions defining the functional relationships between the attributes of a generic data source and those of a generic EDM application database;

entering into the migration specification utility the attributes of each data source as determined by its associated source metadata, and the attributes of the user defined EDM application database, to define an EDM application transformation expression which identifies the steps necessary to present data from each data source to the EDM application database; and

generating the executable software program code for the EDM application in accordance with the EDM application transformation expression.

20. The apparatus of claim 19, wherein said generating includes:

using an extract, transform, and load (ETL) utility to generate the executable software program code directly from said EDM application transformation expression.

21. The apparatus of claim 20, wherein:

said plurality of entities of said data repository further include entities which define the metadata characteristic of a generic ETL utility.

22. The apparatus of claim 20, wherein said generating includes:

providing the EDM application transformation expression in the protocol of the ETL utility.