Apparatus, systems and methods for relational database replication and proprietary data transformation
First Claim
1. Computer apparatus for creating a destination database from a given source database, comprising:
- an extractor for extracting catalog information and data from a given source database having certain data indicated as proprietary data to create a destination database, the source and destination databases being relational databases, and the extractor configured to filter the data extracted from the source database based on configuration values specific to a particular extract process that indicate a schema-level filter, which specifies a size of the destination database as a specific size in megabytes or as a percentage of a size of the source database;
driving tables, which are an optimum number of tables in the source database for which filtering is to be applied to limit the data extracted from the tables, the filtering of the driving tables driving a selection from the remainder of the tables in the source database based on relationships of the driving tables with other tables in the source database, the relationships being represented by a hierarchy of tables;
the extractor configured to (i) build the hierarchy of tables based on physical referential integrity defined by foreign key relationships among the tables and based on logical relationships among the tables, (ii) designate tables representing leaf-level tables in the hierarchy of tables as the driving tables, and (iii) assign to each table a percent filter indicating a percentage of the rows in the table that are to be extracted, the value of the percentage being equal to the percentage of the size of the source database in an event the schema-level filter so specifies or being a calculated percentage that is determined in an event the schema-level filter specifies a specific size in megabytes; and
a data transformer coupled to the extractor for preventing the proprietary data from being propagated from the source database to the destination database during extraction such that the destination database is populated by a referentially intact set of filtered data from the source database that is free of proprietary information by applying to data of a foreign key of one table a same transformation applied to data of a related primary key of another table, the foreign and related primary keys and application of the same transformation to the data thereof being determined through traversal of the hierarchy of tables.
1 Assignment
0 Petitions
Accused Products
Abstract
The present invention provides data replication and transformation. The invention system creates a replica version of a relational database based upon information obtained from a source database. The system populates the replica database with all or a portion of the data from the source database and transforms the data during the extraction process using a standard set of algorithms. This keeps any proprietary information in the source database from reaching the replica database while still retaining that source database'"'"'s referential integrity. The system supports the process of reloading the replica databases interactively or on a scheduled basis running as a batch process. Included is a global network component that coordinates the reload among the various groups of users accessing the replica database and supports the addition of new or enhanced transform algorithms through the present invention web portal, as they become available.
-
Citations
20 Claims
-
1. Computer apparatus for creating a destination database from a given source database, comprising:
-
an extractor for extracting catalog information and data from a given source database having certain data indicated as proprietary data to create a destination database, the source and destination databases being relational databases, and the extractor configured to filter the data extracted from the source database based on configuration values specific to a particular extract process that indicate a schema-level filter, which specifies a size of the destination database as a specific size in megabytes or as a percentage of a size of the source database; driving tables, which are an optimum number of tables in the source database for which filtering is to be applied to limit the data extracted from the tables, the filtering of the driving tables driving a selection from the remainder of the tables in the source database based on relationships of the driving tables with other tables in the source database, the relationships being represented by a hierarchy of tables; the extractor configured to (i) build the hierarchy of tables based on physical referential integrity defined by foreign key relationships among the tables and based on logical relationships among the tables, (ii) designate tables representing leaf-level tables in the hierarchy of tables as the driving tables, and (iii) assign to each table a percent filter indicating a percentage of the rows in the table that are to be extracted, the value of the percentage being equal to the percentage of the size of the source database in an event the schema-level filter so specifies or being a calculated percentage that is determined in an event the schema-level filter specifies a specific size in megabytes; and a data transformer coupled to the extractor for preventing the proprietary data from being propagated from the source database to the destination database during extraction such that the destination database is populated by a referentially intact set of filtered data from the source database that is free of proprietary information by applying to data of a foreign key of one table a same transformation applied to data of a related primary key of another table, the foreign and related primary keys and application of the same transformation to the data thereof being determined through traversal of the hierarchy of tables. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A computer system for creating a destination database from a given source database, comprising:
-
a navigator for viewing database information, configuring database level parameters and defining configuration parameters associated with a specific extract process that indicate a schema-level filter, which specifies a size of the destination database as a specific size in megabytes or as a percentage of a size of the source database; an extraction assembly for carrying out the extract process to extract data from the source database and use said extracted data to populate and produce the destination database, the source and destination databases being relational databases, and the extraction assembly configured to filter the data extracted from the source database based on the configuration parameters; driving tables, which are an optimum number of tables in the source database for which filtering is to be applied to limit the data extracted from the tables, the filtering of the driving tables driving a selection from the remainder of the tables in the source database based on relationships of the driving tables with other tables in the source database, the relationships being represented by a hierarchy of tables; and the extraction assembly configured to (i) build the hierarchy of tables based on physical referential integrity defined by foreign key relationships among the tables and based on logical relationships among the tables, (ii) designate tables representing leaf-level tables in the hierarchy of tables as the driving tables, and (iii) assign to each table a percent filter indicating a percentage of the rows in the table that are to be extracted, the value of the percentage being equal to the percentage of the size of the source database in an event the schema-level filter so specifies or being a calculated percentage that is determined in an event the schema-level filter specifies a specific size in megabytes. - View Dependent Claims (15, 16, 17)
-
-
18. A method for creating a destination database from a given source database, comprising the computer implemented steps of:
-
a. extracting catalog information and data from the given source database to create the destination database, the source and destination databases being relational databases, and the source database having certain data indicated as proprietary data; filtering the data extracted from the source database based on configuration values specific to a particular extract process that indicate a schema-level filter, which specifies a size of the destination database as a specific size in megabytes or as a percentage of a size of the source database; automatically selecting driving tables, which are an optimum number of tables in the source database for which filtering is to be applied to limit the data extracted from the tables, the filtering of the driving tables driving a selection from the remainder of the tables in the source database based on relationships of the driving tables with other tables in the source database, the relationships being represented by a hierarchy of tables, and the automatic selection of the driving tables including; (i) building the hierarchy of tables based on physical referential integrity defined by foreign key relationships among the tables and based on logical relationships among the tables; (ii) designating tables representing leaf-level tables in the hierarchy of tables as the driving tables; and (iii) assigning to each table a percent filter indicating a percentage of the rows in the table that are to be extracted, the value of the percentage being equal to the percentage of the size of the source database in an event the schema-level filter so specifies or being a calculated percentage that is determined in an event the schema-level filter specifies a specific size in megabytes; and b. preventing the proprietary data from being propagated from the source database to the destination database during extraction by applying to data of a foreign key of one table a same transformation applied to data of a related primary key of another table, the foreign and related primary keys and application of the same transformation to the data thereof being determined through traversal of the hierarchy of tables; c. the steps of extracting and preventing providing the destination database to be populated by a referentially intact set of filtered data free of proprietary information. - View Dependent Claims (19, 20)
-
Specification