RAPID CACHING AND DATA DELIVERY SYSTEM AND METHOD

• US 20090198892A1
• Filed: 02/01/2008
• Published: 08/06/2009
• Est. Priority Date: 02/01/2008
• Status: Active Grant

• Current Assignee: Liberty Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: American Express Travel Related Services Company, Inc. (American Express Company)
• Inventors: Dantu, Stephen K., Khalfin, Marat S., Alvarez, Eddie J., Thomas, Deep, Durvasula, Sastry VSM
• Granted Patent: US 8,032,485 B2
• Time in Patent Office: Days
• US Class Current: 711/118
• CPC Class Codes: G06F 12/0875 with dedicated cache, e.g. instruction or stack
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses

Abstract

The initial systems analysis of a new data source fully defines each data element and also designs, tests and encodes complete data integration instructions for each data element. A metadata cache stores the data element definition and data element integration instructions. The metadata cache enables a comprehensive view of data elements in an enterprise data architecture. When data is requested that includes data elements defined in a metadata cache, the metadata cache and associated software modules automatically generate database elements to fully integrate the requested data elements into existing databases.

Claims

1. A method for data integration comprising:

receiving a first request for a first subset of data, wherein said data includes at least one of raw data and data elements relating to said raw data, and wherein said data includes said first subset;

receiving data from a data source;

assessing said data for integration into an enterprise data architecture;

integrating said first subset into said enterprise data architecture based upon said step of assessing;

creating metadata related to said data and based upon said step of assessing;

caching said data and said metadata to create cached data and cached metadata;

receiving a second request for a second subset of data;

reading cached metadata; and

,

integrating said second subset into said enterprise data architecture based upon said metadata.

2. The method of claim 1, wherein said second subset includes a portion of said first subset.

3. The method of claim 1, further comprising transmitting a notification of said step of integrating at least one of said first subset and said second subset.

4. The method of claim 1, wherein creating metadata further comprises creating data transformation scripts and data loading scripts comprising instructions for integrating said data into said enterprise data architecture.

5. The method of claim 1, further comprising packaging said metadata into an envelope.

6. The method of claim 1, further comprising packaging said metadata into an envelope and storing said envelope in a metadata cache.

7. The method of claim 1, further comprising packaging said metadata into an envelope, wherein said envelope comprises at least one of:

said data source, description of a data schema for said data source, data type information, data extraction logic, a data constraint, a validation rule, a default value, a business rule, a data interdependency, data transformation logic, a data load instruction, performance data, statistical data, a business rule, and a physical relationship with said enterprise data schema.

8. The method of claim 1, wherein said second request does not include data elements included in said first request.

9. The method of claim 1, wherein said integrating at least one of:

said first subset and said second subset further comprises creating a database.

10. The method of claim 1, wherein said integrating at least one of:

said first subset and said second subset further comprises creating a database and monitoring said database.

11. The method of claim 1, wherein said integrating at least one of:

said first subset and said second subset further comprises creating a database, monitoring said database to obtain database performance tuning information, and tuning said database based upon said database performance tuning information.

12. The method of claim 1, further comprising transmitting said cached metadata.

13. A machine-readable medium having stored thereon a plurality of instructions integrating data, said plurality of instructions when executed by at least one processor, cause said processor to perform a method comprising the steps of:

receiving a first request for a first subset of data, wherein said data includes at least one of raw data and data elements relating to said raw data, and wherein said data includes said first subset;

receiving data from a data source;

assessing said data for integration into an enterprise data architecture;

integrating said first subset into said enterprise data architecture based upon said step of assessing;

creating metadata related to said data and based upon said step of assessing;

caching said data and said metadata to create cached data and cached metadata;

receiving a second request for a second subset of data;

reading cached metadata; and

,

integrating said second subset into said enterprise data architecture based upon said metadata.