RAPID CACHING AND DATA DELIVERY SYSTEM AND METHOD

• US 20120221606A1
• Filed: 05/09/2012
• Published: 08/30/2012
• Est. Priority Date: 02/01/2008
• Status: Active Grant

• Current Assignee: Liberty Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: American Express Travel Related Services Company, Inc. (American Express Company)
• Inventors: Alvarez, Eddie J., Dantu, Stephen K., Durvasula, Sastry VSM, Khalfin, Marat S., Thomas, Deep
• Granted Patent: US 8,515,899 B2
• Time in Patent Office: Days
• US Class Current: 707/803
• CPC Class Codes: G06F 12/0875 with dedicated cache, e.g. instruction or stack
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses

Abstract

The initial systems analysis of a new data source fully defines each data element and also designs, tests and encodes complete data integration instructions for each data element. A metadata cache stores the data element definition and data element integration instructions. The metadata cache enables a comprehensive view of data elements in an enterprise data architecture. When data is requested that includes data elements defined in a metadata cache, the metadata cache and associated software modules automatically generate database elements to fully integrate the requested data elements into existing databases.

Claims

1. A computer-based method comprising:

analyzing, by a data integration computer and based upon a first data element subset, enterprise data architecture and cached data to determine a second data element subset comprising data elements present in said cached data but not present in said enterprise data architecture, wherein said cached data is derived from data;

analyzing, by said computer and based upon first raw data, said enterprise data architecture and said cached data to determine second raw data, said second raw data being present in said cached data but not present in said enterprise data architecture; and

extracting, by said computer and based upon data extraction logic, said second raw data.

2. The method of claim 1, further comprising integrating, by said computer, said second data element subset into said enterprise data architecture.

3. The method of claim 1, further comprising integrating, by said computer, said second raw data into said enterprise data architecture.

4. The method of claim 1, further comprising caching, by said computer, said enterprise data architecture and said data to create said cached enterprise data architecture and said cached data.

5. The method of claim 1, wherein said data comprises data element definitions associated with data elements and at least one of said data extraction logic and data transformation logic.

6. The method of claim 1, wherein a first subset is integrated into said enterprise data architecture.

7. The method of claim 6, wherein a second subset includes a portion of said first subset.

8. The method of claim 7, wherein said first subset comprises a first data element subset and a first raw data subset derived from first data.

9. The method of claim 8, wherein said first data includes at least one of raw data and data elements relating to said raw data.

10. The method of claim 8, wherein said second subset comprises a second data element subset and a second raw data subset derived from said first data.

11. The method of claim 8, wherein said first data includes said first raw data subset and said first data element subset.

12. The method of claim 7, further comprising transmitting a notification of said integrating at least one of said first subset and said second subset.

13. The method of claim 1, further comprising creating data by creating data transformation scripts and data loading scripts comprising instructions for integrating first data into said enterprise data architecture.

14. The method of claim 1, further comprising packaging said data into an envelope.

15. The method of claim 1, further comprising packaging said data into an envelope and storing said envelope in a data cache.

16. The method of claim 1, further comprising packaging said data into an envelope, wherein said data comprises at least one of:

a data source, description of a data schema for said data source, data type information, data extraction logic, a data constraint, a validation rule, a default value, a business rule, a data interdependency, data transformation logic, a data load instruction, performance data, statistical data, a business rule, and a physical relationship with said enterprise data schema.

17. The method of claim 1, wherein a second request includes data elements not included in a first request.

18. The method of claim 7, wherein said integrating at least one of:

said first subset and said second subset further comprises creating a database.

19. The method of claim 7, wherein said integrating at least one of:

said first subset and said second subset further comprises creating a database and monitoring said database.

20. The method of claim 7, wherein said integrating at least one of:

said first subset and said second subset further comprises creating a database, monitoring said database to obtain database performance tuning information, and tuning said database based upon said database performance tuning information.

21. An article of manufacture including a non-transitory, tangible computer readable storage medium having instructions stored thereon that, in response to execution by a computer-based data integration system, cause said computer-based system to perform operations comprising:

analyzing, by said computer-based system and based upon a first data element subset, enterprise data architecture and cached data to determine a second data element subset comprising data elements present in said cached data but not present in said enterprise data architecture, wherein said cached data is derived from data;

analyzing, by said computer-based system and based upon first raw data, said enterprise data architecture and said cached data to determine second raw data, said second raw data being present in said cached data but not present in said enterprise data architecture; and

extracting, by said computer-based system and based upon data extraction logic, said second raw data.

22. A system comprising:

a processor for integrating data,

a tangible, non-transitory memory configured to communicate with said processor,

said tangible, non-transitory memory having instructions stored thereon that, in response to execution by said processor, cause said processor to perform operations comprising;

analyzing, by said processor and based upon a first data element subset, enterprise data architecture and cached data to determine a second data element subset comprising data elements present in said cached data but not present in said enterprise data architecture, wherein said cached data is derived from data;

analyzing, by said processor and based upon first raw data, said enterprise data architecture and said cached data to determine second raw data, said second raw data being present in said cached data but not present in said enterprise data architecture; and

extracting, by said processor and based upon data extraction logic, said second raw data.