APPARATUS AND METHOD FOR FORMING A HOMOGENOUS TRANSACTION DATA STORE FROM HETEROGENEOUS SOURCES

• US 20070299885A1
• Filed: 05/11/2007
• Published: 12/27/2007
• Est. Priority Date: 05/12/2006
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Pareek, Alok, Lessiv, Andrei, Fish, Eric, Kutay, Ali
• Granted Patent: US 7,831,574 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses

Abstract

A computer readable storage medium includes executable instructions to gather first committed transactional record from a first transaction log associated with a first data source. The first committed transactional record is converted to a homogeneous format to form a first homogeneous transactional record with a common transaction record header containing transactional context information from the first data source. A second committed transactional record is collected from a second transaction log associated with a second data source with a format different than the first data source. The second committed transactional record is changed to the homogeneous format to form a second homogeneous transactional record with a common transaction record header containing transactional context information from the second data source. The first homogeneous transactional record is combined with the second homogeneous transactional record to form a homogeneous transaction data store.

Claims

1. A computer readable storage medium, comprising executable instructions to:

gather a first committed transactional record from a first transaction log associated with a first data source;

convert the first committed transactional record to a homogeneous format to form a first homogeneous transactional record with a common transaction record header containing transactional context information from the first data source;

collect a second committed transactional record from a second transaction log associated with a second data source with a format different than the first data source

change the second committed transactional record to the homogeneous format to form a second homogeneous transactional record with a common transaction record header containing transactional context information from the second data source; and

combine the first homogeneous transactional record with the second homogeneous transactional record to form a homogeneous transaction data store.

2. The computer readable storage medium of claim 1 wherein the homogeneous transaction data store is distributed across a set of files containing committed transactions in historical order.

3. The computer readable storage medium of claim 1 wherein the homogeneous format includes a common transaction record header, including at least three of:

a transaction identification from the originating data source;

the order and offset of a commit record from the originating data source transaction log;

the original time of commit posting at the originating data source;

the number of data source operations within the common transaction record; and

the number of bytes occupied by the common transaction record on disk.

4. The computer readable storage medium of claim 1 further comprising executable instructions to query the homogeneous transaction data store.

5. The computer readable storage medium of claim 1 further comprising executable instructions to assess from the homogeneous transaction data store temporal data access.

6. The computer readable storage medium of claim 1 further comprising executable instructions to assess from the homogeneous transaction data store data compliance.

7. The computer readable storage medium of claim 1 further comprising executable instructions to assess from the homogeneous transaction data store fraud.

8. The computer readable storage medium of claim 1 further comprising executable instructions to assess from the homogeneous transaction data store audit information.

9. The computer readable storage medium of claim 1 further comprising executable instructions to define capture agents to form the first homogeneous transaction record and the second homogeneous transaction record.

10. The computer readable storage medium of claim 9 wherein the capture agents identify incremental transactional changes from database transaction logs.

11. The computer readable storage medium of claim 9 wherein the capture agents form a sequence of transaction files.

12. The computer readable storage medium of claim 9 wherein the capture agents identify incremental transactional changes from data pumps.

13. The computer readable storage medium of claim 9 wherein the capture agents use application program interfaces to facilitate access to custom data sources.

14. The computer readable storage medium of claim 1 further comprising executable instructions to partition the homogeneous transaction data store across multiple transaction trails and disks.

15. The computer readable storage medium of claim 1 further comprising executable instructions to utilize a heartbeat mechanism to order records across transaction trails.

16. The computer readable storage medium of claim 1 further comprising executable instructions to partition the homogeneous transaction data store according to table name.

17. The computer readable storage medium of claim 9 wherein the capture agents maintain redundant trail copies with a trail directory to facilitate scalability and fault tolerance.

18. The computer readable storage medium of claim 9 wherein the capture agents form transaction trail headers.

19. The computer readable storage medium of claim 18 wherein the transaction trail headers include:

a source host identifier;

information specifying a timestamp-range representing the time span of all committed records within the transaction trail;

information specifying the first and last committed transactions within the transaction trail; and

a trail sequence number.

20. The computer readable storage medium of claim 1 further comprising executable instructions to process the homogeneous transaction data store across a master trail server and worker trail servers to facilitate disk read parallelism and filter processing.

21. The computer readable storage medium of claim 1 further comprising executable instructions to repeatedly apply a single structure query language query against a sequence of transactions in a transaction trail.