Chasing engine for data transfer

• US 7,480,676 B2
• Filed: 04/03/2006
• Issued: 01/20/2009
• Est. Priority Date: 04/01/2005
• Status: Active Grant

• Current Assignee: Schlumberger Technology Corporation (Schlumberger NV)
• Original Assignee: Schlumberger Technology Corporation (Schlumberger NV)
• Inventors: Wald, Joe
• Primary Examiner(s): Ali; Mohammad
• Assistant Examiner(s): FILIPCZYK, MARCIN R
• Application Number:
  US11/397,494
• Publication Number:
  US 20060224637A1
• Time in Patent Office: 1,023 Days
• Field of Search: 707 1- 3, 707/10, 707100-102, 707200-204, 707/9, 713/182, 715/200
• US Class Current: 1/1
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

A data repository defined by a single logical specification includes a data store comprising a plurality of tables and a view of the plurality of tables, a data dictionary comprising a meta-table and means to generate the data store, wherein the meta-table is generated from the single logical specification defining the data repository, and a chase engine configured to transfer a plurality of data within the data store, wherein the chase engine generates a plurality of chasing procedures based on a chasing specification, wherein the chasing procedures define a manner to transfer the plurality of data.

Claims

1. A system for transferring data comprising:

a processor; and

a memory operating under the control of the processor and storing a data repository defined by a single logical specification, comprising;

a first data store comprising a plurality of tables, a view of the plurality of tables, and a seed entity;

a data dictionary comprising a meta-table and means to generate the first data store, wherein the meta-table is generated from the single logical specification defining the data repository;

a first chase rule comprising the seed entity, a first related entity, a first link between the seed entity and the first related entity, and a direction of the first link;

a chasing specification defined by the first chase rule; and

a chase engine configured to generate a plurality of chasing procedures from the chasing specification and transfer the seed entity from the first data store to a second data store by executing said plurality of chasing procedures,

wherein a first chasing procedure of the plurality of chasing procedures requires transferring the first related entity to the second data store.

2. The system of claim 1, wherein the chasing engine is further configured to identify and report an inconsistency in the chasing specification.

3. The system of claim 1, wherein the first data store belongs to a first database platform and the second data store belongs to a second database platform.

4. The system of claim 1, wherein the chasing engine supports incremental data transfer and focused incremental data transfer.

5. The system of claim 1, further comprising:

a second chase rule identifying a second related entity, the first related entity, a second link between the second related entity and the first related entity, and a direction of the second link,

wherein the chasing specification is further defined by the second chase rule, and

wherein a second chasing procedure of the plurality of chasing procedures requires transferring the second entity to the second data store.

6. The system of claim 5, further comprising:

a first cycle comprising the first link and the second link;

a first offset attribute associated with the first chase rule; and

a second offset attribute associated with the second chase rule,

wherein the first offset attribute and the second offset attribute specify an order of the first chase rule and the second chase rule in the first cycle.

7. The system of claim 6, further comprising:

a third chase rule identifying a third related entity, the seed entity, a third link between the third related entity and the seed entity, and a direction of the third link,

wherein the third chase rule is associated with a second cycle,

wherein the chasing specification is further defined by the third chase rule, and

wherein the first cycle and the second cycle are traversed in parallel to generate the plurality of chasing procedures.

8. The system of claim 6, further comprising:

a dependency graph comprising a plurality of edges induced by at least one selected from a group consisting of the first chase rule and the second chase rule.

9. The system of claim 1, further comprising:

a move table comprising the seed entity and the first related entity, wherein the plurality of chasing procedures are generated using the move table.

10. A method of transferring data, comprising:

obtaining a seed entity to transfer from a first data store to a second data store;

obtaining a chasing specification defining at least one domain of a data repository comprising the first data store, wherein the chasing specification is defined by a first chasing rule identifying the seed entity, a first related entity, a first link between the seed entity and the first related entity, and a direction of the first link;

generating a plurality of chasing procedures based on the chasing specification after obtaining the seed entity; and

transferring the seed entity from the first data store to the second data store by executing the plurality of chasing procedures,

wherein a first chasing procedure of the plurality of chasing procedures requires transferring the first related entity to the second data store.

11. The method of claim 10, wherein the chasing specification is further defined by:

a second chase rule specifying a second related entity, the first related entity, a second link between the second related entity and the first related entity, and a direction of the second link,

wherein a second chasing procedure of the plurality of chasing procedures requires transferring the second related entity to the second data store.

12. The method of claim 11, wherein the chasing specification is further defined by:

a head entity identifying a first cycle comprising the first link and the second link;

a first offset attribute associated with the first chase rule; and

a second offset attribute associated with the second chase rule,

wherein the first offset attribute and the second offset attribute specify an order of the first chase rule and the second chase rule in the first cycle.

13. The method of claim of claim 12, wherein the chasing specification is further defined by:

a third chase rule identifying a third related entity, the seed entity, a third link between the third related entity and the seed entity, and a direction of the third link,

wherein the third chase rule is associated with a second cycle, and

wherein the first cycle and the second cycle are traversed in parallel to generate the plurality of chasing procedures.

14. The method of claim 12, wherein generating the plurality of chasing procedures comprises:

generating a dependency graph comprising an edge induced by the first chase rule;

summing a rank of the head entity in the dependency graph with the first offset attribute to generate a rank of the first chase rule.

15. The method of claim 10, wherein generating the plurality of chasing procedures comprises:

populating a move table including the seed entity and the first entity based on the chasing specification;

applying a filter to the move table to identify an updated entity.

16. A computer readable medium encoded with a computer program for transferring data, the computer program executable by a computer and comprising functionality to:

obtain a seed entity to transfer from a first data store to a second data store;

obtain a chasing specification defining at least one domain of a data repository comprising the first data store, wherein the chasing specification is defined by a first chase rule identifying the seed entity, a first related entity, a first link between the seed entity and the first related entity, and a direction of the first link;

generate a plurality of chasing procedures based on the chasing specification after obtaining the seed entity; and

transfer the seed entity from the first data store to the second data store by executing the plurality of chasing procedures,

wherein a first chasing procedure of the plurality of chasing procedures requires transferring the first related entity to the second data store.

17. The computer readable medium of claim 16, wherein the chasing specification is further defined by:

a second chase rule specifying a second related entity, the first related entity, a second link between the second related entity and the first related entity, and a direction of the second link,

wherein a second chasing procedure of the plurality of chasing procedures requires transferring the second related entity to the second data store.

18. The computer readable medium of claim 17, wherein the chasing specification is further defined by:

a head entity identifying a first cycle comprising the first link and the second link;

a first offset attribute associated with the first chase rule; and

a second offset attribute associated with the second chase rule,

wherein the first offset attribute and the second offset attribute specify an order of the first chase rule and the second chase rule in the first cycle.

19. The computer readable medium of claim 18, wherein the chasing specification is further defined by:

a third chase rule identifying a third related entity, the seed entity, a third link between the third related entity and the seed entity, and a direction of the third link,

wherein the third chase rule is associated with a second cycle, and

wherein the first cycle and the second cycle are traversed in parallel to generate the plurality of chasing procedures.

20. The computer readable medium of claim 16, wherein the computer program further comprises functionality to:

populate a move table including the seed entity and the first entity based on the chasing specification;

apply a filter to the move table to identify an updated entity.