Real time sessions in an analytic application

• US 20020198902A1
• Filed: 06/25/2001
• Published: 12/26/2002
• Est. Priority Date: 06/25/2001
• Status: Active Grant

• Current Assignee: Informatica LLC (Informatica, Inc. (California))
• Original Assignee: Informatica Corporation
• Inventors: Gupta, Sanjeev K., Ma, Sen, Sankaran, Mohan
• Granted Patent: US 6,789,096 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99954 Version management

Abstract

A method and system thereof for performing real time transformations of dynamically increasing databases is described. A session, identified as a real time session, is initialized. The real time session repeatedly executes a persistent (e.g., continually running) data transport pipeline of the analytic application. The data transport pipeline extracts data from a changing database, transforms the data, and writes the transformed data to storage (e.g., a data warehouse or data mart). The data transport pipeline is executed at the end of each time interval in a plurality of contiguous time intervals occurring during the real time session. The data transport pipeline remains running after it is executed, until the real time session is completed. Accordingly, new data are transformed in a timely manner, and processing resources are not consumed by having to repeatedly re-establish (re-initialize) the data transport pipeline.

Claims

1. In an analytic application, a method for incrementally transforming in real time a dynamically changing operational database, said method comprising the steps of:

a) initializing a session, said session identified as a real time session, said real time session for repeatedly executing a data transport pipeline of said analytic application; and

b) executing said data transport pipeline at the end of each time interval in a plurality of contiguous time intervals occurring during said real time session, wherein said data transport pipeline is persisted after each execution of said data transport pipeline until said real time session is completed.

2. The method as recited in claim 1 wherein said step of executing said data transport pipeline comprises the steps of:

reading data from a source database;

transforming said data according to a transformation operation of said analytic application; and

committing transformed data to a target database.

3. The method as recited in claim 2 wherein said transformation operation is a source operation, a target operation, an expression operation, an aggregation operation, a filter operation, a rank operation, an update strategy operation, a sequence operation, a joiner operation, a lookup operation, a stored procedure operation, an external procedure operation, or a normalizer operation.

4. The method as recited in claim 2 wherein data in said source database is received from a messaging system coupled to said data transport pipeline.

5. The method as recited in claim 1 wherein said step b) comprises the steps of:

determining elapsed time beginning at said step of initializing; and

executing said data transport pipeline when said elapsed time is equal to a length of a time interval, such that said step of executing is performed at the end of each time interval.

6. The method as recited in claim 1 wherein a flag is used to indicate the end of each said time interval, wherein said flag is for indicating data received during said time interval are to be processed according to said data transport pipeline.

7. The method as recited in claim 1 wherein said real time session is automatically terminated in response to a triggering event.

8. A computer system comprising:

a bus;

a processor coupled to said bus; and

a memory unit coupled to said bus, said processor for executing a method for incrementally transforming in real time a dynamically changing operational database in an analytic application, said method comprising the steps of;

a) initializing a session, said session identified as a real time session, said real time session for repeatedly executing a data transport pipeline of said analytic application; and

b) executing said data transport pipeline at the end of each time interval in a plurality of contiguous time intervals occurring during said real time session, wherein said data transport pipeline is persisted after each execution of said data transport pipeline until said real time session is completed.

9. The computer system of claim 8 wherein said step of executing said data transport pipeline comprises the steps of:

reading data from a source database;

transforming said data according to a transformation operation of said analytic application; and

committing transformed data to a target database.

10. The computer system of claim 9 wherein said transformation operation is a source operation, a target operation, an expression operation, an aggregation operation, a filter operation, a rank operation, an update strategy operation, a sequence operation, a joiner operation, a lookup operation, a stored procedure operation, an external procedure operation, or a normalizer operation.

11. The computer system of claim 9 wherein data in said source database is received from a messaging system coupled to said data transport pipeline.

12. The computer system of claim 8 wherein said step b) of said method comprises the steps of:

determining elapsed time beginning at said step of initializing; and

executing said data transport pipeline when said elapsed time is equal to a length of a time interval, such that said step of executing is performed at the end of each time interval.

13. The computer system of claim 8 wherein a flag is used to indicate the end of each said time interval, wherein said flag is for indicating data received during said time interval are to be processed according to said data transport pipeline.

14. The computer system of claim 8 wherein said real time session is automatically terminated in response to a triggering event.

16. The computer-usable medium of claim 15 wherein said computer-readable program code embodied therein causes a computer system to perform the step of:

reading data from a source database;

transforming said data according to a transformation operation of said analytic application; and

committing transformed data to a target database.

17. The computer-usable medium of claim 16 wherein said transformation operation is a source operation, a target operation, an expression operation, an aggregation operation, a filter operation, a rank operation, an update strategy operation, a sequence operation, a joiner operation, a lookup operation, a stored procedure operation, an external procedure operation, or a normalizer operation.

18. The computer-usable medium of claim 16 wherein data in said source database is received from a messaging system coupled to said data transport pipeline.

19. The computer-usable medium of claim 15 wherein said computer-readable program code embodied therein causes a computer system to perform the steps of:

determining elapsed time beginning at said step of initializing; and

executing said data transport pipeline when said elapsed time is equal to a length of a time interval, such that said step of executing is performed at the end of each time interval.

20. The computer-usable medium of claim 15 wherein a flag is used to indicate the end of each said time interval, wherein said flag is for indicating data received during said time interval are to be processed according to said data transport pipeline.

21. The computer-usable medium of claim 15 wherein said real time session is automatically terminated in response to a triggering event.

15. A computer-usable medium having computer-readable program code embodied therein for causing a computer system to perform a method for incrementally transforming in real time a dynamically changing operational database in an analytic application, said method comprising the steps of:

a) initializing a session, said session identified as a real time session, said real time session for repeatedly executing a data transport pipeline of said analytic application; and

b) executing said data transport pipeline at the end of each time interval in a plurality of contiguous time intervals occurring during said real time session, wherein said data transport pipeline is persisted after each execution of said data transport pipeline until said real time session is completed.