Technique for supplying a data warehouse whilst ensuring a consistent data view

• US 7,599,968 B2
• Filed: 03/17/2006
• Issued: 10/06/2009
• Est. Priority Date: 01/20/2006
• Status: Active Grant

• Current Assignee: UBS Business Solutions AG (UBS Ag)
• Original Assignee: UBS Ag
• Inventors: Pagnussat, Daniel, Loacker, Hansbeat
• Primary Examiner(s): Vital, Pierre M
• Assistant Examiner(s): LIAO, JASON G
• Application Number:
  US11/377,362
• Publication Number:
  US 20070174236A1
• Time in Patent Office: 1,299 Days
• Field of Search: 707/2, 707/201, 707/202, 707/204
• US Class Current: 1/1
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users

Abstract

A technique is described for supplying a data warehouse while ensuring a consistent data view. The system comprises an interface for receiving data elements which are assigned to individual transactions and require processing before delivery into the data warehouse. Also provided is a process control component for analyzing the data elements with regard to the respective assignment to particular processing operations and for controlling the distribution of the data elements to associated processing branches. At least one first buffer storage is disposed downstream from the process control component, for the temporary storage of the distributed data elements, time-control information being assigned to each stored data element. The processing of the data elements read-out from the at least one buffer storage is effected in two or more parallel processing branches, a processing of the data elements that is non-dependent on the time control information being effected in a first mode, and a prioritizing processing of particular data elements that is based on the time control information being provided in a second mode. A transfer component, which transfers the processed data elements to the data warehouse, is disposed down-stream from the processing branches.

Claims

1. A system for supplying a data warehouse with data generated in the context of transaction-based Online Transaction Processing (OLTP), whilst ensuring a consistent data view per time threshold, comprising:

at least one processing unit for accessing memory and executing computer executable instructions;

an interface having computer executable instructions stored in memory that when executed, are configured to receive data elements which are assigned to individual OLTP transactions and require processing before delivery into the data warehouse;

a process control component having computer executable instructions stored in memory that when executed, are configured to;

determine the respective assignment of a data element to a particular processing operation;

control the distribution of the data elements to associated processing branches; and

distribute a plurality of data elements assigned to a single OLTP transaction to different processing branches;

at least one first buffer storage provided downstream from the process control component, configured to temporarily store the distributed data elements, wherein time-control information is assigned to each stored data element;

two or more processing branches having computer executable instructions stored in memory in which there is respectively provided a processing component having computer executable instructions stored in memory that when executed, are configured to process the data elements read-out from the at least one first buffer storage, the processing branches having different throughput times;

at least one mode control component having computer executable instructions stored in memory that when executed, are configured to, in a first mode, initiate a processing of the data elements that is non-dependent on the time control information and, in a second mode, processes particular data elements with time control information that is older than a predefined time threshold, and holding in abeyance the processing of particular data elements with time control information younger than the predefined time threshold until the predefined time threshold is reached such that the consistent data view per time threshold is maintained in the second mode of operation; and

a transfer component having computer executable instructions stored in memory, which is disposed downstream from the processing branches, the computer executable instructions of the transfer component, when executed, being configured to transfer the processed data elements to the data warehouse.

2. The system according to claim 1, further comprising at least one second buffer storage disposed downstream from the processing branches, configured to temporarily store the processed data elements.

3. The system according to claim 1, wherein a separate first and/or a separate second buffer storage is assigned to each processing branch.

4. The system according to claim 1, wherein the first and/or second buffer storage is realized as a queue.

5. The system according to claim 1, wherein the first and/or second buffer storage is realized as a batch file.

6. The system according to claim 1, wherein the first and/or second buffer storage is realized as an interface database.

7. The system according to claim 1, wherein a common first and/or second buffer storage is assigned to a plurality of processing threads per processing branch.

8. The system according to claim 1, wherein there is assigned to the data elements in at least one first and/or second buffer storage status information which indicates the read-out status of the data elements.

9. The system according to claim 1, wherein there is assigned to the data elements in at least one first and/or second buffer storage status information which indicates the processing status of the data elements.

10. The system according to claim 1, wherein numerical identifiers are assigned to the transactions and/or data elements, and the data elements, sorted according to the numerical identifiers, are contained in the at least one first and/or second buffer storage.

11. The system according to claim 1, wherein the process control component is further configured to assign the time control information to the data elements received via the interface.

12. The system according to claim 1, further comprising at least one transaction control component having computer executable instructions stored in memory and disposed upstream from the interface, the computer executable instructions of the at least one transaction control component, when executed, being configured to control transactions in real time.

13. The system according to claim 12, wherein the process control component is further configured to communicate with the transaction control component in real time for the purpose of validating the transactions and/or the data elements.

14. The system according to claim 1, further comprising:

a timing component having computer executable instructions stored in memory that when executed, are configured to provide time control information.

15. A method for supplying a data warehouse with data generated in the context of transaction-based Online Transaction Processing (OLTP), whilst ensuring a consistent data view per time threshold, comprising:

receiving data elements which are assigned to individual OLTP transactions and require processing before delivery into the data warehouse;

determining the respective assignment of a data element to a particular processing operation and controlling of the distribution of the data elements to associated processing branches with different throughput times, wherein the distribution is performed such that a plurality of data elements assigned to a single OLTP transaction are distributed to different processing branches;

temporary storage of the distributed data elements, wherein time-control information is assigned to each stored data element;

processing of the stored data elements in the respectively associated processing branch, wherein processing of the data elements that is non-dependent on the time control information is effected in a first mode and processing particular data elements with time control information that is older than a predefined time threshold, and holding in abeyance the processing of particular data elements with time control information younger than the predefined time threshold until the predefined time threshold is reached such that the consistent data view per time threshold is maintained in a second mode; and

transferring of the processed data elements to the data warehouse.

16. The method according to claim 15, wherein in the second mode there are processed with priority those data elements whose assigned time control information does not exceed a time threshold.

17. The method according to claim 15, wherein a switchover from the second mode to the first mode is effected to ensure continuous supplying of the data warehouse.

18. The method according to claim 15, wherein the switchover between the first mode and the second mode is effected in a time-controlled manner.

19. The method according to claim 15, wherein the processing in the processing branches is effected according to the principle of Straight-Through Processing.

20. The method according to claim 15, wherein the processing of the data elements comprises posting operations.

21. A plurality of computers, at least one computer of the plurality of computers comprising a memory for storing computer executable instructions and a processing unit for accessing memory and executing computer executable instructions, the computer executable instructions comprising a computer program product having program code means for implementing the method according to claim 15.

22. A computer readable medium that stores the computer program product according to claim 21.