DISTRIBUTED, REAL-TIME ONLINE ANALYTICAL PROCESSING (OLAP)

• US 20140101093A1
• Filed: 06/26/2013
• Published: 04/10/2014
• Est. Priority Date: 06/26/2012
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: Medio Systems, Inc. (Nokia Corporation)
• Inventors: Lanphear, Damon, Biswas, Prabuddha
• Granted Patent: US 9,053,160 B2
• Time in Patent Office: Days
• US Class Current: 707/602
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses

Abstract

Source data of an event stream is parsed and supplemented with additional data from reference data sources, producing an enriched event stream from the parsed event stream data. The data records of the enriched event stream are partitioned into data fields designated as a dimension partition and a metric partition, which are partitioned into sub-dimension projections mapped to a plurality of storage keys, such that each of the storage keys includes one or more placeholder wildcard values and each of the storage keys is stored into a database of the computer system by the computer processor. The stored storage keys are then aggregated onto a two-dimensional coordinate vector such that, if the computer processor identifies a permuted storage key having metric values for which a corresponding storage key already exists in the database, then the computer processor aggregates the metric values of the identified storage key with the metric values of the corresponding storage key, and if the computer processor does not identify the permuted storage key as having a corresponding storage key that already exists in the database, then the computer processor writes the metric values of the permuted storage key into the database, comprising initial values for the key combination of dimension values.

Claims

1. A method of online analytical processing (OLAP) of a computer system, the method comprising:

enriching a data record read from a source database of the computer system by a computer processor with one or more enriched data fields comprising data read from a reference database by the computer processor, wherein the data record includes multiple data fields, such that the enriched data record includes one or more of the multiple data fields of the data record and one or more of the enriched data fields;

partitioning the multiple data fields of the enriched data record into a dimension partition and a metric partition;

permuting the dimension partition and the metric partition of the enriched data record into sub-dimension projections that are mapped to a plurality of storage keys, such that each of the storage keys includes one or more placeholder wildcard values and each of the storage keys is stored into a database of the computer system by the computer processor;

aggregating the storage keys of a permuted sub-dimension projections onto a two-dimensional coordinate vector such that, if the computer processor identifies a permuted storage key having metric values for which a corresponding storage key already exists in the database, then the computer processor aggregates the metric values of the identified storage key with the metric values of the corresponding storage key, and if the computer processor does not identify the permuted storage key as having a corresponding storage key that already exists in the database, then the computer processor writes the metric values of the permuted storage key into the database, comprising initial values for the key combination of dimension values.

2. The method as in claim 1, further comprising:

storing a plurality of the aggregated storage keys across two or more host computer processors, such that the stored aggregated storage keys comprise a distributed associative array.

3. The method as in claim 1, further comprising:

storing the aggregated storage keys of the permuted sub-dimension projection on a persistent medium of the computer processor; and

storing the aggregated storage keys of the permuted sub-dimension projection in cache memory of the computer processor.

4. A computer system for online analytical processing (OLAP) of event stream data, the computer system comprising:

a network communications block adapted for communications by the computer system with a computer network;

a computer processor configured to execute program instructions for

enriching a data record read from a source database of the computer system by the computer processor with one or more enriched data fields comprising data read from a reference database by the computer processor, wherein the data record includes multiple data fields, such that the enriched data record includes one or more of the multiple data fields of the data record and one or more of the enriched data fields,

partitioning the multiple data fields of the enriched data record into a dimension partition and a metric partition,

permuting the dimension partition and the metric partition of the enriched data record into sub-dimension projections that are mapped to a plurality of storage keys, such that each of the storage keys includes one or more placeholder wildcard values and each of the storage keys is stored into a database of the computer system by the computer processor, and

aggregating the storage keys of a permuted sub-dimension projections onto a two-dimensional coordinate vector such that, if the computer processor identifies a permuted storage key having metric values for which a corresponding storage key already exists in the database, then the computer processor aggregates the metric values of the identified storage key with the metric values of the corresponding storage key, and if the computer processor does not identify the permuted storage key as having a corresponding storage key that already exists in the database, then the computer processor writes the metric values of the permuted storage key into the database, comprising initial values for the key combination of dimension values.

5. The computer system as in claim 4, wherein the computer processor is further configured to execute program instructions for storing a plurality of the aggregated storage keys across two or more host computer processors, such that the stored aggregated storage keys comprise a distributed associative array.

6. The computer system as in claim 4, wherein the computer processor is further configured to execute program instructions for storing the aggregated storage keys of the permuted sub-dimension projection on a persistent medium of the computer processor, and for storing the aggregated storage keys of the permuted sub-dimension projection in cache memory of the computer processor.