System with a data aggregation module generating aggregated data for responding to OLAP analysis queries in a user transparent manner
First Claim
1. An on-line analytical processing (OLAP) system, comprising:
- a data aggregation module for servicing queries directed towards high dimensionality sparse data sets, said data aggregation module including;
(1) a multi-dimensional datastore,(2) an hierarchy transformation module for receiving an original hierarchical database structure of said OLAP system defining parent-child relationships of levels within dimensions and converting said hierarchical database structure into a functionally equivalent hierarchical database structure optimized for rapid aggregation, storage and retrieval of sparse data; and
(3) an aggregation engine for aggregating said large data sets, including sparse data, according to said functionally equivalent hierarchy.
9 Assignments
0 Petitions
Accused Products
Abstract
A system for supporting OLAP analysis over a network. The system comprises an OLAP server for enabling an OLAP user to perform OLAP analysis via interaction with a client machine on the network. The system also includes a data aggregation module comprising a multi-dimensional datastore, an aggregation engine integrated with the multi-dimensional datastore, and a first interface for loading base data from a data source to the aggregation engine. The aggregation engine performs data aggregation operations on loaded base data, generates aggregated data from the base data, and stores the aggregated data in the multi-dimensional datastore. A second interface receives requests for OLAP analysis from the OLAP server, accesses the aggregation engine to retrieve from the multi-dimensional datastore, aggregated data corresponding to requests, and communicates the retrieved aggregated data to the OLAP server for query servicing, in a manner transparent to the OLAP user.
-
Citations
24 Claims
-
1. An on-line analytical processing (OLAP) system, comprising:
a data aggregation module for servicing queries directed towards high dimensionality sparse data sets, said data aggregation module including; (1) a multi-dimensional datastore, (2) an hierarchy transformation module for receiving an original hierarchical database structure of said OLAP system defining parent-child relationships of levels within dimensions and converting said hierarchical database structure into a functionally equivalent hierarchical database structure optimized for rapid aggregation, storage and retrieval of sparse data; and (3) an aggregation engine for aggregating said large data sets, including sparse data, according to said functionally equivalent hierarchy. - View Dependent Claims (2, 3, 4)
-
5. An on-line analytical processing (OLAP) system, comprising:
a data aggregation module to service queries directed towards high dimensionality sparse data sets, said data aggregation module including; (1) a multi-dimensional datastore, (2) an hierarchy transformation module a for converting an original hierarchical database structure of said OLAP system into a functionally equivalent hierarchical database structure optimized for large data sets, including sparse data having a low density of data points, said functionally equivalent hierarchical database structure being used to perform data indexing and aggregation operations such that groups of related data points at different stages of an aggregation process are organized into sub-units of memory storage that are individually accessible from a memory space of said multi-dimensional datastore; and (3) an aggregation engine having a mode of operation in which data is aggregated on-the-fly to service a query statement by identifying a set of said sub-units of memory storage having partially pre-aggregated data to perform an aggregation on-the-fly, loading said set of sub-units of memory storage into a main memory, and performing an aggregation on-the-fly to service the query statement. - View Dependent Claims (6, 7, 8, 9, 10, 11, 12, 13, 14)
-
15. An on-line analytical processing (OLAP) system, comprising:
a data aggregation module for use as a complimentary accelerator with an OLAP server to improve the servicing of high dimensionality sparse data sets, said data aggregation module including; (1) a multi-dimensional datastore, (2) a load and indexing module for organizing said multi-dimensional datastore into records capable of being independently loaded into a main memory with each record having a size that is small compared with a maximum size of said multi-dimensional data store and each record corresponding to an autonomic segment, each autonomic segment storing base data or aggregated data with autonomic segments corresponding to partially pre-aggregated data capable of being rolled up in different sequences; and (3) an aggregation engine performing data aggregation utilizing said autonomic segments to limit the amount of simultaneously handled data, said aggregation module having a mode of operation in which data is aggregated on-the-fly to service a given query statement by determining a rollup order of a set of records having autonomic segments capable of being rolled up to service the given query statement, loading into a main memory said set of records, and performing a data aggregation operation to service the query request. - View Dependent Claims (16, 17, 18, 19, 20)
-
21. A method for accelerating the servicing of high dimensionality sparse data sets in an on-analytical processing (OLAP) system, comprising:
-
receiving an original hierarchical database structure of said OLAP system; and converting said original hierarchical database structure into a functionally equivalent hierarchical database structure optimized for performing data storage and aggregation operations on large data sets, including sparse data. - View Dependent Claims (22, 23)
-
-
24. A method for accelerating the servicing of high dimensionality sparse data sets in an on-analytical processing (OLAP) system, comprising:
-
receiving an hierarchical database structure of said OLAP system; organizing an aggregation process for autonomic segments capable of being rolled up in different rollup orders; performing a partial pre-aggregation according to said aggregation process to generate an initial set of autonomic segments stored as data records in a memory space of a multi-dimensional datastore; in response to receiving a query statement requiring data that has not been pre-aggregated, determining a rollup order of a subset of autonomic segments to service the query statement on-the-fly based on said initial set of autonomic segments; loading into a main memory records corresponding to said subset of said initial set of autonomic records require to perform said rollup; and performing said rollup to aggregate data on-the-fly to service the query statement.
-
Specification