Method of and apparatus for data aggregation utilizing a multidimensional database and multi-stage data aggregation operations

• US 20050060325A1
• Filed: 05/19/2004
• Published: 03/17/2005
• Est. Priority Date: 02/28/2000
• Status: Abandoned Application

• Current Assignee: MEC Management, LLC (Three Affiliated Tribes of the Fort Berthold Reservation, North Dakota)
• Original Assignee: MEC Management, LLC (Three Affiliated Tribes of the Fort Berthold Reservation, North Dakota)
• Inventors: Bakalash, Reuven, Shaked, Guy, Caspi, Joseph
• Application Number:
  US10/849,501
• Publication Number:
  US 20050060325A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24539 using cached or materialised query results
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: Y10S 707/954 Relational
• CPC Class Codes: Y10S 707/957 Multidimensional
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

An improved method of and apparatus for aggregating data including a scalable multi-dimensional database (MDDB) storing multidimensional data logically organized along N dimensions and a high performance aggregation engine that performs multi-stage data aggregation operations on the multidimensional data. A first stage of such data aggregation operations is performed along a first dimension of the N dimensions; and a second stage of such data aggregation operations is performed for a given slice in the first dimension along another dimension of the N dimensions. Such multi-stage data aggregation operations achieve a significant increase in system performance (e.g. deceased access/search time). The MDDB and high performance aggregation engine of the present invention may be integrated into a standalone data aggregation server supporting an OLAP system (one or more OLAP servers and clients), or may be integrated into a database management system (DBMS), thus achieving improved user flexibility and ease of use. The improved DBMS system of the present invention can be used to realize an improved Data Warehouse for supporting on-line analytical processing (OLAP) operations or to realize an improved informational database system, operational database system, or the like.

Claims

1-151. -151. (canceled)

152. A data aggregation module comprising an aggregation engine and a multidimensional datastore, wherein the multidimensional datastore stores multidimensional data logically organized along N dimensions, and wherein the aggregation engine performs data aggregation operations on said multidimensional data by:

(a) performing a first stage of data aggregation operations along a first dimension of said N dimensions; and

(b) performing a second stage of aggregation operations for a given slice in the first dimension along another dimension of said N dimensions.

153. The data aggregation module of claim 152, wherein the second stage of aggregation operations involves recursive data aggregation operations for slices in said N dimensions.

154. The data aggregation module of claim 152, further comprising a data loading mechanism for loading data from a database, and a storage handler for storing in the multidimensional datastore the data loaded from the database and the aggregated data generated by the aggregation engine.

155. The data aggregation module of claim 152, further comprising an interface for receiving queries generated by a requester, and control logic that, upon determining that the multi-dimensional datastore does not contain data required to service a given query, controls the aggregation engine to generate aggregated data required to service the given query and controls the aggregation module to return the aggregated data to the requestor.

156. The data aggregation module of claim 152, wherein said interface interfaces to an OLAP server (comprising OLAP analysis logic and presentation logic) and client machines operably coupled to the OLAP server to provide user-directed OLAP analysis, to thereby realize an OLAP system capable of performing data aggregation operations on the data, and storing and managing such data.

157. The data aggregation module of claim 154, wherein said client machines include a web-browser-based user interface that enables user access to the OLAP server.

158. The data aggregation module of claim 152, integral to a DBMS to thereby realize an improved DBMS capable of performing data aggregation operations on the data, and storing and managing such data.

159. The data aggregation module of claim 152, integral to a DBMS operably coupled to a plurality of client machines over a network, to thereby realize a data warehouse capable of performing data aggregation operations on the data, and storing and managing such data.

160. The data aggregation module of claim 152, wherein said interface implements a standard protocol for accessing data.

161. The data aggregation module of claim 160, wherein the standard protocol comprises one of OLDB, OLE-DB, ODBC, SQL, and JDBC.

162. The data aggregation module of claim 152, wherein the aggregation engine stores the resultant data of aggregration operations for the given slice as a record in a data file, wherein location of the record in the data file is stored in a directory.

163. The data aggregation module of claim 162, wherein the directory stores, for a given record, a start address and end address of the record and a physical address of the data file.

164. A method for data aggregation for use with a multidimensional datastore that stores multidimensional data logically organized along N dimensions, the method comprising the steps of:

(a) performing a first stage of data aggregation operations along a first dimension of said N dimensions; and

(b) performing a second stage of aggregation operations for a given slice in the first dimension along another dimension of said N dimensions; and

(c) storing resultant data in said multidimensional datastore.

165. The method of claim 164, wherein step (b) involves recursive data aggregation operations for slices in said N dimensions.

166. The method of claim 164, further comprising the step of loading data from a database, and storing the data loaded from the database in said multidimensional datastore.

167. The method of claim 164, further comprising the steps of:

receiving queries generated by a requester; and

upon determining that the multi-dimensional datastore does not contain data required to service a given query, controlling the aggregation engine to generate aggregated data required to service the given query and returning the aggregated data to the requestor.

168. The method of claim 167, wherein the queries are received over an interface that implements a standard protocol for accessing data.

169. The method of claim 168, wherein the standard protocol comprises one of OLDB, OLE-DB, ODBC, SQL, and JDBC.

170. The method of claim 164, wherein the resultant data of aggregration operations for the given slice is stored as a record in a data file, wherein location of the record in the data file is stored in a directory.

171. The method of claim 170, wherein the directory stores, for a given record, a start address and end address of the record and a physical address of the data file.