Data aggregation module supporting dynamic query responsive aggregation during the servicing of database query requests provided by one or more client machines

• US 8,788,453 B2
• Filed: 10/17/2011
• Issued: 07/22/2014
• Est. Priority Date: 08/04/1999
• Status: Expired due to Term

• Current Assignee: MEC Management, LLC (Three Affiliated Tribes of the Fort Berthold Reservation, North Dakota)
• Original Assignee: Yanicklo Technology LLC (Intellectual Ventures LLC)
• Inventors: Bakalash, Reuven, Shaked, Guy, Caspi, Joseph
• Primary Examiner(s): WILSON, KIMBERLY LOVEL
• Application Number:
  US13/275,242
• Publication Number:
  US 20120089563A1
• Time in Patent Office: 1,009 Days
• Field of Search: 707/603
• US Class Current: 707/603
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

Improved method of and apparatus for aggregating data elements in multidimensional databases (MDDB). In the preferred embodiment, the apparatus is realized in the form of a high-performance stand-alone (i.e. external) aggregation server which can be plugged-into conventional MOLAP systems to achieve significant improvements in system performance. In accordance with the principles of the present invention, the stand-alone aggregation server contains a scalable MDDB and a high-performance aggregation engine that are integrated into the modular architecture of the aggregation server. The stand-alone aggregation server of the present invention can uniformly distribute data elements among a plurality of processors, for balanced loading and processing, and therefore is highly scalable.

Claims

1. A method comprising:

by a computer system comprising computer hardware;

receiving a first multidimensional cube of data, wherein the first multidimensional cube of data is sparse;

initiating a first aggregation process;

conducting the first aggregation process;

receiving a query after the first aggregation process has been initiated and while the first aggregation process is being conducted;

determining a sequence for storing the first multidimensional cube of data in a multidimensional database; and

at least partly in response to the query, performing a query directed, on-the-fly roll-up process while the first aggregation process is still being conducted,

extracting, based at least in part on the determined sequence, a first slice of data for storage in the multidimensional database from the first multidimensional cube of data,

wherein the first slice of data is a second multidimensional cube of data that is a subset of the first multidimensional cube of data.

2. The method of claim 1, wherein a plurality of non-null data points are stored in a first record in the multidimensional database indexed in an ascending order along an aggregation direction.

3. The method of claim 2, further comprising extracting, based at least in part on the determined sequence, a second slice of data for storage from the first multidimensional cube of data.

4. The method of claim 3, wherein non-null data points within the second slice are stored in a second record in the multidimensional database.

5. The method of claim 4, wherein a length of the first record and second record are different and based at least in part on the number of non-null data points that were stored.

6. A computing system comprising:

a processor comprising hardware; and

a computer readable medium storing machine-executable instructions including one or more modules configured for execution by the processor in order to cause the computing system to;

receive a first multidimensional cube of data, wherein the first multidimensional cube of data is sparse;

initiate a first on-the-fly aggregation process;

conduct the first on-the-fly aggregation process;

determine a sequence for storing the first multidimensional cube of data in a multidimensional database;

receive a query after the first on-the-fly aggregation process has been initiated and while the first a on-the-fly aggregation process is being conducted;

determine a sequence for storing the first multidimensional cube of data in a multidimensional database;

at least partly in response to the received query, perform a query directed roll-up process while the first on-the-fly aggregation process is still being conducted; and

extract, based at least in part on the determined sequence, a first slice of data from the first multidimensional cube of data, wherein the first slice of data is a second multidimensional cube of data that is a subset of the first multidimensional cube of data; and

store the extracted first slice of data in the multidimensional database.

7. The system of claim 6, wherein a plurality of non-null data points are stored in a first record in the multidimensional database indexed in an ascending order along an aggregation direction.

8. The system of claim 7, wherein the instructions are further configured to cause the computing system to extract, based at least in part on the determined sequence, a second slice of data for storage from the first multidimensional cube of data.

9. The system of claim 8, wherein non-null data points within the second slice are stored in a second record in the multidimensional database.

10. The system of claim 9, wherein a length of the first record and second record are different and based at least in part on the number of non-null data points that were stored.

11. A non-transitory computer-readable medium encoded with instructions thereon, wherein the instructions are readable by a computing system in order to cause the computing system to perform operations comprising:

receiving a first multidimensional cube of data, wherein the first multidimensional cube of data is sparse;

initiating a first aggregation process;

conducting the first on-the-fly aggregation process;

determining a sequence for storing the first multidimensional cube of data in a multidimensional database;

receiving a query after the first on-the-fly aggregation process has been initiated and while the first aggregation process is being conducted;

at least partly in response to the received query, performing a query directed roll-up process while the first on-the-fly aggregation process is still being conducted;

extracting, based at least in part on the determined sequence, a first slice of data from the first multidimensional cube of data,

wherein the first slice of data is a second multidimensional cube of data that is a subset of the first multidimensional cube of data; and

storing the extracted first slice of data in the multidimensional database.

12. The non-transitory computer-readable medium of claim 11, wherein a plurality of non-null data points are stored in a first record in the multidimensional database indexed in an ascending order along an aggregation direction.

13. The non-transitory computer-readable medium of claim 12, wherein the instructions are further encoded to cause the computing system to perform operations comprising extracting, based at least in part on the determined sequence, a second slice of data for storage from the first multidimensional cube of data.

14. The non-transitory computer-readable medium of claim 13, wherein non-null data points within the second slice are stored in a second record in the multidimensional database.

15. The non-transitory computer-readable medium of claim 14, wherein a length of the first record and second record are different and based at least in part on the number of non-null data points that were stored.