Data aggregation server supporting rapid query response with sparse multi-dimensional data

• US 7,392,248 B2
• Filed: 12/09/2002
• Issued: 06/24/2008
• Est. Priority Date: 08/04/1999
• Status: Expired due to Fees

• Current Assignee: Intellectual Ventures Assets 114 LLC (Intellectual Ventures LLC), MEC Management, LLC (Three Affiliated Tribes of the Fort Berthold Reservation, North Dakota)
• Original Assignee: HyperRoll Israel Limited (Oracle Corporation)
• Inventors: Bakalash, Reuven, Shaked, Guy, Caspi, Joseph
• Primary Examiner(s): Wassum; Luke S.
• Application Number:
  US10/314,868
• Publication Number:
  US 20030200221A1
• Time in Patent Office: 2,024 Days
• Field of Search: 707/5, 707/102
• US Class Current: 1/1
• CPC Class Codes: C03B 2203/36 Dispersion modified fibres, e.g. wavelength or polarisation shifted, flattened or compensating fibres (DSF, DFF, DCF)
• CPC Class Codes: C03B 2205/42 Drawing at high speed, i.e. > 10 m/s
• CPC Class Codes: C03B 2205/55 Cooling or annealing the drawn fibre prior to coating using a series of coolers or heaters
• CPC Class Codes: C03B 37/02718 Thermal treatment of the fibre during the drawing process, e.g. cooling coating C03C25/10
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

Improved method of and apparatus for joining and aggregating data elements integrated within a relational database management system (RDBMS) using a non-relational multi-dimensional data structure (MDD). The improved RDBMS system of the present invention can be used to realize achieving a significant increase in system performance (e.g. deceased access/search time), user flexibility and ease of use. The improved RDBMS system of the present invention can be used to realize an improved Data Warehouse for supporting on-line analytical processing (OLAP) operations or to realize an improved informational database system or the like.

Claims

1. An aggregation server adapted to handle large data volumes and provide rapid query response with sparse data having a tendency to experience data explosion as the number of dimensions increases, said aggregation server comprising:

an aggregation module adapted to aggregate sparse data, said aggregation module operatively coupleable to a relational data store, said aggregation module including a base data loader to load base data from said relational data store, an aggregation engine to aggregate said base data, and a non-relational multi-dimensional data store for storing base data and aggregated data; and

a controller for servicing query statements at any stage of aggregation, said controller in response to determining that data required to service a given query statement has not been pre-aggregated directs said aggregation engine to perform an aggregation on-the-fly to service the given query statement utilizing a query directed rollup in which a multi-dimensional data space is organized as independent segments to permit different rollup orders to service the given query statement, said controller selecting a rollup order based on the given query statement and any partially pre-aggregated data present in said non-relational multi-dimensional data store to service the given query statement.

2. The aggregation server of claim 1, wherein said aggregation server stores data aggregated on-the-fly to build up a store of aggregated data for servicing queries until at least a partial pre-aggregation is completed.

3. The aggregation server of claim 1, wherein said aggregation server selects a rollup order to minimize a processing delay associated with aggregating data on-the-fly to service the given query statement.

4. The aggregation server of claim 1, wherein said aggregation server selects a rollup order to reduce the amount of simultaneously handled data required to aggregate data on-the-fly to service a given query statement.

5. The aggregation server of claim 1, further comprising hierarchy logic, said hierarchy logic resolving hierarchies to reduce data handling performed on sparse data.

6. The aggregation server of claim 1, further comprising a data directory adapted to index sparse data associated with said independent segments for rapid retrieval.

7. The aggregation server of claim 1, wherein after an initial aggregation step said multi-dimensional data space is organized with said independent segments comprising independent multi-dimensional cubes.

8. The aggregation server of claim 1, wherein said aggregation server has a mode of operation in which data is aggregated on-the-fly after a partial pre-aggregation of data is performed.

9. The aggregation server of claim 1, wherein said aggregation server is adapted to perform an aggregation on-the-fly utilizing independent segments corresponding to slices of the multidimensional data that are thin compared with the entire multidimensional data such that data handling required to aggregate data on-the-fly for a given query statement is reduced.

10. The aggregation server of claim 1, wherein said controller is integrated within said aggregation module.

11. The aggregation server of claim 10, wherein said aggregation module comprises:

a first data communication interface for said aggregation module to interface with said relational data store and receive said base data loaded into said aggregation module from said relational data store; and

a second data communication interface for said aggregation module to interface with one or more clients having a query interface for converting queries from end-users into said query statements.

12. The aggregation server of claim 11, wherein said first and second data communication interfaces allow said aggregation module to be installed into a software-based computing environment in a cartridge-like manner.

13. The aggregation server of claim 1, wherein said aggregation module comprises:

a first data communication interface for said aggregation module to interface with said relational data store and receive said base data loaded into said aggregation module from said relational data store; and

a second data communication interface for said aggregation module to interface with one or more clients having a query interface for converting queries from end-users into said query statements.

14. An aggregation server adapted to handle large data volumes and provide rapid query response for sparse data having a tendency to experience data explosion as the number of dimensions increases, said aggregation server comprising:

an aggregation module adapted to aggregate sparse data, said aggregation module operatively coupleable to a relational data store, said aggregation module including a base data loader to load base data from said relational data store, an aggregation engine to aggregate said base data, and a non-relational multi-dimensional data store for storing base data and aggregated data; and

a controller for servicing query statements at any stage of aggregation, said controller in response to determining that data required to service a given query statement has not been pre-aggregated directs said aggregation engine to perform an aggregation on-the-fly to service the given query statement utilizing a query directed rollup in which a multi-dimensional data space is organized as independent segments to permit different rollup orders to service the given query statement, said controller selecting a rollup order based on the given query statement and any partially pre-aggregated data present in said non-relational multi-dimensional data store to service the given query statement;

wherein each of said independent segments represents only a comparatively thin slice of the total multi-dimensional data space to reduce the amount of data that must be handled to perform an aggregation in selected dimensions with respect to an individual segment.

15. The aggregation server of claim 14, wherein said aggregation server selects a rollup order to minimize a processing delay associated with aggregating data on-the-fly to service the given query statement.

16. The aggregation server of claim 14, wherein said aggregation server selects a rollup order to reduce the amount of simultaneously handled data required to aggregate data on-the-fly to service a given query statement.

17. The aggregation server of claim 16, wherein after an initial aggregation step said independent segments comprise independent multi-dimensional cubes.

18. The aggregation server of claim 16, wherein said aggregation server has a mode of operation in which data is aggregated on-the-fly after a partial pre-aggregation of data is performed.

19. The aggregation server of claim 14, wherein said aggregation server stores data aggregated on-the-fly to build up a store of aggregated data in said non-relational multi-dimensional data store to service subsequent query statements.

20. The aggregation server of claim 14, wherein said aggregation server builds up said store for a system requiring only partial pre-aggregation.

21. The aggregation server of claim 14, wherein after an initial aggregation step said independent segments comprise independent multi-dimensional cubes.

22. The aggregation server of claim 14, wherein said aggregation server has a mode of operation in which data is aggregated on-the-fly after a partial pre-aggregation of data is performed.

23. The aggregation server of claim 22, wherein said aggregation server selects a rollup order to minimize a processing delay associated with aggregating data on-the-fly to service the given query statement.

24. The aggregation server of claim 22, wherein said aggregation server selects a rollup order to reduce the amount of simultaneously handled data required to aggregate data on-the-fly to service a given query statement.

25. The aggregation server of claim 14, wherein said aggregation server stores data aggregated on-the-fly to build up a store of aggregated data in said non-relational multi-dimensional data store to service subsequent query statements.

26. The aggregation server of claim 25, wherein said aggregation server builds up said store for a system requiring only partial pre-aggregation.

27. The aggregation server of claim 14, wherein said controller is integrated within said aggregation module.

28. The aggregation server of claim 27, wherein said aggregation module comprises:

a first data communication interface for said aggregation module to interface with said relational data store and receive said base data loaded into said aggregation module from said relational data store; and

a second data communication interface for said aggregation module to interface with one or more clients having a query interface for converting queries from end-users into said query statements.

29. The aggregation server of claim 28 wherein said first and second data communication interfaces allow said aggregation module to be installed into a software-based computing environment in a cartridge-like manner.

30. The aggregation server of claim 14, wherein said aggregation module comprises:

a first data communication interface for said aggregation module to interface with said relational data store and receive said base data loaded into said aggregation module from said relational data store; and

a second data communication interface for said aggregation module to interface with one or more clients having a query interface for converting queries from end-users into said query statements.

31. An aggregation server adapted to handle large data volumes and provide rapid query response for sparse data having a tendency to experience data explosion as the number of dimensions increases, said aggregation server comprising:

an aggregation module adapted to aggregate sparse data, said aggregation module operatively coupled to a relational data store, said aggregation module including a base data loader to load base data, an aggregation engine to aggregate said base data, and a non-relational multi-dimensional data store for storing base data and aggregated data;

hierarchy logic, said hierarchy logic resolving hierarchies to reduce data handling performed on sparse data;

a controller for servicing query statements at any stage of aggregation, said controller in response to determining that data required to service a given query statement has not been pre-aggregated directs said aggregation engine to perform an aggregation on-the-fly to service the given query statement utilizing a query directed rollup in which a multi-dimensional data space is organized as independent segments to permit different rollup orders to be utilized to service the given query statement, said controller selecting a rollup order based on the given query statement and any partially pre-aggregated data present in said non-relational multi-dimensional data store to service the given query statement; and

a data directory adapted to index sparse data associated with said independent segments for rapid retrieval;

wherein each of said independent segments represents only a comparatively thin slice of the total multi-dimensional data space to reduce the amount of data that must be handled to perform an aggregation in selected dimensions with respect to an individual segment.

32. The aggregation server of claim 31, wherein said controller is integrated within said aggregation module.

33. The aggregation server of claim 32, wherein said aggregation module comprises:

a first data communication interface for said aggregation module to interface with said relational data store and receive said base data loaded into said aggregation module from said relational data store; and

a second data communication interface for said aggregation module to interface with one or more clients having a query interface for converting queries from end-users into said query statements.

34. The aggregation server of claim 33, wherein said first and second data communication interfaces allow said aggregation module to be installed into a software-based computing environment in a cartridge-like manner.

35. The aggregation server of claim 31, wherein said aggregation module comprises:

a first data communication interface for said aggregation module to interface with said relational data store and receive said base data loaded into said aggregation module from said relational data store; and

a second data communication interface for said aggregation module to interface with one or more clients having a query interface for converting queries from end-users into said query statements.