Concurrent OLAP-Oriented Database Query Processing Method

• US 20130275364A1
• Filed: 05/16/2012
• Published: 10/17/2013
• Est. Priority Date: 04/17/2012
• Status: Active Grant

• Current Assignee: Renmin University of China
• Original Assignee: Renmin University of China
• Inventors: Wang, Shan, Zhang, Yan-Song
• Granted Patent: US 8,762,407 B2
• Time in Patent Office: Days
• US Class Current: 707/602
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: H04L 9/50 using hash chains, e.g. blockchains or hash trees

Abstract

A concurrent on-line analytical processing (OLAP)-oriented database query processing method is described, for performing, on the basis of predicate vector-based memory OLAP star-join optimization, concurrent OLAP query processing based on a batch query predicate vector bit operation. The concurrent query processing optimization technology is implemented for I/O performance and parallel OLAP processing performance in a database management system, and setting of concurrent OLAP processing load in an optimized way catering to the I/O performance is supported, thereby improving predictable processing performance oriented to diversified OLAP queries and implementing concurrent query star-join bitmap filtering processing based on predicate vector arrays.

Claims

1. A concurrent on-line analytical processing (OLAP)-oriented database query processing method, for performing, on the basis of predicate vector-based memory OLAP star-join optimization, concurrent OLAP query processing based on a batch query predicate vector bit operation, wherein:

the predicate vector-based memory OLAP star-join optimization comprises the following steps;

loading a dimension table in a memory;

vectorizing a predicate operation result in a query;

completing a bitmap filtering operation for a multi-table star-join through a bit operation on predicate vectors of multiple query-related dimension tables, so as to select a fact table tuple satisfying a condition; and

mapping a dimension table primary key to a memory offset address of a memory column-stored dimensional attribute so as to implement direct access to a dimension table group-by attribute value; and

the concurrent OLAP query processing based on the batch query predicate vector bit operation comprises the following steps;

grouping concurrent queries in a designated time window, and performing the queries in a batch processing manner;

using a multi-bit predicate vector to store a predicate operation result of a concurrent query group, wherein each bit of each data item of the predicate vector corresponds to a designated query predicate operation result flag bit;

performing a bit operation in a unit of the multi-bit predicate vector when a star-join bitmap filtering operation is performed, wherein a position of 1 in an operation result represents a query number satisfying predicate conditions in all dimension tables; and

invoking a HASH aggregate processing thread corresponding to 1 in bits of a predicate vector result to complete iterative aggregate calculation on a current fact table tuple.

2. The concurrent OLAP-oriented database query processing method according to claim 1, wherein:

in the case of insufficient memory capacity, the dimension table is memorized according to the following priority;

group-by attributes→

predicate operation attributes→

all dimensional attributes.

3. The concurrent OLAP-oriented database query processing method according to claim 2, wherein:

the group-by attributes and the predicate operation attributes are loaded in an incremental manner in query processing, and in memory dimensional attribute management, a dimensional attribute column with low access frequency is eliminated according to a least recently used (LRU) policy to accommodate a new dimensional attribute column.

4. The concurrent OLAP-oriented database query processing method according to claim 1, wherein:

the dimension table, after being loaded into the memory, is converted into a memory dimensional attribute arrays, array subscripts are one-to-one corresponding to dimension table primary key values, and a foreign key of a fact table is directly mapped to a subscript of the dimensional attribute column; and

a predicate vector is preset for each dimension table, content of the predicate vector is updated when a predicate operation is performed in each query, and a satisfaction state of each dimension table tuple with a current query predicate is marked by 1 and 0.

5. The concurrent OLAP-oriented database query processing method according to claim 4, wherein:

the foreign key of the fact table is mapped to a designated bit in a predicate vector bitmap, and then a bit operation is performed on bit data of multiple predicate vectors, so as to complete filtering determination on a star-join result of multiple dimension tables.

6. The concurrent OLAP-oriented database query processing method according to claim 1, wherein:

a multi-bit predicate vector is preset according to the number of concurrent query tasks, wherein a position of a bit in the predicate vector represents a query number of the concurrent query, and after a predicate operation is completed for a query in the query group, a predicate operation result is recorded at a position corresponding to the query number in each multi-bit vector unit of a predicate vector array.

7. The concurrent OLAP-oriented database query processing method according to claim 1, wherein:

for each sequentially scanned fact table tuple, a designated unit in a predicate vector array of a corresponding dimension table is located directly according to a foreign key value, and then a bit operation is performed on data of the designated unit in the predicate vector array of each dimension table to obtain a global concurrent query join filtering result.

8. The concurrent OLAP-oriented database query processing method according to claim 7, wherein:

HASH aggregate processing threads of corresponding queries are invoked according to positions of 1 in a bit string of the global concurrent query join filtering result, and parallel HASH aggregate calculation is performed on the current fact table tuple according to a dimension table group-by attribute value extracted in real time.

9. The concurrent OLAP-oriented database query processing method according to claim 8, wherein:

in the parallel HASH aggregate calculation, a HASH aggregate processing thread is allocated for each query, and a HASH aggregate table is created in the thread;

multiple HASH aggregate processing threads share processing cores of a multi-core processor, and an operating system allocates processing core resources dynamically for each HASH aggregate processing thread.

10. The concurrent OLAP-oriented database query processing method according to claim 8, wherein:

in the parallel HASH aggregate calculation, a unified HASH aggregate processing interface is set for each concurrent query group, and a HASH aggregate table is set for each query in a unified concurrency query processing thread, so as to process HASH aggregate calculation in a unified way.