FAST QUERY PROCESSING IN COLUMNAR DATABASES WITH GPUS

• US 20160378754A1
• Filed: 06/28/2016
• Published: 12/29/2016
• Est. Priority Date: 06/24/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kanagaratnam, Kajatheepan, Keenleyside, John D., Meraji, Seyed Sina, Kamath, Sunil J.
• Granted Patent: US 10,331,669 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/221 Column-oriented storage; Management thereof
• CPC Class Codes: G06F 16/2455 Query execution

Abstract

According to one exemplary embodiment, a method for processing a query associated with a database is provided. The method may include receiving the query. The method may include estimating a number of groups. The method may include copying a plurality of data from the database to graphics processing unit (GPU) memory. The method may include creating a hash table in GPU memory. The method may include determining if a group associated with the database is present in the hash table. The method may include adding the group to the hash table based on determining that the group is not present in the hash table. The method may include aggregating a value associated with the group in the hash table based on determining that the group is present in the hash table. The method may include determining a plurality of results. The method may then include retrieving the plurality of results.

Claims

1. A method for processing a query associated with a database, the method comprising:

receiving the query;

estimating a number of groups based on the received query and the database;

copying a plurality of data from the database to graphics processing unit (GPU) memory associated with a GPU;

creating a global hash table in GPU memory based on the estimated number of groups;

determining if a group associated with the database is present in the global hash table;

adding the group to the global hash table based on determining that the group is not present in the global hash table;

aggregating a value associated with the group in the global hash table based on determining that the group is present in the global hash table;

determining a plurality of results based on the received query and the global hash table; and

retrieving the determined plurality of results.

2. The method of claim 1, wherein adding the group to the global hash table and aggregating the value associated with the group in the global hash table comprises using an atomic operation associated with the GPU.

3. The method of claim 1, further comprising:

determining if the estimated number of groups does not exceed a threshold value;

creating a plurality of small hash tables in multiprocessor shared memory associated with the GPU based on determining that the estimated number of groups does not exceed the threshold value;

determining if the group associated with the database is present in a small hash table within the plurality of small hash tables;

adding the group to the small hash table within the plurality of small hash tables based on determining that the group is not present in the small hash table;

aggregating the value associated with the group in the small hash table based on determining that the group is present in the small hash table; and

merging the plurality of small hash tables into the global hash table.

4. The method of claim 1, wherein adding the group to the global hash table and aggregating the value associated with the group in the global hash table comprises using a plurality of execution threads associated with the GPU in parallel.

5. The method of claim 1, wherein aggregating the value associated with the group in the global hash table is based on the received query.

6. The method of claim 1, wherein the received query is a group by/aggregate query and the database is a columnar database.

7. The method of claim 1, wherein estimating the number of groups comprises using an optimizer executed in a central processing unit (CPU).