MANAGING COMPLEX QUERIES WITH PREDICATES

• US 20170147642A1
• Filed: 10/12/2016
• Published: 05/25/2017
• Est. Priority Date: 11/25/2015
• Status: Active Grant

• Current Assignee: AirBNB Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Zhang, Liping, Chainani, Naresh K., Chinta, Kiran K.
• Granted Patent: US 9,892,162 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24539 using cached or materialised query results
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/24549 Run-time optimisation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/3338 Query expansion
• CPC Class Codes: G06F 16/335 Filtering based on additional data, e.g. user or group profiles filtering in web context G06F16/9535, G06F16/9536

Abstract

A first query is detected. The first query is for a storage device, and pulls a result set from the storage device using a set of predicates. The first query satisfies a size criterion. A new set of predicates of the result set is identified. The new set of predicates does not include any predicates of the original set of predicates. A second query which is the same as the first query is detected. The new set of predicates is used to determine the result set.

Claims

1. A method of processing complex queries, comprising:

detecting a first query from a first client device directed towards a dataset, wherein the first query includes a set of initial predicates and results in a first result set, wherein an initial set of tuples are initially gathered from the dataset in response to the first query, wherein each tuple of the initial set of tuples satisfies as least one of the predicates, wherein the initial set of tuples is reduced to the result set as predicates of the initial set of predicates eliminate tuples;

storing a query plan and query statistics of the first query in the database;

determining that the initial set of tuples includes an amount of tuples that satisfies a size criterion;

in response to determining that the amount of tuples satisfies the size criterion, identifying a set of columns from the result set that do not correspond to a predicate of the initial set of predicates;

executing both a set of min/max filters and a set of bloom filters on the result set, wherein each bloom filter of the set of bloom filters is derived from a respective column of the set of columns and each min/max filter of the set of min/max filters is derived from a respective column of the set of columns;

determining that a subset of the set of bloom filters would filter a threshold percentage of tuples from the initial set of tuples;

determining that a subset of the min/max filters would filter the threshold percentage of tuples from the initial set of tuples;

identifying both the subset of the set of bloom filters and the subset of the min/max filters as new predicates;

detecting a second query from a second client device directed towards the dataset, wherein the second query is the same as the first query;

evaluating the new predicates immediately before executing the second query;

detecting an indicia of ineffectiveness regarding the new predicates while evaluating the new predicates;

in response to detecting the indicia of ineffectiveness, disabling the new predicates at runtime for the second query, wherein disabling the new predicates results in the new predicates not being used in the second query;

testing the new predicates following the conclusion of the runtime for the second query;

verifying that each of the new predicates filter the threshold amount of tuples from the initial set of tuples;

detecting a third query from a third client device directed towards the dataset, wherein the third query is the same as both the first query and the second query; and

utilizing the new predicates, the query plan, and the query statistics to process the third query and determine a third result set for the third query, wherein the new predicates are used in response to verifying that the new predicates filter the threshold amount of tuples from the initial set of tuples.