QUERY PROGRESS ESTIMATION BASED ON PROCESSED VALUE PACKETS

• US 20100332472A1
• Filed: 06/30/2009
• Published: 12/30/2010
• Est. Priority Date: 06/30/2009
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Inventors: Graefe, Goetz, Kuno, Harumi
• Granted Patent: US 9,836,504 B2
• Time in Patent Office: Days
• US Class Current: 707/736
• CPC Class Codes: G06F 16/24542 Plan optimisation

Abstract

A data processing system performs query progress estimation based on processed value packets. In the illustrative data processing system, a database query processor comprises a query optimizer that creates a query plan, and a database plan executor that executes the query plan and observes intermediate result streams processed as the query plan is executed. A value packet manager anticipates value packets during query optimization, creates value packets as the intermediate result streams are processed, and compares anticipated value packets with created value packets to determine accuracy of the anticipated value packets and estimate query progress.

Claims

1. A data processing system comprising:

a database query processor comprising;

a query optimizer that creates a query plan;

a database plan executor that executes the query plan and observes intermediate result streams processed as the query plan is executed; and

a value packet manager that anticipates value packets during query optimization, creates value packets as the intermediate result streams are processed, and compares anticipated value packets with created value packets to determine accuracy of the anticipated value packets and estimate query progress.

2. The system according to claim 1 wherein:

the value packets comprises data statistics; and

the value packet manager collects and analyzes detailed information about the data statistics and dynamically corrects the data statistics during query processing.

3. The system according to claim 1 wherein:

the database plan executor sorts the intermediate result streams according to predetermine criteria and divides the intermediate result stream into value packets according to sort order.

4. The system according to claim 1 further comprising:

a workload manager that uses the estimate of query progress to determine remaining run-time of active requests and modify overall work schedule according to the estimate.

5. The system according to claim 1 wherein:

the value packet manager preserves detail information in data structures available during query execution, the detail information comprising distribution of unique values in sort keys and number of items or records per unique value.

6. The system according to claim 1 wherein:

the value packet manager determines whether the anticipated value packets are proven to be incorrect during query execution and, if so, reassesses query progress comprising;

estimating remaining work with increased accuracy with processed unique values in sort keys;

adapting workload management policies and reallocating resources;

selectively pausing query execution for later resumption; and

selectively aborting the query plan for re-invoking query optimization with improved information and an improved query plan.

7. The system according to claim 1 wherein:

after a unique value in an intermediate query result'"'"'s sort order and individual items and records associated with the unique value are processed, the value packet manager adjusts number of unique values in the intermediate result stream, and number of records and items associated with the unique value.

8. The system according to claim 1 wherein:

the value packet manager determines a confidence level indicator at an operator level indicative of a progress estimate confidence level.

9. The system according to claim 1 wherein:

the value packet manager uses a plurality of estimation methods at a corresponding plurality of query execution points based on type of operator.

10. The system according to claim 1 wherein:

the value packet manager analyzes operator-level runtime statistics and overall system statistics in combination for estimating query progress.

11. The system according to claim 1 further comprising:

an article of manufacture comprising;

a controller-usable medium having a computer readable program code embodied in a controller for processing data, the computer readable program code further comprising;

code causing the controller to create a query plan;

code causing the controller to execute the query plan and observe intermediate result streams processed as the query plan is executed;

code causing the controller to anticipate value packets during query optimization;

code causing the controller to create value packets as the intermediate result streams are processed; and

code causing the controller to compare anticipated value packets with created value packets to determine accuracy of the anticipated value packets and estimate query progress.

12. The system according to claim 1 further comprising:

a processor that executes the query optimizer, the database plan executor, and the value packet manager.

13. A method for processing data comprising:

processing database queries comprising;

creating a query plan;

executes the query plan;

observing intermediate result streams processed as the query plan is executed;

anticipating value packets during query optimization;

creating value packets as the intermediate result streams are processed; and

comparing anticipated value packets with created value packets to determine accuracy of the anticipated value packets and estimate query progress.

14. A data processing system comprising:

a database query processor in a relational database having a query optimizer that creates a query plan, verifies compliance of a subset of records from the relational database with a query criteria as the query plan is executed, and generates intermediate result streams based on the compliance verification, the database query processor comprising;

a database plan executor that executes the query plan, observes the intermediate result streams, sorts the intermediate result streams according to predetermine criteria, and divides the intermediate result stream into value packets according to sort order; and

a value packet manager that collects and analyzes detailed information about data statistics, dynamically corrects the data statistics during query processing, and creates value packets based on the data statistics as the intermediate result streams are processed.

15. The system according to claim 14 wherein:

the value packet manager anticipates value packets during query optimization, creates value packets as the intermediate result streams are processed, and compares anticipated value packets with created value packets to determine accuracy of the anticipated value packets and estimate query progress.