MANAGING DATABASE RESOURCES USED FOR OPTIMIZING QUERY EXECUTION ON A PARALLEL COMPUTER SYSTEM

• US 20090132541A1
• Filed: 11/19/2007
• Published: 05/21/2009
• Est. Priority Date: 11/19/2007
• Status: Active Grant

• Current Assignee: ServiceNow Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Barsness, Eric Lawrence, Santosuosso, John Matthew, Darrington, David L., Peters, Amanda
• Granted Patent: US 8,095,512 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: Y10S 707/966 Distributed

Abstract

Embodiments of the invention may be used to increase query processing parallelism of an in-memory database stored on a parallel computing system. A group of compute nodes each store a portion of data as part of the in-memory database. Further, a pool of compute nodes may be reserved to create copies of data from the compute nodes of the in-memory database as part of query processing. When a query is received for execution, the query may be evaluated to determine whether portions of in-memory should be duplicated to allow multiple elements of the query (e.g., multiple query predicates) to be evaluated in parallel.

Claims

1. A method for managing database resources on a parallel computer system, comprising:

providing a database distributed across a first group of compute nodes, wherein each compute node in the first group stores a respective portion of the database;

assigning a second group of compute nodes to a pool, wherein each compute node in the pool is available to replicate the respective portion of the database stored on one of the compute nodes in the first group of compute nodes;

in response to receiving a query to execute against the database, allocating one or more compute nodes from the pool to dynamically replicate the portion of the database stored on a corresponding one or more compute nodes of the first group of compute nodes; and

executing, in parallel, the query on at least the one or more compute nodes allocated from the pool and on the corresponding one or more compute nodes of the first group of compute nodes.

2. The method of claim 1, further comprising, prior to allocating the one or more compute nodes from the pool, evaluating the query to identify two or more conditions of the query referencing data records stored on a first compute node of the first group of compute nodes.

3. The method of claim 2, further comprising, executing one of the two or more conditions on the first compute node, and executing another one of the two or more conditions on one of the compute nodes allocated from the pool.

4. The method of claim 1, wherein the pool includes a maximum number of compute nodes available to be allocated to dynamically replicate portions of the database stored on the first group of compute nodes based on a process ID associated with an application submitting the query for execution.

5. The method of claim 1, wherein the pool includes a maximum number of compute nodes available to be allocated to dynamically replicate portions of the database stored on the first group of compute nodes based on an identity a user submitting the query for executing.

6. The method of claim 1, wherein the pool includes a maximum number of compute nodes available to be allocated to dynamically replicate portions of the database stored on the first group of compute nodes based on based on a specified priority assigned to the query.

7. The method of claim 1, further comprising, prior to allocating one or more compute nodes from the pool, determining a predicted decrease in query execution time from replicating respective portions of the database exceeds a specified minimum.

8. The method of claim 1, further comprising, reclaiming the one or more compute nodes allocated from the pool.

9. A computer-readable storage medium containing a program which, when executed, performs an operation for managing database resources on a parallel computer system that includes a database distributed across a first group of compute nodes, wherein each compute node in the first group stores a respective portion of the database, the operation comprising:

assigning a second group of compute nodes to a pool, wherein each compute node in the pool is available to replicate the respective portion of the database stored on one of the compute nodes in the first group of compute nodes;

in response to receiving a query to execute against the database, allocating one or more compute nodes from the pool to dynamically replicate the portion of the database stored on a corresponding one or more compute nodes of the first group of compute nodes; and

executing, in parallel, the query on at least the one or more compute nodes allocated from the pool and on the corresponding one or more compute nodes of the first group of compute nodes.

10. The computer-readable storage medium of claim 9, wherein the operation further comprises, prior to allocating the one or more compute nodes from the pool, evaluating the query to identify two or more conditions of the query referencing data records stored on a first compute node of the first group of compute nodes.

11. The computer-readable storage medium of claim 10, wherein the operation further comprises, executing one of the two or more conditions on the first compute node, and executing another one of the two or more conditions on one of the compute nodes allocated from the pool.

12. The computer-readable storage medium of claim 9, wherein the pool includes a maximum number of compute nodes available to be allocated to dynamically replicate portions of the database stored on the first group of compute nodes based on a process ID associated with an application submitting the query for execution.

13. The computer-readable storage medium of claim 9, wherein the pool includes a maximum number of compute nodes available to be allocated to dynamically replicate portions of the database stored on the first group of compute nodes based on an identity a user submitting the query for executing.

14. The computer-readable storage medium of claim 9, wherein the pool includes a maximum number of compute nodes available to be allocated to dynamically replicate portions of the database stored on the first group of compute nodes based on based on a specified priority assigned to the query.

15. The computer-readable storage medium of claim 9, wherein the operation further comprises, prior to allocating one or more compute nodes from the pool, determining a predicted decrease in query execution time from replicating respective portions of the database exceeds a specified minimum.

16. The computer readable storage medium of claim 9, wherein the operation further comprises, reclaiming the one or more compute nodes allocated from the pool.

17. A parallel computing system, comprising:

a plurality of compute nodes, each having at least a processor and a memory;

the plurality of compute nodes comprising;

a first group of compute nodes, wherein each compute node in the first group stores a respective portion of an in-memory database; and

a second group of compute nodes allocated to a pool;

wherein each compute node in the pool is available to replicate the respective portion of the database stored on one of the compute nodes in the first group of compute nodes; and

a master compute node comprising a database management program configured to;

in response to receiving a query to execute against the in-memory database, allocate one or more compute nodes from the pool to dynamically replicate the portion of the database stored on a corresponding one or more compute nodes of the first group of compute nodes, and

execute, in parallel, the query on at least the one or more compute nodes allocated from the pool and on the corresponding one or more compute nodes of the first group of compute nodes.

18. The system of claim 17, wherein the database management program is further configured to, prior to allocating the one or more compute nodes from the pool, evaluate the query to identify two or more conditions of the query referencing data records stored on a first compute node of the first group of compute nodes.

19. The system of claim 18, wherein the first compute node is configured to execute one of the two or more conditions, and one of the compute nodes allocated from the pool is configured to execute another one of the two or more conditions.

20. The system of claim 17, wherein the pool includes a maximum number of compute nodes available to be allocated to dynamically replicate portions of the database stored on the first group of compute nodes based on a process ID associated with an application submitting the query for execution.

21. The system of claim 17, wherein the pool includes a maximum number of compute nodes available to be allocated to dynamically replicate portions of the database stored on the first group of compute nodes based on an identity a user submitting the query for executing.

22. The system of claim 17, wherein the pool includes a maximum number of compute nodes available to be allocated to dynamically replicate portions of the database stored on the first group of compute nodes based on based on a specified priority assigned to the query.

23. The system of claim 17, wherein the database management program is further configured to, prior to allocating one or more compute nodes from the pool, determine a predicted decrease in query execution time from replicating respective portions of the database exceeds a specified minimum.

24. The system of claim 17, wherein the database management program configured is further to, reclaim the one or more compute nodes allocated from the pool.