Method for parallel query processing with non-dedicated, heterogeneous computers that is resilient to load bursts and node failures

• US 20080059489A1
• Filed: 08/30/2006
• Published: 03/06/2008
• Est. Priority Date: 08/30/2006
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Narang, Inderpal Singh, Han, Wei, Raman, Vijayshankar
• Application Number:
  US11/512,769
• Publication Number:
  US 20080059489A1
• Time in Patent Office: Days
• US Class Current: 707/100
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451

Abstract

A method is provided for query processing in a grid computing infrastructure. The method entails storing data in a data storage system accessible to a plurality of computing nodes. Computationally-expensive query operations are identified and query fragments are allocated to individual nodes according to computing capability. The query fragments are independently executed on individual nodes. The query fragment results are combined into a final query result.

Claims

1. A method for query processing in a grid computing infrastructure, comprising:

storing data in a data storage system accessible to a plurality of individual computing nodes;

identifying specified query operations;

allocating query fragments of a specified query operation to the individual computing nodes according to each of the individual computing nodes computing capability;

independently executing the query fragments on the individual computing nodes in parallel to increase speed of query execution; and

combining the query fragment results into a final query result.

2. The method of claim 1 further comprising:

monitoring computational performance of the individual computing nodes; and

selectively reassigning the query fragments to particular individual computing nodes according to changes in performance of the monitored nodes.

3. The method of claim 1 wherein the individual computing nodes include non-dedicated heterogeneous computing nodes each running different database applications.

4. The method of claim 1 wherein one of the specified query options determines whether a query option is computationally-expensive.

5. The method of claim 4 wherein the computationally expensive query option includes a join and a select-project-join-aggregate group-by block execution.

6. The method of claim 1 wherein the data storage system is a virtualized storage area network.

7. The method of claim 1 wherein no tuples are shipped between nodes.

8. The method of claim 1 further comprising:

retrieving data from the data storage system by the individual computing nodes when needed.

9. The method of claim 1 wherein the data is indexed when it is read-only.

10. The method of claim 1 wherein the query processing includes a computer program saved on a computer-readable medium.

11. The method of claim 1 further comprising determining the relative processing power of the individual computing nodes using a grid load monitor.

12. The method of claim 1 further comprising reassigning a query fragment of the specified query operation on a first individual computing node to a second individual computing node when the first individual computing node fails by not responding after a specified period of time.

13. The method of claim 1 wherein the allocating of query fragments of the specified query operation is divided substantially equally to the individual computing nodes.

14. The method of claim 1 wherein the query fragments are divided according to the use of a liner program when the computing nodes have different processing capabilities.