System and method for configuring a parallel-processing database system

• US 7,668,840 B2
• Filed: 07/02/2007
• Issued: 02/23/2010
• Est. Priority Date: 11/14/2002
• Status: Active Grant

• Current Assignee: LexisNexis Risk Data Management, Inc. (RELX PLC)
• Original Assignee: LexisNexis Risk Data Management, Inc. (RELX PLC)
• Inventors: Hicks, Nigel, Bayliss, David, Smith, Jake, Halliday, Gavin, Poulsen, Ole, Chapman, Richard
• Primary Examiner(s): Wong; Leslie
• Application Number:
  US11/772,634
• Publication Number:
  US 20080010296A1
• Time in Patent Office: 967 Days
• Field of Search: 707/10, 707/101, 707/104.1, 707/E17.032, 707/E17.044, 709/203, 709/208, 709/209, 709/219
• US Class Current: 707/764
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/2471 Distributed queries

Abstract

A system and method for configuring a plurality of processing nodes into a parallel-processing database system are described herein. Each of a plurality of processing nodes connected by a network receive software and one or more configuration files related to the intended function of the processing node. The software may include homogeneous agent software, one or more library dynamic-link libraries (DLL), and the like. The configuration file is used to configure the homogeneous agent to operate as the intended node in a global-results processing matrix, a general-purpose query processing matrix, or a index-base query processing matrix. Another node or nodes may be configured to convert query-based programming code to intermediary source code in a common programming language and then compile the intermediary source code into a dynamic link library (DLL) or other type of executable. The DLL is then distributed among the processing nodes of the processing matrix, whereupon various subsets of the processing nodes execute related portions of the DLL substantially in parallel to generate query results.

Claims

1. A method for configuring a system for processing queries, the method comprising:

manipulating at least one processing node of a plurality of processing nodes to operate as a master node of a global-results processing matrix based at least in part on configuration software;

manipulating at least some of the remaining processing nodes of the plurality of processing nodes to operate as a slave node of the global-results processing matrix based at least in part on configuration software;

manipulating at least one processing node of a subset of the plurality of processing nodes to operate as a designated master node of a second global-results processing matrix;

manipulating each of the remaining processing nodes of the subset to operate as a designated slave node of the second global-results processing matrix;

receiving at least one query;

processing the at least one query by at least one of the global-results processing matrix and the second global-results processing matrix, wherein processing the at least one query is based on allocation of database operations among the processing matrices and timing of query submission; and

outputting at least one query result based on the processed at least one query.

2. The method of claim 1, further comprising manipulating at least one processing node of the plurality of processing nodes to operate as a query server.

3. The method of claim 1, further comprising distributing manipulating at least one processing node of the plurality of processing nodes to operate as a query agent.

4. The method of claim 1, further comprising manipulating at least one processing node of the plurality of processing nodes to operate as a scheduling services module.

5. The method of claim 1, further comprising manipulating at least one processing node of the plurality of processing nodes to operate as a naming services module.

6. The method of claim 1, further comprising manipulating at least one processing node of the plurality of processing nodes to operate as a work-unit reporting module.

7. The method of claim 1, further comprising:

manipulating at least one processing node of a subset of the plurality of processing nodes to operate as a designated master node of a general-purpose query processing matrix; and

manipulating each of the remaining processing nodes of the subset to operate as a designated slave node of the general-purpose query processing matrix.

8. The method of claim 1, further comprising:

manipulating at least one processing node of a subset of the plurality of processing nodes to operate as a designated master node of an index-based query processing matrix; and

manipulating each of the remaining processing nodes of the subset to operate as a designated slave node of the index-based query processing matrix.

9. The method of claim 1, further comprising:

selecting a subset of processing nodes;

designating at least one processing node of the subset as a master node of a global-results processing matrix; and

designating each of the remaining plurality of processing nodes of the subset as a slave node of the global-results processing matrix.

10. The method of claim 1, wherein the system comprises a first administrator workstation and a second administrator workstation, such that the configuration software received at the at least one processing node for manipulating at least one processing node of a plurality of processing nodes to operate as a master node of a global-results processing matrix is received from the first administrator workstation and the configuration software received at the at least some of the remaining processing nodes for manipulating at least some of the remaining processing nodes of the plurality of processing nodes to operate as a slave node of the global-results processing matrix is received from the second administrator workstation.

21. The method of claim 1, wherein the configuration software received at the at least one processing node for manipulating at least one processing node of a plurality of processing nodes to operate as a master node of a global-results processing matrix comprises at least one master node configuration file and the configuration software for manipulating at least some of the remaining processing nodes of the plurality of processing nodes to operate as a slave node of the global-results processing matrix comprises at least one slave node configuration file.

11. A database configuration system comprising:

a plurality of processing nodes interconnected via a network to the database configuration system, each processing node having at least one processor, at least one memory, and at least one disk storage; and

wherein the system is configured to;

manipulate at least one processing node of a subset of the plurality of processing nodes to operate as a designated master node of a global-results processing matrix based at least in part on configuration software;

manipulate at least some of the remaining processing nodes of the subset to operate as a designated slave node of the global-results processing matrix based at least in part on configuration software;

manipulate at least one processing node of a subset of the plurality of processing nodes to operate as a designated master node of a second global-results processing matrix; and

manipulate each processing node of the subset to operate as a designated slave node of the second global-results processing matrix.

12. The system of claim 11, wherein the system is further configured to manipulate at least one processing node to operate as a query server, wherein the query server receives at least one query and processes the at least one query based on the global-results processing matrix.

13. The system of claim 11, wherein the system is further configured to manipulate at least one processing node to operate as a query agent.

14. The system of claim 11, wherein the system is further configured to manipulate at least one processing node to operate as a scheduling services module.

15. The system of claim 11, wherein the system is further configured to manipulate at least one processing node to operate as a work-unit reporting module.

16. The system of claim 11, wherein the system is further configured to:

manipulate at least one processing node of a subset of the plurality of processing nodes to operate as a designated master node of a general-purpose query processing matrix; and

manipulate each processing node of the subset to operate as a designated slave node of the general-purpose query processing matrix.

17. The system of claim 11, wherein the administration workstation is further configured to:

manipulate at least one processing node of a subset of the plurality of processing nodes to operate as a designated master node of an index-based query processing matrix; and

manipulate each processing node of the subset to operate as a designated slave node of the index-based query processing matrix.

18. The system of claim 11, wherein the system is further configured to verify a correct operation of the global-results processing matrix after manipulating the at least one processing node.

19. The system of claim 11, wherein the system is further configured to:

selecting a subset of processing nodes;

designating at least one processing node of the subset as a master node of a global-results processing matrix; and

designating each of the plurality of processing nodes of the subset as a slave node of the global-results processing matrix.

20. The system of claim 11, wherein the system comprises:

a first administrator workstation operably connected to the network and being configured to manipulate at least one processing node of a subset of the plurality of processing nodes to operate as a designated master node of a global-results processing matrix based at least in part on configuration software received at the at least one processing node from the first administrator workstation; and

a second administrator workstation operably connected to the network and being configured to manipulate at least some of the remaining processing nodes of the subset to operate as a designated slave node of the global-results processing matrix based at least in part on configuration software received at the at least some of the remaining processing nodes from the second administrator workstation.