System and method for configuring a parallel-processing database system

• US 20040098373A1
• Filed: 11/14/2002
• Published: 05/20/2004
• Est. Priority Date: 11/14/2002
• Status: Active Grant

• Current Assignee: LexisNexis Risk Data Management, Inc. (RELX PLC)
• Original Assignee: Seisint, Inc.
• Inventors: Bayliss, David, Smith, Jake, Hicks, Nigel, Halliday, Gavin, Poulsen, Ole, Chapman, Richard
• Granted Patent: US 7,240,059 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/2471 Distributed queries

Abstract

A system and method for configuring a plurality of processing nodes into a parallel-processing database system are described herein. Each of a plurality of processing nodes connected by a network receive software and one or more configuration files related to the intended function of the processing node. The software may include homogeneous agent software, one or more library dynamic-link libraries (DLL), and the like. The configuration file is used to configure the homogeneous agent to operate as the intended node in a global-results processing matrix, a general-purpose query processing matrix, or a index-base query processing matrix. Another node or nodes may be configured to convert query-based programming code to intermediary source code in a common programming language and then compile the intermediary source code into a dynamic link library (DLL) or other type of executable. The DLL is then distributed among the processing nodes of the processing matrix, whereupon various subsets of the processing nodes execute related portions of the DLL substantially in parallel to generate query results.

Claims

1. In a parallel-processing based database management system comprising a plurality of interconnected processing nodes, a method for configuring a system for processing queries, the method comprising:

distributing software to each processing node of a first subset of the plurality of processing nodes;

distributing a master node configuration file to a processing node of the first subset designated as a master node of a first global-results processing matrix;

distributing a slave node configuration file to each processing node of the first subset designated as a slave node of the first global-results processing matrix;

manipulating, by the software and master node configuration file, the designated master node to operate as the master node of the first global-results processing matrix;

manipulating, by the software and slave node configuration file, each of the designated slave nodes to operate as a slave node of the first global-results processing matrix;

receiving a query in a query-based language source code and compiling at least one executable from the query source code; and

executing, by the configured global-results processing matrix, the compiled at least one executable to process a query.

2. The method of claim 1, further comprising distributing query server software to a processing node of the plurality of processing nodes designated as a query server, wherein the query server software is adapted to manipulate the processing node to operate as a query server.

3. The method of claim 1, further comprising distributing query agent software to a processing node of the plurality of processing nodes designated as a query agent, wherein the query agent software is adapted to manipulate the processing node to operate as a query agent.

4. The method of claim 1, further comprising distributing scheduling services software to a processing node of the plurality of processing nodes designated as a scheduling services module, wherein the query agent software is adapted to manipulate the processing node to operate as a scheduling services module.

5. The method of claim 1, further comprising distributing naming services software to a processing node of the plurality of processing nodes designated as a naming services module, wherein the naming services software is adapted to manipulate the processing node to operate as a naming services module.

6. The method of claim 1, further comprising distributing work-unit reporting software to a processing node of the plurality of processing nodes designated as a work-unit reporting module, wherein the work-unit reporting software is adapted to manipulate the processing node to operate as a work-unit reporting module.

7. The method of claim 1, further comprising:

distributing software to each processing node of a second subset of the plurality of processing nodes;

distributing a master node configuration file to a processing node of the second subset designated as a master node of a second global-results processing matrix; and

distributing a slave node configuration file to each processing node of the second subset designated as a slave node of the second global-results processing matrix;

wherein the software and master node configuration file are adapted to manipulate the designated master node to operate as the master node of the second global-results processing matrix;

wherein the software and slave node configuration file are adapted to manipulate each of the designated slave nodes to operate as a slave node of the second global-results processing matrix.

8. The method of claim 1, further comprising:

distributing software to each processing node of a second subset;

distributing a master node configuration file to processing node of the second subset designated as a master node of a general-purpose query processing matrix;

distributing a slave node configuration file to each processing node of the second subset designated as a slave node of the general-purpose query processing matrix;

wherein the software and master node configuration file are adapted to manipulate the designated master node to operate as a master node of the general-purpose query processing matrix;

wherein the software and slave node configuration file are adapted to manipulate each of the designated slave nodes to operate as a slave node of the general-purpose query processing matrix.

9. The method of claim 1, further comprising:

distributing software to each processing node of a second subset;

distributing a master node configuration file to processing node of the second subset designated as a master node of an index-based query processing matrix; and

distributing a slave node configuration file to each processing node of the second subset designated as a slave node of the index-based query processing matrix;

wherein the software and slave node configuration file are adapted to manipulate each of the designated slave nodes to operate as a slave node of index-based query processing matrix.

10. The method of claim 1, further comprising:

selecting the first subset of processing nodes;

designating a processing node of the first subset as the master node of the first global-results processing matrix; and

designating each of a plurality of processing nodes of the first subset as a slave node of the first global-results processing matrix.

11. In a parallel-processing based database management system, a database configuration system comprising:

a plurality of processing nodes interconnected via a network, each processing node having at least one processor, memory, and at least one disk storage;

an administrator workstation operably connected to the network and being adapted to;

distribute software to each processing node of a first subset of processing nodes;

distribute a master node configuration file to a processing node of the first subset designated as a master node of a first global-results processing matrix; and

distribute a slave node configuration file to each processing node of the first subset designated as a slave node of the first global-results processing matrix;

wherein the software and master node configuration file are adapted to manipulate the designated master node to operate as the master node of the first global-results processing matrix;

wherein the software and slave node configuration file are adapted to manipulate each of the designated slave nodes to operate as a slave node of the first global-results processing matrix.

12. The system of claim 11, wherein the administrator workstation is further adapted to distribute query server software to a processing node designated as a query server, wherein the query server software is adapted to manipulate the designated processing node to operate as a query server, whereby the query server receives a query in a query-based language source code and compiles at least one executable from the query source code, the configured global-results processing matrix executing the compiled at least one executable to process a query.

13. The system of claim 11, wherein the administrator workstation is further adapted to distribute query agent software to a processing node designated as a query agent, wherein the query agent software is adapted to manipulate the designated processing node to operate as a query agent.

14. The system of claim 11, wherein the administrator workstation is further adapted to distribute scheduling services software to a processing node designated as a scheduling services module, wherein the scheduling services software is adapted to manipulate the designated processing node to operate as a scheduling services module.

15. The system of claim 11, wherein the administrator workstation is further adapted to distribute work-unit reporting software to a processing node designated as a work-unit reporting module, wherein the work-unit reporting software is adapted to manipulate the designated processing node to operate as a work-unit reporting module.

16. The system of claim 11, wherein the administration workstation is further adapted to:

distribute software to each processing node of a second subset of processing nodes;

distribute a master node configuration file to a processing node of the second subset designated as a master node of a second global-results processing matrix; and

distribute a slave node configuration file to each processing node of the second subset designated as a slave node of the second global-results processing matrix;

wherein the software and master node configuration file are adapted to manipulate the designated master node of the second subset to operate as the master node of the second global-results processing matrix;

wherein the software and slave node configuration file are adapted to manipulate each of the designated slave nodes of the second subset to operate as a slave node of the second global-results processing matrix.

17. The system of claim 11, wherein the administration workstation is further adapted to:

distribute software to each processing node of a second subset of processing nodes;

distribute a master node configuration file to a processing node of the second subset designated as a master node of a general-purpose query processing matrix; and

distribute a slave node configuration file to each processing node of the second subset designated as a slave node of a general-purpose query processing matrix;

wherein the software and master node configuration file are adapted to manipulate the designated master node of the second subset to operate as the master node of the general-purpose query processing matrix;

wherein the software and slave node configuration file are adapted to manipulate each of the designated slave nodes of the second subset to operate as a slave node of the general-purpose query processing matrix.

18. The system of claim 11, wherein the administration workstation is further adapted to:

distribute software to each processing node of a second subset of processing nodes;

distribute a master node configuration file to a processing node of the second subset designated as a master node of an index-based query processing matrix; and

distribute a slave node configuration file to each processing node of the second subset designated as a slave node of the index-based query processing matrix;

wherein the software and master node configuration file are adapted to manipulate the designated master node of the second subset to operate as a master node of the index-based query processing matrix;

wherein the software and slave node configuration file are adapted to manipulate each of the designated slave nodes of the second subset to operate as a slave node of the index-based query processing matrix.

19. The system of claim 11, wherein the administrator workstation is further adapted to verify a correct operation of the first general-purpose processing matrix after distributing the software and configuration files.

20. The system of claim 11, wherein the administrator workstation is further adapted to:

select the first subset of plurality of processing nodes from the plurality of processing nodes;

designate a processing node of the first subset as the master node of the first global-results processing matrix; and

designate a plurality of processing nodes of the first subset as slave nodes of the first global-results processing matrix.