System and method for configuring a parallel-processing database system

• US 7,240,059 B2
• Filed: 11/14/2002
• Issued: 07/03/2007
• Est. Priority Date: 11/14/2002
• Status: Active Grant

• Current Assignee: LexisNexis Risk Data Management, Inc. (RELX PLC)
• Original Assignee: Seisint, Inc.
• Inventors: Bayliss, David, Smith, Jake, Hicks, Nigel, Halliday, Gavin, Poulsen, Ole, Chapman, Richard
• Primary Examiner(s): Wong; Leslie
• Application Number:
  US10/293,483
• Publication Number:
  US 20040098373A1
• Time in Patent Office: 1,692 Days
• Field of Search: 707/3, 707/10, 709201-203, 709/208, 709/209
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/2471 Distributed queries

Abstract

A system and method for configuring a plurality of processing nodes into a parallel-processing database system are described herein. Each of a plurality of processing nodes connected by a network receive software and one or more configuration files related to the intended function of the processing node. The software may include homogeneous agent software, one or more library dynamic-link libraries (DLL), and the like. The configuration file is used to configure the homogeneous agent to operate as the intended node in a global-results processing matrix, a general-purpose query processing matrix, or a index-base query processing matrix. Another node or nodes may be configured to convert query-based programming code to intermediary source code in a common programming language and then compile the intermediary source code into a dynamic link library (DLL) or other type of executable. The DLL is then distributed among the processing nodes of the processing matrix, whereupon various subsets of the processing nodes execute related portions of the DLL substantially in parallel to generate query results.

Claims

1. In a parallel-processing based database management system comprising a plurality of interconnected processing nodes, a method for configuring a system for processing queries, the method comprising:

distributing software to each processing node of a first subset of the plurality of processing nodes;

distributing a master node configuration file to at least one processing node of the first subset designated as the master node of a first global-results processing matrix;

distributing a slave node configuration file to each processing node of the first subset designated as the slave node of the first global-results processing matrix;

manipulating, by the software and master node configuration file, the designated master node to operate as the master node of the first global-results processing matrix;

manipulating, by the software and slave node configuration file, each of the designated slave nodes to operate as the slave node of the first global-results processing matrix;

receiving a query in a query-based language source code and compiling at least one executable from the query source code;

executing, by the first global-results processing matrix, the compiled at least one executable to process the query; and

completing the execution to process the query.

2. The method of claim 1, further comprising distributing query server software to at least one processing node of the plurality of processing nodes designated as a query server, wherein the query server software is configured to manipulate the processing node to operate as the query server.

3. The method of claim 1, further comprising distributing query agent software to at least one processing node of the plurality of processing nodes designated as a query agent, wherein the query agent software is configured to manipulate the processing node to operate as the query agent.

4. The method of claim 1, further comprising distributing scheduling services software to at least one processing node of the plurality of processing nodes designated as a scheduling services module, wherein the scheduling services software is configured to manipulate the processing node to operate as the scheduling services module.

5. The method of claim 1, further comprising distributing naming services software to at least one processing node of the plurality of processing nodes designated as a naming services module, wherein the naming services software is configured to manipulate the processing node to operate as the naming services module.

6. The method of claim 1, further comprising distributing work-unit reporting software to at least one processing node of the plurality of processing nodes designated as a work-unit reporting module, wherein the work-unit reporting software is configured to manipulate the processing node to operate as the work-unit reporting module.

7. The method of claim 1, further comprising:

distributing software to each processing node of a second subset of the plurality of processing nodes;

distributing a master node configuration file to at least one processing node of the second subset designated as the master node of a second global-results processing matrix; and

distributing a slave node configuration file to each processing node of the second subset designated as the slave node of the second global-results processing matrix;

wherein the software and master node configuration file are configured to manipulate the designated master node to operate as the master node of the second global-results processing matrix;

wherein the software and slave node configuration file are configured to manipulate each of the designated slave nodes to operate as a slave node of the second global-results processing matrix.

8. The method of claim 1, further comprising:

distributing software to each processing node of a second subset;

distributing a master node configuration file to at least one processing node of the second subset designated as the master node of a general-purpose query processing matrix;

distributing a slave node configuration file to each processing node of the second subset designated as the slave node of the general-purpose query processing matrix;

wherein the software and master node configuration file are configured to manipulate the designated master node to operate as the master node of the general-purpose query processing matrix;

wherein the software and slave node configuration file are configured to manipulate each of the designated slave nodes to operate as the slave node of the general-purpose query processing matrix.

9. The method of claim 1, further comprising:

distributing software to each processing node of a second subset;

distributing a master node configuration file to at least one processing node of the second subset designated as the master node of an index-based query processing matrix; and

distributing a slave node configuration file to each processing node of the second subset designated as the slave node of the index-based query processing matrix;

wherein the software and slave node configuration file are configured to manipulate each of the designated slave nodes to operate as the slave node of index-based query processing matrix.

10. The method of claim 1, further comprising:

selecting the first subset of processing nodes;

designating at least one processing node of the first subset as the master node of the first global-results processing matrix; and

designating each of the plurality of processing nodes of the first subset as the slave node of the first global-results processing matrix.

11. In a parallel-processing based database management system, a database configuration system comprising:

a plurality of processing nodes interconnected via a network, each processing node having at least one processor, at least one memory, and at least one disk storage;

an administrator workstation operably connected to the network and being configured to;

distribute software to each processing node of a first subset of processing nodes;

distribute a master node configuration file to at least one processing node of the first subset designated as the master node of a first global-results processing matrix; and

distribute a slave node configuration file to each processing node of the first subset designated as the slave node of the first global-results processing matrix;

wherein the software and master node configuration file are configured to manipulate the designated master node to operate as the master node of the first global-results processing matrix;

wherein the software and slave node configuration file are configured to manipulate each of the designated slave nodes to operate as the slave node of the first global-results processing matrix.

12. The system of claim 11, wherein the administrator workstation is further configured to distribute query server software to at least one processing node designated as a query server, wherein the query server software is configured to manipulate the designated processing node to operate as a query server, whereby the query server receives a query in a query-based language source code and compiles at least one executable from the query source code, the configured global-results processing matrix executing the compiled at least one executable to process a query.

13. The system of claim 11, wherein the administrator workstation is further configured to distribute query agent software to at least one processing node designated as a query agent, wherein the query agent software is configured to manipulate the designated processing node to operate as the query agent.

14. The system of claim 11, wherein the administrator workstation is further configured to distribute scheduling services software to at least one processing node designated as a scheduling services module, wherein the scheduling services software is configured to manipulate the designated processing node to operate as the scheduling services module.

15. The system of claim 11, wherein the administrator workstation is further configured to distribute work-unit reporting software to at least one processing node designated as a work-unit reporting module, wherein the work-unit reporting software is configured to manipulate the designated processing node to operate as the work-unit reporting module.

16. The system of claim 11, wherein the administration workstation is further configured to:

distribute software to each processing node of a second subset of processing nodes;

distribute a master node configuration file to at least one processing node of the second subset designated as the master node of a second global-results processing matrix; and

distribute a slave node configuration file to each processing node of the second subset designated as the slave node of the second global-results processing matrix;

wherein the software and master node configuration file are configured to manipulate the designated master node of the second subset to operate as the master node of the second global-results processing matrix;

wherein the software and slave node configuration file are configured to manipulate each of the designated slave nodes of the second subset to operate as the slave node of the second global-results processing matrix.

17. The system of claim 11, wherein the administration workstation is further configured to:

distribute software to each processing node of a second subset of processing nodes;

distribute a master node configuration file to at least one processing node of the second subset designated as the master node of a general-purpose query processing matrix; and

distribute a slave node configuration file to each processing node of the second subset designated as the slave node of a general-purpose query processing matrix;

wherein the software and master node configuration file are configured to manipulate the designated master node of the second subset to operate as the master node of the general-purpose query processing matrix;

wherein the software and slave node configuration file are configured to manipulate each of the designated slave nodes of the second subset to operate as a the slave node of the general-purpose query processing matrix.

18. The system of claim 11, wherein the administration workstation is further configured to:

distribute software to each processing node of a second subset of processing nodes;

distribute a master node configuration file to at least one processing node of the second subset designated as the master node of an index-based query processing matrix; and

distribute a slave node configuration file to each processing node of the second subset designated as the slave node of the index-based query processing matrix;

wherein the software and master node configuration file are configured to manipulate the designated master node of the second subset to operate as the master node of the index-based query processing matrix;

wherein the software and slave node configuration file are configured to manipulate each of the designated slave nodes of the second subset to operate as the slave node of the index-based query processing matrix.

19. The system of claim 11, wherein the administrator workstation is further configured to verify a correct operation of the first general-purpose processing matrix after distributing the software and configuration files.

20. The system of claim 11, wherein the administrator workstation is further configured to:

select the first subset of plurality of processing nodes from the plurality of processing nodes;

designate at least one processing node of the first subset as the master node of the first global-results processing matrix; and

designate a plurality of processing nodes of the first subset as slave nodes of the first global-results processing matrix.