System and method for pre-compiling a query and pre-keying a database system

• US 7,693,826 B1
• Filed: 06/11/2004
• Issued: 04/06/2010
• Est. Priority Date: 06/11/2004
• Status: Active Grant

• Current Assignee: LexisNexis Risk Data Management, Inc. (RELX PLC)
• Original Assignee: Seisint, Inc.
• Inventors: Cobbett-Smith, Jacob, Halliday, Gavin C., Poulsen, Ole D., Bayliss, David A., Hicks, Nigel G., Chapman, Richard K.
• Primary Examiner(s): Mahmoudi; Tony
• Assistant Examiner(s): Reyes; Mariela D
• Application Number:
  US10/866,270
• Time in Patent Office: 2,125 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

A parallel database system capable of deploying a pre-compiled query and pre-keying data associated with the pre-compiled query includes at least one master node. The at least one master node is operable to store and execute a pre-compiled query that is capable of resolving a data request received by the parallel database system. The system further includes a plurality of slave nodes coupled to the at least one master node. In this particular embodiment, each of the plurality of slave nodes is operable to store one or more key parts. The one or more key parts include data capable of resolving a portion of the pre-compiled query.

Claims

1. A database system, including memory and at least one processor, capable of deploying a pre-compiled query and pre-keying data associated with the pre-compiled query, the system comprising:

at least one master node operable to store a pre-compiled query and a top level key, the pre-compiled query being stored on the master node in advance of receiving a plurality of data requests capable of being resolved by the pre-compiled query;

a communication channel; and

a plurality of slave nodes coupled to the at least one master node by the communication channel, each of the plurality of slave nodes operable to store one or more key parts comprising a portion of a data file sorted based on the pre-compiled query in advance of receiving the plurality of data requests,

wherein the top level key associates a particular one of the key parts with one of the plurality of slave nodes that stores the particular one of the key parts, and

wherein the at least one master node executes the pre-compiled query by communicating the data requests over the communication channel to the plurality of slave nodes, and the plurality of slave nodes perform operations responsive to the data requests to resolve the data requests by accessing the one or more key parts.

2. The system of claim 1, wherein each of the plurality of slave nodes operates to store the pre-compiled query.

3. The system of claim 2, wherein the pre-compiled query is stored on the at least one master node and each of the plurality of slave nodes in advance of a user providing the data request to the database system.

4. The system of claim 1, wherein the pre-compiled query operates to resolve routine data requests that are capable of having variations in input parameters.

5. The system of claim 1, wherein the pre-compiled query operates to resolve routine data requests that are capable of having variations in intermediate results of the pre-compiled query.

6. The system of claim 1, wherein the pre-compiled query comprises an annotated query execution graph and one or more executables.

7. The system of claim 1, wherein the top level key is capable of identifying a communication channel that communicates with one or more of the plurality of slave nodes that store a particular key part.

8. The system of claim 1, wherein each of the plurality of slave nodes comprise one or more memory modules capable of storing the one or more key parts associated with a sorted table.

9. The system of claim 8, wherein the memory module comprises a hard disk.

10. The system of claim 1, wherein each of the one or more key parts comprises a portion of a sorted data file.

11. The system of claim 1, wherein at least some of the one or more key parts are stored on at least some of the plurality of slave nodes in advance of a user requesting data from such key parts.

12. The system of claim 1, further comprising a plurality of master nodes, each of the plurality of master nodes operable to store the pre-compiled query and one or more top level keys associated with the pre-compiled query.

23. The database system according to claim 1, wherein the top level key identifies, for a first field identified by the pre-compiled query, a range of values for each of the one or more key parts.

24. The database system according to claim 1, wherein the master node stores the one or more key parts.

13. A method of forming a database system, the method comprising:

deploying a pre-compiled query to at least one master node, the pre-compiled query being stored on the master node in advance of receiving a plurality of data requests capable of being resolved by the pre-compiled query;

pre-keying the database system, wherein pre-keying the database system comprises;

deploying a plurality of key parts to a plurality of slave nodes, each of the plurality of key parts comprising a portion of a data file sorted based on the pre-compiled query in advance of receiving the plurality of data requests;

deploying one or more top level keys associated with the pre-compiled query to the at least one master node, the one or more top level keys associating a particular one of the key parts with one of the plurality of slave nodes that stores the particular one of the key parts, and

configuring the at least one master node to execute the pre-complied query by communicating the data requests over a communication channel to the plurality of slave nodes, and configuring the plurality of slave nodes to perform operations responsive to the data requests to resolve the data requests by accessing the one or more key parts.

14. The method of claim 13, wherein the pre-compiled query is deployed to each of the plurality of slave nodes.

15. The method of claim 13, wherein deploying the pre-compiled query is done in advance of a user providing the data request to the database system.

16. The method of claim 13, wherein the pre-compiled query operates to resolve a routine data request that is capable of having variations in input parameters.

17. The method of claim 13, wherein the one or more top level keys identify a communication channel that communicates with one or more of the plurality of slave nodes that store a particular key part.

18. The method of claim 13, deploying the plurality of key parts comprises:

communicating each key part to the database system; and

storing each key part on at least one memory module coupled to a particular one or more slave nodes.

19. The method of claim 18, wherein the memory module comprises a hard disk.

20. The method of claim 13, wherein deploying the plurality of key parts is done in advance of a user providing the data request to the database system.

21. The method of claim 13, further comprising distributing the pre-compiled query and the one or more top level keys to each of a plurality of master nodes coupled to the at least master node.

22. A computer-implemented system for pre-keying a database system, comprising:

one or more computer processors operably connected to one or more computer-readable media, the computer-readable media containing instructions to cause the one or more processors to;

deploy a pre-compiled query to at least one master node, the pre-compiled query being stored on the master node in advance of receiving a plurality of data requests capable of being resolved by the pre-compiled query;

deploy a plurality of key parts to a plurality of slave nodes, each of the plurality of key parts comprising a portion of a data file sorted based on the pre-compiled query in advance of receiving the plurality of data requests; and

deploy one or more top level keys associated with the pre-compiled query to the at least one master node, the one or more top level keys including the first field and associating a particular one of the key parts with one of the plurality of slave nodes that stores the particular one of the key parts,

wherein the at least one master node is configured to execute the pre-compiled query by communicating the data requests over a communication channel to the plurality of slave nodes, and the plurality of slave nodes perform operations responsive to the data requests to resolve the data requests by accessing the one or more key parts.