System and method for processing query requests in a database system

• US 7,917,495 B1
• Filed: 06/11/2004
• Issued: 03/29/2011
• Est. Priority Date: 06/11/2004
• Status: Active Grant

• Current Assignee: LexisNexis Risk Data Management, Inc. (RELX PLC)
• Original Assignee: Seisint, Inc.
• Inventors: Halliday, Gavin C., Poulsen, Ole D., Bayliss, David A., Hicks, Nigel G., Cobbett-Smith, Jacob, Chapman, Richard K.
• Primary Examiner(s): Cottingham; John R
• Assistant Examiner(s): Reyes; Mariela D
• Application Number:
  US10/866,204
• Time in Patent Office: 2,482 Days
• Field of Search: 707/713, 707/718
• US Class Current: 707/713
• CPC Class Codes: G06F 16/24532 of parallel queries

Abstract

In certain embodiments, a database system for processing a query request comprises at least one master node operable to store a precompiled query that is capable of resolving a query request received by the database system. The at least on master node is further operable to receive a query request comprising one or more parameters and associated with the precompiled query, and to communicate a request to perform one or more activities associated with the precompiled query. The system further comprises a plurality of slave nodes coupled to the at least one master node, each of the slave nodes operable to store one or more key parts each comprising data capable of resolving a portion of the precompiled query. At least one of the slave nodes is operable to receive the request communicated by the at least one master node and to process the request communicated by the at least one master node.

Claims

1. A database system for processing a query request, comprising:

at least one master node operable to;

store a precompiled query that is capable of resolving a query request received by the database system, each precompiled query being a query that has been deployed on the database system in advance of a user executing such query on the database system;

receive a query request comprising one or more parameters used by the precompiled query to resolve the query request;

communicate a request to a slave node to perform one or more activities associated with the precompiled query and one or more top level keys associated with the precompiled query, the one or more top level keys identifying the location of a key part within the database system; and

a plurality of slave nodes coupled to the at least one master node, each of the plurality of slave nodes operable to store one or more key parts each comprising stored data, an address associated with the stored data, and an identifier for identifying the location of additional data that satisfies at least a portion of the precompiled query, wherein the stored data is sorted according to combinations of the one or more parameters used by the precompiled query to resolve the query request, at least one of the plurality of slave nodes operable to;

receive the request communicated by the at least one master node; and

process the request communicated by the at least one master node.

2. The system of claim 1, wherein the precompiled query is operable to resolve routine or standard data requests that are capable of having variations in input parameters.

3. The system of claim 1, wherein the precompiled query is operable to resolve routine or standard data requests that are capable of having variations in intermediate results of the precompiled query.

4. The system of claim 1, wherein:

the precompiled query comprises an annotated query execution graph and one or more executables; and

the at least one master node is operable to read the annotated query execution graph to determine the one or more activities associated with the precompiled query to communicate.

5. The system of claim 1, wherein:

the at least one master node is operable to access at least one of the top level keys to identify a communication channel that communicates with one or more of the plurality of slave nodes that store a particular key part.

6. The system of claim 5, wherein the at least one master node is operable to communicate the request to perform the one or more activities associated with the precompiled query on the identified communication channel, the request comprising one or more of the parameters of the query request.

7. The system of claim 6, wherein:

one or more of the plurality of slave nodes are each operable to receive the request communicated by the master node on the communication channel; and

at least one of the one or more slave nodes is operable to process the request communicated by the master node by performing the one or more activities of the request.

8. The system of claim 7, wherein:

at least one of the one or more activities comprises an index read of one or more key parts; and

the slave node performs the at least one activity by retrieving appropriate data in the one or more key parts for resolving at least a portion of the query request based at least on the one or more parameters of the request.

9. The system of claim 7, wherein the at least one slave node is operable to perform the one or more activities by executing appropriate portions of one or more executable files operable to perform the one or more activities.

10. The system of claim 1, wherein at least one of the plurality of slave nodes is operable to return one or more results to the at least one master node after processing at least a portion of the request communicated by the at least one master node.

11. The system of claim 10, wherein the at least one master node is operable to perform one or more local activities in response to receiving the one or more results.

12. The system of claim 11, wherein the one or more local activities comprise one or more of:

a sort of at least a portion of the returned results;

a de-duplication of at least a portion of the returned results; and

a roll-up of at least a portion of the returned results.

13. The system of claim 1, wherein each of the one or more key parts comprises a portion of a sorted data file.

14. The system of claim 1, wherein at least some of the one or more key parts are stored on at least some of the plurality of slave nodes in advance of a user requesting data from such key parts.

15. The system of claim 1, further comprising a plurality of master nodes, each of the plurality of master nodes operable to store the precompiled query and one or more top level keys associated with the precompiled query.

16. The system of claim 1, wherein each of the plurality of slave nodes is operable to store the precompiled query.

17. The system of claim 1, wherein the precompiled query is stored on the at least one master node in advance of a user providing the query request to the database system.

18. A method for processing a query request on a database system, the database system comprising one or more master nodes and a plurality of slave nodes coupled to the one or more master nodes, the method comprising:

receiving a query request comprising one or more parameters used by a precompiled query to resolve the query request, the precompiled query being stored on at least one master node of the database system and operable to resolve a query request received by the database system, each precompiled query being a query that has been deployed on the database system in advance of a user executing such query on the database system;

communicating from the at least one master node a request to a slave node to perform one or more activities associated with the precompiled query and one or more top level keys associated with the precompiled query, the one or more top level keys identifying the location of a key part within the database system;

receiving the request to perform one or more activities at one or more of the plurality of slave nodes, each of the plurality of slave nodes operable to store one or more key parts each comprising stored data, an address associated with the stored data, and an identifier for identifying the location of additional data that satisfies at least a portion of the precompiled query, wherein the stored data is sorted according to combinations of the one or more parameters used by the precompiled query to resolve the query request; and

processing the request communicated by the at least one master node at the one or more slave nodes.

19. The method of claim 18, wherein the precompiled query is operable to resolve routine or standard data requests that are capable of having variations in input parameters.

20. The method of claim 18, wherein:

the precompiled query comprises an annotated query execution graph and one or more executables; and

the method comprises reading the annotated query execution graph to determine the one or more activities associated with the precompiled query to communicate.

21. The method of claim 18, wherein:

the method further comprises accessing at least one of the top level keys to identify a communication channel that communicates with the one or more slave nodes that store a particular key part.

22. The method of claim 21, comprising communicating the request to perform the one or more activities associated with the precompiled query on the identified communication channel, the request comprising one or more of the parameters of the query request.

23. The method of claim 22, comprising:

receiving the request communicated by the master node on the communication channel at the one or more slave nodes; and

processing the request communicated by the master node on at least one of the one or more slave nodes by performing the one or more activities of the request.

24. The method of claim 23, wherein:

at least one of the one or more activities comprises an index read of one or more key parts; and

the method comprises performing the at least one activity at the at least one slave node by retrieving appropriate data in the one or more key parts for resolving at least a portion of the query request based at least on the one or more parameters of the request.

25. The method of claim 23, comprising performing the one or more activities at the at least one slave node by executing appropriate portions of one or more executable files operable to perform the one or more activities.

26. The method of claim 18, wherein at least one of the plurality of slave nodes is operable to return one or more results to the at least one master node after processing at least a portion of the request communicated by the at least one master node.

27. The method of claim 26, further comprising performing one or more local activities at the at least one master node in response to receiving the one or more results.

28. The method of claim 18, comprising:

storing at least some of the one or more key parts on at least some of the plurality of slave nodes in advance of a user requesting data from such key parts; and

storing, on each of the plurality of slave nodes, the precompiled query in advance of the user requesting data from such key parts.

29. The method of claim 18, comprising storing, on the at least one master node, the precompiled query in advance of a user providing the query request to the database system.

30. A computer-readable storage medium, storing program instructions for performing, when executed by a processor, a method for processing a query request on a database system, wherein the database system comprises one or more master nodes and a plurality of slave nodes coupled to the one or more master nodes, the method comprising:

receiving a query request comprising one or more parameters used by a precompiled query to resolve the query request, the precompiled query being stored on at least one master node of the database system and operable to resolve a query request received by the database system, each precompiled query being a query that has been deployed on the database system in advance of a user executing such query on the database system;

communicating from the at least one master node a request to a slave node to perform one or more activities associated with the precompiled query and one or more top level keys associated with the precompiled query, the one or more top level key identifying the location of a key part within the database system;

receiving the request to perform one or more activities at one or more of the plurality of slave nodes, each of the plurality of slave nodes operable to store one or more key parts each comprising stored data, an address associated with the stored data, and an identifier for identifying the location of additional data that satisfies at least a portion of the precompiled query, wherein the stored data is sorted according to combinations of the one or more parameters used by the precompiled query to resolve the query request; and

processing the request communicated by the at least one master node at the one or more slave nodes.