Query scheduling in a parallel-processing database system

• US 20040098374A1
• Filed: 11/14/2002
• Published: 05/20/2004
• Est. Priority Date: 11/14/2002
• Status: Active Grant

• Current Assignee: LexisNexis Risk Data Management, Inc. (RELX PLC)
• Original Assignee: Seisint, Inc.
• Inventors: Bayliss, David, Smith, Jake, Hicks, Nigel, Halliday, Gavin, Poulsen, Ole, Chapman, Richard
• Granted Patent: US 7,185,003 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

A system and method for scheduling database operations to one or more databases in a parallel-processing database system are described herein. After a query server generates a dynamic-link library (DLL) or other executable representative of one or more database operations to a database, the query server notifies a scheduling services module of the generation of the DLL and submits the DLL to a query agent. The query agent notifies the scheduling services module of its receipt of the DLL. Based on any of a variety of considerations, the scheduling services module schedules a time of execution for the DLL by one or more processing matrices that store the database. At the scheduled time, the scheduling services module directs the query agent to submit the DLL to the indicated processing matrices. The scheduling services module also can be adapted to monitor the execution of previously submitted DLLs by one or more processing matrices and adjust the scheduled times of execution for subsequent DLLs accordingly.

Claims

1. A system for scheduling database operations on at least one database, the system comprising:

a first global-results processing matrix having a plurality of interconnected processing nodes operating in parallel and being adapted to execute an executable on the at least one database, the executable being representative of a query having at least one database operation;

a first query agent operably connected to the first global-results processing matrix and being adapted to manage the execution of the executable by the first global-results processing matrix; and

a scheduling services module operably connected to the query agent and the first global-results processing matrix and being adapted to;

schedule a time for execution of the executable by the first global-results processing matrix; and

direct the first query agent to submit the executable to the first global-results processing matrix for execution at the scheduled time of execution.

2. The system of claim 1, wherein the scheduling services module is further adapted to adjust the first scheduled time of execution for the executable based in part on a difference between an expected execution time and an actual execution time of at least one executable previously executed by the first global-results processing matrix.

3. The system of claim 2, wherein the first query agent is further adapted to provide an indication of a progress of an execution of an executable by the first global-results processing matrix to the scheduling services module.

4. The system of claim 1, further comprising:

a second global-results processing matrix having a plurality of interconnected processing nodes operating in parallel and being adapted to execute a second executable on the at least one database, the second executable being representative of a query having at least one database operation;

a second query agent operably connected to the second global-results processing matrix and the scheduling services module and being adapted to manage the execution of the second executable by the second global-results processing matrix; and

wherein the scheduling services module is further adapted to;

schedule a time for execution of the second executable by the second global-results processing matrix; and

direct the second query agent to submit the second executable to the second global-results processing matrix for execution at the scheduled time of execution.

5. The system of claim 4, wherein the scheduling services module is further adapted to schedule executions of executables at the first global-results processing matrix and the second global-results processing matrix to minimize execution times for the executables.

6. The system of claim 4, wherein the scheduling services module is further adapted to schedule executions of executables at the first global-results processing matrix and the second global-results processing matrix to maximize utilization of the first global-results processing matrix and the second global-results processing matrix.

7. The system of claim 4, wherein the scheduling services module is further adapted to schedule executions of executables at the first global-results processing matrix and the second global-results processing matrix to minimize delays in execution of the executables by the first global-results processing matrix and the second global-results processing matrix.

8. The system of claim 1, further comprising:

a general-purpose query processing matrix having a plurality of interconnected processing nodes operating in parallel and being adapted to execute a third executable on the at least one database, the third executable being representative of a query having at least one database operation;

a second query agent operably connected to the general-purpose query processing matrix and the scheduling services module and being adapted to manage the execution of the third executable by the general-purpose query processing matrix; and

wherein the scheduling services module is further adapted to;

schedule a time for execution of an executable by the general-purpose query processing matrix; and

direct the second query agent to submit the third executable to the general-purpose query processing matrix for execution at the scheduled time of execution.

9. The system of claim 8, wherein the scheduling services module is further adapted to schedule executions of executables at the first global-results processing matrix and the general-purpose query processing matrix to minimize execution times for the executables.

10. The system of claim 8, wherein the scheduling services module is further adapted to schedule executions of executables at the first global-results processing matrix and general-purpose query processing matrix to maximize utilization of the first global-results processing matrix and the general-purpose query processing matrix.

11. The system of claim 8, wherein the scheduling services module is further adapted to schedule executions of executables at the first global-results processing matrix and the second global-results processing matrix to minimize delays in execution of the executables by the first global-results processing matrix and the general-purpose query processing matrix.

12. In a parallel processing based database management system, a method for scheduling execution of compiled executables representing queries having at least one database operation, the method comprising the steps of:

scheduling a time for execution of an executable by a first global-results processing matrix; and

submitting, at the scheduled time, the executable to the first global-results processing matrix for execution.

13. The method of claim 12, further comprising the step of adjusting the scheduled time of the executable by the first global-results processing matrix based in part on a difference between an expected execution time and an actual execution time of at least one executable previously executed by the first global-results processing matrix.

14. The method of claim 12, further comprising the steps of:

scheduling a time for execution of a second executable by a second global-results processing matrix; and

submitting, at the scheduled time, the second executable to the second global-results processing matrix for execution.

15. The method of claim 14, wherein the scheduling of the executions of executables at the first global-results processing matrix and the second global-results processing matrix is determined to minimize execution times for the executables.

16. The method of claim 14, wherein the scheduling of the executions of executables at the first global-results processing matrix and the second global-results processing matrix is determined to maximize utilization of the first global-results processing matrix and the second global-results processing matrix.

17. The method of claim 14, wherein the scheduling of the executions of executables at the first global-results processing matrix and the second global-results processing matrix is determined to minimize delays in execution of the executables by the first global-results processing matrix and the second global-results processing matrix.

18. The method of claim 12, further comprising the steps of:

scheduling a time for execution of a third executable by a general-purpose query processing matrix; and

submitting, at the scheduled time, the third executable to the general-purpose processing matrix for execution.

19. The method of claim 18, wherein the scheduling of the executions of executables at the first global-results processing matrix and the general-purpose query processing matrix is determined to minimize execution times for the executables.

20. The method of claim 18, wherein the scheduling of the executions of executables at the first global-results processing matrix and the general-purpose query processing matrix is determined to maximize utilization of the first global-results processing matrix and the general-purpose query processing matrix.

21. The method of claim 18, wherein the scheduling of the executions of executables at the first global-results processing matrix and the general-purpose query processing matrix is determined to minimize delays in execution of the executables by the first global-results processing matrix and the general-purpose query processing matrix.

22. The method of claim 12, further comprising the steps of:

scheduling a time for execution of a fourth executable by an index-based query processing matrix; and

submitting, at the scheduled time, the fourth executable to the index-based query processing matrix for execution.

23. The method of claim 21, wherein the scheduling of the executions of executables at the first global-results processing matrix and the index-based query processing matrix is determined to minimize execution times for the executables.

24. The method of claim 21, wherein the scheduling of the executions of executables at the first global-results processing matrix and the index-based query processing matrix is determined to maximize utilization of the first global-results processing matrix and the index-based query processing matrix.

25. The method of claim 21, wherein the scheduling of the executions of executables at the first global-results processing matrix and the index-based query processing matrix is determined to minimize delays in execution of the executables by the first global-results processing matrix and the index-based query processing matrix.