Task scheduling for highly concurrent analytical and transaction workloads
First Claim
1. A method, executable by a computer system, comprising:
- processing, by a database management system (DBMS) operating on the computer system, a query in a first thread in a plurality of worker threads to generate a parsing task associated with the query;
parsing, by a parser of the DBMS operating on the computer system, the query based on the parsing task to generate an execution plan comprising a plurality of ordered operations for answering the query;
generating, by the parser of the DBMS operating on the computer system, a task graph based on the execution plan, the task graph comprising a plurality of task nodes corresponding to the plurality of ordered operations;
identifying, by the DBMS operating on the computer system, a first task node in the plurality of task nodes corresponding to a first ordered operation in the plurality of ordered operations determined to be partitionable into sub operations for execution in parallel; and
generating, by an optimizer of the DBMS operating on the computer system, a plurality of partitioned tasks for execution in parallel based on the first task node and a number of available worker threads in the plurality of worker threads.
2 Assignments
0 Petitions
Accused Products
Abstract
Systems and method for a task scheduler with dynamic adjustment of concurrency levels and task granularity are disclosed for improved execution of highly concurrent analytical and transactional systems. The task scheduler can avoid both over commitment and underutilization of computing resources by monitoring and controlling the number of active worker threads. The number of active worker threads can be adapted to avoid underutilization of computing resources by giving the OS control of additional worker threads processing blocked application tasks. The task scheduler can dynamically determine a number of parallel operations for a particular task based on the number of available threads. The number of available worker threads can be determined based on the average availability of worker threads in the recent history of the application. Based on the number of available worker threads, the partitionable operation can be partitioned into a number of sub operations and executed in parallel.
-
Citations
20 Claims
-
1. A method, executable by a computer system, comprising:
-
processing, by a database management system (DBMS) operating on the computer system, a query in a first thread in a plurality of worker threads to generate a parsing task associated with the query; parsing, by a parser of the DBMS operating on the computer system, the query based on the parsing task to generate an execution plan comprising a plurality of ordered operations for answering the query; generating, by the parser of the DBMS operating on the computer system, a task graph based on the execution plan, the task graph comprising a plurality of task nodes corresponding to the plurality of ordered operations; identifying, by the DBMS operating on the computer system, a first task node in the plurality of task nodes corresponding to a first ordered operation in the plurality of ordered operations determined to be partitionable into sub operations for execution in parallel; and generating, by an optimizer of the DBMS operating on the computer system, a plurality of partitioned tasks for execution in parallel based on the first task node and a number of available worker threads in the plurality of worker threads. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A non-transitory computer readable medium storing a program executable by at least one processing unit of a computing system, the program comprising sets of instructions for:
-
processing, by a database management system (DBMS) operating on the computing system, a query in a first thread in a plurality of worker threads to generate a parsing task associated with the query; parsing, by a parser of the DBMS operating on the computing system, the query based on the parsing task to generate an execution plan comprising a plurality of ordered operations for answering the query; generating, by the parser of the DBMS operating on the computing system, a task graph based on the execution plan, the task graph comprising a plurality of task nodes corresponding to the plurality of ordered operations; identifying, by the DBMS operating on the computing system, a first task node in the plurality of task nodes corresponding to a first ordered operation in the plurality of ordered operations determined to be partitionable into sub operations for execution in parallel; and generating, by a optimizer of the DBMS operating on the computing system, a plurality of partitioned tasks for execution in parallel based on the first task node and the concurrency level value. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. A system comprising:
-
a set of processing units; a non-transitory computer readable medium coupled to the set of processing units and comprising instructions, that when executed by at least one processing unit cause the at least one processing unit to be configured to; process, by a database management system (DBMS) operating on the system, a query in a first thread in a plurality of worker threads to generate a parsing task associated with the query; parse, by a parser of the DBMS operating on the system, the query based on the parsing task to generate an execution plan comprising a plurality of ordered operations for answering the query; generate, by the parser of the DBMS operating on the system, a task graph based on the execution plan, the task graph comprising a plurality of task nodes corresponding to the plurality of ordered operations; identify, by the DBMS operating on the system, a first task node in the plurality of task nodes corresponding to a first ordered operation in the plurality of ordered operations determined to be partitionable into sub operations for execution in parallel; and generate, by an optimizer of the DBMS operating on the system, a plurality of partitioned tasks for execution in parallel based on the first task node and a number of available worker threads in the plurality of worker threads. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification