Dynamic thread pool tuning techniques
First Claim
1. A method of dynamically tuning thread pools in a multithreaded server, comprising:
- gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools;
programmatically altering the thread pools by adding an additional thread pool to service the dynamically changeable workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool, such that subsequent requests in at least one of the request classifications are serviced by the added thread pool;
gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and
programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration.
1 Assignment
0 Petitions
Accused Products
Abstract
Thread pools in a multithreaded server are programmatically adjusted, based on observed statistics from the server'"'"'s inbound workload. In a multithreaded server environment, response time to end users is improved while increasing the efficiency of software execution and resource usage. Execution time and wait/queued time are tracked, for various types of requests being serviced by a server. Multiple logical pools of threads are used to service these requests, and inbound requests are directed to a selected one of these pools such that requests of similar execution-time requirements are serviced by the threads in that pool. The number and size of thread pools may be adjusted programmatically, and the distribution calculation (i.e., determining which inbound requests should be assigned to which pools) is a programmatic determination. In preferred embodiments, only one of these variables is adjusted at a time, and the results are monitored to determine whether the effect was positive or negative. The disclosed techniques also apply to tracking and classifying requests by method name (and, optionally, parameters).
72 Citations
55 Claims
-
1. A method of dynamically tuning thread pools in a multithreaded server, comprising:
-
gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools; programmatically altering the thread pools by adding an additional thread pool to service the dynamically changeable workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool, such that subsequent requests in at least one of the request classifications are serviced by the added thread pool; gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17)
-
-
18. A method of dynamically tuning thread pools in a multithreaded server, comprising:
-
gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools;
programmatically altering the thread pools by removing a thread pool from the plurality of thread pools that service the dynamically changeable workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool, such that subsequent requests in none of the request classifications are serviced by the removed thread pool;gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration. - View Dependent Claims (19, 20, 21)
-
-
22. A system for dynamically tuning thread pools in a multithreaded server, comprising:
-
means for gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools; means for programmatically altering the thread pools by removing a thread pool from the plurality of thread pools that service the dynamically changeable workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool, such that subsequent requests in none of the request classifications are serviced by the removed thread pool; means for gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and means for programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration. - View Dependent Claims (23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33)
-
-
34. A system for dynamically tuning thread pools in a multithreaded server, comprising:
-
means for gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools; means for programmatically altering the thread pools by adding an additional thread pool to service the dynamically changeable workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool, such that subsequent requests in at least one of the request classifications are serviced by the added thread pool; means for gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and means for programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration. - View Dependent Claims (35, 36, 37, 38)
-
-
39. A computer program product for dynamically tuning thread pools in a multithreaded server, the computer program product embodied on one or more computer readable media readable by a computing system in a computing environment and comprising:
-
computer-readable program code for gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools; computer-readable program code for programmatically altering the thread pools by adding an additional thread pool to service the dynamically changeable workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool, such that subsequent requests in at least one of the request classifications are serviced by the added thread pool; computer-readable program code for gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and computer-readable program code for programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration. - View Dependent Claims (40, 41, 42, 43, 44, 45, 46, 47)
-
-
48. A computer program product for dynamically tuning thread pools in a multithreaded server, the computer program product embodied on one or more computer readable media readable by a computing system in a computing environment and comprising:
-
computer-readable program code for gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools; computer-readable program code for programmatically altering the thread pools by removing a thread pool from the plurality of thread pools that service the dynamically changeable workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool, such that subsequent requests in none of the request classifications are serviced by the removed thread pool; computer-readable program code for gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and computer-readable program code for programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration. - View Dependent Claims (49, 50, 51, 52, 53, 54)
-
-
55. A method of dynamically tuning thread pools of a multithreaded server, comprising:
-
programmatically gathering baseline operational characteristics of a workload at a multithreaded server, wherein the baseline operational characteristics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools; programmatically altering the thread pools by selectively incrementing or decrementing a number of the thread pools used to service requests of the workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools by computing, for each of the thread pools, an upper bound on execution time of requests in the request classifications to be serviced by that thread pool; programmatically gathering new operational characteristics for the dynamically changeable workload, wherein the new operational characteristics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration; and charging a fee for carrying out the steps of programmatically gathering baseline operational characteristics, programmatically altering, programmatically gathering new operational characteristics, and programmatically reversing steps.
-
Specification