Programmatic response-time based workload distribution techniques
First Claim
1. A computer-implemented method of programmatically distributing workload across thread pools of a multithreaded server, comprising:
- determining a number of available thread pools at the server;
obtaining execution times from historical statistics of a workload on the server;
programmatically distributing the obtained execution times over the number of available thread pools, further comprising;
sorting the obtained execution times;
determining a count of the execution times;
dividing the count of execution times by the number of available thread pools to find a value, “
N”
; and
assigning an upper bound on execution time for each of the available thread pools, according to the sorted execution times when accessed using integer multiples of the value “
N”
as an index; and
using the assigned upper bounds for assigning inbound requests to the available thread pools at run time.
1 Assignment
0 Petitions
Accused Products
Abstract
Workload is programmatically distributed across a set of execution resources. In a multithreaded server environment, response time to end users is improved while increasing the efficiency of software execution and resource usage. Execution time and wait/queued time are tracked, for various types of requests being serviced by a server. Multiple logical pools of threads are used to service these requests, and inbound requests are directed to a selected one of these pools such that requests of similar execution-time requirements are serviced by the threads in that pool. The number and size of thread pools may be adjusted programmatically, and the distribution calculation (i.e., determining which inbound requests should be assigned to which pools) is a programmatic determination. In preferred embodiments, only one of these variables is adjusted at a time, and the results are monitored to determine whether the effect was positive or negative. The disclosed techniques also apply to tracking and classifying requests by method name (and, optionally, parameters).
-
Citations
33 Claims
-
1. A computer-implemented method of programmatically distributing workload across thread pools of a multithreaded server, comprising:
-
determining a number of available thread pools at the server; obtaining execution times from historical statistics of a workload on the server; programmatically distributing the obtained execution times over the number of available thread pools, further comprising; sorting the obtained execution times; determining a count of the execution times; dividing the count of execution times by the number of available thread pools to find a value, “
N”
; andassigning an upper bound on execution time for each of the available thread pools, according to the sorted execution times when accessed using integer multiples of the value “
N”
as an index; andusing the assigned upper bounds for assigning inbound requests to the available thread pools at run time. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. A system for programmatically distributing inbound requests across thread pools in a multithreaded server, comprising:
-
means for determining a number of available thread pools at the server; means for obtaining execution times from historical statistics of a workload on the server; means for programmatically distributing the obtained execution times over the number of available thread pools, further comprising; means for sorting the obtained execution times; means for determining a count of the execution times; means for dividing the count of execution times by the number of available thread pools to find a value, “
N”
; andmeans for assigning an upper bound on execution time for each of the available thread pools, according to the sorted execution times when accessed using integer multiples of the value “
N”
as an index;means for receiving at the server, at run time, inbound requests; and means for assigning the received requests to the available thread pools according to the assigned upper bounds. - View Dependent Claims (16, 17, 18, 19, 20, 21, 22, 23)
-
-
24. A computer program product for programmatically distributing workload across thread pools of a multithreaded server, the computer program product embodied on one or more computer readable media readable by a computing system in a computing environment and comprising:
-
computer-readable program code for determining a number of available thread pools at the server; computer-readable program code for obtaining execution times from historical statistics of a workload on the server; computer-readable program code for programmatically distributing the obtained execution times over the number of available thread pools, further comprising; computer-readable program code for sorting the obtained execution times; computer-readable program code for determining a count of the execution times; computer-readable program code for dividing the count of execution times by the number of available thread pools to find a value, “
N”
; andcomputer-readable program code for assigning an upper bound on execution time for each of the available thread pools, according to the sorted execution times when accessed using integer multiples of the value “
N”
as an index; andcomputer-readable program code for using the assigned upper bounds for assigning inbound requests to the available thread pools at run time. - View Dependent Claims (25, 26, 27, 28, 29, 30, 31, 32)
-
-
33. A method of programmatically distributing workload across thread pools of a multithreaded server, comprising steps of:
-
programmatically monitoring operational characteristics of a workload at a server; programmatically distributing the workload across thread pools of the server, further comprising the steps of; determining a number of available thread pools at the server; obtaining execution times from historical statistics of the workload; programmatically distributing the obtained execution times over the number of available thread pools, further comprising; sorting the obtained execution times; determining a count of the execution times; dividing the count of execution times by the number of available thread pools to find a value, “
N”
; andassigning an upper bound on execution time for each of the available thread pools, according to the sorted execution times when accessed using integer multiples of the value “
N”
as an index;receiving at the server, at run time, inbound requests; and assigning each of the inbound requests to a first one of the available thread pools for which the assigned upper bound is greater than or equal to an expected execution time for the request, or to a last one of the available thread pools if none of the assigned upper bounds is greater than or equal to the expected execution time; and charging a fee for carrying out the programmatically monitoring and programmatically distributing steps.
-
Specification