Dynamic thread pool tuning techniques

• US 7,237,242 B2
• Filed: 12/31/2002
• Issued: 06/26/2007
• Est. Priority Date: 12/31/2002
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hogstrom, Matt R., Daughtrey, Erik A., Blythe, Christopher James, Cuomo, Gennaro A.
• Primary Examiner(s): BULLOCK JR, LEWIS ALEXANDER
• Application Number:
  US10/334,768
• Publication Number:
  US 20040139434A1
• Time in Patent Office: 1,638 Days
• Field of Search: 718/1, 718100-108, 709223-225
• US Class Current: 718/105
• CPC Class Codes: G06F 2209/5018 Thread allocation
• CPC Class Codes: G06F 9/505 considering the load

Abstract

Thread pools in a multithreaded server are programmatically adjusted, based on observed statistics from the server'"'"'s inbound workload. In a multithreaded server environment, response time to end users is improved while increasing the efficiency of software execution and resource usage. Execution time and wait/queued time are tracked, for various types of requests being serviced by a server. Multiple logical pools of threads are used to service these requests, and inbound requests are directed to a selected one of these pools such that requests of similar execution-time requirements are serviced by the threads in that pool. The number and size of thread pools may be adjusted programmatically, and the distribution calculation (i.e., determining which inbound requests should be assigned to which pools) is a programmatic determination. In preferred embodiments, only one of these variables is adjusted at a time, and the results are monitored to determine whether the effect was positive or negative. The disclosed techniques also apply to tracking and classifying requests by method name (and, optionally, parameters).

Claims

1. A method of dynamically tuning thread pools in a multithreaded server, comprising:

gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools;

programmatically altering the thread pools by adding an additional thread pool to service the dynamically changeable workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool, such that subsequent requests in at least one of the request classifications are serviced by the added thread pool;

gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and

programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration.

2. The method according to claim 1, wherein thread pools are logically organized groupings of a physical thread pool.

3. The method according to claim 1, further comprising programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance failed to improve as a result of the programmatic alteration.

4. The method according to claim 1, wherein the rebalancing further comprises the computing, for each of the thread pools, an upper bound on execution time of requests in the request classifications to be serviced by that thread pool.

5. The method according to claim 1, wherein the programmatically reversing further comprises:

programmatically removing the added thread pool and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool, such that subsequent requests in none of the request classifications are serviced by the removed thread pool.

6. The method according to claim 1, wherein the request classifications comprise request types.

7. The method according to claim 1, wherein the request classifications comprise request types and parameter values.

8. The method according to claim 1, wherein the request classifications comprise request types, parameter names, and parameter values.

9. The method according to claim 1, wherein the request classifications comprise method names.

10. The method according to claim 1, wherein the request classifications comprise method names and parameter values.

11. The method according to claim 1, wherein the request classifications comprise method names and parameter names and values.

12. The method according to claim 1, wherein the request classifications comprise Uniform Resource Identifier (“

URI”

) names.

13. The method according to claim 1, wherein the request classifications comprise Uniform Resource Identifier (“

URI”

) names and parameter values.

14. The method according to claim 1, wherein the request classifications comprise processing destinations.

15. The method according to claim 1, wherein the programmatically altering further comprises incrementing a number of threads assigned to a selected one of the thread pools;

and wherein the programmatically reversing further comprises decrementing the number of threads assigned to the selected one of the thread pools.

16. The method according to claim 1, wherein the programmatically altering further comprises decrementing a number of threads assigned to a selected one of the thread pools;

and wherein the programmatically reversing further comprises incrementing the number of threads assigned to the selected one of the thread pools.

17. The method according to claim 1, wherein the baseline run-time statistics and the new run-time statistics further comprise standard deviation of execution time and standard deviation of queued time for servicing the requests comprising the changeable workload.

18. A method of dynamically tuning thread pools in a multithreaded server, comprising:

gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools;

programmatically altering the thread pools by removing a thread pool from the plurality of thread pools that service the dynamically changeable workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool, such that subsequent requests in none of the request classifications are serviced by the removed thread pool;

gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and

programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration.

19. The method according to claim 18, wherein the programmatically reversing further comprises:

programmatically adding another thread pool and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool, such that subsequent requests in at least one of the request classifications are serviced by the added thread pool.

20. The method according to claim 18, wherein the programmatically altering further comprises incrementing a number of threads assigned to a selected one of the thread pools;

and wherein the programmatically reversing further comprises decrementing the number of threads assigned to the selected one of the thread pools.

21. The method according to claim 18, wherein the programmatically altering further comprises decrementing a number of threads assigned to a selected one of the thread pools;

and wherein the programmatically reversing further comprises incrementing the number of threads assigned to the selected one of the thread pools.

22. A system for dynamically tuning thread pools in a multithreaded server, comprising:

means for gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools;

means for programmatically altering the thread pools by removing a thread pool from the plurality of thread pools that service the dynamically changeable workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool, such that subsequent requests in none of the request classifications are serviced by the removed thread pool;

means for gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and

means for programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration.

23. The system according to claim 22, further comprising means for programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance failed to improve as a result of the programmatic alteration.

24. The system according to claim 22, wherein the request classifications comprise request types.

25. The system according to claim 22, wherein the request classifications comprise request types, parameter names, and parameter values.

26. The system according to claim 22, wherein the request classifications comprise method names.

27. The system according to claim 22, wherein the request classifications comprise method names and parameter names and values.

28. The system according to claim 22, wherein the request classifications comprise Uniform Resource Identifier (“

URI”

) names.

29. The system according to claim 22, wherein the request classifications comprise Uniform Resource Identifier (“

URI”

) names and parameter values.

30. The system according to claim 22, wherein the means for programmatically reversing further comprises:

means for programmatically adding another thread pool and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool, such that subsequent requests in at least one of the request classifications are serviced by the added thread pool.

31. The system according to claim 22, wherein the means for programmatically altering further comprises means for incrementing a number of threads assigned to a selected one of the thread pools;

and wherein the means for programmatically reversing further comprises means for decrementing the number of threads assigned to the selected one of the thread pools.

32. The system according to claim 22, wherein the means for programmatically altering further comprises means for decrementing a number of threads assigned to a selected one of the thread pools;

and wherein the means for programmatically reversing further comprises means for incrementing the number of threads assigned to the selected one of the thread pools.

33. The system according to claim 22, wherein the baseline run-time statistics and the new run-time statistics further comprise standard deviation of execution time and standard deviation of queued time for servicing the requests comprising the changeable workload.

34. A system for dynamically tuning thread pools in a multithreaded server, comprising:

means for gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools;

means for programmatically altering the thread pools by adding an additional thread pool to service the dynamically changeable workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool, such that subsequent requests in at least one of the request classifications are serviced by the added thread pool;

means for gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and

means for programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration.

35. The system according to claim 34, wherein the rebalancing further comprises computing, for each of the thread pools, an upper bound on execution time of requests in the request classifications to be serviced by that thread pool.

36. The system according to claim 34, wherein the means for programmatically reversing further comprises:

means for programmatically removing the added thread pool and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool, such that subsequent requests in none of the request classifications are serviced by the removed thread pool.

37. The system according to claim 34, wherein the means for programmatically altering further comprises means for incrementing a number of threads assigned to a selected one of the thread pools;

and wherein the means for programmatically reversing further comprises means for decrementing the number of threads assigned to the selected one of the thread pools.

38. The system according to claim 34, wherein the means for programmatically altering further comprises means for decrementing a number of threads assigned to a selected one of the thread pools;

and wherein the means for programmatically reversing further comprises means for incrementing the number of threads assigned to the selected one of the thread pools.

39. A computer program product for dynamically tuning thread pools in a multithreaded server, the computer program product embodied on one or more computer readable media readable by a computing system in a computing environment and comprising:

computer-readable program code for gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools;

computer-readable program code for programmatically altering the thread pools by adding an additional thread pool to service the dynamically changeable workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool, such that subsequent requests in at least one of the request classifications are serviced by the added thread pool;

computer-readable program code for gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and

computer-readable program code for programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration.

40. The computer program product according to claim 39, wherein thread pools are logically organized groupings of a physical thread pool.

41. The computer program product according to claim 39, further comprising computer-readable program code for programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance failed to improve as a result of the programmatic alteration.

42. The computer program product according to claim 39, wherein the rebalancing further comprises computing, for each of the thread pools, an upper bound on execution time of requests to be serviced by that thread pool.

43. The computer program product according to claim 39, wherein the computer-readable program code for programmatically reversing further comprises:

computer-readable program code for programmatically removing the added thread pool and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool such that subsequent requests in none of the request classifications are serviced by the removed thread pool.

44. The computer program product according to claim 39, wherein the request classifications comprise request types.

45. The computer program product according to claim 39, wherein the request classifications comprise request types and parameter values.

46. The computer program product according to claim 39, wherein the request classifications comprise Uniform Resource Identifier (“

URI”

) names.

47. The computer program product according to claim 39, wherein the computer-readable program code for programmatically altering further comprises computer-readable program code for incrementing a number of threads assigned to a selected one of the thread pools;

and wherein the computer-readable program code for programmatically reversing further comprises computer-readable program code for decrementing the number of threads assigned to the selected one of the thread pools.

48. A computer program product for dynamically tuning thread pools in a multithreaded server, the computer program product embodied on one or more computer readable media readable by a computing system in a computing environment and comprising:

computer-readable program code for gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools;

computer-readable program code for programmatically altering the thread pools by removing a thread pool from the plurality of thread pools that service the dynamically changeable workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool, such that subsequent requests in none of the request classifications are serviced by the removed thread pool;

computer-readable program code for gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and

computer-readable program code for programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration.

49. The computer program product according to claim 48, wherein the request classifications comprise method names.

50. The computer program product according to claim 48, wherein the request classifications comprise method names and parameter names and values.

51. The computer program product according to claim 48, wherein the request classifications comprise processing destinations.

52. The computer program product according to claim 48, wherein the computer-readable program code for programmatically reversing further comprises:

computer-readable program code for programmatically adding another thread pool and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool such that subsequent requests in at least one of the request classifications are serviced by the added thread pool.

53. The computer program product according to claim 48, wherein the computer-readable program code for programmatically altering further comprises computer-readable program code for decrementing a number of threads assigned to a selected one of the thread pools;

and wherein the computer-readable program code for programmatically reversing further comprises computer-readable program code for incrementing the number of threads assigned to the selected one of the thread pools.

54. The computer program product according to claim 48, wherein the baseline run-time statistics and the new run-time statistics further comprise standard deviation of execution time and standard deviation of queued time for servicing the requests comprising the changeable workload.

55. A method of dynamically tuning thread pools of a multithreaded server, comprising:

programmatically gathering baseline operational characteristics of a workload at a multithreaded server, wherein the baseline operational characteristics comprise average execution time and average queued time for executing a first plurality of requests in a plurality of request classifications of the dynamically changeable workload, the requests being serviced by a plurality of thread pools;

programmatically altering the thread pools by selectively incrementing or decrementing a number of the thread pools used to service requests of the workload and rebalancing allocation of the dynamically changeable workload to the plurality of thread pools by computing, for each of the thread pools, an upper bound on execution time of requests in the request classifications to be serviced by that thread pool;

programmatically gathering new operational characteristics for the dynamically changeable workload, wherein the new operational characteristics comprise average execution time and average queued time for executing a second plurality of requests in the plurality of request classifications of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools;

programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration; and

charging a fee for carrying out the steps of programmatically gathering baseline operational characteristics, programmatically altering, programmatically gathering new operational characteristics, and programmatically reversing steps.