Dynamic thread pool tuning techniques

• US 20040139434A1
• Filed: 12/31/2002
• Published: 07/15/2004
• Est. Priority Date: 12/31/2002
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hogstrom, Matt R., Daughtrey, Erik A., Blythe, Christopher James, Cuomo, Gennaro A.
• Granted Patent: US 7,237,242 B2
• Time in Patent Office: Days
• US Class Current: 718/100
• CPC Class Codes: G06F 2209/5018 Thread allocation
• CPC Class Codes: G06F 9/505 considering the load

Abstract

Thread pools in a multithreaded server are programmatically adjusted, based on observed statistics from the server'"'"'s inbound workload. In a multithreaded server environment, response time to end users is improved while increasing the efficiency of software execution and resource usage. Execution time and wait/queued time are tracked, for various types of requests being serviced by a server. Multiple logical pools of threads are used to service these requests, and inbound requests are directed to a selected one of these pools such that requests of similar execution-time requirements are serviced by the threads in that pool. The number and size of thread pools may be adjusted programmatically, and the distribution calculation (i.e., determining which inbound requests should be assigned to which pools) is a programmatic determination. In preferred embodiments, only one of these variables is adjusted at a time, and the results are monitored to determine whether the effect was positive or negative. The disclosed techniques also apply to tracking and classifying requests by method name (and, optionally, parameters).

Claims

1. A method of dynamically tuning thread pools in a multithreaded server, comprising steps of:

gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics pertain to executing a first plurality of requests of the dynamically changeable workload, the requests being serviced by a plurality of thread pools;

programmatically altering the thread pools;

gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics pertain to executing a second plurality of requests of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and

programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration.

2. The method according to claim 1, wherein thread pools are logically organized groupings of a physical thread pool.

3. The method according to claim 1, further comprising the step of programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance failed to improve as a result of the programmatic alteration.

4. The method according to claim 1, wherein the programmatically altering step further comprises the step of adding an additional thread pool to service the dynamically changeable workload.

5. The method according to claim 4, further comprising the step of programmatically rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool, prior to the step of gathering the new run-time statistics.

6. The method according to claim 5, wherein the programmatically rebalancing step further comprises the step of computing an upper bound on execution times of requests to be serviced by the thread pools.

7. The method according to claim 4, wherein the programmatically reversing step further comprises the steps of:

programmatically removing the added thread pool; and

rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool.

8. The method according to claim 1, wherein the baseline run-time statistics and the new run-time statistics are average execution time and average queued time for servicing the requests comprising the changeable workload.

9. The method according to claim 8, wherein the average execution time and average queued time are maintained per request type.

10. The method according to claim 8, wherein the average execution time and average queued time are maintained per request type and parameter values.

11. The method according to claim 8, wherein the average execution time and average queued time are maintained per request type, parameter names, and parameter values.

12. The method according to claim 8, wherein the average execution time and average queued time are maintained per method name.

13. The method according to claim 8, wherein the average execution time and average queued time are maintained per method name and parameter values.

14. The method according to claim 8, wherein the average execution time and average queued time are maintained per method name and parameter names and values.

15. The method according to claim 8, wherein the average execution time and average queued time are maintained per Uniform Resource Identifier (“

URI”

) name.

16. The method according to claim 8, wherein the average execution time and average queued time are maintained per Uniform Resource Identifier (“

URI”

) name and parameter values.

17. The method according to claim 8, wherein the average execution time and average queued time are maintained per processing destination.

18. The method according to claim 1, wherein the programmatically altering step further comprises the step of removing a thread pool from the plurality of thread pools that service the dynamically changeable workload.

19. The method according to claim 18, further comprising the step of programmatically rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool, prior to the step of gathering the new run-time statistics.

20. The method according to claim 18, wherein the programmatically reversing step further comprises the steps of:

programmatically adding another thread pool; and

rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool.

21. The method according to claim 1, wherein the programmatically altering step further comprises the step of incrementing a number of threads assigned to a selected one of the thread pools, prior to the step of gathering new run-time statistics.

22. The method according to claim 21, wherein the programmatically reversing step further comprises the step of decrementing the number of threads assigned to the selected one of the thread pools.

23. The method according to claim 1, wherein the programmatically altering step further comprises the step of decrementing a number of threads assigned to a selected one of the thread pools, prior to the step of gathering new run-time statistics.

24. The method according to claim 23, wherein the programmatically reversing step further comprises the step of incrementing the number of threads assigned to the selected one of the thread pools.

25. The method according to claim 1, wherein the baseline run-time statistics and the new run-time statistics are average execution time, standard deviation of execution time, average queued time, and standard deviation of queued time for servicing the requests comprising the changeable workload.

26. A system for dynamically tuning thread pools in a multithreaded server, comprising:

means for gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics pertain to executing a first plurality of requests of the dynamically changeable workload, the requests being serviced by a plurality of thread pools;

means for programmatically altering the thread pools;

means for gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics pertain to executing a second plurality of requests of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and

means for programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration.

27. The system according to claim 26, further comprising means for programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance failed to improve as a result of the programmatic alteration.

28. The system according to claim 26, wherein the means for programmatically altering further comprises means for adding an additional thread pool to service the dynamically changeable workload.

29. The system according to claim 28, further comprising means for programmatically rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool, prior to operation of the means for gathering the new run-time statistics, by computing an upper bound on execution times of requests to be serviced by the thread pools.

30. The system according to claim 28, wherein the means for programmatically reversing further comprises:

means for programmatically removing the added thread pool; and

means for rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool.

31. The system according to claim 26, wherein the baseline run-time statistics and the new run-time statistics are average execution time and average queued time for servicing the requests comprising the changeable workload.

32. The system according to claim 31, wherein the average execution time and average queued time are maintained per request type.

33. The system according to claim 31, wherein the average execution time and average queued time are maintained per request type, parameter names, and parameter values.

34. The system according to claim 31, wherein the average execution time and average queued time are maintained per method name.

35. The system according to claim 31, wherein the average execution time and average queued time are maintained per method name and parameter names and values.

36. The system according to claim 31, wherein the average execution time and average queued time are maintained per Uniform Resource Identifier (“

URI”

) name.

37. The system according to claim 31, wherein the average execution time and average queued time are maintained per Uniform Resource Identifier (“

URI”

) name and parameter values.

38. The system according to claim 26, wherein the means for programmatically altering further comprises means for removing a thread pool from the plurality of thread pools that service the dynamically changeable workload.

39. The system according to claim 38, further comprising means for programmatically rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool, prior to operation of the means for gathering the new run-0time statistics.

40. The system according to claim 39, wherein the means for programmatically reversing further comprises:

means for programmatically adding another thread pool; and

means for rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool.

41. The system according to claim 26, wherein the means for programmatically altering further comprises means for incrementing a number of threads assigned to a selected one of the thread pools, prior to operation of the means for gathering new run-time statistics;

and wherein the means for programmatically reversing further comprises means for decrementing the number of threads assigned to the selected one of the thread pools.

42. The system according to claim 26, wherein the means for programmatically altering further comprises means for decrementing a number of threads assigned to a selected one of the thread pools, prior to operation of the means for gathering new run-time statistics;

and wherein the means for programmatically reversing further comprises means for incrementing the number of threads assigned to the selected one of the thread pools.

43. The system according to claim 26, wherein the baseline run-time statistics and the new run-time statistics are average execution time, standard deviation of execution time, average queued time, and standard deviation of queued time for servicing the requests comprising the changeable workload.

44. A computer program product for dynamically tuning thread pools in a multithreaded server, the computer program product embodied on one or more computer readable media readable by a computing system in a computing environment and comprising:

computer-readable program code means for gathering baseline run-time statistics for a dynamically changeable workload on a multithreaded server, wherein the baseline statistics pertain to executing a first plurality of requests of the dynamically changeable workload, the requests being serviced by a plurality of thread pools;

computer-readable program code means for programmatically altering the thread pools;

computer-readable program code means for gathering new run-time statistics for the dynamically changeable workload, wherein the new statistics pertain to executing a second plurality of requests of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools; and

computer-readable program code means for programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration.

45. The computer program product according to claim 44, wherein thread pools are logically organized groupings of a physical thread pool.

46. The computer program product according to claim 44, further comprising computer-readable program code means for programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance failed to improve as a result of the programmatic alteration.

47. The computer program product according to claim 44, wherein the computer-readable program code means for programmatically altering further comprises computer-readable program code means for adding an additional thread pool to service the dynamically changeable workload;

and further comprising computer-readable program code means for programmatically rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool, prior to operation of the computer-readable program code means for gathering the new run-time statistics.

48. The computer program product according to claim 47, wherein the computer-readable program code means for programmatically rebalancing further comprises computer-readable program code means for computing an upper bound on execution times of requests to be serviced by the thread pools.

49. The computer program product according to claim 47, wherein the computer-readable program code means for programmatically reversing further comprises:

computer-readable program code means for programmatically removing the added thread pool; and

computer-readable program code means for rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool.

50. The computer program product according to claim 44, wherein the baseline run-time statistics and the new run-time statistics are average execution time and average queued time for servicing the requests comprising the changeable workload.

51. The computer program product according to claim 50, wherein the average execution time and average queued time are maintained per request type.

52. The computer program product according to claim 50, wherein the average execution time and average queued time are maintained per request type and parameter values.

53. The computer program product according to claim 50, wherein the average execution time and average queued time are maintained per method name.

54. The computer program product according to claim 50, wherein the average execution time and average queued time are maintained per method name and parameter names and values.

55. The computer program product according to claim 50, wherein the average execution time and average queued time are maintained per Uniform Resource Identifier (“

URI”

) name.

56. The computer program product according to claim 55, wherein the average execution time and average queued time are maintained per processing destination.

57. The computer program product according to claim 44, wherein the computer-readable program code means for programmatically altering further comprises computer-readable program code means for removing a thread pool from the plurality of thread pools that service the dynamically changeable workload;

and further comprising computer-readable program code means for programmatically rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the removed thread pool, prior to operation of the computer-readable program code means for gathering the new run-time statistics.

58. The computer program product according to claim 57, wherein the computer-readable program code means for programmatically reversing further comprises:

computer-readable program code means for programmatically adding another thread pool; and

computer-readable program code means for rebalancing allocation of the dynamically changeable workload to the plurality of thread pools, to account for the added thread pool.

59. The computer program product according to claim 44, wherein the computer-readable program code means for programmatically altering further comprises computer-readable program code means for incrementing a number of threads assigned to a selected one of the thread pools, prior to operation of the computer-readable program code means for gathering new run-time statistics;

and wherein the computer-readable program code means for programmatically reversing further comprises computer-readable program code means for decrementing the number of threads assigned to the selected one of the thread pools.

60. The computer program product according to claim 44, wherein the computer-readable program code means for programmatically altering further comprises computer-readable program code means for decrementing a number of threads assigned to a selected one of the thread pools, prior to operation of the computer-readable program code means for gathering new run-time statistics;

and wherein the computer-readable program code means for programmatically reversing further comprises computer-readable program code means for incrementing the number of threads assigned to the selected one of the thread pools.

61. The computer program product according to claim 44, wherein the baseline run-time statistics and the new run-time statistics are average execution time, standard deviation of execution time, average queued time, and standard deviation of queued time for servicing the requests comprising the changeable workload.

62. A method of doing business by dynamically tuning thread pools of a multithreaded server, comprising steps of:

programmatically gathering baseline operational characteristics of a workload at a multithreaded server, wherein the baseline operational characteristics pertain to executing a first plurality of requests of the dynamically changeable workload, the requests being serviced by a plurality of thread pools;

programmatically altering the thread pools by selectively incrementing or decrementing one of (1) a number of the thread pools used to service requests of the workload and (2) a number of threads assigned to a particular one of the thread pools;

programmatically gathering new operational characteristics for the dynamically changeable workload, wherein the new operational characteristics pertain to executing a second plurality of requests of the dynamically changeable workload, the requests being serviced by the programmatically altered thread pools;

programmatically reversing the programmatic alteration if a comparison of the new run-time statistics to the baseline run-time statistics indicates that performance was degraded as a result of the programmatic alteration; and

charging a fee for carrying out the steps of programmatically gathering baseline operational characteristics, programmatically altering, programmatically gathering new operational characteristics, and programmatically reversing steps.