Thread scheduling techniques for multithreaded servers

• US 6,427,161 B1
• Filed: 06/12/1998
• Issued: 07/30/2002
• Est. Priority Date: 06/12/1998
• Status: Expired due to Term

• Current Assignee: Trend Micro Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: LiVecchi, Patrick Michael
• Primary Examiner(s): BANANKHAH, MAJID A
• Application Number:
  US09/097,282
• Time in Patent Office: 1,509 Days
• Field of Search: 709/100-108, 709/202, 709/203, 709/315, 709/201, 709/317, 709/318, 709/313
• US Class Current: 718/102
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues

Abstract

A technique, system, and computer program for enhancing performance of a computer running a multithreaded server application. A scheduling heuristic is defined for optimizing the number of available threads. This heuristic alleviates over-scheduling of worker threads by defining a technique to wait to assign an incoming request to a currently-executing thread (upon completion of the thread'"'"'s current work), instead of awakening a blocked thread for the incoming request. Provision is made to ensure no thread waits too long. Two stages are associated with a passive socket, so that a connection is only bound to a worker thread when work arrives for that connection. A new type of socket is defined, for merging input from more than one source and making that merged input available for scheduling. A giveback function is defined, for optimizing assignment of threads to incoming requests when persistent connections are used. Threads that go idle are put onto an idle queue, releasing them from a worker thread.

Claims

1. In a computing environment having a connection to a network, computer readable code readable by a computer system in said environment, for enhancing performance of a multithreaded application, comprising:

a plurality of client requests for connections;

a plurality of worker threads;

a subprocess for receiving said plurality of client requests; and

a subprocess for implementing a scheduling heuristic to alleviate over-scheduling of said worker threads.

2. Computer readable code for enhancing performance of a multithreaded application according to claim 1, wherein:

a first group of said worker threads are active threads, said first group being comprised of changeable ones of said plurality of worker threads, and having a changeable number of said changeable ones, said changeable number being at least one; and

said subprocess for implementing a scheduling heuristic further comprises a subprocess for balancing said changeable number in said first group against a current workload comprised of one or more of said plurality of client requests.

3. Computer readable code for enhancing performance of a multithreaded application according to claim 2, wherein said subprocess for balancing further comprises using an average delay.

4. Computer readable code for enhancing performance of a multithreaded application according to claim 3, wherein said subprocess for balancing further comprises using a maximum delay.

5. Computer readable code for enhancing performance of a multithreaded application according to claim 4, wherein said average delay and said maximum delay are configuration parameters.

6. Computer readable code for enhancing performance of a multithreaded application according to claim 2, wherein:

a second group of said worker threads are blocked threads, said second group being comprised of ones of said plurality of worker threads which are not in said first group; and

said blocked threads are stored in a Last-In, First-Out queue.

28. The method for enhancing performance of a multithreaded application according to claim 1, wherein:

said binding step further comprises using a 2-stage queue; and

said unbinding step further comprises using said 2-stage queue.

29. The method for enhancing performance of a multithreaded application according to claim 28 wherein:

said binding using said 2-stage queue step further comprises the steps of;

moving each of said persistent connections to said second stage when an initial data packet arrives for said connection;

moving each of said persistent connections from said second stage to said first stage when data is received for said connection; and

scheduling said persistent connections from said first stage; and

said unbinding using said 2-stage queue step further comprises the steps of;

moving selected ones of said bound connections from said first stage to said second stage when said selected bound connection goes idle;

closing selected ones of said persistent connections in said second stage, responsive to a maximum idle period being exceeded; and

making said unbound worker thread available to said subprocess for binding.

30. The method for enhancing performance of a multithreaded application according to claim 29, wherein said unbinding step further comprises the step of:

closing further selected ones of said persistent connections in said second stage, responsive to exceeding a maximum number of idle connections.

7. In a computing environment having a connection to a network, computer readable code readable by a computer system in said environment, for enhancing performance of a multithreaded application, comprising:

a plurality of persistent connections;

a plurality of worker threads;

a subprocess for binding selected ones of said persistent connections to selected ones of said worker threads, wherein an execution of said subprocess for binding results in a bound connection; and

a subprocess for unbinding selected ones of said bound connections, wherein an execution of said subprocess for unbinding results in an unbound worker thread.

8. Computer readable code for enhancing performance of a multithreaded application according to claim 7, wherein:

said subprocess for binding further comprises using a 2-stage queue; and

said subprocess for unbinding further comprises using said 2-stage queue.

9. Computer readable code for enhancing performance of a multithreaded application according to claim 8, wherein:

said subprocess for binding using said 2-stage queue further comprises;

a subprocess for moving each of said persistent connections to said second stage when an initial data packet arrives for said connection;

a subprocess for moving each of said persistent connections from said second stage to said first stage when data is received for said connection; and

a subprocess for scheduling said persistent connections from said first stage; and

said subprocess for unbinding using said 2-stage queue further comprises;

a subprocess for moving selected ones of said bound connections from said first stage to said second stage when said selected bound connection goes idle;

a subprocess for closing selected ones of said persistent connections in said second stage, responsive to a maximum idle period being exceeded; and

a subprocess for making said unbound worker thread available to said subprocess for binding.

10. Computer readable code for enhancing performance of a multithreaded application according to claim 9, wherein said subprocess for unbinding further comprises:

a subprocess for closing further selected ones of said persistent connections in said second stage, responsive to exceeding a maximum number of idle connections.

11. A system for enhancing performance of a multithreaded application in a computing environment having a connection to a network, comprising:

a plurality of client requests for connections;

a plurality of worker threads;

means for receiving said plurality of client requests; and

means for implementing a scheduling heuristic to alleviate over-scheduling of said worker threads.

12. The system for enhancing performance of a multithreaded application according to claim 11, wherein:

a first group of said worker threads are active threads, said first group being comprised of changeable ones of said plurality of worker threads, and having a changeable number of said changeable ones, said changeable number being at least one; and

said means for implementing a scheduling heuristic further comprises means for balancing said changeable number in said first group against a current workload comprised of one or more of said plurality of client requests.

13. The system for enhancing performance of a multithreaded application according to claim 12, wherein said means for balancing further comprises using an average delay.

14. The system for enhancing performance of a multithreaded application according to claim 13, wherein said means for balancing further comprises using a maximum delay.

15. The system for enhancing performance of a multithreaded application according to claim 14, wherein said average delay and said maximum delay are configuration parameters.

16. The system for enhancing performance of a multithreaded application according to claim 12, wherein:

a second group of said worker threads are blocked threads, said second group being comprised of ones of said plurality of worker threads which are not in said first group; and

said blocked threads are stored in a Last-In, First-Out queue.

17. A system for enhancing performance of a multithreaded application in a computing environment having a connection to a network, comprising:

a plurality of persistent connections;

a plurality of worker threads;

means for binding selected ones of said persistent connections to selected ones of said worker threads, wherein an execution of said subprocess for binding results in a bound connection; and

means for unbinding selected ones of said bound connections, wherein an execution of said subprocess for unbinding results in an unbound worker thread.

18. The system for enhancing performance of a multithreaded application according to claim 17, wherein:

said means for binding further comprises using a 2-stage queue; and

said means for unbinding further comprises using said 2-stage queue.

19. The system for enhancing performance of a multithreaded application according to claim 18, wherein:

said means for binding using said 2-stage queue further comprises;

means for moving each of said persistent connections to said second stage when an initial data packet arrives for said connection;

means for moving each of said persistent connections from said second stage to said first stage when data is received for said connection; and

means for scheduling said persistent connections from said first stage; and

said means for unbinding using said 2-stage queue further comprises;

means for moving selected ones of said bound connections from said first stage to said second stage when said selected bound connection goes idle;

means for closing selected ones of said persistent connections in said second stage, responsive to a maximum idle period being exceeded; and

means for making said unbound worker thread available to said subprocess for binding.

20. The system for enhancing performance of a multithreaded application according to claim 19, wherein said means for unbinding further comprises:

means for closing further selected ones of said persistent connections in said second stage, responsive to exceeding a maximum number of idle connections.

21. A method for enhancing performance of a multithreaded application in a computing environment having a connection to a network, comprising the steps of:

receiving a plurality of client requests for connections; and

implementing a scheduling heuristic to alleviate over-scheduling of a plurality of worker threads to said plurality of client requests.

22. The method for enhancing performance of a multithreaded application according to claim 21, wherein:

a first group of said worker threads are active threads, said first group being comprised of changeable ones of said plurality of worker threads, and having a changeable number of said changeable ones, said changeable number being at least one; and

said implementing a scheduling heuristic step further comprises balancing said changeable number in said first group against a current workload comprised of one or more of said plurality of client requests.

23. The method for enhancing performance of a multithreaded application according to claim 22, wherein said balancing step further comprises using an average delay.

24. The method for enhancing performance of a multithreaded application according to claim 23, wherein said balancing step further comprises using a maximum delay.

25. The method for enhancing performance of a multithreaded application according to claim 24, wherein said average delay and said maximum delay are configuration parameters.

26. The method for enhancing performance of a multithreaded application according to claim 22, wherein:

a second group of said worker threads are blocked threads, said second group being comprised of ones of said plurality of worker threads which are not in said first group; and

said blocked threads are stored in a Last-In, First-Out queue.

27. A method for enhancing performance of a multithreaded application in a computing environment having a connection to a network, comprising the steps of:

binding selected ones of a plurality of persistent connections to selected ones of a plurality of worker threads, wherein an execution of said binding step results in a bound connection; and

unbinding selected ones of said bound connections, wherein an execution of said unbinding step results in an unbound worker thread.