Generic application server and method of operation therefor

• US 7,051,330 B1
• Filed: 11/21/2000
• Issued: 05/23/2006
• Est. Priority Date: 11/21/2000
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kaler, Christopher G., Kruy, Steven J.
• Primary Examiner(s): An, Meng-Al T.
• Assistant Examiner(s): Ali, Syed J
• Application Number:
  US09/717,675
• Time in Patent Office: 2,009 Days
• Field of Search: 709200-203, 718/100, 718102-104, 718106-108, 719312-314, 719/318
• US Class Current: 718/106
• CPC Class Codes: G06F 9/4843 by program, e.g. task dispatcher, supervisor, operating system

Abstract

A generic application server is capable of simultaneously receiving requests, processing requested work, and returning results using multiple, conceptual thread pools. In addition, functions are programmable as state machines. While executing such a function, when a worker thread encounters a potentially blocking condition, the thread issues an asynchronous request for data, a state transition is performed, and the thread is released to do other work. After the blocking condition is relieved, another worker thread is scheduled to advance to the next function state and continue the function. Multiple priority work queues are used to facilitate completion of functions already in progress. In addition, lower-priority complex logic threads can be invoked to process computationally intense logic that may be necessitated by a request. Throttling functions are also implemented, which control the quantity of work accepted into the server and server response time.

Claims

1. A method in a computer system for servicing requests from one or more client computers, the method comprising:

receiving a request from a client computer;

a first thread processing the request by invoking a receive handler that creates a work item, wherein the first thread is part of a pool of generic threads;

a second thread performing a task specified in the work item by invoking a work handler, wherein the second thread is part of the pool of generic threads;

receiving a result of performing the task;

a third thread returning at least a portion of the result to the client computer by invoking a reply handler, wherein the third thread is part of the pool of generic threads;

determining a size of the result;

when the size of the result is not larger than a cutoff size, storing the result in a partial results cache;

the first thread, the second thread or the third thread indicating that the first thread, the second thread or the third thread has completed a work item; and

if a quantum has not expired for the first thread, the second thread or the third thread, then the first thread, the second thread or the third thread being given an additional work item to perform without relinquishing the central processing unit upon which the first thread, the second thread or the third thread was running.

2. The method as claimed in claim 1, wherein receiving the request comprises:

receiving the request into an input/output port; and

placing a reference in a queue indicating that work is available for the first thread.

3. The method as claimed in claim 2, further comprising:

invoking a receive thread manager when the work is available; and

the receive thread manager scheduling the first thread for execution on one of multiple processors.

4. The method as claimed in claim 3, wherein the receive thread manager schedules the first thread from a queue of available threads.

5. The method as claimed in claim 1, further comprising the first thread placing the work item on a work queue for execution by the second thread.

6. The method as claimed in claim 5, further comprising the first thread placing a reference in a completion port queue, indicating that work is available for the second thread.

7. The method as claimed in claim 1, further comprising the second thread creating and placing a second work item on a reply work queue when the results are received.

8. The method as claimed in claim 7, further comprising the second thread placing a reference in a completion port queue, indicating that work is available for the third thread.

9. The method as claimed in claim 1, further comprising:

receiving input data; and

the first thread storing the input data in a cache that is accessible to the second thread.

10. The method as claimed in claim 1, wherein the results include data, and further comprising storing the data in a cache that is accessible to the third thread.

11. The method as claimed in claim 1, further comprising:

the third thread returning a portion of the result to client computer; and

if a second request is received for an additional portion of the result that is stored in the partial results cache, a fourth thread creating a second work item by invoking another receive handler, wherein the second work item is processed by a fifth thread, which invokes another reply handler.

12. A method in a computer system for servicing requests from multiple client computers, the method comprising:

determining that work is available after receiving a request from a client computer, wherein the request from the client computer is a request to perform a function having multiple states;

when work is available, a first work handler invoked by a first thread looking in a first work queue for a first work item corresponding to the work;

if the first work item is not found in the first work queue, the first work handler looking in a second work queue for the first work item, wherein the computer system includes multiple work queues, including the first work queue and the second work queue, each of the multiple work queues are associated with a priority level;

looking for the first work item first in a work queue associated with a highest priority level; and

if the first work item is not found in the work queue associated with the highest priority level, looking for the first work item in each of the multiple work queues in descending priority order until the first work item is found, wherein each state of the multi-state function is performed by a work handler invoked by a subsequent thread based on work items placed in the multiple work queues, and wherein the work items are placed in higher and higher priority work queues as execution of the multi-state function progresses through the multiple states.

13. The method as claimed in claim 12, further comprising:

receiving a request from a client computer to perform a task;

creating the first work item that specifies the task;

placing the first work item in the second queue; and

indicating that the work is available.

14. The method as claimed in claim 12, further comprising:

the first work handler performing a task specified in the first work item; and

issuing an asynchronous request for data.

15. The method as claimed in claim 14, further comprising:

receiving the data;

placing a second work item on the first work queue; and

indicating that additional work is available.

16. The method as claimed in claim 15, further comprising:

a second work handler invoked by a second thread looking in the first work queue for the second work item when the second work is available; and

performing a second task specified in the second work item.

17. A method in a computer system for servicing requests from multiple client computers, the method comprising:

receiving, from a client computer, a request to perform a first task;

evaluating the first task, by a first handler invoked by a first thread, to determine whether the first task includes complex or long-running logic; and

if the first task includes complex or long-running logic, performing the first task by a second handler invoked by a second thread, wherein the first and second threads are all identical threads within a pool of generic threads, wherein the first thread is a thread within a first group of threads having a first priority level, and the second thread is a thread within a second group of threads having a second priority level that is lower than the first priority level; and

receiving a request to perform a second task;

evaluating the second task to determine whether the second task includes complex or long-running logic; and

if the second task does not include complex or long-running logic and a processor is not available for performing the second task, preempting the second thread and performing the second task by a third thread in the first group of threads.

18. The method as claimed in claim 17, further comprising:

the second handler returning a result of the first task; and

using the result, performing a second task by a third handler invoked by a third thread of the first group of threads.

19. The method as claimed in claim 18, wherein the first task is specified in a first work item and the second task is specified in a second work item, the method further comprising:

the first handler obtaining the first work item from a first work queue; and

the third handler obtaining the second work item from a second work queue.

20. A computer system for servicing requests from one or more client computers, the computer system comprising:

a memory containing an application server which maintains a pool of threads, wherein each thread in the pool of threads is identical and can invoke at least one receive handler, at least one work handler, and at least one reply handler, and the application server further schedules threads in the pool of threads for execution on multiple processors available to the computer system; and

the multiple processors for simultaneously executing multiple threads within the pool of threads.

21. A computer-readable medium holding computer executable instructions, the computer-readable medium for performing a method in a computer system, the method comprising:

maintaining a pool of threads, wherein each thread in the pool of threads is identical and can invoke at least on receive handler, at least one work handler, and at least one reply handler, and the application server further schedules threads in the pool of threads for execution on multiple processors available to the computer system; and

simultaneously executing multiple threads within the pool of threads on multiple processors available to the computer system.