System and method for supporting work sharing muxing in a cluster

• US 9,086,909 B2
• Filed: 01/31/2013
• Issued: 07/21/2015
• Est. Priority Date: 05/17/2011
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Otenko, Oleksandr
• Primary Examiner(s): CAO, DIEM K
• Application Number:
  US13/756,207
• Publication Number:
  US 20140215475A1
• Time in Patent Office: 901 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 9/46 Multiprogramming arrangements
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues

Abstract

A system and method can provide efficient low-latency muxing between servers in the cluster. One such system can include a cluster of one or more high performance computing systems, each including one or more processors and a high performance memory. The cluster communicates over an InfiniBand network. The system can also include a middleware environment, executing on the cluster, which includes one or more application server instances. The system can include one or more selectors, wherein each said selector contains a queue of read-ready file descriptors. Furthermore, the system can include a shared queue, wherein the read-ready file descriptors in each said selector can be emptied into the shared queue. Additionally, a plurality of muxer threads operates to take work from said shared queue.

Claims

1. A system for supporting work sharing multiplexing in a cluster, comprising:

one or more microprocessors;

a plurality of file descriptor caches that each contain one or more read-ready file descriptors;

a plurality of selectors and a plurality of selector queues, wherein each of said plurality of selectors is associated with a different selector queue of said plurality of selector queues, and wherein each of said plurality of selectors polls an associated file descriptor cache of said plurality of file descriptor caches, and writes read-ready file descriptors from the associated file descriptor cache to the associated selector queue of said plurality of selector queues;

a single shared queue coupled to said plurality of selectors, wherein each selector queue of said plurality of selector queues empties file descriptors from the associated selector queue of the plurality of selector queues into the single shared queue such that all read-ready file descriptors polled from the plurality of file descriptor caches by the plurality of selectors are provided to the single shared queue; and

a plurality of worker threads, running on the one or more microprocessors, that operates to process said read-ready file descriptors provided to said single shared queue.

2. The system according to claim 1, wherein said single shared queue includes read-ready file descriptors and related selectors of said plurality of selectors.

3. The system according to claim 2, wherein:

a thread associated with a said selector is blocked when it is waiting for a scan on the associated file descriptor cache to complete.

4. The system according to claim 2, wherein:

a thread associated with a said selector operates to copy the one or more read-ready file descriptors into the one or more selectors.

5. The system according to claim 1, wherein:

each said worker thread maintains a thread-local list of read-ready sockets that it is processing.

6. The system according to claim 1, further comprising:

a request manager that is capable of handling one or more requests from different servers in the cluster.

7. The system according to claim 6, wherein:

the request manager contains multiple queues where the one or more requests prepared by the plurality of worker threads are added to.

8. The system according to claim 1, wherein:

there are more said worker threads than said selectors.

9. The system according to claim 1, wherein:

said worker threads operate to continuously obtain read-ready sockets from the shared queue and process them without the need to get suspended or to perform context switches.

10. The system according to claim 1, wherein:

a worker thread that returns from a said selector can retain one last read-ready socket.

11. A method for supporting work sharing multiplexing in a cluster, comprising:

providing a plurality of file descriptor caches that each contain one or more read-ready file descriptors;

providing a plurality of selectors and a plurality of selector queues, wherein each of said plurality of selectors is associated with an associated file descriptor cache of the plurality of file descriptor caches and a different selector queue of said plurality of selector queues;

providing a single shared queue coupled to said plurality of selectors;

polling said plurality of file descriptor caches with said plurality of selectors wherein each selector polls the associated file descriptor cache;

writing read-ready file descriptors from each selector of said plurality of selectors to the selector queue associated with said each selector;

emptying the read-ready file descriptors from all of the plurality of selector queues into the single shared queue such that all read-ready file descriptors polled from the plurality of file descriptor caches by the plurality of selectors are provided to the single shared queue; and

processing said read-ready file descriptors provided to said single shared queue with a plurality of worker threads.

12. The method according to claim 11, wherein said single shared queue includes read-ready file descriptors and related selectors of said plurality of selectors.

13. The method according to claim 12, further comprising:

configuring a thread associated with each selector to be blocked when it is waiting for a scan on the associated file descriptor cache to complete.

14. The method according to claim 12, further comprising:

configuring a thread associated with each selector to copy the one or more read-ready file descriptors into the one or more selectors.

15. The method according to claim 11, further comprising:

configuring each said worker thread to maintain a thread-local list of read-ready sockets that it is processing.

16. The method according to claim 11, further comprising:

providing a request manager that is capable of handling one or more requests from different servers in the cluster.

17. The method according to claim 16, further comprising:

configuring the request manager to contain multiple queues where the one or more requests prepared by the plurality of worker threads are added to.

18. The method according to claim 11, further comprising:

providing more worker threads than said selectors.

19. The method according to claim 11, further comprising:

configuring one or more said worker threads to continuously obtain read-ready sockets from the selectors and process them without the need to get suspended or to perform context switches.

20. A non-transitory machine readable storage medium having instructions stored thereon for supporting work sharing multiplexing in a cluster, which instructions, when executed, cause a system to perform steps comprising:

providing a plurality of file descriptor caches each containing one or more read-ready file descriptors;

providing a plurality of selectors and a plurality of selector queues, wherein each of said plurality of selectors is associated with an associated file descriptor cache of the plurality of file descriptor caches and a different selector queue of said plurality of selector queues;

providing a single shared queue coupled to said plurality of selectors;

polling said plurality of file descriptor caches with said plurality of selectors wherein each selector polls the associated file descriptor cache;

writing read-ready file descriptors from each selector of said plurality of selectors to the selector queue associated with said each selector;

emptying the read-ready file descriptors from all of the plurality of selector queues into the single shared queue such that all read-ready file descriptors polled from the plurality of file descriptor caches by the plurality of selectors are provided to the single shared queue; and

processing work said read-ready file descriptors provided to said single shared queue with a plurality of worker threads.