Epoll optimisations

• US 9,384,071 B2
• Filed: 06/10/2011
• Issued: 07/05/2016
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: Xilinx, Inc. (Advanced Micro Devices, Inc.), Cellco Partnership, Inc. (Verizon Communications Inc.)
• Original Assignee: SolarFlare Communications Incorporated (Advanced Micro Devices, Inc.)
• Inventors: Pope, Steven L., Riddoch, David J.
• Primary Examiner(s): Sough, H S
• Assistant Examiner(s): Yun, Carina
• Application Number:
  US13/158,176
• Publication Number:
  US 20120254893A1
• Time in Patent Office: 1,852 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 9/48 Program initiating; Program switching, e.g. by interrupt
• CPC Class Codes: G06F 9/545 where tasks reside in different layers, e.g. user- and kernel-space

Abstract

A method for managing I/O event notifications in a data processing system, the data processing system comprising a plurality of applications and an operating system having a kernel and an I/O event notification mechanism operable to maintain a plurality of I/O event notification objects each handling a set of file descriptors associated with one or more I/O resources, the method comprising: for each of a plurality of application-level configuration calls: intercepting at a user-level interface a configuration call from an application to the I/O event notification mechanism for configuring an I/O event notification object; and storing a set of parameters of the configuration call at a data structure, each set of parameters representing an operation on the set of file descriptors handled by the I/O event notification object; and subsequently, on a predetermined criterion being met: the user-level interface causing the plurality of configuration calls to be effected by means of a first system call to the kernel.

Claims

1. A method for managing I/O event notifications in a data processing system, the data processing system comprising a plurality of applications and an operating system having a kernel and an I/O event notification mechanism operable to maintain a plurality of I/O event notification objects each handling a set of file descriptors associated with one or more I/O resources, the method comprising:

for each of a plurality of application-level configuration calls;

a user-level interface intercepting a configuration call from an application to the I/O event notification mechanism for configuring an I/O event notification object; and

storing a set of parameters of the configuration call at a data structure, each set of parameters representing an operation on the set of file descriptors handled by the I/O event notification object;

intercepting at the user-level interface a wait call from the application to the I/O event notification mechanism for causing the I/O event notification object to wait for an event associated with one or more of its file descriptors;

subsequently, on a predetermined criterion being met, the user-level interface causing the plurality of configuration calls to be effected by making a single system call from the user level to the kernel,

wherein the predetermined criterion is reception of a wait call from the application at the user-level interface, and wherein the predetermined criterion is additionally met if no wait call is received at the user-level interface but one of the following occurs;

a predetermined time period expires;

or

storage space required at the data structure for storing the sets of parameters representing operations to be performed on the set of file descriptors handled by the I/O event notification object exceeds a predetermined size.

2. A method as claimed in claim 1, wherein the step of causing the plurality of configuration calls to be effected comprises:

the user-level interface making the single system call to the kernel so as to pass the data structure to a kernel library; and

the kernel library invoking a configuration routine of the I/O event notification mechanism for each set of parameters stored in the data structure so as to configure the I/O event notification object in accordance with the configuration calls.

3. A data processing system comprising a CPU and memory,

the CPU having a plurality of applications executing thereon;

the CPU further having an operating system executing thereon, the operating system having a kernel and an I/O event notification mechanism operable to maintain a plurality of I/O event notification objects each handling a set of file descriptors associated with one or more I/O resources;

the CPU further configured to store in a data structure parameters of application-level configuration calls, each set of parameters representing an operation on the set of file descriptors handled by the I/O event notification object to which the respective configuration call is directed; and

the CPU having a user-level interface executing thereon, the user-level interface configured to intercept a plurality of configuration calls from an application to the I/O event notification mechanism for configuring an I/O event notification object and store parameters of the configuration calls at the data structure,

the user-level interface being further configured to intercept a wait call from the application to the I/O event notification mechanism for causing the I/O event notification object to wait for an event associated with one or more of its file descriptors;

wherein the user-level interface is configured to, on a predetermined criterion being met, cause the plurality of configuration calls to be effected by passing at least part of the data structure into the kernel context and making a system call from the user level to the kernel,

wherein the predetermined criterion is the reception of a wait call from the application at the user-level interface, and wherein the predetermined criterion is additionally met if no wait call is received at the user-level interface but one of the following occurs;

a predetermined time period expires;

or

the storage space required at the data structure for storing the sets of parameters representing operations to be performed on the set of file descriptors handled by the I/O event notification object exceeds a predetermined size.

4. A method for managing I/O event notifications in a data processing system, the data processing system comprising a plurality of applications and an operating system having a kernel and an I/O event notification mechanism operable to maintain a plurality of I/O event notification objects each handling a set of file descriptors associated with one or more I/O resources, the method comprising:

a user-level interface receiving a plurality of application-level configuration calls from one or more applications toward the I/O event notification mechanism, each of the configuration calls including a respective set of parameters representing an operation on a set of file descriptors handled by an I/O event notification object of the I/O event notification mechanism of the operating system;

the user-level interface receiving a wait call sent from an application toward the I/O event notification mechanism for causing the I/O event notification object to wait for an event associated with one or more of its file descriptors;

in response to receipt of each particular one of the application-level configuration calls in the plurality of application-level configuration calls, the user-level interface storing at a data structure in user address space the set of parameters of the particular configuration call;

subsequently, on a predetermined criterion being met, the user-level interface making a single system call from the user level to the kernel so as to pass the data structure to a kernel module; and

the kernel module invoking a configuration routine of the I/O event notification mechanism for the sets of parameters stored in the data structure so as to configure the I/O event notification object in accordance with the configuration calls,

wherein the predetermined criterion is the reception of a wait call from the application at the user-level interface and wherein the predetermined criterion is additionally met if no wait call is received at the user-level interface but one of the following occurs;

a predetermined time period expires;

or

storage space required at the data structure for storing the sets of parameters representing operations to be performed on the set of file descriptors handled by the I/O event notification object exceeds a predetermined size.

5. A method as claimed in claim 4, further comprising, subsequent to each receiving step, returning processing to the application that made the respective configuration call.

6. A method as claimed in claim 4, wherein the receiving and storing steps are performed without invoking any kernel level processes.

7. A method as claimed in claim 4, wherein the I/O event notification mechanism is configured such that each file descriptor of an I/O event notification object is individually configurable by means of a configuration call from the application to the I/O event notification mechanism.

8. A method as claimed in claim 4, wherein the data structure is passed to the kernel module by one of:

passing a reference to the data structure, passing a copy of at least part of the data structure, and passing a representation of the data structure into the kernel context.

9. A method as claimed in claim 4, wherein the I/O event notification mechanism is epoll and the configuration routine is epoll_ctl( ).

10. A method as claimed in claim 4, wherein each operation on the set of file descriptors held by the I/O event notification object is an operation to add a file descriptor to the set, modify an event type to be monitored by the I/O event notification object in respect of a file descriptor of the set, or delete a file descriptor from the set of file descriptors.

11. A method as claimed in claim 4, wherein more than one of the sets of parameters stored at the data structure are associated with the same file descriptor of the I/O event notification object, further comprising:

combining into a single operation those operations represented by the more than one set of parameters; and

replacing the operations represented by those sets of parameters with the single operation.

12. A method as claimed in claim 11, wherein the operations represented by the more than one set of parameters stored at the data structure cancel each other out, and wherein the single operation is do nothing.

13. A method as claimed in claim 11, wherein the combining step is performed by the user-level interface or the kernel module.

14. A method as claimed in claim 4, further comprising, subsequent to the step of making a single system call, the user-level interface causing the I/O event notification object to wait for an event associated with one or more of the file descriptors of the I/O event notification object by means of a subsequent system call to the kernel.

15. A method as claimed in claim 14, wherein the subsequent system call is to the kernel module and the step of causing the I/O event notification object to wait for an event comprises invoking a wait routine of the I/O event notification mechanism.

16. A method as claimed in claim 15, wherein the I/O event notification mechanism is epoll and the wait routine is epoll_wait( ) or epoll_pwait( ).

17. A method as claimed in claim 14, wherein the single and subsequent system calls are made by means of a single unified system call from the user-level interface to the kernel.

18. A method as claimed in claim 4, wherein the operating system comprises a plurality of I/O event notification objects and the user-level interface relates to a single I/O event notification object.

19. A method as claimed in claim 4, wherein the I/O event notification object handles a set of file descriptors corresponding to a network interface device.

20. A method as claimed in claim 19, wherein the file descriptors correspond to receive or transmit buffers.

21. A method as claimed in claim 4, wherein the user-level interface is provided at a user-level network stack.

22. A method as claimed in claim 4, further comprising the step of not enabling interrupts at the operating system in respect of file descriptors on which the operations represented by the sets of parameters stored at the data structure are operations to add or modify a file descriptor managed by the user-level network stack.

23. A data processing system comprising a CPU and memory, the CPU having a plurality of applications executing thereon;

the CPU further having an operating system executing thereon, the operating system having a kernel and an I/O event notification mechanism operable to maintain a plurality of I/O event notification objects each handling a set of file descriptors associated with one or more I/O resources;

the CPU having a user-level interface executing thereon, the user-level interface configured to receive a plurality of application-level configuration calls from one or more applications toward the I/O event notification mechanism, each of the configuration calls including a respective set of parameters representing an operation on a set of file descriptors handled by an I/O event notification object of the I/O event notification mechanism of the operating system;

the user-level interface further configured to store at a data structure in user address space the set of parameters of each of the configuration calls in the plurality of application-level configuration calls;

the user-level interface being further configured to receive a wait call sent from an application toward the I/O event notification mechanism for causing the I/O event notification objection to wait for an event associated with one or more of its file descriptors;

the user-level interface further configured to, on a predetermined criterion being met, make a single system call from the user level to the kernel so as to pass to a kernel module the data structure having stored therein the sets of parameters of each of the configuration calls in the plurality of application-level configuration calls; and

the kernel module configured to invoke a configuration routine of the I/O event notification mechanism for the sets of parameters stored in the data structure so as to configure the I/O event notification object in accordance with the configuration calls,

wherein the predetermined criterion is the reception of a wait call from the application at the user-level interface and wherein the predetermined criterion is additionally met if no wait call is received at the user-level interface but one of the following occurs;

a predetermined time period expires;

or

the storage space required at the data structure for storing the sets of parameters representing operations to be performed on the set of file descriptors handled by the I/O event notification object exceeds a predetermined size.