Dynamic optimization for IP forwarding performance

• US 10,284,502 B2
• Filed: 10/09/2016
• Issued: 05/07/2019
• Est. Priority Date: 10/12/2015
• Status: Active Grant

• Current Assignee: Mellanox Technologies Limited (NVIDIA Corporation)
• Original Assignee: Mellanox Technologies Limited (NVIDIA Corporation)
• Inventors: Degani, Yuval, Tor, Yohad
• Primary Examiner(s): Gidado, Rasheed
• Application Number:
  US15/289,176
• Publication Number:
  US 20170104697A1
• Time in Patent Office: 940 Days
• CPC Class Codes: H04L 49/9094 Arrangements for simultaneous transmit and receive, e.g. simultaneous reading/writing from/to the storage element

Abstract

A host connected to at least one data network has a processor having a plurality of cores, and a memory. A network interface controller is coupled to the host, and configured to transmit and receive data packets via multiple distinct physical ports. The host and the network interface controller are cooperative upon receiving a packet for storing the packet in a receive buffer of the memory, deciding in the host, responsively to a destination identifier in the packet, to forward the packet from the host to the at least one data network via another one of the physical ports, and selecting one of the cores to perform a send operation.

Claims

1. A method, comprising the steps of:

coupling a network interface controller to a host, the host comprising a processor having a plurality of cores, including a first core and a second core, the network interface controller being configured to transmit and receive data packets over a data network via multiple distinct physical ports that are connected to at least one data network;

upon receiving a packet belonging to a flow from the network via one of the physical ports assigning a first core to store the packet in a receive buffer of a memory having pages, the pages having virtual start addresses;

deciding in the host, responsively to a destination identifier in the packet, to perform a send operation on the packet;

selecting the second core to perform the send operation by excluding the first core from selection;

with the second core, executing the send operation to forward the packet from the host into the at least one data network via another of the physical ports; and

using new packets belonging to the flow as the packet, iterating the steps of;

receiving a packet and storing the packet with the first core; and

with the second core executing new send operations to forward the new packets exclusively via the other physical port.

2. The method according to claim 1, wherein deciding in the host comprises:

determining that a memory element of the packet and the receive buffer are co-located on a common page of the memory; and

wherein

performing a send operation comprises referencing the receive buffer containing the memory element while avoiding copying the receive buffer to a send buffer.

3. The method according to claim 2, wherein the step of determining that the memory element of the packet and the receive buffer are co-located on the common page comprises determining that the memory element and the receive buffer are on pages having a common start address.

4. The method according to claim 1, further comprising:

inserting a unique signature in the receive buffer; and

determining that a memory element of the packet and the receive buffer share the unique signature; and

performing a send operation comprises referencing the receive buffer containing the memory element to forward the packet while avoiding copying the receive buffer to a send buffer.

5. The method according to claim 1, wherein receiving a packet and performing a send operation on the packet are performed by respective instances of a network driver, and wherein deciding in the host comprises making a determination that a thread handle of a prior send operation in the flow and a thread handle of a receive operation for the packet are identical.

6. The method according to claim 5, further comprising:

responsively to the determination initiating a request for the new send operations for the packet in a running thread context of the receive operation for the packet, wherein the new send operations and the receive operation have a common thread handle.

7. The method according to claim 1, further comprising selecting the second core from the plurality of cores according to a fixed rotation schedule and excluding the first core from the schedule.

8. The method according to claim 1, further comprising selecting the second core by identifying an idle core.

9. The method according to claim 1, wherein at least a portion of the cores are processing elements in a logical processor.

10. The method according to claim 1, wherein selecting the second core comprises selecting only cores allocated for receive side scaling.

11. The method according to claim 1, wherein selecting the second core comprises selecting only cores assigned to receiving packets from queues.

12. An apparatus, comprising:

a host connected to at least one data network, the host comprising a processor having a plurality of cores, including a first core and a second core, and a memory having pages, the pages having virtual start addresses;

a network interface controller coupled to the host, and configured to transmit and receive data packets over the at least one data network via multiple distinct physical ports, the host and the network interface controller cooperative for;

upon receiving a packet belonging to a flow from the at least one data network via one of the physical ports assigning the first core to store the packet in a receive buffer of the memory;

deciding in the host, responsively to a destination identifier in the packet, to perform a send operation on the packet;

selecting the second core to perform the send operation, by excluding the first core from selection;

with the second core, executing the send operation to forward the packet from the host into the at least one data network via another of the physical ports; and

using new packets belonging to the flow as the packet, iterating the steps of;

receiving a packet and storing the packet with the first core; and

with the second core executing new send operations to forward the new packets exclusively via the other physical port.

13. The apparatus according to claim 12, wherein deciding in the host comprises:

determining that a memory element of the packet and the receive buffer are co-located on a common page of the memory; and

wherein

performing a send operation comprises referencing the receive buffer containing the memory element while avoiding copying the receive buffer to a send buffer.

14. The apparatus according to claim 13, wherein the step of determining that the memory element of the packet and the receive buffer are co-located on the common page comprises determining that the memory element and the receive buffer are on pages having a common start address.

15. The apparatus according to claim 12, wherein the host and the network interface controller are cooperative for:

inserting a unique signature in the receive buffer; and

determining that a memory element of the packet and the receive buffer share the unique signature; and

performing a send operation comprises referencing the receive buffer containing the memory element to forward the packet while avoiding copying the receive buffer to a send buffer.

16. The apparatus according to claim 12, wherein receiving a packet and performing a send operation on the packet are performed by respective instances of a network driver, and wherein deciding in the host comprises making a determination that a thread handle of a prior send operation in the flow and a thread handle of a receive operation for the packet are identical.

17. The apparatus according to claim 16, wherein the host and the network interface controller are cooperative for:

responsively to the determination initiating a request for the new send operations for the packet in a running thread context of the receive operation for the packet, wherein the new send operations and the receive operation have a common thread handle.

18. The apparatus according to claim 12, wherein selecting the second core is performed according to a fixed rotation schedule and by excluding the first core from the schedule.

19. The apparatus according to claim 12, wherein selecting the second core is performed by identifying an idle core.

20. The apparatus according to claim 12, wherein at least a portion of the cores are processing elements in a logical processor.

21. A method, comprising the steps of:

coupling a network interface controller to a host, the host comprising a processor having a plurality of cores, the network interface controller being configured to transmit and receive data packets over a data network via multiple distinct physical ports that are connected to at least one data network;

upon receiving a packet from the network via one of the physical ports storing the packet in a receive buffer of a memory having pages, the pages having virtual start addresses;

inserting a unique signature in the receive buffer; and

determining that a memory element of the packet and the receive buffer share the unique signature; and

deciding in the host, responsively to a destination identifier in the packet, to perform a send operation on the packet to forward the packet from the host to the at least one data network via another one of the physical ports; and

selecting one of the cores to perform the send operation, by referencing the receive buffer containing the memory element to forward the packet while avoiding copying the receive buffer to a send buffer.

22. The method according to claim 21, wherein the packet is a member of a flow of packets, wherein receiving a packet and performing a send operation on the packet are performed by respective instances of a network driver, and wherein deciding in the host comprises making a determination that a thread handle of a prior send operation in the flow and a thread handle of a receive operation for the packet are identical.

23. The method according to claim 22, further comprising:

responsively to the determination initiating a request for a new send operation for the packet in a running thread context of the receive operation for the packet, wherein the new send operation and the receive operation have a common thread handle.