Mapped FIFO buffering

• US 9,008,113 B2
• Filed: 03/24/2011
• Issued: 04/14/2015
• Est. Priority Date: 12/20/2010
• Status: Active Grant

• Current Assignee: Xilinx, Inc. (Advanced Micro Devices, Inc.)
• Original Assignee: SolarFlare Communications Incorporated (Advanced Micro Devices, Inc.)
• Inventors: Pope, Steven L., Riddoch, David, Kitariev, Dmitri
• Primary Examiner(s): LY, ANH VU H
• Application Number:
  US13/070,924
• Publication Number:
  US 20120155256A1
• Time in Patent Office: 1,482 Days
• Field of Search: None
• US Class Current: 370/463
• CPC Class Codes: G06F 13/128 for dedicated transfers to a network for protocol converters G06F13/387
• CPC Class Codes: H04L 47/6275 based on priority
• CPC Class Codes: H04L 49/3018 Input queuing
• CPC Class Codes: H04L 49/3045 Virtual queuing

Abstract

A network interface device for connection between a network and a data processing system, the network interface device comprising: an I/O interface for connection to a data processing system; a set of physical data ports for connection to a network; a unified memory comprising a plurality of buffers; a plurality of ingress ports operable to receive data packets for buffering at the unified memory, a first subset of the plurality of ingress ports being configured to receive data packets on a transmit path from said I/O interface, and a second subset of the plurality of ingress ports being configured to receive data packets on a receive path from said set of physical data ports; a memory manager configured to store representations of a plurality of virtual queues held in the unified memory, each virtual queue being a linked logical sequence of buffers of the unified memory; and an ingress interface configured to service the ingress ports in a predetermined order and write data packets received at the ingress ports to buffers of the unified memory selected by the memory manager; wherein the memory manager is arranged to select buffers of the unified memory so as to cause the ingress interface to populate the plurality of virtual queues with data packets, and the ingress interface is arranged to contiguously write data packets into the linked logical sequence of buffers representing each virtual queue.

Claims

1. A network interface device for connection between a network and a data processing system, the network interface device comprising:

an I/O interface for connection to a data processing system;

a set of physical data ports for connection to a network;

a unified memory comprising a plurality of buffers for buffering data packets directed to the data processing system before the data packets are provided to the I/O interface;

a plurality of ingress ports operable to receive data packets for buffering at the unified memory, a first subset of the plurality of ingress ports being configured to receive data packets on a transmit path from said data processing system, and a second subset of the plurality of ingress ports being configured to receive data packets on a receive path from said set of physical data ports;

a memory manager configured to store representations of a plurality of virtual queues held in the unified memory, each virtual queue being a linked logical sequence of buffers of the unified memory; and

an ingress interface configured to service the ingress ports in a predetermined order and write data packets received at the ingress ports to buffers of the unified memory selected by the memory manager;

wherein the memory manager is arranged to select buffers of the unified memory so as to cause the ingress interface to populate the plurality of virtual queues with data packets, and the ingress interface is arranged to contiguously write data packets into the linked logical sequence of buffers representing each virtual queue and the ingress interface is configured to delineate data packets by writing an end-of-packet control word such that each buffer is able to store a plurality of data packets and such that a single data packet is able to span a plurality of buffers.

2. A network interface device as claimed in claim 1, wherein the memory manager is configured to define one or more virtual queues for each ingress port such that data packets received at an ingress port are written to virtual queues associated with that ingress port.

3. A network interface device as claimed in claim 1, wherein the memory manager is configured to define for at least some of the ingress ports a plurality of virtual queues, two or more of the plurality of virtual queues having different priority levels.

4. A network interface device as claimed in claim 3, wherein the ingress interface is configured to, on receiving at an ingress port a data packet that includes a priority identifier, assign the data packet to a virtual queue associated with that ingress port and having a priority level corresponding to the priority identifier.

5. A network interface device as claimed in claim 4, wherein the memory manager supports a priority lookup table defining correspondences between possible priority identifiers of data packets received at the ingress ports and the priority levels of the virtual queues, and the ingress interface is configured to, on receiving at an ingress port a data packet that includes a priority identifier, cause the priority lookup table to be accessed so as to identify the virtual queue having the corresponding priority level for that data packet.

6. A network interface device as claimed in claim 4, wherein the priority identifiers are one of:

Class of Service identifiers in accordance with the IEEE 802.1Qbb flow control standard;

a Quality of Service parameter; and

an identifier of a network endpoint having an associated priority level.

7. A network interface device as claimed in claim 6, wherein, for each ingress port, the lookup table defines a one-to-one correspondence between the priority levels of the virtual queues and the Classes of Service supported by the data flows over the ingress port, with each priority level having a respective virtual queue.

8. A network interface device as claimed in claim 4, further comprising a management controller configured to perform flow control and/or congestion avoidance mechanisms on a flow of data packets received at an ingress port of the network interface device in dependence on the number of buffers in the virtual queue associated with that ingress port and having a priority level corresponding to the priority identifiers of the data packets of the flow.

9. A network interface device as claimed in claim 1, further comprising a third subset of the plurality of ingress ports configured to receive data packets from a management controller.

10. A network interface device as claimed in claim 1, further comprising at least one transmit engine configured to provide data packets to the first subset of ingress ports and to one or more of the set of physical data ports of the network interface device, each transmit engine being operable to receive data over the I/O interface by means of a DMA transfer.

11. A network interface device as claimed in claim 10, wherein the ingress interface is configured to not write to the virtual queues held in the unified memory those data packets received at the first subset of ingress ports that are directed only to network endpoints accessible over the physical data ports of the network interface device.

12. A network interface device as claimed in claim 10, wherein the ingress interface is configured to write to the virtual queues held in the unified memory only those data packets received at the first subset of ingress ports that are directed to network endpoints accessible over the I/O interface of the network interface device.

13. A network interface device as claimed in claim 10, wherein at least one ingress port of the first subset receives data packets from a plurality of transmit engines.

14. A network interface device as claimed in claim 1, further comprising a set of burst buffers arranged to buffer data packets received from the physical data ports of the network interface device.

15. A network interface device as claimed in claim 1, further comprising an egress interface configured to service the virtual queues in sequence according to a specified service order maintained in dependence on one or both of:

the number of buffers in each of the virtual queues, and a priority level of each of the virtual queues.

16. A network interface device as claimed in claim 15, wherein the egress interface is configured to promote to the head of the sequence specified in the service order those virtual queues that include buffers from their respective private buffer pool, such virtual queues having a first egress priority.

17. A network interface device as claimed in claim 16, wherein the egress interface is configured to promote to the head of the sequence specified in the service order those virtual queues whose length exceeds a predetermined number of buffers but that do not include buffers from their respective private pools, such virtual queues having a second egress priority.

18. A network interface device as claimed in claim 17, wherein the egress interface is configured to maintain a timer for each virtual queue, each timer indicating the length of time elapsed since the egress interface last serviced the respective virtual queue, and the egress interface is further configured to promote to the head of the sequence specified in the service order those virtual queues whose timer exceeds a predetermined timeout value, such virtual queues having a third egress priority.

19. A network interface device as claimed in claim 18, wherein, if more than one virtual queue is to be promoted to the head of the sequence specified in the service order, the virtual queues so promoted are serviced in accordance with their respective priorities, with virtual queues having a first priority being serviced before virtual queues having a second priority and virtual queues having a second priority being serviced before virtual queues having a third priority.

20. A network interface device as claimed in claim 19, wherein the egress interface is operable to promote a virtual queue so as to comply with a quality of service scheme in operation over data flows of the respective ingress port.

21. A network interface device as claimed in claim 15, wherein the egress interface is configured to, on emptying a buffer, cause the memory manager to release that buffer from the respective virtual queue.

22. A network interface device as claimed in claim 15, wherein the egress interface is configured to provide de-queued data packets to one or more receive engines of the network interface device operable to transmit data over the I/O interface by means of a DMA transfer.

23. A network interface device as claimed in claim 1, wherein each virtual queue is defined by a linked list held at a descriptor table of the memory manager.

24. A network interface device as claimed in claim 1, wherein the unified memory comprises a plurality of RAM banks configured such that data is written to the unified memory in a striped arrangement.

25. A network interface device as claimed in claim 1, wherein the unified memory comprises a plurality of single port RAM cells.

26. A network interface device for connection between a network and a data processing system, the network interface device comprising:

an I/O interface for connection to a data processing system;

a set of physical data ports for connection to a network;

a unified memory comprising a plurality of buffers for buffering data packets directed to the data processing system before the data packets are provided to a the I/O interface;

a plurality of ingress ports operable to receive data packets for buffering at the unified memory, a first subset of the plurality of ingress ports being configured to receive data packets on a transmit path from said I/O interface, and a second subset of the plurality of ingress ports being configured to receive data packets on a receive path from said set of physical data ports;

a memory manager configured to store representations of a plurality of virtual queues held in the unified memory, each virtual queue being a linked logical sequence of buffers of the unified memory; and

an ingress interface configured to service the ingress ports in a predetermined order and write data packets received at the ingress ports to buffers of the unified memory selected by the memory manager;

wherein the memory manager is arranged to select buffers of the unified memory so as to cause the ingress interface to populate the plurality of virtual queues with data packets, and the ingress interface is arranged to contiguously write data packets into the linked logical sequence of buffers representing each virtual queue and the memory manager is configured to allocate buffers not linked to a virtual queue to one or more free buffer pools and to, on a virtual queue becoming full, append a buffer selected from one of the one or more free buffer pools to that virtual queue.

27. A network interface device as claimed in claim 26, wherein the one or more free buffer pools comprise a common free buffer pool, from which any of the virtual queues is able to receive a buffer, and for each virtual queue a private free buffer pool, from which only the respective virtual queue is able to receive a buffer.

28. A network interface device as claimed in claim 27, wherein the memory manager is configured to maintain for each virtual queue a common pool count of the total number of buffers added from the common pool buffer and to select a free buffer pool in accordance with the following rules:

if the common buffer pool is not empty and the common pool count of the virtual queue does not exceed a predetermined common buffer pool allocation parameter, then select the common buffer pool; and

otherwise select the private buffer pool of the virtual queue.

29. A network interface device as claimed in claim 28, further comprising a management controller configured to, when a virtual queue associated with an ingress port of the second subset includes buffers from its respective private buffer pool, cause an Ethernet priority pause frame or a congestion notification message to be transmitted from the network interface device so as to reduce the rate at which data packets are received at that ingress port.

30. A network interface device as claimed in claim 28, further comprising a management controller configured to, when a virtual queue associated with an ingress port of the first subset includes buffers from its respective private buffer pool, cause the rate at which data packets are received onto the transmit path to be throttled so as to reduce the rate at which data packets are received at that ingress port.

31. A network interface device as claimed in claim 27, wherein the memory manager is configured to maintain for each virtual queue a private buffer pool size parameter indicating a minimum size for the private buffer pool of the respective virtual queue, the memory manager being further configured to, on releasing a buffer from a virtual queue, allocate that buffer to the respective private buffer pool if that private buffer pool is smaller than the minimum size indicated in its private buffer pool size parameter, and otherwise allocate that buffer to the common buffer pool.

32. A network interface device for connection between a network and a data processing system, the network interface device comprising:

an I/O interface for connection to a data processing system;

a set of physical data ports for connection to a network;

a unified memory comprising a plurality of buffers for buffering data packets directed to the data processing system before the data packets are provided to a the I/O interface;

a plurality of ingress ports operable to receive data packets for buffering at the unified memory, a first subset of the plurality of ingress ports being configured to receive data packets on a transmit path from said I/O interface, and a second subset of the plurality of ingress ports being configured to receive data packets on a receive path from said set of physical data ports;

a memory manager configured to store representations of a plurality of virtual queues held in the unified memory, each virtual queue being a linked logical sequence of buffers of the unified memory; and

an ingress interface configured to service the ingress ports in a predetermined order and write data packets received at the ingress ports to buffers of the unified memory selected by the memory manager;

wherein the memory manager is arranged to select buffers of the unified memory so as to cause the ingress interface to populate the plurality of virtual queues with data packets, and the ingress interface is arranged to contiguously write data packets into the linked logical sequence of buffers representing each virtual queue and the memory manager is configured to maintain for each virtual queue active buffer parameters representing an active buffer to which packet data received at an associated ingress port is to be written and a write offset in the active buffer at which writing is to commence.

33. A network interface device as claimed in claim 32, wherein the buffers of the unified memory are of a fixed size and the ingress interface is configured to, on writing a data packet and if the write offset in the active buffer reaches a predetermined value indicating that the active buffer is almost full, receive from the memory manager updated active buffer parameters representing a new buffer appended to the linked logical sequence of buffers of a virtual queue to which the ingress interface is writing the data packet.

34. A method for buffering data packets at a network interface device, the network interface device having an I/O interface for connection to a data processing system, a set of one or more physical data ports for connection to a network, a unified memory comprising a plurality of buffers and a plurality of ingress ports operable to receive data packets for buffering at the unified memory, the plurality of ingress ports including a first subset of ingress ports configured to receive data packets on a transmit path from said data processing system and a second subset of ingress ports configured to receive data packets on a receive path from said set of physical data ports, the method comprising:

buffering data packets directed to the data processing system before the data packets are provided to the I/O interface by;

servicing the ingress ports in a predetermined order so as to receive data packets from the ingress ports;

selecting for each data packet received at an ingress port a buffer of a virtual queue held in the unified memory, each virtual queue being a linked logical sequence of buffers of the unified memory; and

writing data packets received at the ingress ports to the selected buffers of the unified memory such that the linked logical sequence of buffers representing each virtual queue is contiguously populated with data packets;

servicing the virtual queues in sequence according to a specified service order so as to provide data packets de-queued from the virtual queues at an egress port; and

directing data packets received at the egress port to one or more of;

a receive engine for delivery over the I/O interface;

a virtualised interface of the network interface device, and the transmit path for delivery over a physical data port of the NIC;

wherein the service order is maintained in dependence on one or both of;

the number of buffers in each of the virtual queues, and a priority level of each of the virtual queues.