Ingress based headroom buffering for switch architectures

• US 9,467,522 B2
• Filed: 09/27/2013
• Issued: 10/11/2016
• Est. Priority Date: 07/19/2013
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: Matthews, William Brad, Kwan, Bruce Hui
• Primary Examiner(s): Chea, Philip
• Assistant Examiner(s): Chen, Wuji
• Application Number:
  US14/040,556
• Publication Number:
  US 20150026361A1
• Time in Patent Office: 1,110 Days
• US Class Current: 1/1
• CPC Class Codes: H04L 47/11 Identifying congestion
• CPC Class Codes: H04L 47/12 Avoiding congestion; Recovering from congestion
• CPC Class Codes: H04L 47/26 using explicit feedback to the source, e.g. choke packets
• CPC Class Codes: H04L 47/30 in combination with information about buffer occupancy at either end or at transit nodes
• CPC Class Codes: H04L 47/32 by discarding or delaying data units, e.g. packets or frames
• CPC Class Codes: H04L 49/00 Packet switching elements
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54

Abstract

A network device performs ingress based headroom buffering. The network device may be configured as an output queue switch and include a main packet buffer that stores packet data according to a destination egress port. The network device may implement one or more ingress buffers associated with ingress data ports in the network device. The ingress buffers may be separate from the main packet buffer. The network device may identify a flow control condition triggered by an ingress data port, such as when an amount of data stored in the main packet buffer received through the ingress data port exceeds a fill threshold. In response, the network device may send a flow control message to a link partner to cease sending network traffic through the ingress data port. The network device may store in-flight data from the link partner in an ingress buffer instead of the main packet buffer.

Claims

1. A method comprising:

in a network device;

maintaining a packet buffer configured to store network traffic received through a first ingress port of a first network interface from over a network, and destined for an egress port;

determining that the network traffic stored in the packet buffer exceeds a fill threshold, and in response;

sending a headroom buffering indication to the first network interface; and

selectively allocating as additional buffer memory, by the first network interface, at least a portion of an ingress buffer allocated to the ingress port to store in-flight packet data not yet received from over the network by the first network interface, the ingress buffer being separate from the packet buffer and being allocated to store in-flight packet data instead of the packet buffer.

2. The method of claim 1, further comprising:

sending a flow control message over the network to a link partner linked to the network device through the first network interface, the flow control message instructing the link partner to stop communication through the first network interface.

3. The method of claim 2, further comprising:

determining that the network traffic stored in the packet buffer has fallen below a resumption fill level, and in response;

sending a resume flow control message to the link partner to resume communication through the first network interface.

4. The method of claim 1, further comprising, after sending the headroom buffering indication:

selectively storing the in-flight packet data received by the first network interface into the ingress buffer instead of forwarding the in-flight packet data to the packet buffer.

5. The method of claim 4, where selectively storing the in-flight packet data comprises:

storing a first packet belonging to a lossless traffic class; and

dropping a second packet belonging to a lossy traffic class.

6. The method of claim 1, further comprising:

determining that the packet data received by the first network interface and stored in the packet buffer has fallen below a resumption fill level, and in response;

sending a resumption indication to the first network interface to resume forwarding network traffic received by the first network interface to the packet buffer.

7. The method of claim 1, further comprising:

determining that the network traffic received by the first network interface and stored in the packet buffer has fallen below an resumption fill level, and in response;

forwarding the in-flight packet data stored in the ingress buffer to the packet buffer.

8. The method of claim 7, further comprising sending a resume flow control message over the network to a link partner to resume communication when an amount of in-flight packet data stored in the ingress buffer falls below an emptying threshold.

9. The method of claim 1, further comprising allocating a first portion of the ingress buffer for storing the in-flight data and a second portion of the ingress buffer for other ingress functionality.

10. The method of claim 1, where the fill threshold accounts for an amount of packet data from the ingress port that can be processed in a packet processing pipeline.

11. The method of claim 1, further comprising allocating a respective input buffer for multiple network interfaces in the network device.

12. A device comprising:

a packet memory configured to store network traffic received over a network, the network traffic stored according to destination data ports for the network traffic;

data port logic comprising;

a processor;

an ingress data port; and

a packet buffer associated with the ingress data port; and

where the data port logic is executable by the processor to forward network traffic received through the ingress data port for processing and storage in the packet buffer;

buffering logic executable by the processor to;

determine that an amount of the network traffic received through the ingress data port and stored in the packet buffer exceeds a threshold amount, and in response;

send a buffering indication to the data port logic; and

where the data port logic is further executable by the processor to, in response to receipt of the buffering indication, selectively allocate at least a portion of an input buffer as additional storage of network traffic and store subsequent network traffic received through the ingress data port into the allocated at least a portion of the input buffer instead of forwarding the subsequent network traffic for processing and storage in the packet buffer.

13. The device of claim 12, where the buffering logic is further operable to send a flow control message to a link partner linked to the data port, the flow control message instructing the link partner to stop sending data to the data port.

14. The device of claim 13, where the buffering logic is further operable to:

determine that the amount of network traffic received through the input port and stored in the packet buffer has fallen below a resumption fill level, and in response;

sending another flow control message to the link partner to resume communication.

15. The device of claim 12, where the data port logic is operable to store the subsequent network traffic by:

storing a first packet belonging to a lossless traffic class; and

dropping a second packet belonging to a lossy traffic class.

21. The device of claim 12, wherein the input buffer is allocated to the data port, and the data port logic is further executable by the processor to forward In flight data packets stored in the input buffer to the packet buffer for subsequent processing and storage in the packet buffer.

16. A device comprising:

a network interface configured to communicate across a network with a link partner, the network interface comprising a plurality of ingress ports; and

a distributed packet memory architecture comprising;

a primary packet memory comprising;

a first memory block configured to store packet data received through a first ingress port and destined for a first egress port;

a second memory block configured to store packet data received through the first ingress port and destined for a second egress port different from the first egress port; and

an ingress buffer separate from the packet memory and allocated to the first ingress port; and

system logic in communication with the packet memory and the ingress buffer, the system logic configured to;

selectively allocate a portion of the ingress buffer as additional memory space for inflight packet data not yet received from over a network at the first ingress port, an amount of the ingress buffer allocated as additional memory space instead of an allocation in the first memory block and second memory block.

17. The device of claim 16, where the system logic is further configured to:

determine a flow control condition triggered when an amount of packet data from the first ingress port stored in the first memory block exceeds a fill threshold, and in response;

send a flow control message to a link partner linked to the first ingress port to request cessation of sending packet data to the first ingress port.

18. The device of claim 17, where the system logic is further configured to:

selectively allocate by sending a buffer allocation request to the ingress buffer; and

where the ingress buffer is configured to respond by;

selectively storing the in-flight data received from the link partner after sending the flow control message in the portion of the ingress buffer allocated as additional memory space.

19. The device of claim 18, where the system logic is configured to selectively store the in-flight data by:

receiving a packet; and

storing the packet in the ingress buffer when the destination egress port for the packet is the first egress port; and

forwarding the packet for processing and storing in the second memory block when the destination egress port for the packet is the second egress port.

20. The device of claim 18, where the system logic is configured to selectively store the in-flight data by:

storing a first packet that belongs to a lossless traffic class; and

dropping a second packet that belongs to a lossy traffic class.

22. The device of claim 16, wherein the system logic is further configured to:

forward the packet data from the first ingress port to the ingress buffer, and from the ingress buffer to either the first memory block or the second memory block.