Network device architecture for consolidating input/output and reducing latency

• US 7,830,793 B2
• Filed: 03/30/2005
• Issued: 11/09/2010
• Est. Priority Date: 10/22/2004
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Edsall, Thomas, Bergamasco, Davide, Gai, Silvano, Bonomi, Flavio, Dutt, Dinesh
• Primary Examiner(s): Chou; Albert T
• Application Number:
  US11/094,877
• Publication Number:
  US 20060087989A1
• Time in Patent Office: 2,050 Days
• Field of Search: 370/412, 370/230, 370/231, 370/233, 370/415, 370/487
• US Class Current: 370/230
• CPC Class Codes: H04L 12/413 with random access, e.g. carrier-sense multiple-access with collision detection [CSMA-CD]
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/32 by discarding or delaying data units, e.g. packets or frames
• CPC Class Codes: H04L 47/33 using forward notification
• CPC Class Codes: H04L 47/39 Credit based
• CPC Class Codes: H04L 49/90 Buffering arrangements

Abstract

The present invention provides methods and devices for implementing a Low Latency Ethernet (“LLE”) solution, also referred to herein as a Data Center Ethernet (“DCE”) solution, which simplifies the connectivity of data centers and provides a high bandwidth, low latency network for carrying Ethernet and storage traffic. Some aspects of the invention involve transforming FC frames into a format suitable for transport on an Ethernet. Some preferred implementations of the invention implement multiple virtual lanes (“VLs”) in a single physical connection of a data center or similar network. Some VLs are “drop” VLs, with Ethernet-like behavior, and others are “no-drop” lanes with FC-like behavior. Some preferred implementations of the invention provide guaranteed bandwidth based on credits and VL. Active buffer management allows for both high reliability and low latency while using small frame buffers. Preferably, the rules for active buffer management are different for drop and no drop VLs.

Claims

1. A method for processing more than one type of network traffic in a single network device, the method comprising:

partitioning buffers of the network device into first buffer spaces for storing frames received on a first virtual lane of a physical link of the network device and second buffer spaces for storing frames received on a second virtual lane of the physical link of the network device;

receiving a plurality of frames into the physical link of the network device, wherein each frame indicates either a first virtual lane or a second virtual lane based on whether or not, respectively, the frame indicates a protocol that recovers from dropping of frames; and

for each received frame, applying either a first set of rules or a second set of rules with respect to the received frame based on whether the received frame specifies the first virtual lane or the second virtual lane, respectively, wherein the first set of rules causes the received frame to be dropped or stored in the first buffer based on whether or not, respectively, the first buffer spaces has been filled a predetermined amount, wherein the second set of rules inhibit dropping of the received frame and causes the received frame to be stored in the second buffer spaces in response to latency.

2. The method of claim 1, further comprising the step of assigning a guaranteed minimum amount of buffer space per virtual lane.

3. The method of claim 1, wherein the frames comprise Ethernet frames that specify the first virtual lane and Fibre Channel frames that specify the second virtual lane.

4. The method of claim 1, wherein the first set of rules or the second set of rules causes an explicit congestion notification to be transmitted from the network device in response to latency of the first frames.

5. The method of claim 4, wherein the explicit congestion notification is sent to one of a source device or an edge device.

6. The method of claim 4, wherein the explicit congestion notification is sent via one of a data frame or a control frame.

7. The method of claim 1, wherein the second set of rules causes an explicit congestion notification to be transmitted from the network device in response to latency of second frames and the second set of rules inhibit dropping of the received frame without causing congestion notification to be transmitted from the network device in response to latency of first frames.

8. The method of claim 1, wherein the partitioning step comprises dynamically partitioning buffers according to one or more factors that include buffer occupancy, time of day, traffic loads, congestion, guaranteed minimum bandwidth allocation, known tasks requiring greater bandwidth or maximum bandwidth allocation.

9. The method of claim 1, wherein the second set of rules causes frames to be dropped to avoid deadlocks.

10. The method of claim 1, wherein the first set of rules applies a probabilistic drop function in response to latency.

11. The method of claim 10, wherein the second set of rules comprises implementing a buffer-to-buffer crediting scheme for the second frames.

12. The method of claim 11, wherein the buffer-to-buffer crediting scheme comprises crediting according to frame size.

13. The method of claim 11, wherein buffer-to-buffer credits are managed by an arbiter.

14. The method of claim 11, wherein the second set of rules comprises implementing the buffer-to-buffer crediting scheme within a network device and using PAUSE frames for flow control on the second virtual lane.

15. The method of claim 1, further comprising the step of storing the first frames and the second frames in virtual output queues (“

VOQs”

), wherein each VOQ is associated with a destination port/virtual lane pair.

16. The method of claim 15, further comprising the step of performing buffer management in response to at least one of VOQ length, buffer occupancy per virtual lane, overall buffer occupancy and packet age.

17. The method of claim 1, further comprising the step of performing buffer management in response to at least one of buffer occupancy per virtual lane and packet age.

18. The method of claim 1 wherein the first set of rules defines the first virtual lane as a drop virtual lane, and the second set of rules defines the second virtual lane as a no-drop virtual lane.

19. A network device, comprising:

means for partitioning buffers of the network device into first buffer spaces for storing frames received on a first virtual lane of a physical link of the network device and second buffer spaces for storing frames received on a second virtual lane of the physical link of the network device;

means for receiving a plurality of frames into the physical link of the network device, wherein each frame indicates either a first virtual lane or a second virtual lane based on whether or not, respectively, the frame indicates a protocol that recovers from dropping of frames; and

means, for each received frame, applying either a first set of rules or a second set of rules with respect to the received frame based on whether the received frame specifies the first virtual lane or the second virtual lane, respectively, wherein the first set of rules causes the received frame to be dropped or stored in the first buffer based on whether or not, respectively, the first buffer spaces has been filled a predetermined amount, wherein the second set of rules inhibit dropping of the received frame and causes the received frame to be stored in the second buffer spaces in response to latency.

20. A network device, comprising:

a plurality of ports configured for receiving frames on a plurality of physical links;

a plurality of line cards, each line card in communication with one of the plurality of ports and configured to do the following;

receive frames from a first one of the physical links of the plurality of ports, wherein each frame indicates either the first virtual lane or the second virtual lane based on whether or not, respectively, the frame indicates a protocol that recovers from dropping of frames;

identify first frames received on a first virtual lane of the first physical link and second frames received on a second virtual lane of the first physical link;

partition buffers into first buffer spaces for storing the identified first frames and second buffer spaces for storing the identified second frames; and

for each received frame, apply either a first set of rules or a second set of rules with respect to the received frame based on whether the received frame specifies the first virtual lane or the second virtual lane, respectively, wherein the first set of rules causes the received frame to be dropped or stored in the first buffer based on whether or not, respectively, the first buffer spaces has been filled a predetermined amount, wherein the second set of rules inhibit dropping of the received frame and causes the received frame to be stored in the second buffer spaces in response to latency.

21. A method for carrying more than one type of traffic in a single network device, the method comprising:

identifying received frames as first frames received on first virtual lanes and second frames received on second virtual lanes based on whether or not, respectively, the received frames indicate a protocol that recovers from dropping of frames;

dynamically partitioning buffers of the network device into first buffer spaces having first virtual queues (VOQ'"'"'s) for storing the first frames and second buffer spaces having second virtual queues (VOQ'"'"'s) for storing the second frames, wherein the buffers are dynamically partitioned according to one or more factors including overall buffer occupancy, buffer occupancy per virtual lane, time of day, traffic loads, congestion, guaranteed minimum bandwidth allocation, known tasks requiring greater bandwidth or maximum bandwidth allocation; and

for each received frame, applying either a first set of rules or a second set of rules with respect to the received frame based on whether the received frame has been identified on the first virtual lane or on the second virtual lane, respectively, wherein the first set of rules causes the received frame to be dropped or stored in the first VOQs of the first buffer spaces based on whether or not, respectively, the first buffer has been filled a predetermined amount, wherein the second set of rules inhibit dropping of the received frame and causes the received frame to be stored in the second VOQs of the second buffer spaces in response to latency.

22. A network device, comprising:

means for identifying received frames as first frames received on first virtual lanes and second frames received on second virtual lanes based on whether or not, respectively, the received frames indicate a protocol that recovers from dropping of frames;

means for dynamically partitioning buffers of the network device into first buffer spaces having first virtual queues (VOQ'"'"'s) for storing the first frames and second buffer spaces having second virtual queues (VOQ'"'"'s) for storing the second frames, wherein the buffers are dynamically partitioned according to one or more factors including overall buffer occupancy, buffer occupancy per virtual lane, time of day, traffic loads, congestion, guaranteed minimum bandwidth allocation, known tasks requiring greater bandwidth or maximum bandwidth allocation; and

for each received frame, applying either a first set of rules or a second set of rules with respect to the received frame based on whether the received frame has been identified on the first virtual lane or on the second virtual lane, respectively, wherein the first set of rules causes the received frame to be dropped or stored in the first VOQs of the first buffer spaces based on whether or not, respectively, the first buffer has been filled a predetermined amount, wherein the second set of rules inhibit dropping of the received frame and causes the received frame to be stored in the second VOQs of the second buffer spaces in response to latency.

23. A network device, comprising:

a plurality of ports configured for receiving frames on a plurality of physical links;

a plurality of line cards, each line card in communication with one of the plurality of ports and configured to do the following;

identify first frames received frames as first frames received on first virtual lanes and second frames received on second virtual lanes based on whether or not, respectively, the received frames indicate a protocol that recovers from dropping of frames;

dynamically partition buffers of the network device into first buffer spaces having first virtual queues (VOQ'"'"'s) for storing the first frames and second buffer spaces having second virtual queues (VOQ'"'"'s) for storing the second frames, wherein the buffers are dynamically partitioned according to one or more factors including overall buffer occupancy, buffer occupancy per virtual lane, time of day, traffic loads, congestion, guaranteed minimum bandwidth allocation, known tasks requiring greater bandwidth or maximum bandwidth allocation; and

for each received frame, apply either a first set of rules or a second set of rules with respect to the received frame based on whether the received frame has been identified on the first virtual lane or on the second virtual lane, respectively, wherein the first set of rules causes the received frame to be dropped or stored in the first VOQs of the first buffer spaces based on whether or not, respectively, the first buffer has been filled a predetermined amount, wherein the second set of rules inhibit dropping of the received frame and causes the received frame to be stored in the second VOQs of the second buffer spaces in response to latency.