Fibre channel switch

• US 20050047334A1
• Filed: 06/21/2004
• Published: 03/03/2005
• Est. Priority Date: 06/13/2001
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: McData Services Corp. (Broadcom, Inc.)
• Inventors: Tornetta, Anthony G., Wright, James C., Paul, Harry V., Cantwell, Larry, Gonzalez, Henry J., Schmidt, Steven G., Nallur, Govindaswamy, Carlsen, Scott, Koellner, Gregory L., Pearson, Jereld W., Workman, Jason
• Granted Patent: US 7,042,842 B2
• Time in Patent Office: Days
• US Class Current: 370/229
• CPC Class Codes: H04L 12/5601 Transfer mode dependent, e.g. ATM
• CPC Class Codes: H04L 2012/5605 Fibre
• CPC Class Codes: H04L 2012/5665 Interaction of ATM with other protocols
• CPC Class Codes: H04L 2012/5679 Arbitration or scheduling
• CPC Class Codes: H04L 2012/5681 Buffer or queue management
• CPC Class Codes: H04L 2012/5683 for avoiding head of line blocking
• CPC Class Codes: H04L 49/101 using crossbar or matrix
• CPC Class Codes: H04L 49/251 Cut-through or wormhole routing
• CPC Class Codes: H04L 49/3072 Packet splitting
• CPC Class Codes: H04L 49/357 Fibre channel switches
• CPC Class Codes: H04L 49/505 Corrective measures
• CPC Class Codes: H04L 49/608 ATM switches adapted to switch variable length packets, e.g. IP packets
• CPC Class Codes: H04Q 11/0478 Provisions for broadband connections

Abstract

A Fibre Channel switch is presented that tracks the congestion status of destination ports in an XOFF mask at each input. A mapping is maintained between virtual channels on an ISL and the destination ports to allow changes in the XOFF mask to trigger a primitive to an upstream port that provides virtual channel flow control. The XOFF mask is also used to avoid sending frames to a congested port. Instead, these frames are stored on a single deferred queue and later processed in a manner designed to maintain frame ordering. A routing system is provided that applies multiple routing rules in parallel to perform line speed routing. The preferred switch fabric is cell based, with techniques used to manage path maintenance for variable length frames and to adapt to varying transmission rates in the system. Finally, the switch allows data and microprocessor communication to share the same crossbar network.

Claims

1. A method for sending a variable length data packet through a downstream switch having a plurality of destination ports comprising:

a) tracking a congestion status of the destination ports in an XOFF mask;

b) using changes in the XOFF mask values to trigger a flow control primitive to an upstream switch to control flow over a virtual channel to the downstream switch;

c) routing the data packet to a particular destination port by selecting an applicable routing rule from a plurality of routing rules based on a destination field within the data packet;

d) if the particular destination port has a congested status in the XOFF mask, storing the data packet in a deferred queue for later processing after a change in the XOFF mask;

e) transmitting the data packet over a cell-based switch fabric to the particular destination port; and

f) transmitting over the same cell-based switch fabric a communication destined for a microprocessor that services the particular destination port.

2. The method of claim 1, wherein step a) further comprises:

i) maintaining at each ingress port a lookup table having multiple entries, each entry containing a congestion status for a different destination in the switch, each lookup table containing entries for all available destinations in the switch, each lookup table returning the congestion status in response to a status query for a particular destination,

ii) maintaining at a credit module an indicator of an amount of data submitted for each destination,

iii) when the indicator passes a threshold value, sending a congestion update from the credit module to a first lookup table, the congestion update containing a destination identifier and an updated congestion status, and

iv) updating the entry in the first lookup table corresponding to the destination identifier using the updated congestion status.

3. The method of claim 1, wherein step b) further comprises:

i) establishing a map between each virtual channel and the possible destination ports for that virtual channel, and

ii) comparing the map and the XOFF mask status for all of the possible destination ports to determine a flow control status for each virtual channel.

4. The method of claim 1, wherein step c) further comprises:

a) applying multiple routing rules to the data packet in parallel, each rule;

i) determining whether the rule applies to the data packet, and, at the same time, and

ii) determining a routing result for the data packet, the result including an address for internal routing within the switch; and

b) selecting a preferred result from among all of the results obtained from the multiple routing rules.

5. The method of claim 1, wherein step d) further comprises:

i) forwarding the data packet to the destination port if the congestion status is in the XOFF mask is non-congested,

ii) storing the data packet in a deferred queue if the congestion status is congested, and

iii) repeating the above steps for additional data packets, such that multiple packets that are routed to a plurality of different congested destination ports are stored in the same deferred queue.

6. The method of claim 1, wherein the cell-based switch fabric requires cells having a data payload and a packet length field, wherein step e) further comprises:

i) constructing a first data cell from the data frame, the first data cell containing a full data payload and a packet length value in the packet length field, the packet length value being indicative of the number of data cells in the data frame, and

ii) constructing a second data cell from the data frame, the second data cell containing a partially filled data payload and a valid data value in the packet length field, the valid data value being indicative of the amount of valid data in the data payload.

7. The method of claim 1, wherein step f) further comprises:

i) assigning the data packet destined for the particular destination port a first class of service level;

ii) assigning processor data destined for the microprocessor a second class of service level; and

iii) sending the data packet and the processor data over an identical crossbar using a traffic shaping algorithm that treats traffic according to its class of service level.

8. The method of claim 1, further comprising

g) maintaining packet order at an egress from the cell-based switch fabric by:

i) storing packets received from a first source in a first buffer;

ii) storing a first indicator in a storage mechanism whenever one of the packets is stored in the first buffer;

iii) storing packets received from a second source in a second buffer;

iv) storing a second indicator on the storage mechanism whenever one of the packets is stored in the second buffer;

v) removing packets from the first and second buffer using the indicators stored in the storage mechanism to determine whether a next packet is removed from the first or second buffer.

9. A data switch for switching a variable-length data packet comprising:

a) a plurality of ports,

b) a cell-based crossbar switching fabric in communication with the ports;

c) an XOFF mask containing the status of each port on the data switch;

d) a routing module having routing rules that determine routing information based on a header in the data packet;

e) a queuing module that receives the routing information and stores data packets destined for a congested port in a deferred queue;

f) a segmentation module that segments the data packet into at least a first and second fixed-size data cells, the first data cell containing a full data payload and the second data cell containing a partially filled data payload;

g) a microprocessor;

h) a memory subsystem for queuing data for submission to the crossbar switching fabric into a plurality of output queues having different classes of service for data directed to one of the ports and data directed to one of the microprocessors; and

i) a virtual input queue mapping between a first virtual channel on an interswitch link and a subset of the plurality of output queues.

10. A data switch comprising:

a) a plurality of ports receiving and transmitting variable-length data packets;

b) a buffer in communication with at least one of the ports;

c) a plurality of microprocessors that control the ports;

d) a cell-based switch crossbar in communication with at least two of the ports;

e) a routing means for determining a destination port for the data packets;

f) a congestion indicator means indicating the congestion status of the plurality of ports;

g) a deferred queuing means for avoiding head of line blocking in the buffer by storing a subset of the received data packets in a deferred queue;

h) a cell segmentation means for segmenting one of the variable length data packets into a plurality of fixed-sized data cells;

i) data transmission means for transmitting data cells over the crossbar, the data transmission means transmitting data to at least one of the ports and at least one of the microprocessors over the same crossbar; and

j) interswitch link flow control means for providing flow control on a single virtual channel within an interswitch link communicating with the data switch.

11. The data switch of claim 10 wherein the congestion indicator means further comprises:

i) an indicating means at each port for indicating the congestion status of the plurality of ports, and

ii) a congestion signaling means for signaling a need to update the input indicating means with a new congestion status for at least one port.

12. The data switch of claim 10 wherein the deferred queuing means further comprises a queue controller means for directing the data packet to the deferred queue if the congestion status for the destination port of the data packet is congested and for directing the data packet to the destination port if the congestion status is non-congested.

13. The data switch of claim 10, wherein cell segmentation means further comprises:

i) means for constructing a first data cell from the variable-length data packet, the first data cell containing a full data payload and a packet length value in the packet length field, the packet length value being set large enough to represent a maximum sized data packet,

ii) means for establishing a path through the cell based switch fabric and associating with the path a duration determined by the packet length value,

iii) means for constructing a second data cell from the variable-length data packet, the second data cell containing a partially filled data payload and a valid data value in the packet length field, the valid data value indicating the number of valid data bytes in the partially filled data payload, and

iv) means for reconstructing the data packet at least in part from the transmitted first and second data cells, the reconstructed data packet containing data from the entire full data payload of the first data cell and the valid data bytes of the second data cell data payload.

14. The data switch of claim 10 wherein the interswitch link flow control means further comprises:

i) means for establishing an interswitch link with an upstream switch, the interswitch link having a plurality of virtual channels;

ii) means for determining a flow control status for each virtual channel in the interswitch link based on the congestion indicator means; and

iii) means for communicating a change in flow control status for a particular virtual channel to the upstream switch.