Fibre channel arbitrated loop bufferless switch circuitry to increase bandwidth without significant increase in cost

• US 20030118040A1
• Filed: 01/21/2003
• Published: 06/26/2003
• Est. Priority Date: 11/19/1998
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Alistair D. Black, Kurt Chan
• Inventors: Black, Alistair D., Chan, Kurt
• Granted Patent: US 7,362,769 B2
• Time in Patent Office: Days
• US Class Current: 370/403
• CPC Class Codes: H04J 3/245 in which the allocation protocols between more than two stations share the same transmission medium stations for satellite systems H04B7/185
• CPC Class Codes: H04L 49/101 using crossbar or matrix
• CPC Class Codes: H04L 49/357 Fibre channel switches

Abstract

A switch, switched architecture and process for transferring data through an FCAL switch is disclosed. The switch uses multiple switch control circuits each coupled to one FCAL network and all connected to a crossbar switch. The switch control circuits are coupled together by a protocol bus for coordination purposes. Local conversations can occur on each FCAL loop and crossing conversations through the switch can occur concurrently. The OPN primitive is used to establish the connection before any data is transferred thereby eliminating the need for buffer memory in the switch control circuits. The destination address of each OPN is used to address a lookup table in each switch control circuit to determine if the destination node is local. If not, the destination is looked up and a connection request made on the protocol bus. If the remote port is not busy, it sends a reply which causes both ports to establish a data path through the backplane crossbar switch.

Claims

1. A bufferless switch for coupling to a plurality of FCAL nets and having a crossbar switch and FCAL loop interface port circuits structured to use the OPN and RRDY primitives of the FCAL protocol for hold back flow control to eliminate the need for a buffer and said ports and crossbar switch structured to provide multiple simultaneous loop tenancies.

4. The apparatus of claim 1 wherein said port circuits are each integrated on a single integrated circuit with a portion of said crossbar switch.

2. A bufferless switch for coupling to a plurality of FCAL nets and having a crossbar switch and FCAL loop interface port circuits structured to use the OPN and RRDY primitives of the FCAL protocol for hold back flow control to eliminate the need for a buffer and said ports and crossbar switch structured to provide dual simplex communication across said switch.

5. The apparatus of claim 2 wherein said port circuits are each integrated on a single integrated circuit with a portion of said crossbar switch.

3. A bufferless switch for coupling to a plurality of FCAL nets and having a crossbar switch and FCAL loop interface port circuits structured to use the OPN and RRDY primitives of the FCAL protocol for hold back flow control to eliminate the need for a buffer and said ports and crossbar switch structured to provide multiple simultaneous loop tenancies and dual simplex communication across said switch, each said port including a local bypass data path which can be switched to keep primitives and data frames generated on a local FCAL net coupled to said port routed so as to stay on said local FCAL net such that local loop tenancies on each local FCAL net coupled to a port which do not involve transmission of data or primitives from one port to another can occur simultaneously, and each port including local learning memory means for storing a routing table and scoreboard table, and functioning to fully cache the 8-bit addresses of all nodes on each said local FCAL net coupled to a port along with the ID of the port coupled to each node in said routing table and the busy, available or no privilege status of each other port and whether each busy port is available for dual simplex communications in said scoreboard table, each port structured to write the contents of each said routing table and scoreboard table by learning either from watching local traffic in said port bound for local destination nodes or remote traffic leaving said port and crossing said crossbar switch bound for remote destination nodes or by conducting an active discovery process, each port structured to write the contents of said scoreboard table by learning from watching messages posted on a protocol bus, each said port structured to use destination addresses in OPN primitives transmitted by nodes on said local FCAL net as search keys to search said routing table to determine whether the destination node is local or remote and to which port it is connected and, if local, to switch said local bypass to keep the loop tenancy confined to said local FCAL net, or, if remote, to generate a connection request message to said port coupled to said destination node, each said port being structured to receive such connection requests and respond by arbitrating for control of its local FCAL net and pick a data path through said crossbar switch and generate a reply message naming the data path through said crossbar switch to use in exchanging data frames and primitives, each said port also being structured to respond to receipt or generation of said reply message by generating commands to establish said data path named in said reply message through said crossbar switch, each said port also structured to circulate a fairness token on a fairness token bus coupled to all such ports and to used said fairness token to increase its priority level of access privilege to busy nodes when the fairness token is in possession of the port so that no port can be starved, and then to circulate said fairness token to a neighboring port such that all ports eventually get said fairness token.

6. The apparatus of claim 3 wherein said port circuits are each integrated on a single integrated circuit with a portion of said crossbar switch.

7. A switch for selectively, concurrently coupling the nodes on a plurality of a Fibre Channel Arbitrated Loops together comprising:

a multiplexed bus;

a crossbar switch;

a plurality of switch control means, each for coupling to one local FCAL network, and each coupled to said multiplexed bus and coupled to said crossbar switch, each for receiving OPN primitives from a source node on the local FCAL network to which it is coupled and determining if the destination node identified in said OPN is on said local FCAL network, and, if so, routing the OPN to the destination node via the local FCAL network, but if the destination node is not on said local FCAL, transmitting a destination node location request and the destination address of the destination node identified in said OPN primitive to all said other switch control means via a channel on said multiplexed bus, and for scanning messages received back from said other switch control means via a channel on said multiplexed bus to determine which switch control means and local FCAL network to which flow control primitives and data are to be transferred to accomplish the desired data transfer, and when a grant message is received from another switch control means indicating said other switch control means is the place to send flow control primitives and data and indicating that the FCAL to which said other switch control means is coupled is available for a loop tenancy, for cooperating with said other switch control means to establish a data transfer path across said crossbar switch, and using said data transfer path to transfer flow control primitives between said source and destination nodes, and each switch control means also for receiving destination node location requests and destination addresses, and using said destination address to search a table of destination addresses of nodes on the local FCAL of said switch control means to determine if the destination node identified in said destination node location request is on the local FCAL to which said switch control means is coupled, and, if so, for arbitrating for control of said local FCAL and, when control is won, sending a grant message back to said switch control means which initiated said destination node location request identifying the switch control means coupled to the local FCAL on which said destination node is resident and for sending one or more messages to said switch control means which initiated said destination node location request for purposes of controlling establishment of said data transfer path across said crossbar switch.

8. A packet switching switch for coupling a plurality of Fibre Channel NL nodes and/or Fibre Channel Arbitrated Loop network (herafter FCAL nets) together to all concurrent data exchanges between a plurality of pairs comprised of one NL node or FCAL net and one other NL node or FCAL net, comprising:

a crossbar switch;

a protocol bus;

a plurality of port circuits each having an input and an output for connecting to an FCAL net coupled to one or more NL nodes and capable of implementing a Fibre Channel loop protocol, and each having a crossbar switch port coupled to said crossbar switch;

means coupled to said protocol bus for maintaining a scoreboard table containing at least status information and a routing table either centrally located or in each port circuit, said routing table containing data mapping destination addresses of NL nodes to port IDs;

and wherein said port circuits are coupled to said means for maintaining a scoreboard table and routing table and wherein said port circuits function to establish connections between themselves by using destination addresses in OPN primitives received from source nodes to search said routing table to determine the ID of a remote port coupled to the destination node having the destination address in the OPN and, using that port ID to search said scoreboard table to determine status of the remote port, and then exchanging messages with said remote port to cause it to arbitrate for and take control of its local FCAL net and establish a particular channel through said crossbar switch and use said channel to transmit primitives and data frames between said source node and said destination node.

9. The apparatus of claim 8 where said switch is structured to be capable of operating at high throughput rate, but neither said crossbar switch nor said port circuits have the amount of memory that would be needed in a fabric switch to operate at the same throughput rate.

10. The apparatus of claim 8 wherein said port circuits are not FL_ports.

11. The apparatus of claim 8 wherein said port circuits have no error recovery protocols and are structured to use hold back flow control such that frames are not transmitted until the connection is established through said crossbar switch to the destination node and is ready to stream the data directly from the source node to the destination node without ever buffering the data and never dropping a frame.

12. The apparatus of claim 8 wherein each said port circuit is structured to use the destination address tag of an FCAL net OPN primitive as a routing address and search key to search said routing table of destination addresses and structured to establish and update the contents of said routing table by learning the locations of destination nodes by watching traffic passing through said port, said routing table comprising a destination address and the ID of the particular port to which data and primitives of an FCAL loop tenancy are to be sent to communicate with the NL node having that destination address, said port circuits also structured to use the routing information derived from said search to search said scoreboard table and exchange messages with said remote port over a protocol bus to cause said remote port to begin arbitrating and to select a particular data path through said crossbar switch and to control switching by said crossbar switch to establish the selected data path.

13. The apparatus of claim 8 wherein said crossbar switch is distributed and a portion of it is integrated together with one or more of said port circuits on a single integrated circuit die.

14. A process for communicating data between a source node and a destination node through one or more ports of a switch in a Fibre Channel protocol network, comprising:

1) in a source node, arbitrating for and winning control of a first FCAL net and transmitting an OPN primitive thereon, said OPN primitive having a destination address of a destination node therein;

2) receiving and latching at a first port of said switch said OPN primitive from said source node coupled to said first port by said first FCAL net;

3) using the destination address in said OPN primitive as a search key to search a routing table to find the location of a destination node having said destination address or the ID of a port coupled by an FCAL net to said destination node, or both, and, if the destination node is coupled to said first port, passing said OPN primitive to said destination node via said first port via a local bypass data path coupling an input of the first port to an output of the first port coupled to the first FCAL net, but, if said destination node is coupled to a second port other than said first port, controlling a crossbar switch to establish a data path between said first and second ports and determining if said second port is available, and, if so, sending said OPN primitive to said second port indicating traffic is waiting to be sent to said destination node and latching said OPN in said second port;

4) in said second node, arbitrating for control of a second FCAL net coupled to said second port;

5) when control of said second FCAL net is won following said arbitration, forwarding said OPN to said destination node;

6) receiving an RRDY primitive or a CLS primitive from said destination node in said second port and transmitting said primitive to said source node through said crossbar switch, said first port and said first FCAL net; and

7) continuing to pass data frames and primitives between said source and destination nodes, until a CLS primitive is transmitted by either said source node or said destination node, and then closing said data path through said crossbar switch and relinquishing control of said first and second FCAL nets.

15. The process of claim 14 wherein step 1 further comprises the steps of transmitting from said source node one or more RRDY primitives immediately following transmission of said OPN primitive, each RRDY primitive representing one frame buffer of credit which has been reserved in memory to store a frame of data transmitted back from said destination node to said source node, and wherein step 2 further comprises the step of storing any RRDY primitives that were transmitted following said OPN transmitted by said source node, and wherein step 5 further comprises the step of forwarding any RRDY primitives received from said source node to said destination node, and wherein step 6 further comprises the steps of receiving any data frames output by said destination node in response to receipt of one or more RRDY primitives from said source node and transmitting said data frames to said source node through said one or more ports of said switch.

16. The process of claim 14 wherein the step of determining if said second port is available in step 5 comprises the steps of using said ID of the port coupled to said destination node determined by searching said routing table with the destination address in the OPN primitive received from the source address as a search key to search a scoreboard table which stores port status information for every port and using said status information to determine if the port coupled to said destination node is available where said status information indicates whether or not the FCAL net on which the destination node resides is currently busy in a loop tenancy.

17. A packet switching switch for coupling a plurality of Fibre Channel NL nodes and/or Fibre Channel Arbitrated Loop network (herafter FCAL nets) together to all concurrent data exchanges between a plurality of pairs comprised of one NL node or FCAL net and one other NL node or FCAL net, comprising:

a crossbar switch;

a protocol bus;

a plurality of port circuits each having an input and an output for connecting to an FCAL net coupled to one or more NL nodes and capable of implementing a Fibre Channel loop protocol, and each having a crossbar switch port coupled to said crossbar switch;

circuitry in each said port circuit and coupled to said protocol bus for maintaining a scoreboard table containing at least status information and a routing table containing data mapping destination addresses of NL nodes to port IDs;

and wherein said port circuits function to establish connections between themselves by using destination addresses in-OPN primitives received from source nodes to search said routing table to determine the ID of a remote port coupled to the destination node having the destination address in the OPN and, using that port ID to search said scoreboard table to determine status of the remote port, and then exchanging messages with said remote port to cause it to arbitrate for and take control of its local FCAL net and establish a data path via a particular channel through said crossbar switch and use said channel to transmit primitives and data frames between said source node and said destination node.

18. The apparatus of claim 17 wherein each said port circuit includes means for storing and updating the contents of said routing table using a passive learning process by watching traffic through the port.

19. The apparatus of claim 17 wherein each said port circuit includes means for storing and updating the contents of said routing table using an active discovery process of sending out inquiries to all nodes on the network.

20. A packet switching switch for coupling a plurality of Fibre Channel NL nodes and/or Fibre Channel Arbitrated Loop network (herafter FCAL nets) together to all concurrent data exchanges between a plurality of pairs comprised of one NL node or FCAL net and one other NL node or FCAL net, comprising:

a crossbar switch;

a protocol bus;

a plurality of port circuits each having an input and an output for connecting to an FCAL net coupled to one or more NL nodes and capable of implementing a Fibre Channel loop protocol, and each having a crossbar switch port coupled to said crossbar switch;

circuitry in each said port circuit and coupled to said protocol bus for maintaining a routing table containing data mapping destination addresses of NL nodes to port IDs;

and wherein said port circuits function to establish connections between themselves by latching any OPN and RRDY primitives received from source nodes and using destination addresses in OPN primitives received from said source nodes to search said routing table to determine the ID of a remote port coupled to the destination node having the destination address in the OPN, and then establishing a data path through said crossbar switch to said remote port and wherein each said port when it receives an OPN and any following RRDYs from another port via said crossbar switch, functioning to latch said OPN and RRDY and arbitrate for and take control of its local FCAL net, and continuing to try to take conrol of its local FCAL net until control is achieved, and when control is achieved, sending said OPN and any latched RRDYs to the destination node, and thereafter using said data path through said crossbar switch to send primitives and data frames back and forth between said source and destination nodes until the transaction is completed without ever storing any data frames.

21. The apparatus of claim 20 wherein each said port circuit includes means for storing and updating the contents of said routing table using a passive learning process by watching traffic through the port.

22. The apparatus of claim 20 wherein each said port circuit includes means for storing and updating the contents of said routing table using an active discovery process of sending out inquiries to all nodes on the network.

23. An apparatus comprising:

a plurality of port means for coupling to a plurality of FCAL nets, each FCAL net having one or more NL nodes thereon; and

switching means coupled to said plurality of port means for coupling selectable pairs of said port means together by separate channels; and

each said port means including means for receiving an OPN from a source node, determining which port the destination node identified in the OPN is coupled to, and communicating with the port coupled to the destination node to select one of said channels through said switching means and establish said channel for communication of primitives and data frames between said source node and said destination node.

24. The apparatus of claim 23 wherein each said port means has a priority level for access to busy destination nodes and wherein each port means includes means for receiving and forwarding a fairness token such that said fairness token circulates to all other said port means, and wherein each said port means includes means for using said fairness token to increase the priority level of said port to a highest priority level when said fairness token is in possession of said port.

25. A switch for coupling a plurality of FCAL nets, comprising:

a plurality of separate backplane data paths; and

a plurality of port integrated circuits selectably coupled together by said separate backplane data paths, each integrated circuit having integrated thereon a port circuit comprising an FCAL net interface for coupling to one FCAL net and a portion of a crossbar switch coupled to said FCAL net interface and structured so as to capable of coupling said port to any selectable one of said separate backplane data paths.

26. The apparatus of claim 25 wherein each said port integrated circuit has a priority level for access to busy destination nodes and wherein each port integrated circuit includes means for receiving and forwarding a fairness token such that said fairness token circulates to all other said port integrated circuits, and wherein each said port integrated circuit includes means for using said fairness token to increase the priority level of said port integrated circuit to a highest priority level when said fairness token is in possession of said port integrated circuit.

27. In a distributed system comprised of a plurality of nodes which are data consumers and data senders coupled by a switch and a plurality of Fibre Channel Arbitrated Loops, a process for establishing one or more concurrent data transfers between different pairs of nodes comprising the steps:

for each desired data transfer, sending to said switch from a source node a request to start a loop tenancy with a destination node, said request being in the form of a Fibre Channel Arbitrated Loop (hereafter FCAL) primitive and including a tag which identifies the destination node with which data is to be exchanged;

using said request and tag in said switch to locate the FCAL upon which said destination node is resident, establishing a data transfer path between the FCAL upon which said source node is resident and the FCAL upon which said destination node is resident.

28. In a distributed system comprised of a plurality of nodes which are data consumers and data senders coupled by a switch and a plurality of Fibre Channel Arbitrated Loops, a process for establishing one or more concurrent data transfers between different pairs of nodes comprising the steps:

(1) for each desired data transfer, sending to said switch from a source node a request to start a loop tenancy with a destination node, said request being in the form of a Fibre Channel Arbitrated Loop (hereafter FCAL) primitive and including a tag which identifies the destination node with which data is to be exchanged;

(2) receiving said request and tag in a switching circuit coupled to the FCAL upon which said source node is resident and using said tag to determine from a table of nodes resident on said FCAL upon which said source node is resident if said destination node is resident thereon;

(3) if said destination node is not resident on said FCAL upon which said source node is resident, using a control channel on a multiplexed bus having data transfer channels and control channels to circulate said request and tag to other switching circuits coupled each of said other FCALs;

(4) in each said other switching circuit, using said tag to determine from a table of nodes resident on said FCAL to which said switching circuit is coupled if said destination node is resident on a local FCAL coupled to said switching circuit;

(5) generating in the switching circuit coupled to the FCAL upon which said destination node is resident a grant message identifying the FCAL upon which said destination node is present by inclusion of the identification code of the switching circuit coupled to the FCAL on which said destination node is present in said grant message;

(6) assiging one or more data transfer channels of said multiplexed bus to carry data of said desired data transfer so as to establish a data transfer path between said switching circuit coupled to said FCAL upon which said source node is present and the switching circuit coupled to said FCAL upon which said destination node is present to carry the data and FCAL primitives of the desired FCAL protocol data transfer; and

(7) using said switching circuits coupled to said FCALs upon which said source and destination nodes are resident to couple an RRDY FCAL protocol primitive from said destination node to said source node after said data transfer path across said multiplexed bus has been established to trigger and control the flow of data, and placing the data of said desired data transfer in one or more of said data transfer channels of said multiplexed bus to carry out said data transfer.

29. The process of claim 28 wherein said multiplexed bus is a time division multiplexed bus wherein some timeslots are dedicated as control channels and other timeslots are dedicated to carrying data, and wherein step (3) is accomplished by placing said request and tag in a first one of said control channel timeslots known to all switching circuits, and wherein step (5) is accomplished by placing said grant message in a second one of said control channel timeslots known to all switching circuits, and wherein step (6) is accomplished by assigning one of said timeslots dedicated to carrying data to carry the data of the desired data transfer by locating an unused data timeslot and broadcasing the switching circuit identification code of the switching circuit coupled to the FCAL upon which said destination node is resident to all other switching circuits such that the switching circuit coupled to the FCAL upon which said source node is resident knows which timeslot to used to transfer further primitives and frame data to said switching circuit coupled to said FCAL upon which said destination node is resident, and wherein step (7) is accomplished by receiving data frames from said source node, marking the beginning and ending of every frame with a delimiter, breaking each frame into timeslot sized pieces, and placing the pieces data of said desired data transfer into the timeslot assigned in step (6) for transfer to said switching circuit coupled to said FCAL upon which said destination node is resident.

30. In a distributed system comprised of a plurality of nodes which are data consumers and data senders coupled by a switch and a plurality of Fibre Channel Arbitrated Loops, a process for establishing one or more concurrent data transfers between different pairs of nodes comprising the steps:

(1) for each desired data transfer, sending to said switch from a source node a request to start a loop tenancy with a destination node after arbitrating for and winning control of the FCAL upon which said source node is resident, said request being in the form of FCAL OPN primitive and including a tag which identifies the destination node with which data is to be exchanged;

(2) receiving said request and tag in a switching circuit coupled to the FCAL upon which said source node is resident and using said tag to determine from a table of nodes resident on said FCAL upon which said source node is resident if said destination node is also resident thereon;

(3) if said destination node is not resident on said FCAL upon which said source node is resident, using a control channel on a multiplexed bus having data transfer channels and control channels to broadcast a destination node location request and the destination address of the destination node identified in said OPN primitive, said request requesting other switching circuits coupled each of said other FCALs to search their tables to determine which of them is coupled to the FCAL to which the destination node is connected;

(4) in each said other switching circuit, using said destination address as a search key to search a table of addresses of nodes resident on said FCAL to which said other switching circuit is coupled

(5) if said destination node is resident on a local FCAL coupled to said switching circuit and is available for a transaction, generating a grant message in the switching circuit coupled to the FCAL upon which said destination node is resident, said grant message identifying the FCAL upon which said destination node is present by inclusion of the identification code of the switching circuit coupled to the FCAL on which said destination node is present in said grant message;

(6) establishing a data transfer path between said switching circuit coupled to said FCAL upon which said source node is present and the switching circuit coupled to said FCAL upon which said destination node is present to carry the data and FCAL primitives of the desired FCAL protocol data transfer; and

(7) using said switching circuits coupled to said FCALs upon which said source and destination nodes are resident and the data path established in step 6 to couple an RRDY FCAL protocol primitive from said destination node to said source node after said data transfer path has been established to trigger and control the flow of data, and transferring data over said data transfer path between said source and destination nodes.

31. The process of claim 30 wherein step 6 is accomplished by receiving the grant message at said switching circuit coupled to said FCAL upon which said source node is resident and generating suitable steering signals to a crossbar switch which functions to selectively couple all switching circuits to each other under the influence of steering signals received from said switching circuits, said steering signals received from said switching circuit coupled to said FCAL upon which said source node is resident causing said crossbar switch to establish a data path connection between said switching circuit coupled to said FCAL upon which said source node is resident and said switching circuit coupled to the FCAL on which the destination node is resident.

32. A switch for selectively, concurrently coupling the nodes on a plurality of a Fibre Channel Arbitrated Loops together comprising:

a multiplexed bus;

a plurality of switch control means, each for coupling to one local FCAL network, and each coupled to said multiplexed bus, each for receiving OPN primitives from a source node on the local FCAL network to which it is coupled and determining if the destination node identified in said OPN is on said local FCAL network, and, if so, routing the OPN to the destination node via a local traffic data path in said switch control means and the local FCAL network, but if the destination node is not on said local FCAL, transmitting a destination node location request and the destination address of the destination node identified in said OPN primitive to all said other switch control means via a channel on said multiplexed bus, and for scanning messages received back from said other switch control means to determine which switch control means and local FCAL network to which flow control primitives and data are to be transferred to accomplish the desired data transfer, and when a grant message is received from another switch control means indicating said other switch control means is the place to send flow control primitives and data and indicating that the FCAL to which said other switch control means is coupled is available for a loop tenancy, for cooperating with said other switch control means to establish a data transfer path across said multiplexed bus, and using said data transfer path to transfer flow control primitives between said source and destination nodes, and each switch control means also for receiving destination node location requests and destination addresses, and using said destination address to search a table of destination addresses of nodes on the local FCAL of said switch control means to determine if the destination node identified in said destination node location request is on the local FCAL to which said switch control means is coupled, and, if so, for arbitrating for control of said local FCAL and, when control is won, sending a grant message back to said switch control means which initiated destination node location request identifying the switch control means coupled to the local FCAL on which said destination node is resident and for sending one or more messages to said switch control means which initiated said destination node location request for purposes of controlling establishment of said data transfer path.

33. A switch for selectively coupling pairs of nodes on a plurality of FCAL nets to each other for one or more simultaneous conversations, comprising:

data path circuitry having a plurality of parallel point-to-point transmit/receive channels and multiplexing circuitry to selectively couple individual ones of said transmit/receive channels to transmit-receive terminals;

a protocol bus;

a plurality of ports each port for interfacing a local FCAL net to said data path circuitry, each port coupled to said protocol bus and comprising;

a serializer/deserializer circuit having terminals for coupling to an FCAL net and having demultiplexing circuitry to convert incoming differential, encoded serial data from said FCAL net to multi-bit characters and having clock recovery circuitry to recover a receive clock therefrom and having multiplexing circuitry for receiving a plurality of multi-bit characters and converting them into a differential, encoded serial data stream;

an elastic buffer to receive multi-bit characters from and send multi-bit characters to said serializer/deserializer circuit and absorb the differences in transmit and receive rate;

a routing table memory storing mappings between destination node addresses and port IDs of the port coupled to each destination node;

scoreboard means for storing status information;

one or more state machine means coupled to said elastic buffer, said routing table memory, said scoreboard means and said protocol bus and at least to said transmit-receive terminals of said data path circuitry for using said routing table memory contents for determining whether destination nodes identified in OPN primitives received from source nodes on said local FCAL net are local or remote and using the data content of said scoreboard means to determine if connection to-a destination port coupled to said destination node is permissible and transmitting a connection request on said protocol bus to said destination port requesting a connection thereto and using the contents of a connect response frame output by said state machine means of said destination port on said protocol bus for controlling said data path circuitry to implement full duplex connections between said local source and remote destination nodes or to implement full or half duplex connections to remote destination nodes, and for implementing logic and control functions to control said data path circuitry to carry out a dual simplex connections so as to send data and primitives from a source node on said local FCAL net to a remote dual simplex destination node coupled to some other port which is a source node in another half duplex loop tenancy and receive buffer credit flow control primitives back from said dual simplex destination node and to respond to each said buffer credit flow control primitive by transmitting a frame of data to said dual simplex destination node, said state machine means also structured for sending flow control primitives to a third port for transmission to a third node and receive OPN and CLS primitives and data frames from said third node when said third port has established a dual simplex connection to said port through said data path circuitry, and transmit data frames and CLS primitives received from said third node to said source node if and only if said data frames are directed to said source node as indicated by said OPN received from said third node.

34. The apparatus of claim 33 wherein said state machine means includes means for receiving configuration commands at a management interface port causing said state machine means to control said data path circuitry to assign and couple one or more transmit-receive channels to a corresponding number of one or more receiving ports on a continuing basis until no other transmit-receive channels are available to service conversations between pairs of ports and then release one or more of said assigned channels to service a conversation between pairs of ports which currently do not have an assigned channel.

35. The apparatus of claim 33 wherein said state machine means in every port includes means for receiving configuration commands at a management interface port causing said state machine means to control said data path circuitry to assign and couple a transmit-receive channel to said port if it is coupled to either the source node or destination node of a desired loop tenancy defined by an initial OPN primitive transmitted by said source node and to maintain said connection until either said loop tenancy is terminated, and, thereafter, to control said data path circuitry to release said channel and to control said state machine to publish a responder channel idle data frame on said protocol bus.

36. The apparatus of claim 33 wherein each said state machine means in every port includes means for circulating a fairness token to guarantee remote access fairness to ports by rotating a fairness token that guarantees high priority access among all ports.

37. The apparatus of claim 33 wherein each said state machine means in every port includes means for escalating a no-priority request to a low-priority request based upon the number of consecutive denials of access issued by a state machine means of a remote port.

38. The apparatus of claim 36 wherein each said state machine means in every port includes means for escalating a no-priority request to a low-priority request based upon the number of consecutive denials of access issued by a state machine means of a remote port.

39. The apparatus of claim 33 wherein every state machine means further comprises means to originate and receive broadcast sequences.

40. A bufferless switch for coupling to a plurality of FCAL nets and having a crossbar switch and FCAL loop interface port circuits structured to use the OPN and RRDY primitives of the FCAL protocol for hold back flow control to eliminate the need for a buffer and said ports and crossbar switch structured to provide multiple simultaneous loop tenancies during switch mode or to couple all FCAL nets together as one big FCAL net during hub mode, and wherein said switch can be switched back and forth between hub mode and switch mode by going through an intermediate locked mode.

41. A process for carrying out dual simplex communications in a FCAL switch coupling a plurality of FCAL nets, comprising:

receiving and latching an OPN and at least the source node address thereof identifying the address of said source node and receiving any following buffer credit RRDY primitives and either storing said RRDYs or counting the number thereof at a source port from a source node on the local FCAL net coupled to said source port, said OPN directed to a remote destination node on the FCAL net coupled to a remote port;

establishing a front channel connection through a backplane between said source port and said destination port;

converting said OPN to half duplex and transmitting said half duplex OPN to said destination OPN over said front channel;

receiving any RRDY primitives from said destination node and transmitting them to said source node via a back channel;

receiving a connection request message from a third port coupled to a third node and granting said connection request by a connection grant message to said third port coupled to said third node, said connection grant message naming a backplane channel to use for a back channel connection between said source node and said third node;

establishing a backchannel connection between said source port and said third port through said backplane channel named in said connection grant message;

transmitting an OPN directed to said source node from said third node to said source port;

comparing the destination node address in said OPN from said third port to the node address of the OPN originally received by said source port, and, if the addresses match, quashing the OPN from said third port and either transmitting one stored RRDY received from said source node to said third node or transmitting a message which causes said third port to generate an RRDY and transmit it to said third node;

transmitting one frame of data from said third node to said source node via said back channel connection; and

continuing to intercept RRDY primitives transmitted by said source node and send them to said third node and to receive frames of data transmitted in response to receipt at said third node of said RRDYs and transmit said frames to said source node via said back channel connection and mix said frames of data on said back channel connection with RRDY primitives received from said destination node.

42. A process for dual simplex communication using a switch coupling a plurality of FCAL nets, comprising:

establishing a front channel half duplex data path between a source node and destination node on different FCAL nets and stripping and storing or counting any buffer credit RRDY primitives output by said source node and not transmitting them to said destination node;

establishing a back channel data path between a third node and said source node but not transmitting to said source node any OPN primitive emitted by said third node, and transmitting a number of RRDYs either equal to the number of RRDYs output by said source node or the number of RRDYs needed by said third node to send all the data it has to said source node before closing said back channel connection, transmission of said RRDYs being one at a time; and

receiving any RRDYs transmitted by said destination node and mixing them in with data frames and/or primitives transmitted on said back channel by said third node so as to exercise flow control on transmissions of data frames from said source node to said destination node.

43. A process for dual simplex communication using a switch coupling a plurality of FCAL nets, comprising:

1) receiving a full duplex OPN (Dest, Src) from a source node designating a destination node address, Dest, and a source node address, Src, and converting the full duplex format OPN (Dest, Src) to a simplex or half duplex format OPN (Dest, Dest) and establishing a front channel connection to said destination node;

2) storing the full duplex OPN including at least the source node address in a source port of said switch coupled to said source node;

3) stripping any buffer credit RRDYs output by said source node and not transmitting said RRDYs output by said source node to said destination node of said front channel connection, but either storing or counting said RRDYs output by said source node and making them available to a third node to send data to said source node either by transmitting stored RRDYs output by said source node one at a time to said third node or by conveying the number of RRDYs output by said source node to a third port coupled to said third node and synthesizing in said third node a number of RRDYs equal to the value of said count and sending said RRDYs to said third node one at a time;

4) updating the status data in a scoreboard memory of each port to indicate which busy ports are coupled to source nodes of front channel connections and thus are available to receive frames in dual simplex communication from third nodes;

5) establishing a back channel connection from said third node to said source node of said front channel connection and transmitting an OPN from said third node to said source port, and using said stored source node address from said OPN transmitted by said source node and stored in said source port as an access key to compare to the destination node address of said OPN transmitted to said source port by said third node via said back channel connection and allowing dual simplex communication to proceed on said back channel if said source node address matches said destination node address of said OPN from said third node, but if there is no match, blocking dual simplex communication with this particular third node and closing said back channel connection;

6) deleting said OPN from said third node in said source port and never transmitting it to said source node if the destination address of the OPN from said third node matches the stored source address of the stored OPN from said source node; and

7) receiving data frames transmitted by said third node at said source port with one data frame transmitted in response to each RRDY received by said third node, and mixing said data frames in with RRDYs received from said destination node via said front channel connection and transmitting the combined data frames and RRDYs to said source node via said back channel connection.

44. A switch apparatus comprising a plurality of integrated circuits each comprising a slice of an FCAL switch and coupled together to form said FCAL switch, each integrated circuit having a plurality of port circuits, each port circuit having substantially less buffer memory to store data than prior art FL ports in prior art fabric switches, each said integrated circuit also comprising a portion of a distributed, scaleable crossbar switch such that any number of said integrated circuit slices can be coupled together to form an FCAL switch which has as many ports as are necessary for the size of network in which the switch is to be used, limited only by the available FCAL address space of 128 nodes.

45. The apparatus of claim 44 wherein said crossbar switch is structured such that when all of said integrated circuits are coupled together, a complete crossbar switch is formed, said integrated circuit or circuits including a lookup table and circuitry to use the destination address of an OPN primitive arriving from a local FCAL net coupled to a port circuit to access said lookup table to determine the FCAL net and port coupled to a destination node having that destination address and to determine whether to make connections so as to forward the OPN back onto the local FCAL net from which the OPN arrived and keep any subsequent data frames and primitives on the local FCAL net or to forward the OPN to another port on the same integrated circuit for coupling onto its FCAL net and transmit subsequent data frames and primitives between the FCAL net coupled to the port which received the OPN and the FCAL net coupled to another port on the same integrated circuit having the destination node identified in said OPN or to make connections through said crossbar switch to forward said open to a port on another integrated circuit and an FCAL net coupled to said port and said destination node and to transfer any subsequent data frames and primitives between said FCAL nets coupled to different integrated circuits through said crossbar switch and the ports coupled to said FCAL nets.

46. The switch of claim 44 wherein said port circuits and crossbar switch circuitry are structured so as to be capable of cooperating to implement dual simplex communications.

47. The switch of claim 44 wherein said switch contains a memory for storing configuration data and wherein said configuration data contains programmable data indicating whether or not dual simplex communication is allowed, and wherein said port circuits and crossbar switch circuitry are structured so as to be capable of cooperating to implement dual simplex communications when said configuration data indicates dual simplex communication is allowed.

48. A process for insuring fairness is achieved in an switch having a plurality of switch ports, each coupled to a an FCAL net having one or more nodes coupled thereto, comprising:

assigning each switch port a priority level for use in arbitration between conflicting connect requests from multiple switch ports to the same destination port;

circulating a fairness token among said switch ports so that each switch port receives said fairness token, holds it for a time and then forwards it to another switch port;

assigning a switch port that has said fairness token in its possession the highest of said priority levels.

49. The process of claim 48 further comprising the steps of:

keeping a camp list for busy ports which have received multiple connect requests from different switch ports;

when a switch port in possession of said fairness token makes a connect request to a switch port with a camp list which is not full, adding the ID of said switch port having said fairness token to said camp list;

when access to said switch port with the camp list is granted to the switch port in possession of the fairness token, forwarding said fairness token to another switch port.

50. An FCAL buffered switch for coupling to a plurality of FCAL nets and having a crossbar switch and FCAL loop interface port circuits structured to use the destination address of the OPN to find the destination node and destination port and including switching circuitry coupled to said crossbar switch by by multiple paths including at least a straight through path and paths through one or more buffer memorys to exchange data frames and FCAL primitives with one or more destination nodes and structured to use one or more buffer memories to store data output by a source node destined for a destination node which has a busy status at the time the data is output by said source node, said ports and crossbar switch structured to provide multiple simultaneous loop tenancies.

51. An FCAL buffered switch for coupling to a plurality of FCAL nets and having a crossbar switch and FCAL loop interface port circuits structured to use the destination address of the OPN to find the destination node and destination port and including switching circuitry coupled to said crossbar switch by by multiple paths including at least a straight through path and paths through one or more buffer memorys to exchange data frames and FCAL primitives with one or more destination nodes and structured to use one or more buffer memories to store data output by a source node destined for a destination node which has a busy status at the time the data is output by said source node, said ports and crossbar switch structured to provide multiple simultaneous loop tenancies and said ports and crossbar switch also structured to provide dual simplex communication across said switch.

52. An FCAL buffered switch for coupling to a plurality of FCAL nets and having a crossbar switch and FCAL loop interface port circuits structured to use the destination address of the OPN to find the destination node and destination port and including switching circuitry coupled to said crossbar switch by by multiple paths including at least a straight through path and paths through one or more buffer memorys to exchange data frames and FCAL primitives with one or more destination nodes and structured to use one or more buffer memories to store data output by a source node destined for a destination node which has a busy status at the time the data is output by said source node, said ports and crossbar switch structured to provide multiple simultaneous loop tenancies and said ports and crossbar switch also structured to provide dual simplex communication across said switch, each said port including a local bypass data path which can be switched to keep primitives and data frames generated on a local FCAL net coupled to said port routed so as to stay on said local FCAL net such that local loop tenancies on each local FCAL net coupled to a port which do not involve transmission of data or primitives from one port to another can occur simultaneously, and each port including local learning memory means for storing a routing table and scoreboard table, and functioning to fully cache the 8-bit addresses of all nodes on each said local FCAL net coupled to a port along with the ID of the port coupled to each node in said routing table and the busy, available or no privilege status of each other port and whether each busy port is available for dual simplex communications in said scoreboard table, each port structured to write the contents of each said routing table and scoreboard table by learning either from watching local traffic in said port bound for local destination nodes or remote traffic leaving said port and crossing said crossbar switch bound for remote destination nodes or by conducting an active discovery process, each port structured to write the contents of said scoreboard table by learning from watching messages posted on a protocol bus, each said port structured to use destination addresses in OPN primitives transmitted by nodes on said local FCAL net as search keys to search said routing table to determine whether the destination node is local or remote and to which port it is connected and, if local, to switch said local bypass to keep the loop tenancy confined to said local FCAL net, or, if remote, to generate a connection request message to said port coupled to said destination node, each said port being structured to receive such connection requests and respond by arbitrating for control of its local FCAL net and pick a data path through said crossbar switch and generate a reply message naming the data path through said crossbar switch to use in exchanging data frames and primitives, each said port also being structured to respond to receipt or generation of said reply message by generating commands to establish said data path named in said reply message through said crossbar switch, each said port also structured to circulate a fairness token on a fairness token bus coupled to all such ports and to used said fairness token to increase its priority level of access privilege to busy nodes when the fairness token is in possession of the port so that no port can be starved, and then to circulate said fairness token to a neighboring port such that all ports eventually get said fairness token.