System and method for implementing LAN within shared I/O subsystem

• US 7,356,608 B2
• Filed: 06/28/2002
• Issued: 04/08/2008
• Est. Priority Date: 05/06/2002
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: QLogic Corporation (Marvell Technology Group Limited)
• Inventors: Rimmer, Todd
• Primary Examiner(s): Flynn; Nathan
• Assistant Examiner(s): DENNISON, JERRY B
• Application Number:
  US10/186,191
• Publication Number:
  US 20030208633A1
• Time in Patent Office: 2,111 Days
• Field of Search: 709/212, 709/215, 709/238, 709/239, 709/245, 709/250, 719/311, 719/312
• US Class Current: 709/238
• CPC Class Codes: H04L 45/742 Route cache; Operation thereof
• CPC Class Codes: H04L 49/3009 Header conversion, routing tables or routing tags
• CPC Class Codes: H04L 49/358 Infiniband Switches
• CPC Class Codes: H04L 49/602 Multilayer or multiprotocol switching, e.g. IP switching
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1017 based on a round robin mechanism
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/18 Multiprotocol handlers, e.g. single devices capable of handling multiple protocols
• CPC Class Codes: H04L 69/22 Parsing or analysis of headers
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A shared I/O subsystem for providing network protocol management for a plurality of computer systems. The shared I/O subsystem includes a plurality of I/O interfaces where each of the I/O interfaces operatively couples one of the computer systems to the shared I/O subsystem. The shared I/O subsystem also includes an I/O management link that operatively interconnects the I/O interfaces, and a link layer switch that communicatively couples to each of the I/O interfaces. The link layer switch receives a data packet from one of the I/O interfaces and directs the data packet to one or more of the other ones of the I/O interfaces. The I/O interfaces may form a local area network within the shared I/O subsystem.

Claims

1. A shared I/O subsystem for providing network protocol management for a plurality of computer systems comprising:

a plurality of I/O interfaces, each of the I/O interfaces operatively coupling one or more of the computer systems to the shared I/O subsystem;

the plurality of computer systems including a plurality of initiating servers that initiate I/O requests;

an I/O management link operatively interconnecting the I/O interfaces; and

a link layer switch communicatively coupled to each of the I/O interfaces, where the link layer switch receives a data packet from one of the I/O interfaces and directs the data packet to one or more of the other ones of the I/O interfaces;

wherein the I/O interfaces form a local area network within the shared I/O subsystem; and

at least one of the I/O interfaces includes;

(a) a plurality of virtual ports where each virtual port is operationally coupled to a virtual network interface card (NIC) via a virtual I/O bus, where the virtual NIC is executed by a computer system coupled to the I/O interface;

(b) a target channel adapter (TCA) that communicates with a host channel adapter (HCA) of the computer system;

(c) a switching function that uses a forwarding table for routing inbound packets received from a network and destined for one or more of the plurality of computer systems and for routing outbound packets originating from one or more of the plurality of computer systems destined for a network device or a virtual port of an I/O interface; and

(d) a shared physical NIC that is shared by the plurality of virtual NICs of the plurality of computer systems to send and receive network packets via a network link where the virtual NIC on an initiating server allows the initiating server to communicate via the shared physical NIC as if the shared physical NIC was dedicated only to the initiating server;

wherein the virtual NIC receives an outbound packet from a software stack that is executed above the virtual NIC stack in a computer system from among the plurality of computer systems and the virtual NIC;

(a) encapsulates the outbound packet into a combination of send/receive and remote direct memory access (RDMA) operations that forms the virtual I/O bus between each virtual port and the virtual NIC; and

(b) sends the encapsulated operations to the HCA and the HCA then sends the encapsulated operations to the TCA; and

the encapsulated send/receive and RDMA operations are reassembled into an outgoing packet by the virtual port associated with the virtual NIC; and

the virtual port forwards the outgoing packet to the switching function for delivery to another virtual port or to the shared NIC for delivery to another network device.

2. The shared I/O subsystem of claim 1, wherein each of the I/O interfaces is independently detachable from one another.

3. The shared I/O subsystem of claim 1, wherein the I/O management link comprises an InfiniBand Maintenance Link (IBML).

4. The shared I/O subsystem of claim 1, wherein the data packet follows an IB ML protocol.

5. The shared I/O subsystem of claim 4, wherein the data packet includes a user configurable portion, and wherein the user configuration portion is populated by an Ethernet packet.

6. The shared I/O subsystem of claim 5, wherein the user configurable portion includes data associated with at least one of a HTTP protocol, an SNMP protocol, a Telnet protocol, and an FTP protocol.

7. The shared I/O subsystem of claim 1, wherein at least one of the computer systems comprises an Ethernet network.

8. The shared I/O subsystem of claim 1, wherein at least one of the computer systems comprises a Fibre Channel Storage Network System.

9. The shared I/O subsystem of claim 1, wherein at least one of the computer systems comprises a IBML network.

10. The shared I/O subsystem of claim 1, wherein the data packet follows multiple network layer protocols.

11. The shared I/O subsystem of claim 1, wherein one of the plurality of initiating servers uses an operating system that is different from an operating system used by one other of the plurality of initiating servers.

12. The shared I/O subsystem of claim 1, wherein at least one of the computer system is one of a network appliance and a network device.

13. A shared input/output (I/O) subsystem for providing network protocol management for a plurality of computer systems, comprising:

an I/O interface unit from among a plurality of I/O interface units, operationally coupled to one or more computer systems from among the plurality of computer systems via an I/O link;

the I/O interface unit comprising;

a plurality of virtual ports where each virtual port is operationally coupled to a virtual network interface card (NIC) via a virtual I/O bus and the virtual NIC is executed by each of the computer system coupled to the I/O interface unit;

a target channel adapter (TCA) that communicates with a host channel adapter (HCA) of each of the computer system coupled to the I/O interface unit via the I/O link and a fabric;

a switching function that uses a forwarding table for routing inbound packets received from a network and destined for one or more of the plurality of computer systems; and

for routing outbound packets originating from one or more of the plurality of computer systems destined for a network device or a virtual port of an I/O interface unit; and

a shared physical NIC that is shared by a plurality of virtual NICs executed by the plurality of computer systems to send and receive network packets via a network link;

where a virtual NIC in a computer system allows the computer system to communicate via the shared physical NIC as if the shared physical NIC was dedicated only to the computer system;

wherein a virtual NIC of a computer system coupled to the I/O interface unit (a) receives an outbound packet from a software application executed by the computer system;

(b) encapsulates the outbound packet into a combination of send/receive and remote direct memory access (RDMA) operations that forms a virtual I/O bus between a virtual port and the virtual NIC; and

(c) sends the encapsulated operations to the HCA; and

the HCA then sends the encapsulated operations to the TCA; and

the encapsulated send/receive and RDMA operations are reassembled into an outgoing packet by the virtual port associated with the virtual NIC; and

the virtual port forwards the outgoing packet to the switching function for delivery to another virtual port or to the shared NIC for delivery to another network device.

14. The subsystem of claim 13, wherein the shared NIC receives an inbound packet from a network device via the network link and delivers the packet to the switching function and based on a destination address of the inbound packet, the switching function forwards the inbound packet to a virtual port that performs a combination of send/receive and RDMA operations for a logical virtual I/O bus between the virtual port and a virtual NIC;

and the send/receive and RDMA operations are delivered from a TCA to a HCA via the I/O link.

15. The subsystem of claim 14, wherein the virtual NIC delivers the inbound packet to a protocol stack executed above the virtual NIC.

16. The subsystem of claim 13, wherein each virtual port uses a virtual port frame for sending data with variable length and without using interrupt signals, the virtual port frame comprising a first portion for arranging data bits into a virtual port frame;

and a second portion which includes control bits where one of the control bit indicates when packet transmission is complete.

17. The subsystem of claim 16, wherein the virtual frame includes a padded segment whose length is set so that an address of a beginning of an Ethernet frame is computed from an address in the control bits.

18. The subsystem of claim 13, wherein the I/O link is an InfiniBand link.

19. A method for a shared input/output (I/O) subsystem for providing network protocol management for a plurality of computer systems coupled to one of a plurality of I/O interface units, comprising:

receiving an outbound packet initiated by an initiating computer system from among one of the plurality of computer systems; and

a virtual network interface card (NIC) executed in the initiating computer system receives the outbound packet;

encapsulating the outbound packet into a combination of send/receive and remote direct memory access (RDMA) based operations, wherein the virtual NIC encapsulates the operations and delivers the encapsulated operations to a host channel adapter (HCA) that communicates with a target channel adapter (TCA) in one of the I/O interface units via an I/O link;

reassembling the outbound packet from the encapsulated operations, wherein a virtual port in the I/O interface unit reassembles the outbound packet and delivers the outbound packet to a switching function; and

delivering the outbound packet to another virtual port of an I/O interface unit or to a shared physical NIC that is linked to a network using a network link;

wherein the shared physical NIC is shared by a plurality of virtual NICs executed in each of the plurality of computer systems to send and receive network packets via the network link and the virtual NIC on each given initiating computer system allows the initiating computer system to communicate via the shared physical NIC as if the shared physical NIC was dedicated only to said initiating computer system.

20. The subsystem of claim 19, wherein the shared physical NIC receives an inbound packet from a network device via the network link and delivers the packet to the switching function and based on a destination address of the inbound packet, the switching function forwards the inbound packet to a virtual port that performs a combination of send/receive and RDMA operations for a logical virtual I/O bus between the virtual port and a virtual NIC;

and the send/receive and RDMA operations are delivered from a TCA to a HCA via an I/O link.

21. The subsystem of claim 19, wherein each virtual port uses a virtual port frame for sending data with variable length and without using interrupt signals, the virtual port frame comprising a first portion for arranging data bits into a virtual port frame;

and a second portion which includes control bits where one of the control bit indicates when packet transmission is complete.

22. The subsystem of claim 19, wherein the virtual frame includes a padded segment whose length is set so that an address of a beginning of an Ethernet frame is computed from an address in the control bits.