Method and apparatus for shared I/O in a load/store fabric

• US 7,620,064 B2
• Filed: 09/26/2005
• Issued: 11/17/2009
• Est. Priority Date: 01/21/2003
• Status: Active Grant

• Current Assignee: Mellanox Technologies Limited (NVIDIA Corporation)
• Original Assignee: NextIO Incorporated
• Inventors: Khan, Asif, Pagan, Annette, Pekkala, Richard E., Utley, Robert Haskell, Pettey, Christopher J.
• Primary Examiner(s): Nguyen, Brian D
• Application Number:
  US11/235,514
• Publication Number:
  US 20060018342A1
• Time in Patent Office: 1,513 Days
• Field of Search: 370/389, 370/392, 370/419, 370/463, 370/468, 370/401, 370/402, 370/395.5, 709/203, 709/220, 709/213, 710/100, 710/104, 710/305, 710/313
• US Class Current: 370/463
• CPC Class Codes: H04L 49/351 for local area network [LAN], e.g. Ethernet switches
• CPC Class Codes: H04L 49/356 for storage area networks
• CPC Class Codes: H04L 67/54 Presence management, e.g. monitoring or registration for receipt of user log-on information, or the connection status of the users

Abstract

An apparatus and method is provided for allowing I/O devices to be shared and/or partitioned among a plurality of processing complexes within the load/store fabric of each of the processing complexes without requiring modification to the operating system or driver software of the processing complexes. The apparatus and method includes a switch for selectively coupling each of the processing complexes to one or more shared I/O devices. The apparatus and method further includes placing information within packets transmitted between the switch and the I/O devices to identify which of the processing complexes the packets are associated with. The invention further includes an apparatus and method within the shared I/O devices to allow the shared I/O devices to service each of the processing complexes independently.

Claims

1. An apparatus to allow a first computer and a second computer to share a network interface controller, the apparatus comprising:

a shared I/O switch;

a shared Ethernet network interface controller, wherein in lieu of including an Ethernet controller within each of the first computer and the second computer, each of the first computer and the second computer are coupled to share the Ethernet network interface controller in order to communicate on a network using an Ethernet protocol;

a first link coupling the first computer to said shared I/O switch;

a second link coupling the second computer to said shared I/O switch;

a third link coupling said shared I/O switch to the Ethernet network interface controller, said third link utilizing a serial load/store fabric to pass packets originating from both the first computer and the second computer to the Ethernet network interface controller, wherein packets conveyed via said third link include header information which associates each of said packets with either the first computer or the second computer; and

an interface for the Ethernet network interface controller which examines said packets, including said header information, and selects resources for said packets based on said association.

2. The apparatus as recited in claim 1 wherein the first computer and/or the second computer is a blade server.

3. The apparatus as recited in claim 1 wherein each of the first computer and the second computer correspond to a root complex in a PCI express hierarchy, and wherein the Ethernet network interface controller corresponds to an endpoint in the PCI express hierarchy.

4. The apparatus as recited in claim 1 wherein the serial load/store fabric utilizes a modified form of a PCI Express protocol, the modified form of the PCI Express protocol comprising PCI Express packets that have been modified by adding an identification of either the first computer or the second computer.

5. The apparatus as recited in claim 4 wherein said first link and said second link utilize the PCI Express protocol comprising PCI Express packets that do not include said identification.

6. The apparatus as recited in claim 1 wherein said shared I/O switch comprises routing control to add said header information to said packets received from the first computer and the second computer.

7. The apparatus as recited in claim 1 wherein said interface comprises a table which associates dedicated resources with either the first computer or the second computer.

8. The apparatus as recited in claim 7 wherein said interface further comprises a multiplexer for selecting said dedicated resources utilizing information from said table.

9. The apparatus as recited in claim 1 wherein packets from both the first computer and the second computer reside within the Ethernet network interface controller at the same time.

10. A method to allow at least two root complexes to share an endpoint device within a serial load/store fabric, comprising:

receiving original packets, at a switch, from the at least two root complexes;

modifying the received original packets to form modified packets by adding header information which identifies a root complex which is a source of each of the packets;

receiving the modified packets at the endpoint device, wherein the endpoint device is configured to communicate using a first protocol, and wherein in lieu of including dedicated endpoint devices within each of the root complexes that is configured to communicate using the first protocol, each of the root complexes are coupled to share the endpoint device in order to communicate using the first protocol;

at the endpoint device, examining the modified packets to determine which of the at least two root complexes are associated with the original packets;

allocating resources for the original packets based on the association; and

processing the original packets according to said step of allocating.

11. The method to share an endpoint device as recited in claim 10 wherein each of the two root complexes is a root complex in a PCI express hierarchy, and wherein said endpoint device shared by the two root complexes corresponds to an endpoint device in the PCI express hierarchy.

12. The method to share an endpoint device as recited in claim 10 wherein the endpoint device is a shared network interface controller.

13. The method to share an endpoint device as recited in claim 10 wherein the endpoint device is a shared storage controller.

14. The method to share an endpoint device as recited in claim 13 wherein the shared storage controller is a Fibre Channel controller.

15. The method to share an endpoint device as recited in claim 10 wherein each of the two root complexes are configured to convey said original packets using a PCI Express protocol, and wherein said endpoint device is configured to receive the modified packets via an interface configured to communicate using a modified form of the PCI Express protocol which comprises PCI Express packets that have been modified by adding an identification of a root complex that is a source of each of the PCI Express packets.

16. The method to share an endpoint device as recited in claim 15 wherein said PCI Express packets do not include said identification.

17. The method to share an endpoint device as recited in claim 16 wherein the first protocol is at least one of the following:

an Ethernet protocol, and a Fibre Channel protocol.

18. The method to share an endpoint device as recited in claim 10 further comprising:

receiving the original packets from the at least two root complexes into the switch via independent links;

the switch forming the modified packets by incorporating the header information into the original packets; and

transmitting the modified packets from the switch to the endpoint device via a single link.

19. The method to share an endpoint device as recited in claim 10 wherein said step of examining is performed by a bus interface of the endpoint device.

20. The method to share an endpoint device as recited in claim 10 wherein a bus interface in the endpoint device comprises a table to associated the header information with the resources.

21. The method to share an endpoint device as recited in claim 10 wherein the resources comprise a plurality of control registers.

22. The method to share an endpoint device as recited in claim 21 wherein each of the plurality of control registers are dedicated to either of the at least two root complexes.