Switching apparatus and method for providing shared I/O within a load-store fabric

• US 20050025119A1
• Filed: 04/19/2004
• Published: 02/03/2005
• Est. Priority Date: 01/21/2003
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: NextIO Incorporated
• Inventors: Khan, Asif, Pagan, Annette, Utley, Robert Haskell, Pekkala, Richard E., Pettey, Christopher J.
• Granted Patent: US 7,219,183 B2
• Time in Patent Office: Days
• US Class Current: 370/351
• CPC Class Codes: H04L 49/25 Routing or path finding in a switch fabric
• CPC Class Codes: H04L 49/253 using establishment or release of connections between ports
• CPC Class Codes: H04L 49/254 Centralised controller, i.e. arbitration or scheduling
• CPC Class Codes: H04L 49/602 Multilayer or multiprotocol switching, e.g. IP switching
• CPC Class Codes: H04L 49/604 Hybrid IP/Ethernet switches

Abstract

An apparatus and method are provided that enable I/O devices to be shared among multiple operating system domains. The apparatus has a first plurality of I/O ports, a second I/O port, and core logic. The first plurality of I/O ports is coupled to a plurality of operating system domains through a load-store fabric. Each of the first plurality of I/O ports is configured to route transactions between said plurality of operating system domains and the switching apparatus. The second I/O port is coupled to a first shared input/output endpoint, where the first shared input/output endpoint is configured to request/complete said transactions for each of the plurality of operating system domains. The core logic is coupled to the first plurality of I/O ports and the second I/O port. The core logic routes the transactions between the first plurality of I/O ports and the second I/O port.

Claims

1. A switching apparatus for sharing input/output endpoints, the switching apparatus comprising:

a first plurality of I/O ports, coupled to a plurality of operating system domains through a load-store fabric, each configured to route transactions between said plurality of operating system domains and the switching apparatus;

a second I/O port, coupled to a first shared input/output endpoint, wherein said first shared input/output endpoint is configured to request/complete said transactions for each of said plurality of operating system domains; and

core logic, coupled to said first plurality of I/O ports and said second I/O port, configured to route said transactions between said first plurality of I/O ports and said second I/O port.

2. The switching apparatus as recited in claim 1, wherein the switching apparatus routes said transactions according to a protocol that provides for routing of said transactions only for a single operating system domain, and wherein said first plurality of I/O ports communicates with said plurality of operating system domains according to said protocol.

3. The switching apparatus as recited in claim 1, wherein said load-store fabric comprises a point-to-point fabric.

4. The switching apparatus as recited in claim 1, wherein said core logic maps said each of said plurality of operating system domains to a corresponding root complex.

5. The switching apparatus as recited in claim 4, wherein said corresponding root complex is associated with one or more processing complexes, each comprising:

a processing core, configured to process data, wherein said processing core accesses a memory that is configured to store said data.

6. The switching apparatus as recited in claim 5, wherein said data is transferred from said corresponding root complex to a corresponding one of said first plurality of I/O ports according to a protocol that provides for routing of said transactions only for a single operating system domain.

7. The switching apparatus as recited in claim 5, wherein said data is transferred to said root complex from a corresponding one of said first plurality of I/O ports according to a protocol that provides for routing of said transactions only for a single operating system domain.

8. The switching apparatus as recited in claim 1, wherein said load-store fabric comprises hardware, software, or a combination of hardware and software that is configured to route said transactions.

9. The switching apparatus as recited in claim 1, wherein said first shared input/output endpoint comprises a shared network interface controller (NIC).

10. The switching apparatus as recited in claim 9, wherein said shared NIC comprises a shared Ethernet controller.

11. The switching apparatus as recited in claim 1, wherein said first shared input/output endpoint comprises a Fiber Channel controller.

12. The switching apparatus as recited in claim 1, wherein said first shared input/output endpoint comprises a shared RAID controller.

13. The switching apparatus as recited in claim 1, wherein said first shared input/output endpoint is integrated into the switching apparatus.

14. The switching apparatus as recited in claim 1, wherein said first shared input/output endpoint is integrated into a root complex.

15. The switching apparatus as recited in claim 1, wherein said first shared input/output endpoint is integrated into a processing complex.

16. The switching apparatus as recited in claim 2, wherein said protocol comprises PCI Express, and wherein said core logic isolates said transactions over a plurality of PCI Express bus hierarchies according to said each of said plurality of operating system domains.

17. The switching apparatus as recited in claim 1, wherein said core logic is configured to associate each of said transactions with a corresponding one of said plurality of operating system domains (OSDs), said corresponding one of said plurality of OSDs corresponding to one or more root complexes.

18. The switching apparatus as recited in claim 17, wherein said plurality of OSDs comprises 16 OSDs.

19. The switching apparatus as recited in claim 17, wherein said core logic designates said corresponding one of said plurality of OSDs according to a variant of a protocol, wherein said protocol provides for routing of said transactions only for a single operating system domain.

20. The switching apparatus as recited in claim 19, wherein said variant comprises encapsulating an OS domain header within a transaction layer packet that otherwise comports with said protocol.

21. The switching apparatus as recited in claim 20, wherein said transaction layer packet is routed between said second I/O port and said first shared input/output endpoint.

22. The switching apparatus as recited in claim 21, wherein said first shared I/O endpoint is configured to detect said OS domain header and to perform a specified operation according to said protocol exclusively for said one of said plurality of OSDs.

23. The switching apparatus as recited in claim 1, wherein said first shared input/output endpoint designates one of said transactions for a particular one of said plurality of operating system domains by encapsulating an OS domain header within a transaction layer packet, and wherein said transaction layer packet is routed to said core logic via said second I/O port.

24. The switching apparatus as recited in claim 23, wherein said core logic is configured to detect and decapsulate said OS domain header from said transaction layer packet, and is configured to route said one of said transactions to said particular one of said plurality of operating system domains according to protocol that provides for routing of said transactions only for a single operating system domain.

25. The switching apparatus as recited in claim 1, wherein said core logic is integrated within a shared I/O switch.

26. The switching apparatus as recited in claim 1, wherein said core logic is integrated within a root complex.

27. The switching apparatus as recited in claim 1, wherein said core logic is integrated within a processing complex.

28. A shared input/output (I/O) switching mechanism, comprising:

core logic, configured to enable operating system domains to share one or more I/O endpoints, said core logic comprising;

global routing logic, configured to route first transactions to/from said operating system domains, and for routing second transactions to/from said one or more I/O endpoints, wherein each of said second transactions designates an associated one of said operating system domains for which an operation specified by each of said first transactions be performed.

29. The shared I/O switching mechanism as recited in claim 28, wherein said first transactions comport with a protocol that provides exclusively for a single operating system domain within said load-store fabric.

30. The shared I/O switching mechanism as recited in claim 28, wherein said first transactions comport with PCI Express.

31. The shared I/O switching mechanism as recited in claim 30, wherein said core logic isolates said first and second transactions over a plurality of PCI Express bus hierarchies according to each of said operating system domains.

32. The shared I/O switching mechanism as recited in claim 31, wherein said first and second transactions are routed in accordance with addressing mechanisms for a particular one of said plurality of PCI Express bus hierarchies, said particular one of said plurality of PCI Express bus hierarchies corresponding to said associated one of said operating system domains.

33. The shared I/O switching mechanism as recited in claim 28, wherein said core logic associates each of said operating system domains with each of a corresponding root complex.

34. The shared I/O switching mechanism as recited in claim 28, wherein said second transactions comport with a variant of a protocol that provides exclusively for a single operating system domain within said load-store fabric.

35. The shared I/O switching mechanism as recited in claim 34, wherein said variant comprises encapsulating an OS domain header within a transaction layer packet of said each of said second transactions, wherein said each of said second transactions otherwise comports with said protocol.

36. The shared I/O switching mechanism as recited in claim 35, wherein said transaction layer packet is routed between said the shared I/O switching mechanism and one of said one or more I/O endpoints.

37. The shared I/O switching mechanism as recited in claim 36, wherein said one of said one or more I/O endpoints is configured to detect said OS domain header and to perform said operation according to said protocol exclusively for said one of said operating system domains.

38. The shared I/O switching mechanism as recited in claim 28, wherein said the shared I/O switching mechanism comprises a shared I/O switch.

39. The shared I/O switching mechanism as recited in claim 28, wherein one of said one or more I/O endpoints designates one of said second transactions for a particular one of said operating system domains by encapsulating an OS domain header within a transaction layer packet, and wherein said transaction layer packet is routed to the shared I/O switching mechanism.

40. The shared I/O switching mechanism as recited in claim 39, wherein VMAC logic within the shared I/O switching mechanism is configured to detect and decapsulate said OS domain header from said transaction layer packet, and wherein said core logic routes a corresponding one of said first transactions to said particular one of said plurality of operating system domains according to a protocol that provides exclusively for a single operating system domain within said load-store fabric.

41. A method for interconnecting independent operating system domains to a shared I/O endpoint within a load-store fabric, comprising:

via first ports, first communicating with each of the independent operating system domains according to a protocol that provides exclusively for a single operating system domain within the load-store fabric;

via a second port, second communicating with the shared I/O endpoint according to a variant of the protocol to enable the shared I/O endpoint to associate a prescribed operation with a corresponding one of the independent operating system domains; and

via core logic within a switching apparatus, mapping the independent operating system domains to the shared I/O endpoint.

42. The method as recited in claim 41, further comprising:

over a point-to-point fabric, first interconnecting the independent operating system domains to the switching apparatus via the first ports; and

over the point-to-point fabric, second interconnecting the shared I/O endpoint to the switching apparatus via the second port.

43. The method as recited in claim 41, wherein said first communicating comprises:

associating each of the operating system domains with a corresponding root complex.

44. The method as recited in claim 43, wherein the corresponding root complex is associated with one or more processing complexes, each configured to process data, and wherein the data is stored in a memory.

45. The method as recited in claim 44, further comprising:

transferring the data to from the corresponding root complex according to the protocol.

46. The method as recited in claim 44, further comprising:

transferring the data to the corresponding root complex according to the protocol.

47. The method as recited in claim 41, wherein said first communicating comprises:

employing PCI Express as the protocol.

48. The method as recited in claim 47, wherein said mapping comprises:

isolating transactions over a plurality of PCI Express bus hierarchies according to each of the operating system domains.

49. The method as recited in claim 48, wherein said mapping further comprises:

routing the transactions in accordance with addressing mechanisms for a particular one of the plurality of PCI Express bus hierarchies, the particular one of said plurality of PCI Express bus hierarchies corresponding to the corresponding one of the operating system domains.

50. The method as recited in claim 41, wherein said second communicating comprises:

employing the variant of the protocol to associate a unique root complex that with the corresponding one of the operating system domains.

51. The method as recited in claim 50, wherein said employing comprises:

encapsulating an OS domain header within a transaction layer packet that otherwise comports with the protocol, wherein the value of the OS domain header designates the corresponding one of the operating system domains.

52. The method as recited in claim 51, wherein said second communicating further comprises:

routing the transaction layer packet between the second port and the shared I/O endpoint.

53. The method as recited in claim 52, wherein said second communicating further comprises:

within the I/O endpoint, detecting the OS domain header; and

performing the prescribed operation according to the protocol, wherein said performing is accomplished exclusively for the corresponding one of the operating system domains.

54. The method as recited in claim 41, wherein said second communicating comprises:

within the shared I/O endpoint, designating a transaction for a particular one of the operating system domains by embedding an OS domain header within a transaction layer packet; and

first transmitting the transaction layer packet to the second port.

55. The method as recited in claim 54, wherein said second communicating comprises:

within the switching apparatus, detecting and removing the OS domain header from the transaction layer packet.

56. The method as recited in claim 55, wherein said first communicating comprises:

second transmitting the transaction layer packet to the particular one of the operating system domains according to the protocol.