Fibre channel controller shareable by a plurality of operating system domains within a load-store architecture

• US 20050157725A1
• Filed: 01/27/2005
• Published: 07/21/2005
• Est. Priority Date: 01/21/2003
• Status: Active Grant

• Current Assignee: Mellanox Technologies Limited (NVIDIA Corporation)
• Original Assignee: NextIO Incorporated
• Inventors: Pettey, Christopher J.
• Granted Patent: US 7,512,717 B2
• Time in Patent Office: Days
• US Class Current: 370/392
• CPC Class Codes: G06F 13/12 using hardware independent of the central processor, e.g. channel or peripheral processor

Abstract

A Fibre Channel (FC) controller shareable by a plurality of operating system domains (OSDs) within a load-store architecture is disclosed. The controller includes a plurality of control/status register (CSR) banks. A respective one of the CSR banks is used by each OSD to request the controller to perform I/O operations with remote FC devices. A load-store bus interface receives from a load-store bus load and store transactions from each OSD. Each transaction includes an OSD identifier identifying the OSD that initiated the transaction. The bus interface directs the transactions to the respective CSR bank based on the OSD identifier. A FC port obtains a distinct FC port identifier for each OSD and transceives FC frames with the remote FC devices using the distinct FC port identifier for each OSD in response to the I/O operation requests. In one embodiment, the controller includes a shared I/O switch coupling the OSDs thereto.

Claims

1. A Fibre Channel (FC) controller shareable by a plurality of operating system domains (OSDs) within a load-store architecture, comprising:

a plurality of control/status register (CSR) banks, wherein a respective one of said plurality of CSR banks is used by each of the plurality of OSDs to request the shared FC controller to perform I/O operations with remote FC devices;

a load-store bus interface, coupled to said plurality of CSR banks, for coupling to a load-store bus, configured to receive, via said load-store bus, load and store transactions from each of the plurality of OSDs, each of said load and store transactions including an OSD identifier for identifying one of the plurality of OSDs that initiated said transaction, wherein said load-store bus interface directs each of said load and store transactions to said respective one of said plurality of CSR banks based on said OSD identifier; and

a FC port, coupled to said load-store bus interface, configured to obtain a distinct FC port identifier for each of the plurality of OSDs, and to transceive FC frames with said remote FC devices using said distinct FC port identifier for each of the plurality of OSDs in response to said I/O operation requests.

2. The shareable FC controller as recited in claim 1, wherein each of said load and store transactions further includes a memory address for specifying a control/status register in said respective one of said plurality of CSR banks to which said load-store bus interface directs said transaction.

3. The shareable FC controller as recited in claim 2, wherein said memory address is a source or destination memory address of a load or store instruction, respectively, executed by a processor of said one of the plurality of OSDs.

4. The shareable FC controller as recited in claim 2, wherein the shareable FC controller is selected as a destination of said load and store transactions based on said memory address.

5. The shareable FC controller as recited in claim 4, wherein the shareable FC controller is selected as a destination of said load and store transactions based on said memory address rather than based on a device identifier in said load and store transactions specifying the shareable FC controller as said destination.

6. The shareable FC controller as recited in claim 1, wherein each of said load and store transactions further includes an I/O address for specifying a control/status register in said respective one of said plurality of CSR banks to which said load-store bus interface directs said transaction.

7. The shareable FC controller as recited in claim 6, wherein said I/O address is a source or destination I/O address of a load or store instruction, respectively, executed by a processor of said one of the plurality of OSDs.

8. The shareable FC controller as recited in claim 6, wherein the shareable FC controller is selected as a destination of said load and store transactions based on said I/O address.

9. The shareable FC controller as recited in claim 8, wherein the shareable FC controller is selected as a destination of said load and store transactions based on said I/O address rather than based on a device identifier in said load and store transactions specifying the shareable FC controller as said destination.

10. The shareable FC controller as recited in claim 1, wherein each of the plurality of OSDs comprises a system load-store memory map, wherein each of said system load-store memory maps comprises a location of said respective one of said plurality of CSR banks.

11. The shareable FC controller as recited in claim 10, wherein said location of said respective one of said plurality of CSR banks is configurable by each of said plurality of OSDs within said system load-store memory map.

12. The shareable FC controller as recited in claim 10, wherein said location of said respective one of said plurality of CSR banks may be different for each of said plurality of OSD system load-store memory maps.

13. The shareable FC controller as recited in claim 10, wherein said location of said respective one of said plurality of CSR banks in said system load-store memory map does not survive a reset of the respective one of the plurality of OSDs.

14. The shareable FC controller as recited in claim 1, wherein said load-store bus interface is further configured to generate interrupts on said load-store bus to the plurality of OSDs.

15. The shareable FC controller as recited in claim 14, wherein the sharable FC controller generates said interrupts in response to completion of said requests to perform I/O operations with remote FC devices.

16. The shareable FC controller as recited in claim 15, wherein each of said interrupts include said OSD identifier for identifying the one of the plurality of OSDs that requested the shared FC controller to perform said I/O operation.

17. The shareable FC controller as recited in claim 15, wherein said interrupts comprise a message transmitted on said load-store bus, wherein said message includes said OSD identifier.

18. The shareable FC controller as recited in claim 17, wherein said message comprises a PCI message signaled interrupt (MSI) modified to include said OSD identifier.

19. The shareable FC controller as recited in claim 18, wherein said message comprises a PCI Express message signaled interrupt (MSI) packet modified to include said OSD identifier.

20. The shareable FC controller as recited in claim 1, wherein said transaction comprises a PCI Express request packet modified to include said OSD identifier.

21. The shareable FC controller as recited in claim 1, wherein said load-store bus interface selects one of said plurality of CSR banks as a target of each of said load and store transactions based on said OSD identifier identifying one of the plurality of OSDs that initiated said transaction.

22. The shareable FC controller as recited in claim 1, wherein said load-store bus interface is further configured to initiate transactions on said load-store bus targeted at the plurality of OSDs, wherein said load-store bus interface supplies an OSD identifier within each of said transactions targeted at the plurality of OSDs for identifying said one of the plurality of OSDs at which said transaction is targeted.

23. The shareable FC controller as recited in claim 1, further comprising:

at least one direct memory access controller (DMAC), coupled to said load-store bus interface, configured to request said load-store bus interface to transfer data on said load-store bus to respective ones of the plurality of OSDs specified by said DMAC.

24. The shareable FC controller as recited in claim 1, wherein said CSR banks comprise storage elements for storing data.

25. The shareable FC controller as recited in claim 24, wherein said storage elements comprise memory.

26. The shareable FC controller as recited in claim 24, wherein said storage elements comprise registers.

27. The shareable FC controller as recited in claim 1, wherein said load-store bus interface is configured for coupling to a serial load-store bus.

28. The shareable FC controller as recited in claim 27, wherein said serial load-store bus comprises a packet-based serial load-store bus.

29. The shareable FC controller as recited in claim 27, wherein said serial load-store bus comprises a PCI Express+bus.

30. The shareable FC controller as recited in claim 29, wherein said PCI Express+bus comprises a PCI Express bus for transmitting PCI Express transactions including said OSD identifier for identifying said one of the plurality of OSDs that initiated said transaction.

31. The shareable FC controller as recited in claim 29, wherein said PCI Express+bus comprises a PCI Express bus for transmitting PCI Express transactions including an OSD header.

32. The shareable FC controller as recited in claim 31, wherein said OSD header includes said OSD identifier for identifying said one of the plurality of OSDs that initiated said transaction.

33. The shareable FC controller as recited in claim 1, wherein said load-store bus interface is configured for coupling to a HyperTransport bus.

34. The shareable FC controller as recited in claim 1, wherein said load-store bus interface is configured for coupling to a VME bus.

35. The shareable FC controller as recited in claim 1, wherein said load-store bus interface is configured for coupling to a RapidIO bus.

36. The shareable FC controller as recited in claim 1, wherein said load-store bus interface is configured for coupling to an AGP bus.

37. The shareable FC controller as recited in claim 1, wherein the plurality of OSDs comprises a plurality of processors coupled to a processor bus, wherein said load-store bus interface is configured for coupling to said processor bus.

38. The shareable FC controller as recited in claim 37, wherein each of the plurality of processors provides a processor ID on said processor bus with each of said load-store transactions, wherein said OSD ID comprises said processor ID.

39. The shareable FC controller as recited in claim 1, further comprising:

a processor, coupled to said plurality of CSR banks, configured to process said requests to perform said I/O operations.

40. The shareable FC controller as recited in claim 1, further comprising:

a second FC port, coupled to said load-store bus interface, configured to obtain a second distinct FC port identifier for each of the plurality of OSDs, and to transceive FC frames with said remote FC devices using said second distinct FC port identifier for each of the plurality of OSDs in response to said I/O operation requests.

41. The shareable FC controller as recited in claim 1, further comprising:

association logic, coupled to said FC port, for associating each of said FC frames with its respective one of the plurality of OSDs based on said distinct port identifier.

42. The shareable FC controller as recited in claim 41, wherein said association logic comprises a mapping table for mapping each said port identifier to said OSD identifier uniquely identifying said respective OSD.

43. The shareable FC controller as recited in claim 41, wherein said association logic comprises a mapping table for mapping each said port identifier to a respective queue of operations of said FC port associated with said respective OSD.

44. The shareable FC controller as recited in claim 41, wherein said association logic comprises a mapping table for mapping each said port identifier to an outstanding I/O request issued to the shareable FC controller by said respective OSD.

45. The shareable FC controller as recited in claim 1, wherein said FC port is configured to drop incoming FC frames having a D_ID field value not matching any of said distinct port identifiers obtained for the plurality of OSDs.

46. The shareable FC controller as recited in claim 1, wherein said FC port comprises a FC NL_Port for communication on a FC arbitrated loop, wherein said FC NL_Port obtains a respective FC AL_PA for each of the plurality of OSDs, wherein said distinct port identifier for each of the plurality of OSDs comprises said respective FC AL_PA.

47. The shareable FC controller as recited in claim 1, wherein said FC port comprises a FC NL_Port for communication on a FC arbitrated loop, wherein said FC NL_Port appears as multiple FC NL_Ports to a FC fabric by performing multiple logins to said FC fabric.

48. The shareable FC controller as recited in claim 1, wherein said FC port comprises a FC NL_Port for communication on a FC arbitrated loop, wherein said FC NL_Port appears as multiple FC end nodes to a FC fabric by performing multiple logins to said FC fabric.

49. The shareable FC controller as recited in claim 1, wherein said FC port comprises a FC NL_Port for communication on a FC arbitrated loop, wherein said FC NL_Port appears as multiple FC end nodes to a FC arbitrated loop by acquiring multiple AL_PAs.

50. The shareable FC controller as recited in claim 1, wherein each of the plurality of OSDs comprises:

at least one processor;

an operating system executing on said at least one processor;

a load instruction executable by said at least one processor for loading data from said respective one of said plurality of CSR banks into said at least one processor; and

a store instruction executable by said at least one processor for storing data from said at least one processor to said respective one of said plurality of CSR banks.

51. The shareable FC controller as recited in claim 50, wherein said operating system comprises a device driver including said load and store instruction.

52. The shareable FC controller as recited in claim 1, wherein said load-store bus interface is configured to concurrently receive said load and store transactions from the plurality of OSDs.

53. The shareable FC controller as recited in claim 1, wherein said FC port comprises a FC N_Port linked to a FC F_Port supporting a FC Multiple N_Port_ID Assignment capability, wherein said FC N_Port obtains from said F_Port a respective FC N_Port identifier for each of the plurality of OSDs.

54. The shareable FC controller as recited in claim 1, wherein the controller is configured to determine whether a FC F_Port to which said FC port is linked supports a FC Multiple N_Port_ID Assignment capability, to obtain a distinct FC N_Port_ID for each of the plurality of OSDs if the FC F_Port supports the FC Multiple N_Port_ID Assignment capability, and to obtain a distinct FC NL_Port_ID for each of the plurality of OSDs if the FC F_Port does not support the FC Multiple N_Port_ID Assignment capability.

55. The shareable FC controller as recited in claim 1, wherein the controller is configured to determine that one of the plurality of OSDs has been reset, and to disclaim the distinct FC port identifier for the one of the plurality of OSDs that has been reset, in response thereto.

56. A Fibre Channel (FC) controller shareable by a plurality of operating system domains (OSDs) within a load-store architecture, comprising:

a plurality of I/O ports, coupled to receive load-store transactions from a corresponding plurality of OSDs;

a plurality of control/status register (CSR) banks, for programming by respective ones of the plurality of OSDs;

switching logic, coupled to said plurality of I/O ports and said plurality of CSR banks, configured to route each of said transactions to respective ones of said plurality of CSR banks based on which of said plurality of I/O ports received said transaction; and

a FC port, coupled to said plurality of CSR banks, for coupling to a FC fabric, configured to obtain for each of the plurality of OSDs a distinct respective FC port identifier from said FC fabric, and to transceive FC frames with remote FC devices using said distinct respective FC port identifier for each of the plurality of OSDs in response to said load-store transactions.

57. The shareable FC controller as recited in claim 56, wherein each of said load-store transactions includes a memory address, wherein said switching logic is further configured to route each of said load-store transactions to one of a plurality of control/status registers within said respective one of said plurality of CSR banks based on said memory address.

58. A Fibre Channel (FC) controller shareable by a plurality of operating system domains (OSDs) within a load-store architecture, comprising:

a FC port, for coupling to a FC link, configured to obtain a respective port identifier for each of the plurality of OSDs, and configured to accept from said FC link FC frames having D_ID field values matching said port identifiers obtained for the plurality of OSDs;

association logic, coupled to said FC port, for associating each of said FC frames with its respective one of the plurality of OSDs based on said matching port identifier; and

a load-store bus interface, coupled to said association logic, for coupling to a load-store bus, for initiating at least one transaction on said load-store bus to transfer at least a portion of each of said FC frames to one of the plurality of OSDs associated with said matching port identifier of said accepted FC frame, said transaction including an OSD identifier uniquely identifying said respective OSD among the plurality of OSDs.

59. The shareable FC controller as recited in claim 58, wherein said association logic comprises a mapping table for mapping each said port identifier to said OSD identifier uniquely identifying said respective OSD.

60. The shareable FC controller as recited in claim 58, wherein said association logic comprises a mapping table for mapping each said port identifier to a respective queue of outstanding operations on said FC link associated with said respective OSD.

61. The shareable FC controller as recited in claim 58, wherein said association logic comprises a mapping table for mapping each said port identifier to an outstanding I/O request issued to the shareable FC controller by said respective OSD.

62. The shareable FC controller as recited in claim 58, wherein said FC port is configured to drop FC frames on said FC link having a D_ID field value not matching any of said port identifiers obtained for the plurality of OSDs.

63. The shareable FC controller as recited in claim 58, further comprising:

at least one direct memory access controller (DMAC), coupled to said load-store bus interface, for commanding said load-store bus interface to initiate said at least one transaction on said load-store bus to store at least a portion of said FC frame to said respective OSD.

64. The shareable FC controller as recited in claim 63, wherein said at least one DMAC provides to said load-store bus interface said OSD identifier uniquely identifying said respective OSD for inclusion in said transaction.

65. The shareable FC controller as recited in claim 58, wherein said FC port comprises a FC NL_Port for communication on a FC arbitrated loop, wherein said FC NL_Port obtains a respective FC AL_PA for each of the plurality of OSDs, wherein said respective port identifier for each of the plurality of OSDs comprises said respective FC AL_PA.

66. The shareable FC controller as recited in claim 65, wherein said FC NL_Port obtains said respective port identifier for each of the plurality of OSDs by logging into a FC fabric using said respective FC AL_PA and a FC Node_Name associated with the respective OSD.

67. The shareable FC controller as recited in claim 66, wherein the shareable FC controller supplies said unique FC Node_Name associated with said respective OSD.

68. The shareable FC controller as recited in claim 66, wherein the shareable FC controller receives said unique FC Node_Name associated with said respective OSD from said respective OSD.

69. The shareable FC controller as recited in claim 65, wherein said FC NL_Port obtains said respective port identifier for each of the plurality of OSDs by logging into a FC fabric using said respective FC AL_PA and a FC Port_Name associated with the respective OSD.

70. The shareable FC controller as recited in claim 69, wherein the shareable FC controller supplies said unique FC Port_Name associated with said respective OSD.

71. The shareable FC controller as recited in claim 69, wherein the shareable FC controller receives said unique FC Port_Name associated with said respective OSD from said respective OSD.

72. The shareable FC controller as recited in claim 58, wherein said FC port comprises a FC NL_Port for communication on a FC arbitrated loop, wherein said FC NL_Port appears as multiple FC NL_Ports to a FC fabric by performing multiple logins to said FC fabric.

73. The shareable FC controller as recited in claim 58, wherein said FC port comprises a FC NL_Port for communication on a FC arbitrated loop, wherein said FC NL_Port appears as multiple FC end nodes to a FC fabric by performing multiple logins to said FC fabric.

74. The shareable FC controller as recited in claim 58, wherein said FC port comprises a FC NL_Port for communication on a FC arbitrated loop, wherein said FC NL_Port appears as multiple FC end nodes to a FC arbitrated loop by acquiring multiple AL_PAs.

75. The shareable FC controller as recited in claim 58, wherein said FC port obtains said respective port identifier for each of the plurality of OSDs from a FC fabric.

76. The shareable FC controller as recited in claim 58, wherein said FC port obtains said respective port identifier for each of the plurality of OSDs by logging into a FC fabric using a FC Port_Name associated with the respective OSD, wherein the shareable FC controller supplies said unique FC Port_Name associated with said respective OSD.

77. The shareable FC controller as recited in claim 58, wherein said FC port obtains said respective port identifier for each of the plurality of OSDs by logging into a FC fabric using a FC Port_Name associated with the respective OSD, wherein the shareable FC controller receives said unique FC Port_Name associated with said respective OSD from said respective OSD.

78. The shareable FC controller as recited in claim 58, wherein said FC port obtains said respective port identifier for each of the plurality of OSDs by logging into a FC fabric using a FC Node_Name associated with the respective OSD, wherein the shareable FC controller supplies said unique FC Node_Name associated with said respective OSD.

79. The shareable FC controller as recited in claim 58, wherein said FC port obtains said respective port identifier for each of the plurality of OSDs by logging into a FC fabric using a FC Node_Name associated with the respective OSD, wherein the shareable FC controller receives said unique FC Node_Name associated with said respective OSD from said respective OSD.

80. The shareable FC controller as recited in claim 58, further comprising:

a plurality of control/status register (CSR) banks, wherein a respective one of said plurality of CSR banks is used by each of the plurality of OSDs to request the shared FC controller to perform I/O operations with remote FC devices;

wherein said load-store bus interface is configured to receive second load-store transactions from each of the plurality of OSDs, each of said second load-store transactions including a second OSD identifier for identifying one of the plurality of OSDs that initiated said second load-store transaction, wherein said load-store bus interface directs each of said second load-store transactions to said respective one of said plurality of CSR banks based on said second OSD identifier.

81. The shareable FC controller as recited in claim 80, wherein the controller transfers said at least a portion of each of said FC frames to said respective OSD in response to said requests to perform said I/O operations.

82. The shareable FC controller as recited in claim 80, wherein said load-store bus interface is further configured to generate interrupts on said load-store bus to the plurality of OSDs, wherein each of said interrupts include said OSD identifier for identifying the one of the plurality of OSDs that requested the shared FC controller to perform said I/O operation.

83. The shareable FC controller as recited in claim 58, wherein the plurality of OSDs comprises a plurality of processors coupled to a processor bus, wherein said load-store bus interface is configured for coupling to said processor bus.

84. The shareable FC controller as recited in claim 83, wherein each of the plurality of processors provides a processor ID on said processor bus with each of said at least one transaction, wherein said OSD ID comprises said processor ID.

85. The shareable FC controller as recited in claim 58, wherein said FC port comprises a FC N_Port linked to a FC F_Port supporting a FC Multiple N_Port_ID Assignment capability, wherein said FC N_Port obtains from said F_Port a respective FC N_Port identifier for each of the plurality of OSDs.

86. The shareable FC controller as recited in claim 58, wherein the controller is configured to determine whether a FC F_Port to which said FC port is linked supports a FC Multiple N_Port_ID Assignment capability, to obtain a distinct FC N_Port_ID for each of the plurality of OSDs if the FC F_Port supports the FC Multiple N_Port_ID Assignment capability, and to obtain a distinct FC NL_Port_ID for each of the plurality of OSDs if the FC F_Port does not support the FC Multiple N_Port_ID Assignment capability.

87. The shareable FC controller as recited in claim 58, wherein the controller is configured to determine that one of the plurality of OSDs has been reset, and to disclaim the distinct FC port identifier for the one of the plurality of OSDs that has been reset, in response thereto.