Multi-root I/O virtualization using separate management facilities of multiple logical partitions

• US 8,359,415 B2
• Filed: 05/05/2008
• Issued: 01/22/2013
• Est. Priority Date: 05/05/2008
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Aaron C., Freimuth, Douglas M., Recio, Renato J., Thurber, Steven M.
• Primary Examiner(s): Tsai, Henry
• Assistant Examiner(s): BARTELS, CHRISTOPHER A.
• Application Number:
  US12/114,989
• Publication Number:
  US 20090276773A1
• Time in Patent Office: 1,723 Days
• Field of Search: None
• US Class Current: 710/62
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

Mechanisms are provided for implementing a multi-root PCI manager (MR-PCIM) in a multi-root I/O virtualization management partition (MR-IMP) to control the shared functionality of an multi-root I/O virtualization (IOV) enabled switch fabric and multi-root IOV enabled I/O adapter (IOA) through the base functions (BF) of the switches and IOAs. A hypervisor provides device-independent facilities to the code running in the I/O Virtualization Management Partition (IMP), Multi-Root (MR)-IMP and client partitions. The MR-IMP may include device specific code without the hypervisor needing to sacrifice its size, robustness, and upgradeability. The hypervisor provides the virtual intermediary functionally for the sharing and control of the switch and IOA'"'"'s control functions.

Claims

1. A method, in a data processing system, for managing input/output virtualization (IOV) management operations, comprising:

creating at least one multiple root I/O virtualization management partition (MR-IMP) in the data processing system, wherein the MR-IMP is a logical partition of the data processing system that is assigned to one or more base functions (BFs) and operates to control multiple-root shared functionality of one or more IOV enabled I/O adaptors via the one or more BFs;

controlling IOV management operations within the data processing system using the at least one MR-IMP, wherein a multiple root I/O virtualization (MR-IOV) manager application of the MR-IMP controls access to the one or more BFs of one or more I/O virtualization enabled I/O adaptors of the data processing system;

creating at least one IOV management partition (IMP) in the data processing system, wherein the IMP is a logical partition of the data processing system that is a separate logical partition from the MR-IMP, and is assigned to the one or more physical functions (PFs) of the one or more IOV enabled I/O adaptors and operates to manage shared resources of the one or more IOV enabled I/O adaptors via the one or more PFs;

creating one or more client partitions on one or more root nodes of the data processing system;

controlling IOV management operations within the data processing system using the at least one IMP, wherein the IMP controls access to one or more physical functions (PFs) of the one or more IOV enabled I/O adaptors, but is not configured to access the BFs of the one or more IOV enabled I/O adaptors; and

associating at least one virtual function (VF) of the one or more IOV enabled I/O adaptors with the one or more client partitions, wherein;

the one or more BFs and one or more PFs comprise physical circuit logic in the one or more IOV enabled I/O adaptors that is configurable based on data stored in an associated configuration space in a memory of the one or more IOV enabled I/O adaptors,

the at least one VF is a function, within the one or more IOV enabled adaptors, that shares one or more physical adaptor resources, is provided in a sharable resource pool of the IOV enabled adaptor, and is a sink for operations from one or more system images of the data processing system, and

the IMP is permitted to perform IOV management operations on the PFs of the one or more IOV enabled I/O adaptors and the one or more client partitions are not permitted to perform IOV management operations on the PFs.

2. The method of claim 1, further comprising:

discovering that an I/O fabric associated with the data processing system is a multiple root (MR) I/O fabric;

booting the MR-IMP in response to discovering that the I/O fabric is an MR I/O fabric such that the MR-IMP executes a MR IOV management application; and

configuring, by the MR IOV management application, a MR configuration space using one or more configuration spaces of the base functions of the one or more IOV enabled I/O adaptors.

3. The method of claim 2, further comprising:

reporting, from a hypervisor to a hardware management console (HMC), that the I/O fabric is a MR I/O fabric; and

issuing a command from the HMC to the MR IOV management application to query the I/O fabric for MR I/O fabric configuration information.

4. The method of claim 3, wherein if the MR I/O fabric comprises a plurality of MR-IMPs, the HMC selects one MR-IMP to operate as a master MR-IMP and all other MR-IMPs operate as secondary to the master MR-IMP.

5. The method of claim 4, further comprising:

determining if the master MR-IMP is operating properly; and

replacing the master MR-IMP with a secondary MR-IMP, which thereafter operates as a master MR-IMP, if the master MR-IMP is determined to not be operating properly.

6. The method of claim 3, wherein configuring a MR configuration space further comprises:

discovering, by the MR IOV management application, a configuration of the MR I/O fabric;

generating MR I/O fabric configuration information based on the discovery of the configuration of the MR I/O fabric, the MR I/O fabric configuration information comprising information regarding the base functions and physical functions of the one or more IOV enabled I/O adaptors; and

reporting the MR I/O fabric configuration information to the HMC.

7. The method of claim 6, further comprising:

presenting the MR I/O fabric configuration information to a user via the HMC;

receiving user input indicating which physical functions to assign to which root nodes of the data processing system; and

configuring the base functions, via the at least one MR-IMP, based on the assignment of physical functions to root nodes specified in the user input.

8. The method of claim 1, further comprising receiving an input from a hardware management console (HMC) directing performance of the I/O virtualization management operations.

9. The method of claim 8, wherein the input from the HMC is a command to initiate a hot-plug operation for adding a new I/O adaptor to the data processing system, and wherein performing the I/O virtualization management operations comprises:

discovering, by the MR IOV management application, the new I/O adaptor and its associated one or more new base functions and one or more new physical functions; and

re-configuring, by the MR IOV management application, the MR configuration space based on the addition of the one or more new base functions and one or more new physical functions.

10. The method of claim 8, wherein the input from the HMC is a command to initiate a hot-plug operation for removing an existing I/O adaptor from the data processing system, and wherein performing the I/O virtualization management operations comprises:

identifying at least one client partition in the one or more client partitions having resources associated with a virtual function of the I/O adaptor to be removed;

removing the virtual function and resources associated with the virtual function from use by the at least one client partition;

removing, from use by the IMP, a physical function and resources associated with the physical function;

removing, from use by the MR-IMP, a base function associated with the I/O adaptor to be removed; and

powering down the I/O adaptor to be removed.

11. A computer program product comprising a non-transitory computer useable medium having a computer readable program, wherein the computer readable program, when executed on a data processing system, causes the data processing system to:

create at least one multiple root I/O virtualization management partition (MR-IMP) in the data processing system, wherein the MR-IMP is a logical partition of the data processing system that is assigned to one or more base functions (BFs) and operates to control multiple-root shared functionality of one or more IOV enabled I/O adaptors via the one or more BFs;

control input/output virtualization (IOV) management operations within the data processing system using the at least one MR-IMP, wherein a multiple root I/O virtualization (MR-IOV) manager application of the MR-IMP controls access to the one or more BFs of one or more I/O virtualization (IOV) enabled I/O adaptors of the data processing system;

create at least one IOV management partition (IMP) in the data processing system, wherein the IMP is a logical partition of the data processing system that is a separate logical partition from the MR-IMP, and is assigned to the one or more physical functions (PFs) of the one or more IOV enabled I/O adaptors and operates to manage shared resources of the one or more IOV enabled I/O adaptors via the one or more PFs;

create one or more client partitions on one or more root nodes of the data processing system;

control IOV management operations within the data processing system using the at least one IMP, wherein the IMP controls access to one or more physical functions (PFs) of the one or more IOV enabled I/O adaptors, but is not configured to access the BFs of the one or more IOV enabled I/O adaptors; and

associate at least one virtual function (VF) of the one or more IOV enabled I/O adaptors with the one or more client partitions, wherein;

the one or more BFs and one or more PFs comprise physical circuit logic in the one or more IOV enabled I/O adaptors that is configurable based on data stored in an associated configuration space in a memory of the one or more IOV enabled I/O adaptors,

the at least one VF is a function, within the one or more IOV enabled adaptors, that shares one or more physical adaptor resources, is provided in a sharable resource pool of the IOV enabled adaptor, and is a sink for operations from one or more system images of the data processing system, and

the IMP is permitted to perform IOV management operations on the PFs of the one or more IOV enabled I/O adaptors and the one or more client partitions are not permitted to perform IOV management operations on the PFs.

12. The computer program product of claim 11, wherein the computer readable program further causes the computing device to:

discover that an I/O fabric associated with the data processing system is a multiple root (MR) I/O fabric;

boot the MR-IMP in response to discovering that the I/O fabric is an MR I/O fabric such that the MR-IMP executes a MR IOV management application; and

configure, by the MR IOV management application, a MR configuration space using one or more configuration spaces of the base functions of the one or more IOV enabled I/O adaptors.

13. The computer program product of claim 12, wherein the computer readable program further causes the computing device to:

report, from a hypervisor to a hardware management console (HMC), that the I/O fabric is a MR I/O fabric; and

issue a command from the HMC to the MR IOV management application to query the I/O fabric for MR I/O fabric configuration information.

14. The computer program product of claim 13, wherein if the MR I/O fabric comprises a plurality of MR-IMPs, the HMC selects one MR-IMP to operate as a master MR-IMP and all other MR-IMPs operate as secondary to the master MR-IMP.

15. The computer program product of claim 14, wherein the computer readable program further causes the computing device to:

determine if the master MR-IMP is operating properly; and

replace the master MR-IMP with a secondary MR-IMP, which thereafter operates as a master MR-IMP, if the master MR-IMP is determined to not be operating properly.

16. The computer program product of claim 13, wherein the computer readable program further causes the computing device to configure a MR configuration space further by:

discovering, by the MR IOV management application, a configuration of the MR I/O fabric;

generating MR I/O fabric configuration information based on the discovery of the configuration of the MR I/O fabric, the MR I/O fabric configuration information comprising information regarding the base functions and physical functions of the one or more IOV enabled I/O adaptors; and

reporting the MR I/O fabric configuration information to the HMC.

17. The computer program product of claim 16, wherein the computer readable program further causes the computing device to:

present the MR I/O fabric configuration information to a user via the HMC;

receive user input indicating which physical functions to assign to which root nodes of the data processing system; and

configure the base functions, via the at least one MR-IMP, based on the assignment of physical functions to root nodes specified in the user input.

18. A data processing system, comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to;

create at least one multiple root I/O virtualization management partition (MR-IMP) in the data processing system, wherein the MR-IMP is a logical partition of the data processing system that is assigned to one or more base functions (BFs) and operates to control multiple-root shared functionality of one or more IOV enabled I/O adaptors via the one or more BFs;

control input/output virtualization (IOV) management operations within the data processing system using the at least one MR-IMP, wherein a multiple root I/O virtualization (MR-IOV) manager application of the MR-IMP controls access to the one or more BFs of one or more IOV enabled I/O adaptors of the data processing system;

create at least one IOV management partition (IMP) in the data processing system, wherein the IMP is a logical partition of the data processing system that is a separate logical partition from the MR-IMP, and is assigned to the one or more physical functions (PFs) of the one or more IOV enabled I/O adaptors and operates to manage shared resources of the one or more IOV enabled I/O adaptors via the one or more PFs;

create one or more client partitions on one or more root nodes of the data processing system;

control IOV management operations within the data processing system using the at least one IMP, wherein the IMP controls access to one or more physical functions (PFs) of the one or more IOV enabled I/O adaptors, but is not configured to access the BFs of the one or more IOV enabled I/O adaptors; and

associate at least one virtual function (VF) of the one or more IOV enabled I/O adaptors with the one or more client partitions, wherein;

the one or more BFs and one or more PFs comprise physical circuit logic in the one or more IOV enabled I/O adaptors that is configurable based on data stored in an associated configuration space in a memory of the one or more IOV enabled I/O adaptors,

the at least one VF is a function, within the one or more IOV enabled adaptors, that shares one or more physical adaptor resources, is provided in a sharable resource pool of the IOV enabled adaptor, and is a sink for operations from one or more system images of the data processing system, and

the IMP is permitted to perform IOV management operations on the PFs of the one or more IOV enabled I/O adaptors and the one or more client partitions are not permitted to perform I/O virtualization management operations on the PFs.