Input/Output (I/O) Virtualization System

• US 20110029695A1
• Filed: 03/31/2010
• Published: 02/03/2011
• Est. Priority Date: 04/01/2009
• Status: Active Grant

• Current Assignee: SanDisk Technologies LLC (Western Digital Corporation)
• Original Assignee: Fusion-IO Incorporated (Western Digital Corporation)
• Inventors: Malwankar, Kiron, Kishore, Karagada R., Kirkpatrick, Peter E.
• Granted Patent: US 8,412,860 B2
• Time in Patent Office: Days
• US Class Current: 710/9
• CPC Class Codes: G06F 13/4022 using switching circuits, e.g. switching matrix, connection or expansion network G06F13/4009 takes precedence
• CPC Class Codes: G06F 2213/0058 Bus-related hardware virtualisation

Abstract

An Input/Output (IO) Virtualization (IOV) system couples or connects multiple host computers and IO devices to a managed transport fabric to provide IO virtualization. The host computers may run any operating system to provide a virtualized environment for guest operating systems. The host interface to the IOV system is PCI-Express (PCIe). The IO devices are PCIe based to provide maximum compatibility with industry standard devices, but are not so limited. The IOV system comprises a management central processor unit (MCPU) coupled to transport fabric. The IOV system comprises device interfaces coupled to the transport fabric and to independent input/output (IO) devices. Each device interface couples to the IO device of the independent IO devices. The IOV system comprises host interfaces coupled to the transport fabric. Each host interface couples to a host computer of the independent host computers and exposes functions of the independent IO devices to the host computer.

Claims

1. A system comprising:

a management central processor unit (MCPU) coupled to transport fabric;

a plurality of device interfaces coupled to the transport fabric and to a plurality of independent input/output (IO) devices, wherein each device interface couples to the IO device of the plurality of independent IO devices; and

a plurality of host interfaces coupled to the transport fabric, wherein each host interface couples to a host computer of a plurality of independent host computers and exposes functions of the plurality of independent IO devices to the host computer.

2. The system of claim 1, wherein the plurality of independent IO devices are coupled to the transport fabric using a Peripheral Component Interconnect Express (PCIe) interface.

3. The system of claim 1, wherein the MCPU is connected directly to the transport fabric.

4. The system of claim 1, wherein the MCPU is coupled to the transport fabric with a management interface.

5. The system of claim 1, wherein the MCPU initializes the transport fabric, the device interfaces, and the host interfaces.

6. The system of claim 1, wherein the MCPU initializes the device interfaces and the host interfaces.

7. The system of claim 1, wherein the device interfaces and the host interfaces comprise an encapsulation protocol for Peripheral Component Interconnect Express (PCIe) traffic to traverse the transport fabric.

8. The system of claim 1, wherein the device interfaces and the host interfaces comprise a decapsulation protocol for PCIe traffic to traverse the transport fabric.

9. The system of claim 1, wherein the MCPU discovers and enumerates the plurality of independent IO devices.

10. The system of claim 9, wherein at least one of the plurality of independent IO devices is hot-plugged during system operation.

11. The system of claim 9, wherein the MCPU initializes the plurality of independent IO devices.

12. The system of claim 11, wherein the MCPU runs a privileged driver for each IO device of the plurality of independent IO devices, wherein the privileged driver controls the IO device.

13. The system of claim 1, wherein the plurality of independent IO devices include an IO device with one endpoint function (EPF), wherein the MCPU assigns the IO device with one EPF to a single host computer, wherein the single host computer has control over the EPF.

14. The system of claim 1, wherein the plurality of independent IO devices include an IO device with a plurality of endpoint functions (EPFs), wherein the IO device with the plurality of EPFs is shared across a set of host computers of the plurality of independent host computers.

15. The system of claim 14, wherein a privileged driver of the MCPU controls a physical EPF of the IO device with the plurality of EPFs.

16. The system of claim 14, wherein virtual EPFs assigned to host computers of the set of host computers are controlled by non-privileged drivers of the assigned host computers.

17. The system of claim 14, wherein non-privileged physical EPFs assigned to host computers of the set of host computers are controlled by non-privileged drivers of the assigned host computers.

18. The system of claim 1, wherein each host interface comprises a proxy including configuration data.

19. The system of claim 18, wherein the configuration data comprises a bus-device-function (BDF) number in a domain of the host computer.

20. The system of claim 18, wherein the configuration data comprises a memory range identifier.

21. The system of claim 18, wherein the configuration data comprises an endpoint function (EPF) identifier of an EPF of at least one IO device reserved for use by the corresponding host computer.

22. The system of claim 18, wherein the configuration data comprises a destination address of a device interface corresponding to an endpoint function (EPF) of at least one IO device reserved for use by the corresponding host computer.

23. The system of claim 18, wherein configuration accesses to the proxy are directed to the MCPU, wherein the MCPU controls configuration of the EPF and updates the host computer with current configuration status.

24. The system of claim 18, wherein IO accesses to the proxy are sent directly to the IO device via the transport fabric.

25. The system of claim 1, wherein the host interface comprises a PCIe device enabling communications between the plurality of independent host computers.

26. The system of claim 25, wherein the communications comprise memory-oriented communications and connection-oriented communications.

27. The system of claim 1, wherein each device interface couples to the IO device of the plurality of independent IO devices via a native device driver of the IO device.

28. A system comprising:

a plurality of device interfaces coupled to transport fabric and to a plurality of independent input/output (IO) devices, wherein each device interface couples to the IO device of the plurality of independent IO devices; and

a plurality of host interfaces coupled to the transport fabric, wherein each host interface couples to a host computer of a plurality of independent host computers and exposes functions of the plurality of independent IO devices to the host computer.

29. The system of claim 28, wherein the plurality of independent IO devices are coupled to the transport fabric using a Peripheral Component Interconnect Express (PCIe) interface.

30. The system of claim 28, wherein the device interfaces and the host interfaces comprise an encapsulation protocol for Peripheral Component Interconnect Express (PCIe) traffic to traverse the transport fabric.

31. The system of claim 28, wherein the device interfaces and the host interfaces comprise a decapsulation protocol for PCIe traffic to traverse the transport fabric.

32. The system of claim 28, comprising a privileged driver for each IO device of the plurality of independent IO devices, wherein the privileged driver controls the IO device.

33. The system of claim 28, wherein the plurality of independent IO devices include an IO device with one endpoint function (EPF), wherein the IO device with one EPF is used by a single host computer that controls the EPF.

34. The system of claim 28, wherein the plurality of independent IO devices include an IO device with a plurality of endpoint functions (EPFs), wherein the IO device with the plurality of EPFs is shared across a set of host computers of the plurality of independent host computers.

35. The system of claim 34, wherein a privileged driver controls a physical EPF of the IO device with the plurality of EPFs.

36. The system of claim 34, wherein virtual EPFs assigned to host computers of the set of host computers are controlled by non-privileged drivers of the assigned host computers.

37. The system of claim 34, wherein non-privileged physical EPFs assigned to host computers of the set of host computers are controlled by non-privileged drivers of the assigned host computers.

38. The system of claim 28, wherein each host interface comprises a proxy including configuration data.

39. The system of claim 38, wherein the configuration data comprises a bus-device-function (BDF) number in a domain of the host computer.

40. The system of claim 38, wherein the configuration data comprises a memory range identifier.

41. The system of claim 38, wherein the configuration data comprises an endpoint function (EPF) identifier of an EPF of at least one IO device reserved for use by the corresponding host computer.

42. The system of claim 38, wherein the configuration data comprises a destination address of a device interface corresponding to an endpoint function (EPF) of at least one IO device reserved for use by the corresponding host computer.

43. A method comprising:

coupling a plurality of device interfaces to transport fabric and to a plurality of independent input/output (IO) devices, the coupling including each device interface coupling to the IO device of the plurality of independent IO devices;

coupling a plurality of host interfaces to the transport fabric, the coupling including each host interface coupling to a host computer of a plurality of independent host computers; and

exposing functions of the plurality of independent IO devices to the plurality of independent host computers via the plurality of device interfaces and the plurality of host interfaces.

44. The method of claim 43, wherein the coupling of the plurality of independent IO devices to the transport fabric comprises a Peripheral Component Interconnect Express (PCIe) interface.

45. The method of claim 43, comprising initializing the transport fabric, the device interfaces, and the host interfaces.

46. The method of claim 43, comprising initializing the device interfaces and the host interfaces.

47. The method of claim 43, comprising an encapsulation protocol for Peripheral Component Interconnect Express (PCIe) traffic to traverse the transport fabric, wherein the device interfaces and the host interfaces include the encapsulation protocol.

48. The method of claim 43, comprising a decapsulation protocol for PCIe traffic to traverse the transport fabric, wherein the device interfaces and the host interfaces include the decapsulation protocol.

49. The method of claim 43, comprising discovering the plurality of independent IO devices.

50. The method of claim 43, comprising enumerating the plurality of independent IO devices.

51. The method of claim 43, comprising initializing the plurality of independent IO devices.

52. The method of claim 43, comprising running a privileged driver for each IO device of the plurality of independent IO devices, the privileged driver controlling the IO device.

53. The method of claim 43, comprising assigning an IO device having one endpoint function (EPF) to a single host computer, the single host computer controlling the EPF, the plurality of independent IO devices including the IO device with one EPF.

54. The method of claim 43, comprising sharing an IO device having a plurality of endpoint functions (EPFs) across a set of host computers of the plurality of independent host computers, the plurality of independent IO devices including the IO device with the plurality of EPFs.

55. The method of claim 54, comprising, using a privileged driver, controlling a physical EPF of the IO device with the plurality of EPFs.

56. The method of claim 54, comprising, using non-privileged drivers of the assigned host computers, controlling virtual EPFs assigned to host computers of the set of host computers.

57. The method of claim 54, comprising, using non-privileged drivers of the assigned host computer, controlling non-privileged physical EPFs assigned to host computers of the set of host computers.

58. The method of claim 43, comprising providing a proxy corresponding to each host computer, the proxy including configuration data.

59. The method of claim 58, comprising controlling configuration of at least one endpoint function (EPF) of at least one IO device reserved for use by the corresponding host computer and updating the host computer.

60. The method of claim 58, wherein the configuration data comprises a bus-device-function (BDF) number in a domain of the host computer.

61. The method of claim 58, wherein the configuration data comprises a memory range identifier.

62. The method of claim 58, wherein the configuration data comprises an endpoint function (EPF) identifier of an EPF of at least one IO device reserved for use by the corresponding host computer.

63. The method of claim 58, wherein the configuration data comprises a destination address of a device interface corresponding to an endpoint function (EPF) of at least one IO device reserved for use by the corresponding host computer.