Remote shared server peripherals over an Ethernet network for resource virtualization

• US 9,973,446 B2
• Filed: 08/20/2009
• Issued: 05/15/2018
• Est. Priority Date: 08/20/2009
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Cohen, Ariel, Krishnamurthi, Ashok, Krishnamurthy, Viswanath, Salzmann, Frank, Allison, David S., Tang, Cheng
• Primary Examiner(s): Joshi, Suraj
• Assistant Examiner(s): Lin, Sherman
• Application Number:
  US12/544,744
• Publication Number:
  US 20130138836A1
• Time in Patent Office: 3,190 Days
• Field of Search: 709250, 709226
• CPC Class Codes: H04L 49/351 for local area network [LAN], e.g. Ethernet switches

Abstract

Provided is a novel approach for connecting servers to peripherals, such as NICs, HBAs, and SAS/SATA controllers. Also provided are systems for arranging peripherals within one or more I/O directors, which are connected to the servers over an Ethernet network. Such arrangement allows sharing the same resource among multiple servers.

Claims

1. A system for virtualizing I/O resources, comprising:

a first Ethernet network;

a second Ethernet network;

a plurality of servers connected with the first Ethernet network, the plurality of servers comprising one or more processors and memory; and

an I/O director connected with the first Ethernet network and the second Ethernet network, the I/O director providing one or more common, shared Network Interface Controllers (NICs) and Host Bus Adapters (HBAs) to the plurality of servers, the I/O director comprising;

a plurality of Ethernet network interfaces connected to the plurality of servers over the first Ethernet network, the plurality of servers associated with a plurality of virtual network interface cards executed by the I/O director and shared by the plurality of servers;

a first plurality of output ports connected to a first plurality of external devices over the second Ethernet network;

a network processor operable to map the plurality of virtual network interface cards with the first plurality of output ports, wherein the network processor is further operable to maintain a plurality of media access control (MAC) addresses for the plurality of servers;

a management module communicating with each of the plurality of servers using a management protocol over the first Ethernet network, wherein the servers;

establish communication with the I/O director using the management protocol; and

discover remote I/O resources on the second Ethernet network using the management protocol;

a directory service communicating with each of the plurality of servers using the management protocol over the first Ethernet network, wherein;

the plurality of servers each comprise virtual network interface card drivers;

the plurality of servers discover the remote I/O resources of the second Ethernet network by transmitting multicast discovery packets according to the management protocol from the virtual network interface card drivers to the directory service, the multicast discovery packets comprising server identifiers and corresponding server attributes; and

the directory service processes the multicast discovery packets, creates server objects based at least in part on the multicast discovery packets, determines whether the I/O director retains corresponding server profiles, and responds to the plurality of servers with unicast packets containing first information facilitating establishment of connections with a session manager of the I/O director;

a virtual network interface card module comprising learning logic configured to establish a mapping of the plurality of virtual network interface cards to the one or more MAC addresses, at least in part by populating a mapping table as data packets arrive from one or more servers of the plurality of servers; and

the session manager of the I/O director, in response to one server of the plurality of servers establishing a connection with the session manager using the first information of the unicast packets, uploads to the one server of the plurality of servers second information on one or more of the virtual network interface cards and/or virtual host bus adapters allocated to the one server.

2. The system of claim 1, wherein mapping of the plurality of virtual network interface cards with the first plurality of output ports is performed dynamically.

3. The system of claim 1, further comprising a second plurality of output ports connected to a second plurality of external devices over a first Fibre Channel network.

4. The system of claim 3, wherein the plurality of servers is associated with a plurality of virtual host bus adapters and wherein the network processor is operable to map the plurality of virtual host bus adapters with the second plurality of output ports.

5. The system of claim 4, wherein mapping of the plurality of virtual host bus adapters with the second plurality of output ports is performed dynamically.

6. The system of claim 4, wherein the plurality of servers comprises virtual host bus adapter drivers.

7. The system of claim 6, wherein the virtual host bus adapter drivers are hardware implemented.

8. The system of claim 4, wherein at least one of the plurality of servers is virtualized to provide one or more virtual machines and wherein the one or more virtual machines are associated with the plurality of virtual host bus adapters.

9. The system of claim 4, further comprising a target channel adapter, wherein the target channel adapter is configured to remove transport protocol headers from data packets received from the plurality of servers, to assign internal headers to the data packets, and to send the data packets with the internal headers to the virtual host bus adapters.

10. The system of claim 4, further comprising a virtual host bus adapter module, wherein the virtual host bus adapter module includes the virtual host bus adapters, a buffer manager, and a queue manager.

11. The system of claim 10, wherein the virtual host bus adapter module is implemented in one or more elements selected from a group consisting of a field-programmable gate array (FPGA) and an application-specific integrated circuit (ASIC).

12. The system of claim 4, wherein Quality of Service is provided by the system for each virtual host bus adapter of the plurality of virtual host bus adapters.

13. The system of claim 1, further comprising a second plurality of output ports connected to a second plurality of external devices, wherein the plurality of servers is associated with a plurality of adapters selected from a group consisting of serial ATA (SATA) adapters, serial attached SCSI (SAS) adapters, RAID adapters, Fibre Channel host bus adapters, and iSCSI host bus adapters, and wherein the network processor is operable to map the plurality of adapters with the second plurality of output ports.

14. The system of claim 1, wherein the first Ethernet network is lossless Ethernet.

15. The system of claim 1, wherein the first Ethernet network is lossy Ethernet.

16. The system of claim 1, wherein the virtual network interface card drivers are hardware implemented.

17. The system of claim 1, further comprising an Ethernet switch providing the plurality of Ethernet network interfaces.

18. The system of claim 1, wherein the plurality of servers communicate with the I/O director using Internet Wide Area RDMA Protocol (iWARP).

19. The system of claim 1, wherein the plurality of servers communicate with the I/O director using Reliable Connection (RC) protocol.

20. The system of claim 1, wherein at least one of the plurality of servers is virtualized to provide one or more virtual machines and wherein the one or more virtual machines are associated with the plurality of virtual network interface cards.

21. The system of claim 20, wherein Quality of Service is provided by the system for each of the one or more virtual machines.

22. The system of claim 1, wherein the I/O director provides communication among servers within the plurality of servers.

23. The system of claim 22, wherein the communication among the servers within the plurality of servers is performed using one or more protocols selected from a group consisting of Sockets Direct Protocol (SDP), Reliable Datagram Sockets (RDS), and Message Passing Interface (MPI).

24. The system of claim 1, further comprising a target channel adapter, wherein the target channel adapter is configured to remove transport protocol headers from data packets received from the plurality of servers, to assign internal headers to the data packets, and to send the data packets with the internal headers to the virtual network interface cards.

25. The system of claim 1, wherein the virtual network interface card module includes the virtual network interface cards, a buffer manager, a queue manager, classifier logic, and logic for switching between the virtual network interface cards.

26. The system of claim 25, wherein the learning logic is configured to:

examine data packets received from one or more servers of the plurality of servers on one or more virtual network interface cards of the plurality of virtual network interface cards,

identify from the data packets one or more MAC addresses of the plurality of MAC addresses, and

map the identified one or more MAC addresses to the one or more virtual network interface cards.

27. The system of claim 26, wherein the system is configured to direct data packets received from the first plurality of external devices to the one or more virtual network interface cards based on the mapping of the one or more MAC addresses and the one or more virtual network interface cards.

28. The system of claim 27, wherein the direction of the data packets received from the first plurality of external devices to the one or more virtual network interface cards is performed by the logic for switching between the virtual network interface cards.

29. The system of claim 25, wherein the virtual network interface card module is implemented in one or more elements selected from the group consisting of a field-programmable gate array (FPGA) and an application-specific integrated circuit (ASIC).

30. The system of claim 25, wherein the queue manager is configured to enforce Quality of Service to the plurality of servers over the first Ethernet network.

31. The system of claim 1, wherein the system is configured to provide Quality of Service to the plurality of servers over the first Ethernet network.

32. The system of claim 31, wherein the Quality of Service comprises a committed data rate guarantee and a peak data rate guarantee.

33. The system of claim 31, wherein the plurality of servers comprises a first server and a second server and wherein a first level of the Quality of Service provided to the first server is different than a second level of the Quality of Service provided to the second server.

34. The system of claim 31, wherein the Quality of Service is provided for each virtual network interface card of the plurality of virtual network interface cards.