Pass-through network interface controller configured to support latency sensitive virtual machines

• US 9,552,216 B2
• Filed: 08/25/2014
• Issued: 01/24/2017
• Est. Priority Date: 08/26/2013
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Zheng, Haoqiang, Singaravelu, Lenin, Agarwal, Shilpi, Hecht, Daniel Michael, Smith, Garrett
• Primary Examiner(s): KESSLER, GREGORY AARON
• Application Number:
  US14/468,138
• Publication Number:
  US 20150058847A1
• Time in Patent Office: 883 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 13/24 using interrupt G06F13/32 takes precedence
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45575 Starting, stopping, suspending or resuming virtual machine instances
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 2209/5017 Task decomposition
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/4812 by interrupt, e.g. masked
• CPC Class Codes: G06F 9/4887 involving deadlines, e.g. rate based, periodic
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/5033 considering data affinity
• CPC Class Codes: G06F 9/5088 involving task migration
• CPC Class Codes: H04L 43/0894 Packet rate
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 47/56 implementing delay-aware scheduling
• CPC Class Codes: H04L 47/801 Real time traffic

Abstract

A host computer has a plurality of virtual machines executing therein under the control of a hypervisor, where the host also includes a physical network interface controller (NIC). An interrupt controller detects an interrupt generated by the physical NIC, where the interrupt corresponds to a virtual machine. If the virtual machine has exclusive affinity to one or more physical central processing units (CPUs), then the interrupt is forwarded to the virtual machine. If the virtual machine does not have exclusive affinity, then a process in the hypervisor is invoked to forward the interrupt to the virtual machine.

Claims

1. In a host computer having a plurality of virtual machines executing therein under the control of a hypervisor, the host including a physical network interface controller (NIC), a method of processing interrupts from the physical NIC, the method being performed by the host computer and comprising:

detecting an interrupt generated by the physical NIC upon reception of a packet by the physical NIC;

determining a virtual machine to which the interrupt corresponds;

determining, by the hypervisor, whether the virtual machine has exclusive affinity to a physical central processing unit (CPU);

if the virtual machine has exclusive affinity to a physical CPU, then forwarding the interrupt to the virtual machine; and

if the virtual machine does not have exclusive affinity to a physical CPU, then invoking a process in the hypervisor to forward the interrupt to the virtual machine.

2. The method of claim 1, wherein the physical NIC is configured to operate in a passthrough mode that enables transmission of packets between the physical NIC and a virtual machine, wherein the packets are not forwarded between the physical NIC and the virtual machine by the hypervisor.

3. The method of claim 2, wherein the physical NIC includes a plurality of ports, each port configured to receive packets corresponding to one of the virtual machines.

4. The method of claim 1, further comprising:

after determining, by the hypervisor, whether the virtual machine has exclusive affinity to a physical CPU, wherein the determining comprises;

reading an exclusive affinity indicator for the virtual machine; and

determining whether the exclusive affinity indicator is a predetermined value,

programming an interrupt controller based on the determining.

5. The method of claim 4, wherein, if the virtual machine has exclusive affinity, then the forwarding of the interrupt comprises:

determining, by the interrupt controller, a first physical CPU, wherein a virtual CPU of the virtual machine has exclusive affinity to the first physical CPU; and

posting, by the interrupt controller, the interrupt to the first physical CPU.

6. The method of claim 4, further comprising, if the virtual machine does not have exclusive affinity, then posting the interrupt, by the interrupt controller, to a second physical CPU that is different from the first physical CPU.

7. The method of claim 6, wherein no virtual CPU has exclusive affinity to the second physical CPU.

8. A non-transitory computer-readable medium comprising instructions executable by a host computer, the host computer having a plurality of virtual machines executing therein under the control of a hypervisor, and the host including a physical network interface controller (NIC), where the instructions, when executed, cause the host computer to perform a method of processing interrupts from the physical NIC, the method comprising:

detecting an interrupt generated by the physical NIC upon reception of a packet by the physical NIC;

determining a virtual machine to which the interrupt corresponds;

determining, by the hypervisor, whether the virtual machine has exclusive affinity to a physical central processing unit (CPU);

if the virtual machine has exclusive affinity to a physical CPU, then forwarding the interrupt to the virtual machine; and

if the virtual machine does not have exclusive affinity to a physical CPU, then invoking a process in the hypervisor to forward the interrupt to the virtual machine.

9. The computer-readable medium of claim 8, wherein the physical NIC is configured to operate in a passthrough mode that enables transmission of packets between the physical NIC and a virtual machine, wherein the packets are not forwarded between the physical NIC and the virtual machine by the hypervisor.

10. The computer-readable medium of claim 9, wherein the physical NIC includes a plurality of ports, each port configured to receive packets corresponding to one of the virtual machines.

11. The computer-readable medium of claim 8, wherein the method further comprises:

after determining, by the hypervisor, whether the virtual machine has exclusive affinity to a physical CPU, wherein the determining comprises;

reading an exclusive affinity indicator for the virtual machine; and

determining whether the exclusive affinity indicator is a predetermined value,

programming an interrupt controller based on the determining.

12. The computer-readable medium of claim 11, wherein, if the virtual machine has exclusive affinity, then said forwarding of the interrupt comprises:

determining, by the interrupt controller, a first physical CPU, wherein a virtual CPU of the virtual machine has exclusive affinity to the first physical CPU; and

posting, by the interrupt controller, the interrupt to the first physical CPU.

13. The computer-readable medium of claim 11, further comprising, if the virtual machine does not have exclusive affinity, then posting the interrupt, by the interrupt controller, to a second physical CPU that is different from the first physical CPU.

14. The computer-readable medium of claim 13, wherein no virtual CPU has exclusive affinity to the second physical CPU.

15. A virtualized computing system, comprising:

a host computer, the host computer having a plurality of virtual machines executing therein under the control of a hypervisor;

a physical network interface controller (NIC); and

an interrupt controller, wherein the hypervisor and interrupt controller are configured to perform a method of processing interrupts from the physical NIC, the method comprising;

detecting an interrupt generated by the physical NIC upon reception of a packet by the physical NIC;

determining a virtual machine to which the interrupt corresponds;

determining, by the hypervisor, whether the virtual machine has exclusive affinity to a physical central processing unit (CPU);

if the virtual machine has exclusive affinity to a physical CPU, then forwarding the interrupt to the virtual machine; and

if the virtual machine does not have exclusive affinity, then invoking a process in the hypervisor to forward the interrupt to the virtual machine.

16. The system of claim 15, wherein the physical NIC is configured to operate in a passthrough mode that enables transmission of packets between the physical NIC and a virtual machine, wherein the packets are not forwarded between the physical NIC and the virtual machine by the hypervisor.

17. The system of claim 16, wherein the physical NIC includes a plurality of ports, each port configured to receive packets corresponding to one of the virtual machines.

18. The system of claim 15, wherein the method further comprises:

after determining, by the hypervisor, whether the virtual machine has exclusive affinity to a physical CPU, wherein the determining comprises;

reading an exclusive affinity indicator for the virtual machine; and

determining whether the exclusive affinity indicator is a predetermined value,

programming an interrupt controller based on the determining.

19. The system of claim 18, wherein, if the virtual machine has exclusive affinity, then said forwarding of the interrupt comprises:

determining, by the interrupt controller, a first physical CPU to which a virtual CPU of the virtual machine has exclusive affinity; and

posting, by the interrupt controller, the interrupt to the first physical CPU.

20. The system of claim 18, wherein the method further comprises, if the virtual machine does not have exclusive affinity, then posting, by the interrupt controller, the interrupt to a second physical CPU that is different from the first physical CPU.