Large receive offload for virtual machines

• US 9,742,682 B2
• Filed: 03/11/2014
• Issued: 08/22/2017
• Est. Priority Date: 03/11/2014
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Jain, Jayant, Sengupta, Anirban, Wu, Weiqing
• Primary Examiner(s): Nguyen, Brian D
• Assistant Examiner(s): BHATTI, HASHIM S
• Application Number:
  US14/205,121
• Publication Number:
  US 20150263974A1
• Time in Patent Office: 1,260 Days
• Field of Search: None
• CPC Class Codes: H04L 47/36 by determining packet size, e.g. maximum transfer unit [MTU]
• CPC Class Codes: H04L 47/621 Individual queue per connection or flow, e.g. per VC
• CPC Class Codes: H04L 47/82 Miscellaneous aspects

Abstract

A network interface controller (NIC) that includes a set of receive NIC queues capable of performing large receive offload (LRO) operations by aggregating incoming receive packets is provided. Each NIC queue turns on or off its LRO operation based a set of LRO enabling rules or parameters, whereby only packets that meet the set of rules or parameters will be aggregated in the NIC queue. Each NIC queue is controlled by its own set of LRO enabling rules such that the LRO operations of the different NIC queues can be individually controlled.

Claims

1. A computing device serving as a host machine for a plurality of virtual machines, the computing device comprising:

a physical network interface controller (PNIC), including at least a first queue and a second queue, wherein the first queue is associated with a first set of the plurality of the virtual machines, the second queue is associated with a second set of the plurality of the virtual machines, and the first queue and the second queue are associated with respective first rule and second rule in a set of rules that are specified by an application programming interface (API) for the PNIC; and

wherein the PNIC is for receiving first data packets and second data packets from a network, for enabling packet aggregation at the first queue based on first header content of the first data packets and the first rule, and for disabling packet aggregation at the second queue based on second header content of the second data packets and the second rule; and

a virtualization machine monitor (VMM) for forwarding aggregated first data packets from the first queue to the first set of the plurality of virtual machines and for forwarding non-aggregated second data packets from the second queue to the second set of the plurality of the virtual machines, wherein the VMM uses the API for the PNIC to specify the set of rules.

2. The computing device of claim 1, wherein the VMM uses the API to specify the first rule for the first queue and the second rule for the second queue.

3. The computing device of claim 1, wherein the set of rules comprises the first rule for filtering the first data packets based on the first header content and enabling a packet aggregation operation on the first data packets at the first queue.

4. The computing device of claim 3, wherein the set of rules further comprises the second rule for filtering the second data packets based on the second header content and disabling a packet aggregation operation on the second data packets at the second queue.

5. The computing device of claim 1, wherein the set of rules comprises the first rule to activate a packet aggregation operation on the first data packets destined for a particular virtual machine from the first set of the plurality of the virtual machines.

6. The computing device of claim 1, wherein the set of rules comprises the first rule to enable a packet aggregation operation on the first data packets that belongs to a particular connection session.

7. The computing device of claim 1, wherein the set of rules comprises the first rule to enable packet aggregation operation at the first queue when a set of criteria is met.

8. The computing device of claim 7, wherein the set of criteria comprise a protocol identifier, a source IP address, a destination IP address, a source transport identifier, and a destination transport identifier.

9. A computing device serving as host machine for a plurality of virtual machines, the computing device comprising a set of processing units and a computer readable medium storing a computer program, the computer program executing on one or more processing units, said computer program comprising sets of instructions for:

providing a set of packet aggregation rules to a physical network interface controller (PNIC), the PNIC comprising a plurality of queues for providing data received from a network to the plurality of virtual machines, wherein a first queue from the plurality of queues is associated with a first virtual machine, a second queue from the plurality of queues is associated with a second virtual machine, a first rule in the set of packet aggregation rules is for enabling packet aggregation at the first queue based on first header content of first data packets from the network, and a second rule in the set of packet aggregation rules is for disabling packet aggregation at the second queue based on second header content of second data packets from the network; and

forwarding aggregated first data packets from the first queue in the PNIC to the first virtual machine and non-aggregated second data packets from the second queue in the PNIC to the second virtual machine.

10. The computing device of claim 9 further comprising receiving the set of packet aggregation rules from a controller for the network.

11. The computing device of claim 9, wherein the set of packet aggregation rules comprises the first rule for filtering the first data packets based on the first header content and enabling a packet aggregation operation on the first data packets at the first queue.

12. The computing device of claim 11, wherein the set of packet aggregation rules further comprises the second rule for filtering the second data packets based on the second header content and disabling a packet aggregation operation on the second data packets at the second queue.

13. The computing device of claim 9, wherein the first virtual machine consumes the aggregated first data packets and the second virtual machine forwards the non-aggregated second data packets to another node in the network.

14. The computing device of claim 13, wherein the second virtual machine is a router.

15. The computing device of claim 9, wherein the set of packet aggregation rules comprises the first rule to activate a packet aggregation operation on the first data packets destined for the first virtual machine.

16. The computing device of claim 15, wherein the first rule to enable the packet aggregation operation on the first data packets destined for the first virtual machine, wherein the first header content specifies a destination media access control (MAC) address of the first virtual machine.

17. The computing device of claim 15, wherein the first rule to enable the packet aggregation operation on the first data packets destined for the first virtual machine, wherein the first header content specifies a destination internet protocol (IP) address of the first virtual machine.

18. The computing device of claim 9, wherein the set of packet aggregation rules comprises the first rule to enable a packet aggregation operation on the first data packets that belong to a particular connection session.

19. The computing device of claim 9, wherein the set of packet aggregation rules comprises the first rule to enable a packet aggregation operation at the first queue when a set of criteria is met.

20. The computing device of claim 19, wherein the set of criteria comprise a protocol identifier, a source IP address, a destination IP address, a source transport identifier, and a destination transport identifier.

21. A non-transitory computer readable storage medium storing a program for execution on one or more processors, the program comprising sets of instructions for:

operating a plurality of virtual machines;

setting a plurality of packet aggregation rules for a plurality of queues in a physical network interface controller (PNIC) by using an application programming interface (API) of the PNIC, wherein a first queue from the plurality of queues is associated with a first virtual machine, a second queue from the plurality of queues is associated with a second virtual machine, a first rule of the plurality of packet aggregation rules is for enabling packet aggregation at the first queue based on first header content of first data packets from the network, and a second rule of the plurality of packet aggregation rules is for disabling packet aggregation at the second queue based on second header content of second data packets from the network; and

retrieving aggregated first data packets from the first queue and non-aggregated second data packets from the second queue.

22. The non-transitory computer readable storage medium of claim 21, wherein the program further comprises a set of instruction for operating a plurality of network stacks, wherein a particular network stack for a particular virtual machine receives an interrupt from the API of the PNIC when an aggregated packet is ready to be retrieved from a particular queue.

23. The non-transitory computer readable storage medium of claim 21, wherein the first rule specifies that a particular first data packet is to be aggregated if the particular first data packet is destined for the first virtual machine.

24. The non-transitory computer readable storage medium of claim 21, wherein the first rule specifies that a particular first data packet is to be aggregated if the particular first data packet belongs to a particular connection session.

25. The non-transitory computer readable storage medium of claim 24, wherein the particular connection session is specified by a set of parameters comprising a protocol identifier, a source IP address, a destination IP address, a source transport identifier, and a destination transport identifier.

26. The non-transitory computer readable storage medium of claim 25, wherein at least one of the parameters in the set of parameters is specified as a wildcard.

27. A method for operating a host machine in a virtualized network environment, the host machine hosting a plurality of virtual machines, the method comprising:

identifying a first queue in a physical network interface controller (PNIC) that is buffering first data packets received by the PNIC from a network and destined for a first virtual machine that consumes first data packets;

identifying a second queue in the PNIC that is buffering second data packets received by the PNIC from the network and destined for a second virtual machine that forwards second data packets;

enabling packet aggregation on the first data packets in the first queue based on first header content of the first data packets and a first packet aggregation rule; and

disabling packet aggregation on the second data packets in the second queue based on second header content of the second data packets and of second packet aggregation rule.

28. The method of claim 27, wherein enabling packet aggregation in the first queue comprises setting the first packet aggregation rule for the first queue, wherein disabling packet aggregation in the second queue comprises setting the second packet aggregation rule for the second queue.