ZERO-COPY NETWORK I/O FOR VIRTUAL HOSTS

• US 20070061492A1
• Filed: 08/05/2005
• Published: 03/15/2007
• Est. Priority Date: 08/05/2005
• Status: Active Grant

• Current Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Original Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Inventors: van Riel, Henri
• Granted Patent: US 7,721,299 B2
• Time in Patent Office: Days
• US Class Current: 710/3
• CPC Class Codes: G06F 9/45545 Guest-host, i.e. hypervisor is an application program itself, e.g. VirtualBox
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes
• CPC Class Codes: H04L 63/02 for separating internal from external traffic, e.g. firewalls
• CPC Class Codes: H04L 69/22 Parsing or analysis of headers
• CPC Class Codes: H04L 69/321 Interlayer communication protocols or service data unit [SDU] definitions; Interfaces between layers

Abstract

Techniques for virtualized computer system environments running one or more virtual machines that obviate the extra host operating system (O/S) copying steps required for sending and receiving packets of data over a network connection, thus eliminating major performance problems in virtualized environment. Such techniques include methods for emulating network I/O hardware device acceleration-assist technology providing zero-copy I/O sending and receiving optimizations. Implementation of these techniques require a host O/S to perform actions including, but not limited to: checking of the address translations (ensuring availability and data residency in physical memory), checking whether the destination of a network packet is local (to another virtual machine within the computing system), or across an external network; and, if local, checking whether either the sending destination VM, receiving VM process, or both, supports emulated hardware accelerated-assist on the same physical system. This optimization, in particular, provides a further optimization in that the packet data checksumming operations may be omitted when sending packets between virtual machines in the same physical system.

Claims

1. A method of performing zero copy network Input/Output (I/O) operations in a computer system comprising a host operating system (O/S) executing an application that enables simultaneous operation of one or more guest operating systems (O/Ses) adapted for controlling execution of one or more guest processes, and a network interface device for receiving network packets from or forwarding network packets over a network via said host O/S, said host O/S implementing a method for emulating network I/O hardware device acceleration, said method comprising:

a) receiving a physical memory address location known from the perspective of a guest O/S and corresponding to a guest process operating under control of said guest O/S which is to receive data from a network packet or provide data for assembly in a network packet;

b) performing an address translation to obtain from said physical memory address location known from the perspective of said guest O/S, a corresponding physical memory location accessible by the host O/S; and

,

c) enabling host O/S access to said corresponding physical memory location for one of;

copying data directly thereto from a received network packet and accessing data located thereat for assembly into a network packet.

2. The method as claimed in claim 1, wherein said guest process for receiving data from a network packet or providing data for assembly in a network packet is represented as a virtual memory address located in a virtual memory address space associated with said guest O/S, wherein prior to said receiving, said guest O/S:

performing an address translation to obtain from said virtual memory address, said physical memory address location known from the perspective of a guest O/S; and

,

ensuring residency of said physical memory address location associated with said guest O/S.

3. The method as claimed in claim 1, adapted to perform zero copy network Input/Output (I/O) receive operations, wherein a received network packet is forwarded to said host O/S for processing, said method implemented by said host O/S further comprising:

parsing said received network packet into a header portion and data portions,

determining from said header portion a destination guest process operating under control of said guest O/S for which a data portion of said packet belongs; and

,

forwarding said data portion of said received packet directly from said host O/S to said corresponding physical memory location accessible by the host O/S for direct use by the destination guest process.

4. The method as claimed in claim 3, further comprising:

determining a destination guest O/S kernel buffer address that is to receive said header portion of said received packet;

performing an address translation to obtain from said destination guest O/S kernel buffer address a corresponding physical memory location accessible by the host O/S; and

,

forwarding said header portion directly from said host O/S to said corresponding physical memory location for use by said guest O/S.

5. The method as claimed in claim 4, wherein network packets are communicated to a guest O/S according to a network communications protocol, said method further comprising:

synchronizing network state information associated with said destination guest process with said host O/S.

6. The method as claimed in claim 5, wherein said network communications protocol includes one selected from the group comprising:

TCP, SCTP, SCCP, TCP/IP, IP and IPv6.

7. The method as claimed in claim 4, wherein prior to forwarding a data or header portion of a received network packet, said host O/S further performs:

applying firewall rules specifying one of;

forwarding said header and data portions of said packet, dropping said packet; and

modifying one of said data portion, said header portion or both data and header portions of said network packet prior to forwarding said packet to respective guest destination process and guest O/S.

8. A program storage device tangibly embodying software instructions which are adapted to be executed by a processor to perform a method of performing zero copy network Input/Output (I/O) operations according to claim 1.

9. The method as claimed in claim 1, adapted to perform zero copy network Input/Output (I/O) sending operations, said method comprising:

a guest process providing said data to a physical memory address location known from the perspective of a guest O/S for assembly in a network packet and further, and said guest O/S further providing a header portion at a guest O/S kernel buffer location accessible by said host O/S for assembly in said network packet;

performing an address translation to obtain from said physical memory address location known from the perspective of said guest O/S, a corresponding physical memory location accessible by the host O/S;

said host O/S directly retrieving said data from said physical memory location; and

retrieving said header portion, and

assembling a network packet to be forwarded over said network via said network interface device.

10. A method of performing zero copy network Input/Output (I/O) receive operations in a computer system comprising a host operating system (O/S) executing an application that enables simultaneous operation of one or more guest operating systems adapted for controlling execution of one or more guest processes, and a network interface device for receiving network packets from a network, said network interface device implementing network I/O hardware device acceleration technology of which said host O/S is aware, said method implemented by said host O/S comprising:

a) determining from said guest O/S a guest O/S buffer location that is to receive the received packet;

b) performing an address translation to obtain from said guest O/S buffer location a corresponding physical memory location accessible by the host O/S;

c) informing the network interface device to forward said network packet directly to said physical memory location corresponding to said guest O/S buffer location.

11. the method as claimed in claim 10, wherein said host O/S further instructs said network interface card of certain packet types allowed by firewall rules that may be directly forwarded to the guest O/S.

12. The method as claimed in claim 10, wherein prior to forwarding a network packet to said guest O/S buffer location, said host O/S performing:

receiving a header portion of said received packet and applying firewall rules specifying one of;

forwarding said header and data portions of said packet, dropping said packet;

or modifying one of said data portion, said header portion or both data and header portions of said network packet prior to forwarding said packet.

13. A program storage device tangibly embodying software instructions which are adapted to be executed by a processor to perform a method of performing zero copy network Input/Output (I/O) receive operations according to claim 10.

14. A method of performing zero copy network Input/Output (I/O) receive operations in a computer system comprising a host operating system (O/S) executing an application that enables simultaneous operation of one or more guest operating systems adapted for controlling execution of one or more guest processes, and a network interface device for receiving network packets from a network, said network interface device implementing network I/O hardware device acceleration technology of which said host O/S is aware, said method implemented by said host O/S comprising:

a) receiving, from said guest O/S, a physical memory address location known from the perspective of a guest O/S and corresponding to a guest process operating under control of said guest O/S which is to receive a data portion from a network packet;

b) performing an address translation to obtain from said physical memory address location known from the perspective of said guest O/S, a corresponding physical memory location accessible by the host O/S; and

,

c) informing the network interface device to forward said data portion of said network packet directly to said corresponding physical memory location of said guest process.

15. The method as claimed in claim 14, further comprising:

determining a destination guest O/S kernel buffer address that is to receive a header portion of said received packet;

performing an address translation to obtain from said destination guest O/S kernel buffer address a corresponding physical memory location accessible by the host O/S; and

,

informing the network interface device to forward said header portion of said to said corresponding physical memory location of said destination guest O/S kernel buffer address for use by said guest O/S.

16. The method as claimed in claim 14, wherein said host O/S further instructs said network interface card of certain packet types allowed by firewall rules having header and data portions of which may be directly forwarded to the guest O/S.

17. The method as claimed in claim 14, wherein prior to forwarding a data portion of said network packet to said guest process, said host O/S performing:

receiving a header portion of said received packet and applying firewall rules specifying one of;

forwarding said header and data portions of said packet, dropping said packet;

or modifying one of said data portion, said header portion or, both data and header portions of said network packet prior to forwarding said packet.

18. The method as claimed in claim 14, wherein network packets are communicated to a guest O/S according to a network communications protocol, said method further comprising:

synchronizing network state information associated with said guest process with said host O/S and, between said host O/S and said network interface card.

19. The method as claimed in claim 18, wherein said network communications protocol includes one selected from the group comprising:

TCP, SCTP, SCCP, TCP/IP, IP, IPv6.

20. A program storage device tangibly embodying software instructions which are adapted to be executed by a processor to perform a method of performing zero copy network Input/Output (I/O) receive operations according to claim 14.

21. A method of performing zero copy network Input/Output (I/O) operations in a computer system comprising a host operating system (O/S) executing an application that enables simultaneous operation of one or more guest operating systems (O/Ses) adapted for controlling execution of one or more guest processes, said host O/S adapted for emulating network I/O hardware device acceleration, said method comprising:

a) determining from a header portion of a packet located at a first guest O/S kernel buffer location associated with a first guest O/S and accessible by said host O/S, a destination for a corresponding data portion of said packet, said data portion of said packet located at a physical memory address known from the perspective of a second guest O/S; and

,

b) determining whether said second guest O/S supports said host O/S emulating network I/O hardware device acceleration; and

,

c) if said second guest O/S supports said host O/S emulating network I/O hardware device acceleration, performing address translations to determine physical memory locations that enable host O/S copying of said corresponding data portion of said packet directly to a second guest process and, copying of said header portion directly to a kernel buffer location associated with the second guest O/S.

22. The method as claimed in claim 21, wherein said performing said address translations comprises:

receiving, from said second guest O/S, a physical memory address location known from the perspective of said second guest O/S that corresponds to said second guest process to receive said data portion of said packet; and

,

obtaining from said physical memory address location known from the perspective of said second guest O/S, a corresponding physical memory location accessible by the host O/S.

23. The method as claimed in claim 21, wherein said performing said address translations comprises:

receiving a second guest O/S kernel buffer address that is to receive said header portion of said packet;

obtaining from said second guest O/S kernel buffer address, a corresponding physical memory location accessible by the host O/S.

24. The method as claimed in claim 21, wherein prior to copying a data portion or header portion of a packet to a respective said second guest process and second guest O/S, said host O/S further performing:

applying firewall rules specifying one of;

forwarding said header and data portions of said packet, dropping said packet;

or modifying one of said data portion, said header portion or, both data and header portions of said packet.

25. The method according to claim 21, wherein said first guest O/S providing header and data portions of said packet normally indicates to said host O/S to perform a checksum calculation operation, and said second guest O/S receiving said header and data portions of said packet normally indicates to said host O/S to perform a checksum verification operation prior to receiving said header and data portions of said packet, wherein a need for said host O/S to perform said checksum calculation and verification operations is obviated.

26. A program storage device tangibly embodying software instructions which are adapted to be executed by a processor to perform a method of performing zero copy network Input/Output (I/O) operations according to claim 21.

27. A method of performing zero copy network Input/Output (I/O) operations in a computer system comprising a host operating system (O/S) executing an application that enables simultaneous operation of one or more guest operating systems (O/Ses) adapted for controlling execution of one or more guest processes, said host O/S adapted for emulating network I/O hardware device acceleration, said method comprising:

a) determining from a header portion of a packet located at a first guest O/S kernel buffer location associated with a first guest O/S and accessible by said host O/S, a destination for a corresponding data portion of said packet, said data portion of said packet located at a physical memory address known from the perspective of said host O/S; and

,

b) if said destination is a second guest O/S in said computer system, determining whether said second guest process supports said host O/S emulating network I/O hardware device acceleration; and

,

c) if said second guest O/S does not support said host O/S emulating network I/O hardware device acceleration, performing an address translation to determine a physical memory location that enables host O/S copying of said header portion and corresponding data portion of said packet directly to a kernel buffer location associated with the second guest O/S.

28. The method as claimed in claim 27, wherein said performing said address translations comprises:

receiving a second guest O/S kernel buffer address that is to receive said header portion and corresponding data portion of said packet;

obtaining from said second guest O/S kernel buffer address, a corresponding physical memory location accessible by the host O/S.

29. The method as claimed in claim 27, wherein prior to copying a data portion and header portion of a packet to said second guest O/S, said host O/S further performing:

applying firewall rules specifying one of;

forwarding said header and data portions of said packet, dropping said packet;

or modifying one of said data portion, said header portion or, both data and header portions of said packet.

30. The method according to claim 27, wherein said first guest O/S providing header and data portions of said packet normally indicates to said host O/S to perform a checksum calculation operation, and said second guest O/S receiving said header and data portions of said packet performs a checksum verification operation, wherein a need for said host O/S to perform said checksum calculation operation is obviated.

31. A program storage device tangibly embodying software instructions which are adapted to be executed by a processor to perform a method of performing zero copy network Input/Output (I/O) operations according to claim 27.

32. A method of performing zero copy network Input/Output (I/O) operations in a computer system comprising a host operating system (O/S) executing an application that enables simultaneous operation of one or more guest operating systems (O/Ses) adapted for controlling execution of one or more guest processes, said host O/S adapted for emulating network I/O hardware device acceleration, said method comprising:

a) receiving a packet to be sent from a first guest process for receipt by a second guest process, said packet including a header portion and a data portion;

b) determining from said header portion of said packet, a destination for a corresponding data portion of said packet, said destination located at a physical memory address known from the perspective of a second guest O/S controlling reception of said packet; and

,

c) if said second guest process operating in said computer system supports said host O/S emulating network I/O hardware device acceleration, performing address translations to determine physical memory locations that enable host O/S copying of said corresponding data portion of said packet directly to the second guest process and, copying of said header portion directly to a kernel buffer location associated with a second guest O/S.

33. The method as claimed in claim 32, wherein said performing said address translations comprises:

receiving, from said second guest O/S, a physical memory address location known from the perspective of said second guest O/S that corresponds to said second guest process to receive said data portion of said packet; and

,

obtaining from said physical memory address location known from the perspective of said second guest O/S, a corresponding physical memory location accessible by the host O/S.

34. The method as claimed in claim 32, wherein said performing said address translations comprises:

receiving a second guest O/S kernel buffer address that is to receive said header portion of said packet;

obtaining from said second guest O/S kernel buffer address, a corresponding physical memory location accessible by the host O/S.

35. The method as claimed in claim 32, wherein prior to copying a data portion or header portion of a packet to a respective said second guest process and second guest O/S, said host O/S further performing:

applying firewall rules specifying one of;

forwarding said header and data portions of said packet, dropping said packet;

or modifying one of said data portion, said header portion or, both data and header portions of said packet.

36. The method according to claim 32, wherein said second guest O/S receiving header and data portions of said packet normally indicates to said host O/S to perform a checksum verification operation, wherein a need for said host O/S to perform said checksum verification operation is obviated.

37. A program storage device tangibly embodying software instructions which are adapted to be executed by a processor to perform a method of performing zero copy network Input/Output (I/O) operations according to claim 32.