Method and apparatus for implementing and managing distributed virtual switches in several hosts and physical forwarding elements

• US 8,966,035 B2
• Filed: 04/01/2010
• Issued: 02/24/2015
• Est. Priority Date: 04/01/2009
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Casado, Martin, Ingram, Paul, Amidon, Keith Eric, Balland, Peter J. III, Koponen, Teemu, Pfaff, Benjamin Levy, Pettit, Justin, Gross, Jesse E. IV, Wendlandt, Daniel J.
• Primary Examiner(s): Patel, Haresh N
• Application Number:
  US12/753,044
• Publication Number:
  US 20100257263A1
• Time in Patent Office: 1,790 Days
• Field of Search: 709217-228
• US Class Current: 709/223
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 12/4633 Interconnection of networks using encapsulation techniques, e.g. tunneling
• CPC Class Codes: H04L 12/4641 Virtual LANs, VLANs, e.g. virtual private networks [VPN] LAN interconnection over a bridge based backbone H04L12/462; encapsulation techniques H04L12/4633; routing of packets H04L45/00; packet switches H04L49/00; virtual private networks for security H04L63/0272
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 41/0896 Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities flow or congestion control using dynamic resource allocation, e.g. in-call renegotiation, H04L47/76
• CPC Class Codes: H04L 41/0897 by horizontal or vertical scaling of resources, or by migrating entities, e.g. virtual resources or entities
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 41/122 of virtualised topologies, e.g. software-defined networks [SDN] or network function virtualisation [NFV]
• CPC Class Codes: H04L 45/54 Organization of routing tables
• CPC Class Codes: H04L 45/66 Layer 2 routing, e.g. in Ethernet based MAN's
• CPC Class Codes: H04L 49/00 Packet switching elements
• CPC Class Codes: H04L 49/15 Interconnection of switching modules
• CPC Class Codes: H04L 49/25 Routing or path finding in a switch fabric
• CPC Class Codes: H04L 49/70 Virtual switches
• CPC Class Codes: H04L 61/256 NAT traversal

Abstract

In general, the present invention relates to a virtual platform in which one or more distributed virtual switches can be created for use in virtual networking. According to some aspects, the distributed virtual switch according to the invention provides the ability for virtual and physical machines to more readily, securely, and efficiently communicate with each other even if they are not located on the same physical host and/or in the same subnet or VLAN. According other aspects, the distributed virtual switches of the invention can support integration with traditional IP networks and support sophisticated IP technologies including NAT functionality, stateful firewalling, and notifying the IP network of workload migration. According to further aspects, the virtual platform of the invention creates one or more distributed virtual switches which may be allocated to a tenant, application, or other entity requiring isolation and/or independent configuration state. According to still further aspects, the virtual platform of the invention manages and/or uses VLAN or tunnels (e.g, GRE) to create a distributed virtual switch for a network while working with existing switches and routers in the network. The present invention finds utility in both enterprise networks, datacenters and other facilities.

Claims

1. For a network hypervisor, a method of managing a network comprising physical forwarding elements, the method comprising:

identifying a set of virtual machines communicatively coupled to a set of physical forwarding elements, at least one of the physical forwarding elements in the set of physical forwarding elements also coupled to a virtual machine that is not in the set of virtual machines;

generating a set of flow entries for the set of physical forwarding elements to use to implement a logical forwarding element that is to handle communications between the set of virtual machines, wherein the logical forwarding element maintains isolation between the set of virtual machines and other virtual machines that are coupled to the set of physical forwarding elements but are not in the set of virtual machines; and

sending the generated set of flow entries to the set of physical forwarding elements, wherein a particular physical forwarding element in the set of forwarding elements is for using the set of flow entries to (i) make a set of logical forwarding decisions to identify a logical egress port of the logical forwarding element for a packet received from a virtual machine in the set of virtual machines and (ii) map the identified logical egress port to a physical port of the particular forwarding element through which to send the packet.

2. The method of claim 1, wherein the physical forwarding elements comprise one or more virtual switches.

3. The method of claim 1, wherein the physical forwarding elements comprise one or more physical switches.

4. The method of claim 1, wherein said other virtual machines that are not in the set of virtual machines communicate with each other via another logical forwarding element that is implemented by the set of physical forwarding elements.

5. The method of claim 1, wherein the set of virtual machines are associated with a particular data center tenant, wherein said other virtual machines that are not in the set of virtual machines are associated with another data center tenant.

6. The method of claim 1, wherein said sending the generated set of flow entries comprises:

sending a set of flow entries to a logical flow table of the particular physical forwarding element that specifies logical forwarding decisions that would be made by the particular physical forwarding element to implement the distributed virtual switch;

sending a set of flow entries to a physical flow table of the particular physical forwarding element that specifies physical forwarding decisions that would be made by the particular physical forwarding element; and

determining a mapping between the logical flow table and the physical flow table, wherein the particular physical forwarding element forwards packets using the logical flow table and the physical flow table and the determined mapping.

7. The method of claim 1, wherein the particular physical forwarding element is a first physical forwarding element, wherein mapping the identified logical egress port to the physical port of the first physical forwarding element comprises:

mapping the identified logical egress port to a physical port of a second physical forwarding element of the set of physical forwarding elements; and

making a set of physical forwarding decisions to identify the physical port of the first physical forwarding element through which to send the packet toward the second physical forwarding element.

8. The method of claim 1, wherein the logical forwarding element maintains at least one of counters, port speeds and bisectional bandwidth for logical ports of the logical forwarding element.

9. The method of claim 1, wherein at least two virtual machines of the set of virtual machines are in different subnets.

10. The method of claim 1, further comprising providing tunnels through at least two of the physical forwarding elements.

11. The method of claim 10, wherein the tunnels comprise GRE tunnels.

12. A networking site comprising a plurality of physical forwarding elements, the networking site comprising:

a set of hosts for hosting a set of virtual machines communicatively coupled to a set of physical forwarding elements, at least one of the physical forwarding elements in the set of physical forwarding elements also coupled to a virtual machine that is not in the set of virtual machines; and

a network hypervisor for generating a set of flow entries for the set of physical forwarding elements to use to implement a distributed virtual switch to handle communications between the virtual machines of the set of virtual machines and sending the generated set of flow entries to the set of physical forwarding elements, wherein the distributed virtual switch maintains isolation between the set of virtual machines and other virtual machines that are coupled to the set of physical forwarding elements but are not in the set of virtual machines, wherein a particular physical forwarding element in the set of forwarding elements is for using the set of flow entries to (i) make a set of logical forwarding decisions to identify a logical egress port of the distributed virtual switch for a packet received from a virtual machine in the set of virtual machines and (ii) map the identified logical egress port to a physical port of the particular forwarding element through which to send the packet.

13. The networking site of claim 12, wherein the physical forwarding elements comprise one or more virtual switches.

14. The networking site of claim 12, wherein the physical forwarding elements comprise one or more physical switches.

15. The networking site of claim 12, wherein the distributed virtual switch further handles communications between external hosts and the set of virtual machines.

16. The networking site of claim 12, wherein the particular physical forwarding element comprises a machine readable medium for storing:

a logical flow table that specifies the logical forwarding decisions to identify logical egress ports of the distributed virtual switch; and

a set of mappings between the logical egress ports and the physical ports of the particular physical forwarding element.

17. The networking site of claim 12, wherein the distributed virtual switch maintains at least one of counters, port speeds, and bisectional bandwidth for logical ports of the distributed virtual switch.

18. The networking site of claim 12, wherein at least two virtual machines of the first set of virtual machines are in different subnets.

19. The networking site of claim 12, wherein tunnels are set up through at least two of the physical forwarding elements.

20. The networking site of claim 19, wherein the tunnels comprise GRE tunnels.

21. The networking site of claim 12, wherein the distributed virtual switch is created by modifying a set of flow tables of the set of physical forwarding elements.

22. The networking site of claim 21, wherein the set of flow tables comprises at least one of a logical L2 table, a logical L3 table, and a logical ACL table.

23. The networking site of claim 12,

wherein identifying the physical port of the particular physical forwarding element comprises:

identifying a physical port of another physical forwarding element in the set of physical forwarding elements to which the logical egress port is mapped; and

sending the packet out of the particular physical forwarding element based on the identified physical port of the other physical forwarding element.

24. For a multi-tenant hosting system that uses a plurality of hosts and a plurality of physical forwarding elements to provide different sets of hosted virtual machines for different tenants, a method comprising:

defining a set flow entries for a set of physical forwarding elements to use to implement a distributed virtual switch for one particular tenant, the distributed virtual switch to handle communications between the virtual machines of the particular tenant while isolating the particular tenant'"'"'s virtual machines from the virtual machines of other tenants; and

sending the set of flow entries to the set of forwarding elements, the set of flow entries for populating a set of flow tables of a particular physical forwarding element in the set of physical forwarding elements, wherein the particular physical forwarding element is for making a plurality of lookups on the set of flow tables in order to (i) identify a distributed virtual switch for a particular tenant for a packet from a first virtual machine of the particular tenant to a second virtual machine of the particular tenant, (ii) identify a logical egress port of the distributed virtual switch for the packet, and (iii) identify a physical port of the particular physical forwarding element through which to send the packet out of the particular physical forwarding element based on the identified logical egress port.

25. The method of claim 24, wherein a physical forwarding element comprises a physical switch.

26. The method of claim 24, wherein a physical forwarding element comprises a virtual switch.

27. The method of claim 24, wherein the first virtual machine is in a first virtual local area network (VLAN) and the second virtual machine is in a second VLAN.

28. The method of claim 24, wherein the particular physical forwarding element uses the lookups to add a logical context to the packet, the logical context for identifying a distributed virtual switch for a particular tenant.