Exchange of network state information between forwarding elements

• US 9,331,937 B2
• Filed: 04/18/2013
• Issued: 05/03/2016
• Est. Priority Date: 04/18/2012
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Koponen, Teemu, Padmanabhan, Amar
• Primary Examiner(s): Phan, Tri H
• Application Number:
  US14/348,884
• Publication Number:
  US 20140348161A1
• Time in Patent Office: 1,111 Days
• Field of Search: 370229-240, 370254-258, 370389-402, 370/409, 370412-429, 370/449, 370/453, 370/457, 370461-467, 370/469, 370474-476, 709220-226, 709230-250
• US Class Current: 1/1
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 41/0654 using network fault recovery ring fault isolation or reconfiguration in loop networks without recovery actions by a network management system H04L12/437
• CPC Class Codes: H04L 41/08 Configuration management of networks or network elements address allocation H04L61/50
• CPC Class Codes: H04L 41/0803 Configuration setting
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 41/122 of virtualised topologies, e.g. software-defined networks [SDN] or network function virtualisation [NFV]
• CPC Class Codes: H04L 43/0823 Errors, e.g. transmission errors
• CPC Class Codes: H04L 45/02 Topology update or discovery
• CPC Class Codes: H04L 45/64 using an overlay routing layer
• CPC Class Codes: H04L 45/72 Routing based on the source address
• CPC Class Codes: H04L 49/70 Virtual switches

Abstract

Some embodiments provide a network control system that includes a network controller and a set of hosts on which a set of managed forwarding elements operate. The network controller computes forwarding state information and pushes the computed forwarding state information to a set of managed forwarding elements to define forwarding behaviors of the managed forwarding elements. The managed forwarding elements receive the forwarding state information from the network controller and directly exchange with each other updates to the forwarding state information. The updates are exchanged between the managed forwarding elements without a network controller relaying the updates.

Claims

1. A computer configured as a first host in a network managed by at least one network controller, the computer comprising:

a set of processing units;

a non-volatile storage storing;

a first managed forwarding element for execution by at least one processing unit in the set of processing units, the first managed forwarding element for forwarding data packets to and from virtual machines operating on the first host based on forwarding state records, wherein at least a plurality of the forwarding state records are provided by a network controller; and

a daemon for execution by at least one processing unit in the set of processing units, the daemon for (i) providing, to the first managed forwarding element, a set of forwarding state records required by the first managed forwarding element in order to forward a particular data packet and (ii) requesting forwarding state from a second managed forwarding element operating in a different second host to obtain the required set of forwarding state records from the second forwarding element to forward the particular data packet when the daemon does not have the set of forwarding state records locally available, the second managed forwarding element for forwarding data packets for a second set of virtual machines that operate on the second host.

2. The computer of claim 1, wherein the daemon comprises a lookup module for performing the providing and requesting.

3. The computer of claim 2, wherein the daemon further comprises a registering module for publishing forwarding state records of the first forwarding element to the second forwarding element in order for the second forwarding element to store the published forwarding state records in the second host.

4. The computer of claim 3, wherein the registering module is further for sending a request to the second managed forwarding element to remove a particular forwarding state record from the stored forwarding state records.

5. The computer of claim 1, wherein the first forwarding element is an edge forwarding element that directly interfaces with the virtual machines operating on the first host, wherein said virtual machines are sources and destinations of data packets.

6. The computer of claim 5, wherein the virtual machines are each for execution by at least one processing unit in the set of processing units, the virtual machines for directly interfacing with the first forwarding element within the first host.

7. A non-transitory machine readable medium storing a first managed forwarding element which when executed by at least one processing unit forwards data within a network comprising a plurality of managed forwarding elements, the first managed forwarding element comprising sets of instructions for:

from a network controller that manages the network, receiving forwarding state records computed by the network controller;

determining whether a set of forwarding state records required for forwarding a particular packet exists in the received forwarding state records;

when the required set of forwarding state records do not exist in the received forwarding state records, sending a request to obtain the set of forwarding state records to a second managed forwarding element, wherein the first and second managed forwarding elements receive forwarding state records from the network controller in order to implement a set of logical forwarding elements;

receiving the requested forwarding state records from the second managed forwarding element; and

forwarding the packet using the forwarding state records received from the second managed forwarding element.

8. The non-transitory machine readable medium of claim 7, wherein the first managed forwarding element further comprises a set of instructions for forwarding a second packet using the forwarding state records received from the network controller.

9. The non-transitory machine readable medium of claim 7, wherein the first managed forwarding element further comprises a set of instructions for, when a local forwarding state record is updated, sending the updated forwarding state record to the second managed forwarding element in order for the second managed forwarding element to add the updated forwarding state record to a set of forwarding state records maintained by the second managed forwarding element, wherein the second managed forwarding element sends the updated forwarding state record to a third managed forwarding element when the third managed forwarding element requests the updated forwarding state record and receives forwarding state records from the network controller in order to implement the set of logical forwarding elements.

10. The non-transitory machine readable medium of claim 9, wherein the first managed forwarding element further comprises a set of instructions for sending a request to the second managed forwarding element to remove the updated forwarding state record from its maintained set of forwarding state records when the updated forwarding state record is no longer valid.

11. The non-transitory machine readable medium of claim 7, wherein the first managed forwarding element further comprises sets of instructions for:

receiving a data packet from a third managed forwarding element;

determining whether forwarding state records used by the third managed forwarding element to forward the packet to the first managed forwarding element are valid; and

when the forwarding state records used by the third managed forwarding element are not valid, sending a notification to the third managed forwarding element to indicate to the third managed forwarding element that at least one of its forwarding state records is invalid, wherein the third managed forwarding element receives forwarding state records from the network controller in order to implement the set of logical forwarding elements.

12. A network control system comprising:

a network controller for computing forwarding state information and pushing the computed forwarding state information to a set of managed forwarding elements to define forwarding behaviors of the managed forwarding elements; and

a set of hosts on which the set of managed forwarding elements operate to forward data for sets of virtual machines operating on each host, the managed forwarding elements for receiving the forwarding state information from the network controller and directly exchanging with each other updates to the forwarding state information, wherein the updates are exchanged between the managed forwarding elements without the network controller relaying the updates.

13. The network control system of claim 12, wherein a particular update to the forwarding state information comprises a key-value pair.

14. The network control system of claim 13, wherein the key comprises a first piece of network state information and the value comprises a second piece of network state information bound to the first piece of network state information.

15. The network control system of claim 12, wherein a particular managed forwarding element is for receiving updates to forwarding state information from the set of managed forwarding elements, maintaining the forwarding state information received from the set of managed forwarding elements, receiving forwarding state information requests from the set of managed forwarding elements, and sending responses to the received requests with requested network state information.

16. The network control system of claim 15, wherein the received updates comprise keys and values bound to the keys, a forwarding state information request comprises a key for which a value is required, and the response to the received request comprises the key and the value bound to the key.

17. The network control system of claim 16, wherein the received updates further comprise a time to live that specifies a time duration during which the received updates are valid.

18. The network control system of claim 15, wherein the received updates further comprise instructions to remove a forwarding state record from the maintained forwarding state information.

19. The network control system of claim 12, wherein a first managed forwarding element is for sending a data packet to a second managed forwarding element using received forwarding state information, receiving a notification from a third managed forwarding element that a particular one of its forwarding state records used to send the packet to the second managed forwarding element is invalid, and removing the invalid forwarding state record from its forwarding state information.

20. The network control system of claim 19, wherein the first managed forwarding element is further for relaying the notification to a fourth managed forwarding element in order for the fourth managed forwarding element to remove the invalid forwarding state record from its network forwarding state information.