Communication channel for distributed network control system

• US 9,253,109 B2
• Filed: 01/31/2013
• Issued: 02/02/2016
• Est. Priority Date: 10/25/2011
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Koponen, Teemu, Lambeth, W. Andrew, Thakkar, Pankaj
• Primary Examiner(s): Avellino, Joseph E
• Assistant Examiner(s): MCLEOD, MARSHALL M
• Application Number:
  US13/756,487
• Publication Number:
  US 20130212244A1
• Time in Patent Office: 1,097 Days
• Field of Search: 709/223, 709/204
• US Class Current: 1/1
• CPC Class Codes: G05B 11/01 electric
• CPC Class Codes: G06F 15/177 Initialisation or configuration control processor initialisation G06F9/4405
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 12/4633 Interconnection of networks using encapsulation techniques, e.g. tunneling
• CPC Class Codes: H04L 41/0226 Mapping or translating multiple network management protocols
• CPC Class Codes: H04L 41/042 comprising distributed management centres cooperatively managing the network
• CPC Class Codes: H04L 41/0806 for initial configuration or provisioning, e.g. plug-and-play
• CPC Class Codes: H04L 41/20 Network management software packages
• CPC Class Codes: H04L 41/50 Network service management, e.g. ensuring proper service fulfilment according to agreements
• CPC Class Codes: H04L 45/38 Flow based routing
• CPC Class Codes: H04L 45/42 Centralised routing
• CPC Class Codes: H04L 45/66 Layer 2 routing, e.g. in Ethernet based MAN's
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/50 Queue scheduling
• CPC Class Codes: H04L 47/825 Involving tunnels, e.g. MPLS
• CPC Class Codes: H04L 49/254 Centralised controller, i.e. arbitration or scheduling

Abstract

For a particular controller for managing managed forwarding elements that forward data in a network, a method for computing forwarding state using a set of inputs from a first controller and a second controller that is a back up controller for the first controller is described. The method receives a first subset of the set of inputs from the first controller. After failure of the first controller, the method receives a second subset of the set of inputs from the second controller. At least one input of the second subset of the set of inputs is duplicative of an input in the first subset. The method computes forwarding state using the first and second subsets of the inputs but without using the duplicative input.

Claims

1. For a particular controller that computes forwarding state for managing managed forwarding elements that forward data in a network, a method for computing the forwarding state using a set of inputs from a first controller and a second controller that is a back up controller for the first controller, the method comprising:

receiving a first subset of the set of inputs from the first controller at the particular controller, wherein the particular controller is a master of a set of the managed forwarding elements;

after failure of the first controller, receiving a second subset of the set of inputs from the second controller at the particular controller, at least one input of the second subset of the set of inputs being duplicative of an input in the first subset;

computing forwarding state using the first and second subsets of the set of inputs but without using the duplicative input; and

distributing the computed forwarding state from the particular controller to the set of managed forwarding elements.

2. The method of claim 1, wherein the set of inputs includes an end input that indicates a last input of the set of inputs, the method further comprising removing the duplicative input after receiving the end input from the second controller.

3. The method of claim 1, wherein the forwarding state is customized physical control plane data that defines forwarding behavior of the managed forwarding elements.

4. The method of claim 3, wherein the set of inputs includes universal physical control plane data into which the first controller converted logical control plane data.

5. The method of claim 4, wherein the logical control plane data is converted from user inputs.

6. For a first controller for managing managed forwarding elements that forward data in a network, a method for computing forwarding state to send to a second controller, the method comprising:

generating a first set of outputs that includes the forwarding state, the following state defining forwarding behavior of a set of managed forwarding elements;

sending the outputs of the first set to the second controller as the outputs of the first set are generated;

after sending all of the first set of outputs, sending an indicator that indicates the end of the first set of outputs to the second controller so that the second controller starts processing the outputs only after receiving the indicator; and

generating a second set of outputs at the first controller after sending the indicator.

7. The method of claim 6, wherein each output in the first set of outputs is a data tuple.

8. The method of claim 6, wherein the indicator is included in a last generated output of the set.

9. The method of claim 8, wherein the forwarding state is universal physical control plane data, wherein said generating the set of outputs comprises converting input logical control plane into the universal physical control plane data.

10. The method of claim 9, wherein the second controller converts the universal physical control plane data into customized physical control plane data.

11. A non-transitory machine readable medium storing a program which when executed by at least one processing unit computes forwarding state, for managing a set of managed forwarding elements that forward data in a network, using several sets of inputs from several source controllers, the program comprising sets of instructions for:

receiving a first set of inputs from a first source controller, the first set of inputs comprising a group of inputs that is to be processed separately from another group of inputs;

receiving a second set of inputs from a second source controller, the second set of inputs comprising a group of inputs that is to be processed separately from another group of inputs;

determining whether to combine the first and second sets of inputs into a single group of inputs based on a set of criteria;

combining the first and second set of inputs to form the single group of inputs that is to be processed separately from another group of inputs;

computing forwarding state using the single group of inputs, the forwarding state defining forwarding behaviors of managed forwarding elements; and

distributing the computed forwarding state to the managed forwarding elements.

12. The method non-transitory machine readable medium of claim 11, wherein the set of criteria includes a number of sets of inputs that have been received.

13. The method non-transitory machine readable medium of claim 11, wherein the set of criteria includes an amount of data included in the received inputs.

14. The non-transitory machine readable medium of claim 11, wherein the first and second source controllers are logical controllers each of which manages at least one logical datapath set.

16. The non-transitory machine readable medium of claim 11, wherein the first and second source controllers are physical controllers each of which manages a set of managed forwarding elements.

17. The non-transitory machine readable medium of claim 16, wherein the recipient controller is a chassis controller that manages a particular managed forwarding element that forwards data based on the computed forwarding state.

18. The non-transitory machine readable medium of claim 17, wherein the first and second sets of inputs include universal control plane data.

19. The non-transitory machine readable medium of claim 18, wherein the computed forwarding state includes customized physical control plane data.

15. For a recipient controller for managing a set of managed forwarding elements that forward data in a network, a method for computing forwarding state using several sets of inputs from the several source controllers, the method comprising:

from a first source controller, receiving a first set of inputs comprising a group of inputs that is to be processed separately from another group of inputs;

from a second source controller, receiving a second set of inputs comprising a group of inputs that is to be processed separately from another group of inputs;

combining the first and second set of inputs to form a single group of inputs that is to be processed separately from another group of inputs;

computing forwarding state using the single group of inputs, the forwarding state defining forwarding behaviors of the set of managed forwarding elements,

wherein the recipient controller is a physical controller that is a master of the set of managed forwarding elements that forward data based on the computed forwarding state.

20. The method of claim 15, further comprising determining whether to combine the first and second sets of inputs based on certain criteria.