Reconciliation of network state across physical domains

• US 9,571,304 B2
• Filed: 06/30/2014
• Issued: 02/14/2017
• Est. Priority Date: 07/08/2013
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Koponen, Teemu, Davie, Bruce, Stribling, Jeremy
• Primary Examiner(s): Phunkulh, Bob
• Application Number:
  US14/320,615
• Publication Number:
  US 20150009796A1
• Time in Patent Office: 960 Days
• US Class Current: 1/1
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: H04L 41/0661 by reconfiguring faulty entities
• CPC Class Codes: H04L 41/0695 the faulty arrangement being the maintenance, administration or management system
• CPC Class Codes: H04L 41/0816 the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/0873 Checking configuration conflicts between network elements
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 41/28 Restricting access to network management systems or functions, e.g. using authorisation function to access network configuration
• CPC Class Codes: H04L 41/40 using virtualisation of network functions or resources, e.g. SDN or NFV entities
• CPC Class Codes: H04L 43/0811 by checking connectivity
• CPC Class Codes: H04L 43/20 the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV
• CPC Class Codes: H04L 45/02 Topology update or discovery
• CPC Class Codes: H04L 45/036 Updating the topology between route computation elements, e.g. between OpenFlow controllers
• CPC Class Codes: H04L 45/04 Interdomain routing, e.g. hierarchical routing
• CPC Class Codes: H04L 45/64 using an overlay routing layer
• CPC Class Codes: H04L 63/0263 Rule management
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

Some embodiments provide a method for a first network controller located at a first domain that manages a logical network spanning several physical domains including the first domain. Upon reconnection to a second network controller located at a second physical domain of the several physical domains after a period of disconnect, the method receives a first set of updates indicating modifications to logical network state from the second network controller. The method reconciles any conflicts between (i) the logical network state stored at the first controller, (ii) the first set of updates received from the second network controller, and (iii) updates received from other network controllers located at different physical domains of the several physical domains. The method transmits a second set of updates to the second controller indicating modifications to the logical network state based on the reconciliation of conflicts.

Claims

1. For a first network controller located at a first domain that manages a logical network spanning a plurality of physical domains including the first domain, a method comprising:

upon reconnection to a second network controller located at a second physical domain of the plurality of physical domains after a period of disconnect, receiving a first set of updates indicating modifications to logical network state from the second network controller, wherein the first set of updates are implemented in the second physical domain by the second network controller during the period of disconnect;

reconciling any conflicts between (i) logical network state stored at the first controller, (ii) the first set of updates received from the second network controller, and (iii) updates received from other network controllers located at different physical domains of the plurality of physical domains, wherein the updates received from the other network controllers are implemented by the other network controllers at their respective physical domains prior to the reconciliation of conflicts; and

transmitting a second set of updates to the second controller indicating modifications to the logical network state based on the reconciliation of conflicts.

2. The method of claim 1 further comprising receiving, from a third network controller located at a third physical domain, a third set of updates indicating modifications to the logical network state, wherein the third set of updates are implemented in the third physical domain by the third network controller during the period of disconnect.

3. The method of claim 2 further comprising transmitting a fourth set of updates to the third controller indicating modifications to the logical network state based on the reconciliation of conflicts.

4. The method of claim 3, wherein the second set of updates and the fourth set of updates are different.

5. The method of claim 1, wherein reconciling the conflicts comprises:

identifying a particular update in the first set of updates that conflicts with the logical network state stored at the first network controller; and

selecting the logical network state stored at the first network controller.

6. The method of claim 5, wherein the second set of updates indicates for the second network controller to modify its logical network state reflected in the particular update to match that of the logical network state stored at the first network controller.

7. The method of claim 1, wherein reconciling the conflicts comprises:

identifying a particular update in the first set of updates that conflicts with the logical network state stored at the first network controller; and

selecting the modification to the logical network state indicated in the particular update.

8. The method of claim 7 further comprising modifying the logical network state stored at the first network controller to reflect that of the particular update.

9. The method of claim 1, wherein reconciling the conflicts comprises:

identifying a first update indicating a first modification to the logical network state in the first set of updates that conflicts with a second update indicating a second modification to the logical network state in a third set of updates received from a third controller at a third domain; and

selecting the second modification to the logical network state.

10. The method of claim 9, wherein the second set of updates indicates for the second network controller to modify its logical network state reflected in the first update to match that of the logical network state reflected in the second update, the method further comprising modifying the logical network state stored at the first network controller to reflect that of the second update.

11. The method of claim 9, wherein the first and second modifications indicate different locations for a particular logical port, wherein the second update has a more recent timestamp.

12. The method of claim 9, wherein the first and second modifications indicate different security policies, wherein the second modification indicates a stricter security policy.

13. The method of claim 12, wherein the first update has a more recent timestamp.

14. The method of claim 1, wherein the first network controller is elected as a master controller for the logical network using a consensus protocol.

15. The method of claim 1 further comprising receiving separate updates from each other network controller that was disconnected from the first network controller.

16. The method of claim 1 further comprising, while disconnected from the second network controller, storing a queue of modifications to logical network state made at the first controller.

17. For a first network controller located at a first domain that manages a logical network spanning a plurality of physical domains including the first domain, a method comprising:

while disconnected from a second network controller located at a second one of the physical domains, storing a queue of modifications to state relating to the logical network made at the first controller, wherein the modifications stored in the queue are implemented in the first physical domain by the first network controller while disconnected from the second network controller;

upon reconnection to the second network controller, sending the modifications stored in the queue to the second network controller; and

receiving, from the second network controller, a set of updates to the state based on reconciling the modifications sent to the second network controller with modifications to state relating to the logical network made at other network controllers located at other physical domains while the first network controller was disconnected from the second network controller, wherein the modifications made at other network controllers are implemented by the other network controllers in their respective physical domains prior to the second network controller reconciling the modifications.

18. The method of claim 17 further comprising, prior to disconnection, using a deterministic consensus protocol to determine that the second network controller is a master controller for the logical network.

19. The method of claim 18, wherein the first network controller is a master controller for a second logical network.

20. The method of claim 17, wherein the second network controller reconciles different types of logical network state using different rules.