Managing forwarding of logical network traffic between physical domains

• US 9,559,870 B2
• Filed: 06/30/2014
• Issued: 01/31/2017
• Est. Priority Date: 07/08/2013
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Koponen, Teemu, Davie, Bruce, Stribling, Jeremy
• Primary Examiner(s): Phunkulh, Bob
• Application Number:
  US14/320,617
• Publication Number:
  US 20150009797A1
• Time in Patent Office: 946 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: H04L 41/0661 by reconfiguring faulty entities
• CPC Class Codes: H04L 41/0695 the faulty arrangement being the maintenance, administration or management system
• CPC Class Codes: H04L 41/0816 the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/0873 Checking configuration conflicts between network elements
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 41/28 Restricting access to network management systems or functions, e.g. using authorisation function to access network configuration
• CPC Class Codes: H04L 41/40 using virtualisation of network functions or resources, e.g. SDN or NFV entities
• CPC Class Codes: H04L 43/0811 by checking connectivity
• CPC Class Codes: H04L 43/20 the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV
• CPC Class Codes: H04L 45/02 Topology update or discovery
• CPC Class Codes: H04L 45/036 Updating the topology between route computation elements, e.g. between OpenFlow controllers
• CPC Class Codes: H04L 45/04 Interdomain routing, e.g. hierarchical routing
• CPC Class Codes: H04L 45/64 using an overlay routing layer
• CPC Class Codes: H04L 63/0263 Rule management
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

Some embodiments provide a method for a first network controller located at a first physical domain that manages a logical network spanning several physical domains including the first domain. The method detects that connectivity is lost between the first network controller and a second network controller located in a second one of the physical domains. The method identifies a set of forwarding elements managed by the first network controller that implement the logical network. The method instructs the set of forwarding elements to drop packets for the logical network received from forwarding elements in the second physical domain.

Claims

1. For a first network controller located at a first physical domain that manages a logical network spanning a plurality of physical domains including the first domain, a method comprising:

after a period of time in which the first network controller was unable to communicate with a second network controller located at a second physical domain, determining a current network state for the logical network based on data received from the second network controller and data stored at the first network controller;

distributing current network state data regarding receiving packets to (i) managed forwarding elements located at the first domain and (ii) the second network controller for subsequent distribution by the second network controller to managed forwarding elements located at the second domain;

after distribution of the current network state data regarding receiving packets, subsequently distributing current network state data regarding sending packets to (i) managed forwarding elements located at the first domain and (ii) the second network controller for subsequent distribution by the second network controller to managed forwarding elements located at the second domain.

2. The method of claim 1, wherein determining a current network state comprises:

comparing network state updates received from the second network controller to network state stored at the first controller; and

reconciling conflicts between the received network state updates and the stored network state.

3. The method of claim 2, wherein the distributed network state data comprises (i) changes to the network state made at the first network controller during the period of time in which the first network controller was unable to communicate with the second network controller and (ii) differences between the network state updates received from the second network controller and the current network state after reconciling conflicts.

4. The method of claim 1 further comprising, after distribution of the network state regarding sending packets, distributing a command to remove a previous version of the network state.

5. The method of claim 1 further comprising computing flow entries to direct the managed forwarding elements based on the current network state, wherein the current network state distributed to the managed forwarding elements located at the first domain comprises the flow entries.

6. The method of claim 5, wherein the current network state distributed to the second network controller does not include the flow entries, wherein the second network controller computes flow entries for distribution to the managed forwarding elements located at the second domain.

7. The method of claim 1, wherein the managed forwarding elements in the first physical domain send packets identified with a first version number prior to the distribution of the current network state.

8. The method of claim 7, wherein the managed forwarding elements in the first physical domain send packets identified with the first version number after the distribution of the current network state data regarding receiving packets but prior to the distribution of the current network state data regarding sending packets.

9. The method of claim 8, wherein the managed forwarding elements in the first physical domain send packets identified with a second version number after the distribution of the current network state data regarding sending packets.

10. The method of claim 7, wherein the managed forwarding elements in the second physical domain send packets identified with a second version number prior to the distribution of the current network state.

11. The method of claim 10, wherein the managed forwarding elements in the first and second physical domains send packets identified with a third version number after the distribution of the current network state data regarding sending packets.

12. The method of claim 10, wherein prior to the distribution of the current network state, packets sent from a first managed forwarding element in the first physical domain to a second managed forwarding element in the second physical domain are dropped by the second managed forwarding element based on having an incompatible version number.

13. The method of claim 12, wherein after the distribution of the current network state data, packets sent from the first managed forwarding element to the second managed forwarding element are forwarded by the second managed forwarding element.

14. A non-transitory machine readable medium storing a program for a first network controller located at a first physical domain that manages a logical network spanning a plurality of physical domains including the first domain, the program for execution by at least one processing unit of a controller computer, the program comprising sets of instructions for:

after a period of time in which the first network controller was unable to communicate with a second network controller located at a second physical domain, determining a current network state for the logical network based on data received from the second network controller and data stored at the first network controller;

distributing current network state data regarding receiving packets to (i) managed forwarding elements located at the first domain and (ii) the second network controller for subsequent distribution by the second network controller to managed forwarding elements located at the second domain;

after distribution of the current network state data regarding receiving packets, subsequently distributing current network state data regarding sending packets to (i) managed forwarding elements located at the first domain and (ii) the second network controller for subsequent distribution by the second network controller to managed forwarding elements located at the second domain.

15. The non-transitory machine readable medium of claim 14, wherein the set of instructions for determining a current network state comprises sets of instructions for:

comparing network state updates received from the second network controller to network state stored at the first controller; and

reconciling conflicts between the received network state updates and the stored network state,

wherein the distributed network state data comprises (i) changes to the network state made at the first network controller during the period of time in which the first network controller was unable to communicate with the second network controller and (ii) differences between the network state updates received from the second network controller and the current network state after reconciling conflicts.

16. The non-transitory machine readable medium of claim 14, wherein the managed forwarding elements in the first physical domain (i) send packets identified with a first version number prior to the distribution of the current network state, (ii) send packets identified with the first version number after the distribution of the current network state data regarding receiving packets but prior to the distribution of the current network state data regarding sending packets, and (iii) send packets identified with a second version number after the distribution of the current network state data regarding sending packets.

17. The non-transitory machine readable medium of claim 16, wherein the managed forwarding elements in the second physical domain send packets identified with a third version number prior to the distribution of the current network state.

18. The non-transitory machine readable medium of claim 17, wherein the managed forwarding elements in the first and second physical domains send packets identified with a fourth version number after the distribution of the current network state data regarding sending packets.

19. The non-transitory machine readable medium of claim 17, wherein prior to the distribution of the current network state, packets sent from a first managed forwarding element in the first physical domain to a second managed forwarding element in the second physical domain are dropped by the second managed forwarding element based on having an incompatible version number.

20. The non-transitory machine readable medium of claim 19, wherein after the distribution of the current network state data, packets sent from the first managed forwarding element to the second managed forwarding element are forwarded by the second managed forwarding element.

21. The non-transitory machine readable medium of claim 14, wherein the program further comprises a set of instructions for distributing a command to remove a previous version of the network state after distribution of the network state regarding sending packets.

22. The non-transitory machine readable medium of claim 14, wherein the program further comprises a set of instructions for computing flow entries to direct the managed forwarding elements based on the current network state, wherein the current network state distributed to the managed forwarding elements located at the first domain comprises the flow entries.

23. The non-transitory machine readable medium of claim 22, wherein the current network state distributed to the second network controller does not include the flow entries, wherein the second network controller computes flow entries for distribution to the managed forwarding elements located at the second domain.