Hitless upgrade for network control applications

• US 9,973,382 B2
• Filed: 10/31/2013
• Issued: 05/15/2018
• Est. Priority Date: 08/15/2013
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Padmanabhan, Amar, Stribling, Jeremy, Lambeth, W. Andrew, Gude, Natasha
• Primary Examiner(s): Duong, Christine
• Application Number:
  US14/069,267
• Publication Number:
  US 20150049632A1
• Time in Patent Office: 1,657 Days
• Field of Search: None
• CPC Class Codes: H04L 41/082 the condition being updates or upgrades of network functionality
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416

Abstract

A method for upgrading a set of controller nodes in a controller cluster that manages a plurality of forwarding elements in a way that minimizes dataplane outages. The method of some embodiments upgrades the control applications of a subset of the controller nodes before upgrading a decisive controller node. Once the decisive controller node is upgraded, the method switches the controller cluster to use a new version of the control applications.

Claims

1. A method for upgrading a controller cluster comprising a plurality of controller nodes, the controller cluster for managing a plurality of forwarding elements, each controller node executing a version of a network control application, the method comprising:

upgrading a first subset of the plurality of controller nodes to execute a newer version of the network control application while managing the forwarding elements with a second subset of the plurality of controller nodes that execute an older version of the network control application;

upon completion of the upgrade of the first subset of controller nodes, upgrading a decisive controller node in the second subset of controller nodes to signal the controller cluster to manage the forwarding elements with the first subset of controller nodes;

upgrading the second subset of controller nodes to the newer version of the network control application while managing the forwarding elements with the first subset of controller nodes and the decisive controller node; and

upon completion of the upgrade of the second subset of controller nodes, managing the forwarding elements with the first and second subsets of controller nodes.

2. The method of claim 1, wherein the forwarding elements comprise a plurality of virtual interfaces (VIFs), wherein upgrading the controller nodes of the first subset comprises upgrading a set of forwarding data for each VIF in the plurality of VIFs.

3. The method of claim 1, wherein managing the forwarding elements with the first subset of controller nodes comprises using the first subset of controller nodes to write to a first set of tables on the forwarding elements, wherein managing the forwarding elements with the second subset of controller nodes comprises using the second subset of controller nodes to write to a second, different set of tables on the forwarding elements, wherein the first and second sets of tables are for storing records regarding forwarding behavior of the forwarding elements.

4. The method of claim 3, wherein storing records regarding the forwarding behavior of the forwarding elements comprises storing flows that implement a forwarding behavior on the forwarding elements.

5. The method of claim 4, wherein a particular packet received at a particular forwarding element is forwarded according to one of the first and second sets of tables based on a version bit located in metadata of the particular packet.

6. The method of claim 5, wherein managing the forwarding elements with the second subset further comprises setting a particular value in the version bit for packets passing through the forwarding elements, wherein managing the forwarding elements with the first subset further comprises setting a different value in the version bit for packets passing through the forwarding elements.

7. The method of claim 4, wherein the forwarding elements comprise a set of logical forwarding elements and a set of physical forwarding elements, wherein managing the forwarding elements with the first subset of controller nodes further comprises implementing the set of logical forwarding elements on the set of physical forwarding elements using the newer version of the network control application.

8. The method of claim 7, wherein the set of logical forwarding elements comprises a first set of logical forwarding elements for a first user and a second set of logical forwarding elements for a second user.

9. The method of claim 7 further comprising:

generating a set of flows on the first subset of controller nodes, the set of flows for implementing the set of logical forwarding elements on the set of physical forwarding elements; and

writing the set of flows to the second set of tables on the set of physical forwarding elements.

10. The method of claim 1, wherein the controller cluster performs a master election for the forwarding elements when availability for a controller node changes.

11. A non-transitory machine readable medium storing a program for execution by at least one processing unit, the program for coordinating the upgrade of a controller cluster comprising a plurality of controller nodes, the controller cluster for managing a plurality of forwarding elements, each controller node executing a version of a network control application, the program comprising sets of instructions for:

upgrading a first subset of the plurality of controller nodes to execute a newer version of the network control application while managing the forwarding elements with a second subset of the plurality of controller nodes that execute an older version of the network control application;

upon completion of the upgrade of the first subset of controller nodes, upgrading a decisive controller node in the second subset of controller nodes to signal the controller cluster to manage the forwarding elements with the first subset of controller nodes;

upgrading the second subset of controller nodes to the newer version of the network control application while managing the forwarding elements with the first subset of controller nodes and the decisive controller node; and

upon completion of the upgrade of the second subset of controller nodes, managing the forwarding elements with the first and second subsets of controller nodes.

12. The non-transitory machine readable medium of claim 11, wherein the first subset of controller nodes has the same number of controller nodes as the second subset.

13. The non-transitory machine readable medium of claim 11, wherein the set of instructions for upgrading the first subset comprises sets of instructions for:

determining whether a controller node is a master or a standby controller; and

when the controller node is a standby controller, selecting the standby controller as part of the first subset.

14. The non-transitory machine readable medium of claim 11, wherein the forwarding elements comprise a set of physical forwarding elements and a set of logical forwarding elements, wherein the set of instructions for managing forwarding elements with the first subset of controller nodes comprises a set of instructions for implementing the set of logical forwarding elements on the set of physical forwarding elements.

15. The non-transitory machine readable medium of claim 11, the set of instructions for upgrading the first subset of controller nodes comprises a set of instructions for electing a master controller node from the first subset of controller nodes for each forwarding element.

16. An apparatus comprising:

a processing unit;

a storage for storing a program for coordinating the upgrade of a controller cluster comprising a plurality of controller nodes, the controller cluster for managing a plurality of forwarding elements, each controller node executing a version of a network control application, the program comprising sets of instructions for;

upgrading a first subset of the plurality of controller nodes to execute a newer version of the network control application while managing the forwarding elements with a second subset of the plurality of controller nodes that execute an older version of the network control application;

upon completion of the upgrade of the first subset of controller nodes, upgrading a decisive controller node in the second subset of controller nodes to signal the controller cluster to manage the forwarding elements with the first subset of controller nodes;

upgrading the second subset of controller nodes to the newer version of the network control application while managing the forwarding elements with the first subset of controller nodes and the decisive controller node; and

upon completion of the upgrade of the second subset of controller nodes, managing the forwarding elements with the first and second subsets of controller nodes.

17. The apparatus of claim 16, wherein the set of instructions for upgrading the first subset comprises a set of instructions for identifying no more than half of the controller nodes to be upgraded to the newer version.

18. The apparatus of claim 16, wherein the set of instructions for managing the forwarding elements with the second subset of controller nodes comprises a set of instructions for identifying master and standby controller nodes from the second subset, wherein a standby controller node serves as a backup to a master controller node.

19. The apparatus of claim 16, wherein the program further comprises a set of instructions for removing the older version of the network control applications from the controller nodes.

20. The apparatus of claim 16, wherein the forwarding elements comprise a set of physical forwarding elements and a set of logical forwarding elements, wherein the set of instructions for managing forwarding elements with the first subset of controller nodes comprises a set of instructions for implementing the set of logical forwarding elements on the set of physical forwarding elements.