Stateful services on stateless clustered edge

• US 10,044,617 B2
• Filed: 11/14/2014
• Issued: 08/07/2018
• Est. Priority Date: 11/14/2014
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Parsa, Mike, Jain, Jayant, Hong, Xinhua, Sengupta, Anirban, Fan, Kai-Wei
• Primary Examiner(s): Rinehart, Mark
• Assistant Examiner(s): Dewan, Sanjay K
• Application Number:
  US14/541,501
• Publication Number:
  US 20160142314A1
• Time in Patent Office: 1,362 Days
• CPC Class Codes: H04L 41/0803 Configuration setting
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 43/026 using flow identification
• CPC Class Codes: H04L 45/64 using an overlay routing layer
• CPC Class Codes: H04L 45/66 Layer 2 routing, e.g. in Ethernet based MAN's
• CPC Class Codes: H04L 45/7453 using hashing
• CPC Class Codes: H04L 47/18 End to end
• CPC Class Codes: Y02D 30/50 in wire-line communication networks, e.g. low power modes or reduced link rate

Abstract

In order to enable dynamic scaling of network services at the edge, novel systems and methods are provided to enable addition of add new nodes or removal of existing nodes while retaining the affinity of the flows through the stateful services. The methods provide a cluster of network nodes that can be dynamically resized to handle and process network traffic that utilizes stateful network services. The existing traffic flows through the edge continue to function during and after the changes to membership of the cluster. All nodes in the cluster operate in active-active mode, i.e., they are receiving and processing traffic flows, thereby maximizing the utilization of the available processing power.

Claims

1. For a network comprising a plurality of host computers, each host computer executing a set of virtual machines (VMs), wherein different groups of VMs on different hosts are connected by different logical networks, a method comprising:

performing a stateful service that requires state information for a particular flow that is entering or exiting a particular logical network at a first edge service node of a particular cluster of edge service nodes that are designated for the particular logical network, wherein the first edge service node is selected from the particular cluster of edge service nodes based on a first hash of identifiers of the particular flow;

receiving, at the first edge service node, a packet belonging to the particular flow from a second edge service node, wherein the second edge service node is selected from the particular cluster after at least one edge service node is added to or removed from the particular cluster, the selection based on a second hash of identifiers of the particular flow, wherein at least one edge service node in the cluster executes on a same host computer as a compute VM that is a source or destination of a flow; and

performing the stateful service on the received packet at the first edge service node.

2. The method of claim 1,

wherein a first mapping maps every possible hash value of the first hash to an edge service node in the particular cluster of edge service nodes before the at least one edge service node is added or removed, and

wherein a second mapping maps every possible value of the second hash to an edge service node in the particular cluster of edge service nodes after the at least one edge service node is added or removed, the second mapping being different than the first mapping.

3. The method of claim 1, wherein the first edge service node maintains state information of the particular flow, the state information maintained for the particular flow based on performing the stateful service on previous packets of the particular flow at the first edge service node.

4. The method of claim 1, wherein the particular cluster of edge service nodes provides stateful services at a north-south boundary of the particular logical network.

5. The method of claim 1, wherein the packet is received from the second edge service node based on an entry in an indirection table of the second edge service node that associates the particular flow with the first edge service node.

6. The method of claim 1, wherein the first edge service node has a pinned flow table that includes an entry that corresponds to the particular flow, the entry for preventing the first edge service node from forwarding packets belonging to the particular flow.

7. The method of claim 6, further comprising removing the entry from the pinned flow table when stateful processing of the particular flow is complete.

8. For a network comprising a plurality of host computers, each host computer executing a set of virtual machines (VMs), wherein different groups of VMs on different hosts are connected by different a plurality of logical networks, a method comprising:

performing, at a cluster of edge service nodes, a stateful service that requires state information for flows that are entering or exiting a particular logical network, wherein a state of a particular flow is maintained by a first edge service node in the cluster;

adding an edge service node to the cluster while the cluster is performing the stateful service for the particular flow;

forwarding packets belonging to the particular flow from a second edge service node to the first edge service node when the second edge service node is initially identified to perform the stateful service by a consistent hash function of flow identifiers of the particular flow over the cluster of edge service nodes used by each node in the cluster after an addition of the edge service node, wherein at least one edge service node in the cluster executes on a same host computer as a compute VM that is a source or destination of a flow; and

performing the stateful service on the forwarded packet at the first edge service node.

9. The method of claim 8, wherein the first edge service node is identified to perform the stateful service by a different consistent hash function of flow identifiers of the particular flow over the cluster of edge service nodes used by each node in the cluster before the addition of the edge service node.

10. The method of claim 8, wherein adding the edge service node to the cluster of edge service nodes further comprises adding an entry for the particular flow to an indirection table in the second edge service node, wherein the added entry of the particular flow in the indirection table is for instructing the second edge service node to forward packets belonging to the particular flow to the first edge service node.

11. The method of claim 8, wherein adding the edge service node to the cluster of edge service nodes further comprises adding an entry for the particular flow to a pinned flow table in the first edge service node, wherein the added entry for the particular flow to the pinned flow table is for anchoring packets of the particular flow at the first edge service node.

12. The method of claim 11 further comprising removing the entry from the pinned flow table in the first edge service node when stateful processing of the particular flow is complete.

13. The method of claim 8, wherein the cluster of edge service nodes provides stateful services at a north-south boundary of the particular logical network.

14. The method of claim 8, wherein the second edge service node is the edge service node that was added to the cluster.

15. For a network comprising a plurality of host computers, each host computer executing a set of virtual machines (VMs), wherein different groups of VMs on different hosts are connected by different of logical networks, a non-transitory machine readable medium storing a program for execution by a set of processing units, the program comprising sets of instructions for:

performing, at a cluster of edge service nodes, a stateful service that requires state information for flows that are entering or exiting a particular logical network, wherein a state of a particular flow is maintained by a first edge service node in the cluster;

adding an edge service node to the cluster while the cluster is performing the stateful service for the particular flow;

forwarding packets belonging to the particular flow from a second edge service node to the first edge service node when the second edge service node is initially identified to perform the stateful service by a consistent hash function of flow identifiers of the particular flow over the cluster of edge service nodes used by each node in the cluster after an addition of the edge service node, wherein at least one edge service node in the cluster executes on a same host computer as a compute VM that is a source or destination of a flow; and

performing the stateful service on the forwarded packet at the first edge service node.

16. The non-transitory machine readable medium of claim 15, wherein the first edge service node is identified to perform the stateful service by a different consistent hash function of flow identifiers of the particular flow over the cluster of edge service nodes used by each node in the cluster before the addition of the edge service node.

17. The non-transitory machine readable medium of claim 15, wherein the set of instructions for adding the edge service node to the cluster of edge service nodes further comprises a set of instructions for adding an entry for the particular flow to an indirection table in the second edge service node, wherein the added entry of the particular flow in the indirection table is for instructing the second edge service node to forward packets belonging to the particular flow to the first edge service node.

18. The non-transitory machine readable medium of claim 15, wherein the set of instructions for adding the edge service node to the cluster of edge service nodes further comprises a set of instructions for adding an entry for the particular flow to a pinned flow table in the first edge service node, wherein the added entry for the particular flow to the pinned flow table is for anchoring packets of the particular flow at the first edge service node.

19. The non-transitory machine readable medium of claim 18 further comprising a set of instructions for removing the entry from the pinned flow table in the first edge service node when stateful processing of the particular flow is complete.

20. The non-transitory machine readable medium of claim 15, wherein the set of instructions for performing the stateful service further comprises a set of instructions for providing the stateful service at a north-south boundary of the particular logical network.

21. The non-transitory machine readable medium of claim 15, wherein the second edge service node is the edge service node that was added to the cluster.