Migrating middlebox state for distributed middleboxes

• US 10,235,199 B2
• Filed: 01/04/2017
• Issued: 03/19/2019
• Est. Priority Date: 11/15/2011
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Zhang, Ronghua, Koponen, Teemu, Thakkar, Pankaj, Padmanabhan, Amar, Lambeth, W. Andrew, Casado, Martin
• Primary Examiner(s): Wu, Qing Yuan
• Application Number:
  US15/398,689
• Publication Number:
  US 20170116023A1
• Time in Patent Office: 804 Days
• Field of Search: None
• CPC Class Codes: G06F 15/177 Initialisation or configuration control processor initialisation G06F9/4405
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/455 Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 41/08 Configuration management of networks or network elements address allocation H04L61/50
• CPC Class Codes: H04L 41/0803 Configuration setting
• CPC Class Codes: H04L 41/0806 for initial configuration or provisioning, e.g. plug-and-play
• CPC Class Codes: H04L 41/0813 characterised by the conditions triggering a change of settings
• CPC Class Codes: H04L 41/0823 characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability for optimising operational conditions of wireless networks H04W24/02
• CPC Class Codes: H04L 41/0889 Techniques to speed-up the configuration process
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 45/02 Topology update or discovery
• CPC Class Codes: H04L 45/64 using an overlay routing layer
• CPC Class Codes: H04L 45/74 Address processing for routing
• CPC Class Codes: H04L 49/15 Interconnection of switching modules
• CPC Class Codes: H04L 49/70 : Virtual switches
• CPC Class Codes: H04L 61/2503 : Translation of Internet protocol [IP] addresses
• CPC Class Codes: H04L 61/2517 : using port numbers
• CPC Class Codes: H04L 61/2521 : Translation architectures other than single NAT servers
• CPC Class Codes: H04L 61/256 : NAT traversal
• CPC Class Codes: H04L 63/0218 : Distributed architectures, e.g. distributed firewalls
• CPC Class Codes: H04L 67/1008 : based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30

Abstract

A controller of a network control system for configuring several middlebox instances is described. The middlebox instances implement a middlebox in a distributed manner in several hosts. The controller configures, in a first host, a first middlebox instance to receive a notification from a migration module before a virtual machine (VM) running in the first host migrates to a second host and to send middlebox state related to the VM to the migration module.

Claims

1. For a migration management module that executes on a first host computer, a method comprising:

determining that a machine operating on the first host computer is to be migrated to a second host computer;

from a first middlebox element operating on the first host computer, retrieving state information relating to the machine to be migrated, wherein the first middlebox element provides a middlebox service for a logical middlebox to the machine and is one of a plurality of middlebox elements executing on a plurality of host computers that implement the logical middlebox to provide the middlebox service to a plurality of machines operating on the plurality of host computers, the plurality of machines logically connected through a logical network that comprises the logical middlebox;

sending the retrieved middlebox state information to the second host computer when the machine is migrated to the second host computer, in order for a second middlebox element executing on the second host computer to continue providing the middlebox service to the machine.

2. The method of claim 1, wherein the logical middlebox is one of a load balancer, network address translation, and firewall.

3. The method of claim 1, wherein the logical network further comprises a set of logical forwarding elements that are implemented by a plurality of managed forwarding elements that operate on the plurality of host computers.

4. The method of claim 3, wherein the plurality of machines and the logical middlebox are logically connected to the set of logical forwarding elements.

5. The method of claim 1, wherein the migration management module is a first migration management module, wherein sending the retrieved middlebox state information to the second host computer comprises sending the retrieved middlebox state information to a second migration management module operating on the second host computer.

6. The method of claim 1, wherein the migration management module executes within a hypervisor of the host computer.

7. The method of claim 1, wherein the first and second middlebox elements receive configuration data distributed by a same network controller that is further for distributing data for configuring managed forwarding elements operating on the first and second host computers to implement logical forwarding elements of the logical network that comprises the logical middlebox.

8. The method of claim 1 further comprising facilitating migration of the machine from the first host computer to the second host computer.

9. The method of claim 1, wherein:

the middlebox service is a first middlebox service, the machine is a first machine, the logical middlebox is a first logical middlebox, and the plurality of machines operating on the plurality of host computers are a first plurality of machines operating on a first plurality of host computers;

the first middlebox element provides a second middlebox service for a second logical middlebox to a second machine operating on the first host computer;

the first middlebox element is one of a second plurality of middlebox elements executing on a second plurality of host computers that implement the second logical middlebox to provide the second middlebox service to a second plurality of machines operating on the second plurality of host computers; and

the second plurality of machines are logically connected through a second logical network that comprises the second logical middlebox.

10. The method of claim 9 further comprising:

determining that the second machine is to be migrated to a third host computer;

retrieving state information relating to the second machine from the first middlebox element; and

sending the retrieved state information relating to the second machine to the third host computer when the second machine is migrated to the third host computer.

11. A non-transitory machine readable medium storing a migration management module for execution by at least one processing unit of a first host computer, the migration management module comprising sets of instructions for:

determining that a machine operating on the first host computer is to be migrated to a second host computer;

from a first middlebox element operating on the first host computer, retrieving state information relating to the machine to be migrated, wherein the first middlebox element provides a middlebox service for a logical middlebox to the machine and is one of a plurality of middlebox elements executing on a plurality of host computers that implement the logical middlebox to provide the middlebox service to a plurality of machines operating on the plurality of host computers, the plurality of machines logically connected through a logical network that comprises the logical middlebox;

sending the retrieved middlebox state information to the second host computer when the machine is migrated to the second host computer, in order for a second middlebox element executing on the second host computer to continue providing the middlebox service to the machine.

12. The non-transitory machine readable medium of claim 11, wherein the logical middlebox is one of a load balancer, network address translation, and firewall.

13. The non-transitory machine readable medium of claim 11, wherein the logical network further comprises a set of logical forwarding elements that are implemented by a plurality of managed forwarding elements that operate on the plurality of host computers.

14. The non-transitory machine readable medium of claim 13, wherein the plurality of machines and the logical middlebox are logically connected to the set of logical forwarding elements.

15. The non-transitory machine readable medium of claim 11, wherein the migration management module is a first migration management module, wherein the set of instructions for sending the retrieved middlebox state information to the second host computer comprises a set of instructions for sending the retrieved middlebox state information to a second migration management module operating on the second host computer.

16. The non-transitory machine readable medium of claim 11, wherein the migration management module executes within a hypervisor of the host computer.

17. The non-transitory machine readable medium of claim 11, wherein the first and second middlebox elements receive configuration data distributed by a same network controller that is further for distributing data for configuring managed forwarding elements operating on the first and second host computers to implement logical forwarding elements of the logical network that comprises the logical middlebox.

18. The non-transitory machine readable medium of claim 11, wherein the migration management module further comprises a set of instructions for facilitating migration of the machine from the first host computer to the second host computer.

19. The non-transitory machine readable medium of claim 11, wherein:

the middlebox service is a first middlebox service, the machine is a first machine, the logical middlebox is a first logical middlebox, and the plurality of machines operating on the plurality of host computers are a first plurality of machines operating on a first plurality of host computers;

the first middlebox element provides a second middlebox service for a second logical middlebox to a second machine operating on the first host computer;

the first middlebox element is one of a second plurality of middlebox elements executing on a second plurality of host computers that implement the second logical middlebox to provide the second middlebox service to a second plurality of machines operating on the second plurality of host computers; and

the second plurality of machines are logically connected through a second logical network that comprises the second logical middlebox.

20. The non-transitory machine readable medium of claim 19, wherein the migration management module further comprises sets of instructions for:

determining that the second machine is to be migrated to a third host computer;

retrieving state information relating to the second machine from the first middlebox element; and

sending the retrieved state information relating to the second machine to the third host computer when the second machine is migrated to the third host computer.