Migrating Middlebox State for Distributed Middleboxes

• US 20170116023A1
• Filed: 01/04/2017
• Published: 04/27/2017
• Est. Priority Date: 11/15/2011
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Zhang, Ronghua, Koponen, Teemu, Thakkar, Pankaj, Padmanabhan, Amar, Lambeth, W. Andrew, Casado, Martin
• Granted Patent: US 10,235,199 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 15/177 Initialisation or configuration control processor initialisation G06F9/4405
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/455 Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 41/08 Configuration management of networks or network elements address allocation H04L61/50
• CPC Class Codes: H04L 41/0803 Configuration setting
• CPC Class Codes: H04L 41/0806 for initial configuration or provisioning, e.g. plug-and-play
• CPC Class Codes: H04L 41/0813 characterised by the conditions triggering a change of settings
• CPC Class Codes: H04L 41/0823 characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability for optimising operational conditions of wireless networks H04W24/02
• CPC Class Codes: H04L 41/0889 Techniques to speed-up the configuration process
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 45/02 Topology update or discovery
• CPC Class Codes: H04L 45/64 using an overlay routing layer
• CPC Class Codes: H04L 45/74 Address processing for routing
• CPC Class Codes: H04L 49/15 Interconnection of switching modules
• CPC Class Codes: H04L 49/70 : Virtual switches
• CPC Class Codes: H04L 61/2503 : Translation of Internet protocol [IP] addresses
• CPC Class Codes: H04L 61/2517 : using port numbers
• CPC Class Codes: H04L 61/2521 : Translation architectures other than single NAT servers
• CPC Class Codes: H04L 61/256 : NAT traversal
• CPC Class Codes: H04L 63/0218 : Distributed architectures, e.g. distributed firewalls
• CPC Class Codes: H04L 67/1008 : based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30

Abstract

A controller of a network control system for configuring several middlebox instances is described. The middlebox instances implement a middlebox in a distributed manner in several hosts. The controller configures, in a first host, a first middlebox instance to receive a notification from a migration module before a virtual machine (VM) running in the first host migrates to a second host and to send middlebox state related to the VM to the migration module.

Claims

1-20. -20. (canceled)

21. For a migration management module that executes on a first host machine, a method comprising:

determining that an end machine operating on the first host machine is to be migrated to a second host machine;

from a first middlebox element operating on the first host machine, retrieving state information relating to the end machine to be migrated, wherein the first middlebox element provides a middlebox service for a logical middlebox to the end machine and is one of a plurality of middlebox elements executing on a plurality of host machines that implement the logical middlebox to provide the middlebox service to a plurality of end machines operating on the plurality of host machines, the plurality of end machines logically connected through a logical network that comprises the logical middlebox;

sending the retrieved middlebox state information to the second host machine when the end machine is migrated to the second host machine, in order for a second middlebox element executing on the second host machine to continue providing the middlebox service to the end machine.

22. The method of claim 21, wherein the logical middlebox is one of a load balancer, network address translation, and firewall.

23. The method of claim 21, wherein the logical network further comprises a set of logical forwarding elements that are implemented by a plurality of managed forwarding elements that operate on the plurality of hosts.

24. The method of claim 23, wherein the plurality of end machines and the logical middlebox are logically connected to the logical forwarding elements.

25. The method of claim 21, wherein the migration management module is a first migration management module, wherein sending the retrieved middlebox state information to the second host machine comprises sending the retrieved middlebox state information to a second migration management module operating on the second host.

26. The method of claim 21, wherein the migration management module executes within a hypervisor of the host machine.

27. The method of claim 21, wherein the first and second middlebox elements receive configuration data distributed by a same network controller that is further for distributing data for configuring managed forwarding elements operating on the first and second host machines to implement logical forwarding elements of the logical network that comprises the logical middlebox.

28. The method of claim 21 further comprising facilitating migration of the end machine from the first host machine to the second host machine.

29. The method of claim 21, wherein:

the middlebox service is a first middlebox service, the end machine is a first end machine, the logical middlebox is a first logical middlebox, and the plurality of end machines operating on the plurality of host machines are a first plurality of end machines operating on a first plurality of host machines;

the middlebox element provides a second middlebox service for a second logical middlebox to a second end machine operating in the first host machine;

the first middlebox element is one of a second plurality of middlebox elements executing on a second plurality of host machines that implement the second logical middlebox to provide the second middlebox service to a second plurality of end machines operating on the second plurality of hosts; and

the second plurality of end machines are logically connected through a second logical network that comprises the second logical middlebox.

30. The method of claim 29 further comprising:

determining that the second end machine is to be migrated to a third host machine;

retrieving state information relating to the second end machine from the first middlebox element; and

sending the retrieved state information relating to the second end machine to the third host machine when the second end machine is migrated to the third host machine.

31. A non-transitory machine readable medium storing a migration management module for execution by at least one processing unit of a first host machine, the migration management module comprising sets of instructions for:

determining that an end machine operating on the first host machine is to be migrated to a second host machine;

from a first middlebox element operating on the first host machine, retrieving state information relating to the end machine to be migrated, wherein the first middlebox element provides a middlebox service for a logical middlebox to the end machine and is one of a plurality of middlebox elements executing on a plurality of host machines that implement the logical middlebox to provide the middlebox service to a plurality of end machines operating on the plurality of host machines, the plurality of end machines logically connected through a logical network that comprises the logical middlebox;

sending the retrieved middlebox state information to the second host machine when the end machine is migrated to the second host machine, in order for a second middlebox element executing on the second host machine to continue providing the middlebox service to the end machine.

32. The non-transitory machine readable medium of claim 31, wherein the logical middlebox is one of a load balancer, network address translation, and firewall.

33. The non-transitory machine readable medium of claim 31, wherein the logical network further comprises a set of logical forwarding elements that are implemented by a plurality of managed forwarding elements that operate on the plurality of hosts.

34. The non-transitory machine readable medium of claim 33, wherein the plurality of end machines and the logical middlebox are logically connected to the logical forwarding elements.

35. The non-transitory machine readable medium of claim 31, wherein the migration management module is a first migration management module, wherein the set of instructions for sending the retrieved middlebox state information to the second host machine comprises a set of instructions for sending the retrieved middlebox state information to a second migration management module operating on the second host.

36. The non-transitory machine readable medium of claim 31, wherein the migration management module executes within a hypervisor of the host machine.

37. The non-transitory machine readable medium of claim 31, wherein the first and second middlebox elements receive configuration data distributed by a same network controller that is further for distributing data for configuring managed forwarding elements operating on the first and second host machines to implement logical forwarding elements of the logical network that comprises the logical middlebox.

38. The non-transitory machine readable medium of claim 31, wherein the migration management module further comprises a set of instructions for facilitating migration of the end machine from the first host machine to the second host machine.

39. The non-transitory machine readable medium of claim 31, wherein:

the middlebox service is a first middlebox service, the end machine is a first end machine, the logical middlebox is a first logical middlebox, and the plurality of end machines operating on the plurality of host machines are a first plurality of end machines operating on a first plurality of host machines;

the middlebox element provides a second middlebox service for a second logical middlebox to a second end machine operating in the first host machine;

the first middlebox element is one of a second plurality of middlebox elements executing on a second plurality of host machines that implement the second logical middlebox to provide the second middlebox service to a second plurality of end machines operating on the second plurality of hosts; and

the second plurality of end machines are logically connected through a second logical network that comprises the second logical middlebox.

40. The non-transitory machine readable medium of claim 39, wherein the migration management module further comprises sets of instructions for:

determining that the second end machine is to be migrated to a third host machine;

retrieving state information relating to the second end machine from the first middlebox element; and

sending the retrieved state information relating to the second end machine to the third host machine when the second end machine is migrated to the third host machine.