Live migration of virtual machines using packet duplication

• US 9,880,870 B1
• Filed: 09/24/2015
• Issued: 01/30/2018
• Est. Priority Date: 09/24/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Danilov, Mikhail, Kowalski, Marcin Piotr
• Primary Examiner(s): Puente, Emerson
• Assistant Examiner(s): Labud, Jonathan R
• Application Number:
  US14/864,682
• Time in Patent Office: 859 Days
• Field of Search: None
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

A virtualization management component of a particular host at a virtualized computing determines that a packet duplication phase of a migration of a virtual machine from a source host to the particular host has begun. The virtualization management component detects that a migrated version of the virtual machine, instantiated at the particular virtualization host, has generated a baseline packet directed to a destination address. The virtualization management component sends a first encapsulation packet comprising the baseline packet to an encapsulation intermediary associated with the destination address. The virtualization management component sends a second encapsulation packet comprising the baseline packet to the source virtualization host, with a directive to forward the second encapsulation packet to the encapsulation intermediary.

Claims

1. A method, comprising:

transmitting, by a first virtualization management component of a first virtualization host of a virtual computing service, an indication to a second virtualization management component of a second virtualization host that a packet duplication phase of a migration procedure for a particular guest virtual machine being migrated to the second virtualization host from the first virtualization host has begun, wherein the migration procedure does not include a reboot of the particular guest virtual machine;

in response to determining, by the second virtualization management component during the packet duplication phase, that a migrated version of the particular guest virtual machine instantiated at the second virtualization host has generated a first baseline packet directed to a different guest virtual machine instantiated at a third virtualization host, transmitting;

(a) a first encapsulation packet comprising the first baseline packet from the second virtualization host to the third virtualization host and (b) a second encapsulation packet comprising the first baseline packet from the second virtualization host to the first virtualization host;

in response to receiving, at the first virtualization management component, the second encapsulation packet, forwarding a transformed version of the second encapsulation packet to a third virtualization management component of the third virtualization host;

in response to (a) receiving, at the third virtualization management component, the first encapsulation packet and (b) determining, at the third virtualization management component, that a network mapping database of the third virtualization host does not include a mapping indicating that the particular guest virtual machine is instantiated at the second virtualization host, discarding the first encapsulation packet by the third virtualization management component; and

in response to (a) receiving, at the third virtualization management component, the transformed version of the second encapsulation packet and (b) determining, at the third virtualization management component, that the network mapping database includes a mapping indicating that the particular guest virtual machine is instantiated at the first virtualization host,

extracting the first baseline packet from the transformed version; and

providing the first baseline packet to the different guest virtual machine.

2. The method as recited in claim 1, further comprising:

receiving, at the third virtualization management component after said providing, a new mapping indicating that the particular guest machine is instantiated at the second virtualization host;

replacing, with the new mapping, at the third virtualization management component, the mapping indicating that the particular guest virtual mapping is instantiated at the first virtualization host; and

determining, at the third virtualization management component, based at least in part on the new mapping, that a second baseline packet is to be extracted from a third encapsulation packet received at the third virtualization management component from the second virtualization host.

3. The method as recited in claim 1, wherein the transformed version of the second encapsulation packet indicates that the first baseline packet was generated at the first virtualization host.

4. The method as recited in claim 1, further comprising:

in response to determining, by the second virtualization management component during the packet duplication phase, that a migrated version of the particular guest virtual machine instantiated at the second virtualization host has generated a second baseline packet directed to a particular destination external to the virtual computing service, transmitting;

(a) a third encapsulation packet comprising the second baseline packet from the second virtualization host to an edge router of the virtual computing service and (b) a fourth encapsulation packet comprising the second baseline packet from the second virtualization host to the first virtualization host;

in response to receiving, at the second virtualization management component, the fourth encapsulation packet, forwarding a transformed version of the fourth encapsulation packet to the edge router of the third virtualization host;

in response to (a) receiving, at an encapsulation intermediary component of the edge router, the third encapsulation packet and (b) determining, at the encapsulation intermediary component, that a network mapping database of the edge router does not include a mapping indicating that the particular guest virtual machine is instantiated at the second virtualization host, discarding the third encapsulation packet by the encapsulation intermediary component; and

in response to (a) receiving, at the encapsulation intermediary component, the transformed version of the fourth encapsulation packet and (b) determining, at the encapsulation intermediary component, that the network mapping database of the edge router includes a mapping indicating that the particular guest virtual machine is instantiated at the first virtualization host, transmitting at least a portion of the second baseline packet to the particular destination.

5. The method as recited in claim 1, further comprising:

assigning, by one or more components of the virtual computing service, to the particular guest virtual machine prior to migration of the particular guest virtual machine, a particular network address selected from a range of network addresses of an isolated virtual network of the virtual computing service, wherein the particular network address is not advertised outside the isolated virtual network, and wherein the particular guest virtual machine retains the particular network address after migration to the second virtualization host.

6. A system, comprising:

a plurality of virtualization hosts of a virtual computing service, including a first virtualization host comprising a first virtualization management component, a second virtualization host comprising a second virtualization management component, and a third virtualization host comprising a third virtualization management component;

wherein the second virtualization management component is configured to;

transmit, during a packet duplication phase of a migration of a particular guest virtual machine from the first virtualization host to the second virtualization host;

(a) a first encapsulation packet to the third virtualization host, wherein the first encapsulation packet comprises a first baseline packet generated by a migrated version of the particular guest virtual machine, and wherein the first baseline packet is directed to a different guest virtual machine instantiated at the third virtualization host and (b) a second encapsulation packet to the first virtualization host, wherein the second encapsulation packet comprises the first baseline packet; and

wherein the third virtualization management component is configured to;

based at least in part on an examination of a particular mapping database comprising one or more virtual-machine-to-host mappings, designate, as a valid encapsulation packet, one of;

(a) the first encapsulation packet, or (b) a forwarded version of the second encapsulation packet transmitted by the first virtualization management component to the third virtualization host; and

provide, to the different guest virtual machine, the first baseline packet after extracting the first baseline packet from the valid encapsulation packet.

7. The system as recited in claim 6, further comprising a control plane component implemented at one or more computing devices of the virtual computing service, wherein the control plane component is configured to:

prior to initiation of the migration of the particular guest virtual machine, transmit, to one or more destinations including the third virtualization management component, a first virtual-machine-to-host mapping indicating that the particular guest virtual machine is instantiated at the first virtualization host; and

after instantiation of the migrated version of the particular guest virtual machine at the second virtualization host, transmit, to the one or more destinations, a second virtual-machine-to-host mapping overriding the first virtual-machine-to-host mapping, wherein the second virtual-machine-to-host mapping indicates that the particular guest virtual machine is instantiated at the second virtualization host.

8. The system as recited in claim 7, wherein the one or more destinations include an edge router of the virtual computing service, wherein the edge router is responsible for (a) routing network packets generated within the virtual computing service towards endpoints outside the virtual computing service and (b) routing network packets generated outside the virtual computing service towards guest virtual machines of the virtual computing service.

9. The system as recited in claim 6, wherein the third virtualization management component is configured to:

in response to determining, after receiving the first encapsulation packet, that the particular mapping database does not include a virtual-machine-to-host mapping indicating that the particular guest virtual machine is instantiated at the second virtualization host, discard the first encapsulation packet as an invalid encapsulation packet.

10. The system as recited in claim 6, wherein the third virtualization management component is configured to:

in response to determining, after receiving the forwarded version of the second encapsulation packet, that the particular mapping database does not include a virtual-machine-to-host mapping indicating that the particular guest virtual machine is instantiated at the first virtualization host, discard the forwarded version as an invalid encapsulation packet.

11. The system as recited in claim 6, wherein the first virtualization management component is configured to:

in response to receiving, during the packet duplication phase, a third encapsulation packet directed to the particular guest virtual machine, transmit a forwarded version of the third encapsulation packet to the second virtualization host.

12. The system as recited in claim 6, wherein the first virtualization management component is configured to:

in response to a detection of a triggering condition, determine that the packet duplication phase is to be terminated; and

initiate a transmission of a message to the second virtualization management component, wherein the message indicates that the packet duplication phase is to be terminated.

13. The system as recited in claim 11, wherein the detection of the triggering condition comprises one or more of:

(a) a detection that the time elapsed since the migrated version of the particular guest virtual machine was instantiated at the second virtualization host exceeds a first threshold, or (b) a determination that a rate at which packets directed to the particular guest virtual machine are received at the first virtualization management component has fallen below a second threshold.

14. The system as recited in claim 6, wherein the second virtualization management component is configured to:

after receiving an indication that the packet duplication phase is to be terminated, transmit a third encapsulation packet to the third virtualization host without transmitting a corresponding encapsulation packet to the first virtualization host, wherein the third encapsulation packet comprises a second baseline packet generated by the migrated version of the particular guest virtual machine, and wherein the second baseline packet is directed to the different guest virtual machine.

15. The system as recited in claim 6, wherein the first virtualization management component is configured to:

in response to a detection of a triggering condition, determine that the packet duplication phase is to be initiated; and

initiate a transmission of a message to the second virtualization management component, wherein the message indicates that the packet duplication phase is to be initiated.

16. The system as recited in claim 14, wherein the detection of the triggering condition comprises one or more of:

(a) a detection that contents of at least a portion of memory allocated to the particular guest virtual machine have been copied from the first virtualization host to the second virtualization host or (b) a detection that a propagation of a particular virtual-machine-to-host mapping to one or more destinations including the third virtualization host has been initiated, wherein the particular virtual-machine-to-host mapping indicates that the particular guest virtual machine is instantiated at the second virtualization host.

17. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors implement a virtualization management component at a particular virtualization host of a virtual computing service, wherein the virtualization management component is configured to:

receive an indication that a packet duplication phase of a migration of a particular guest virtual machine from a first virtualization host to the particular virtualization host has been initiated;

detect that a migrated version of the particular guest virtual machine, instantiated at the particular virtualization host, has generated a first baseline packet directed to a destination address, wherein the destination address is not assigned to another guest virtual machine instantiated at the particular virtualization host; and

transmit;

(a) a first encapsulation packet comprising the first baseline packet to an encapsulation intermediary associated with the destination address and (b) a second encapsulation packet comprising the first baseline packet to the first virtualization host, wherein the second encapsulation packet includes a directive to forward the second encapsulation packet to the encapsulation intermediary from the first virtualization host.

18. The non-transitory computer-accessible storage medium as recited in claim 17, wherein the destination address is assigned to a second guest virtual machine instantiated at a different virtualization host of the virtual computing service, and wherein the encapsulation intermediary comprises a virtualization management component of the different virtualization host.

19. The non-transitory computer-accessible storage medium as recited in claim 17, wherein the virtual computing service is implemented at a provider network, wherein the destination address is assigned to a computing device of a different service of the provider network, and wherein the encapsulation intermediary comprises a component of an edge router of the virtual computing service.

20. The non-transitory computer-accessible storage medium as recited in claim 17, wherein the virtualization management component of the particular virtualization host is configured to:

determine that the particular guest virtual machine is to be migrated to a second virtualization host from the particular virtualization host;

in response to a detection of a first triggering condition, (a) determine that the packet duplication phase for migration of the particular guest virtual machine to the second virtualization host is to be initiated and (b) initiate a notification that the packet duplication phase has been initiated to a virtualization management component of the second virtualization host.