System and method for live migration of guest

• US 9,436,751 B1
• Filed: 12/18/2013
• Issued: 09/06/2016
• Est. Priority Date: 12/18/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Serebrin, Benjamin Charles
• Primary Examiner(s): Cao, Phuong Thao
• Application Number:
  US14/132,477
• Time in Patent Office: 993 Days
• Field of Search: 707/624
• US Class Current: 1/1
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/4856 resumption being on a different machine, e.g. task migration, virtual machine migration G06F9/5088 takes precedence

Abstract

A method and apparatus are provided to perform live migration of a guest in a computer system using device assignment. In this method and apparatus, one or more pages of the guest are copied to a target computer system. It is determined which pages have been copied, and what incremental changes have been made to the pages that were copied. For example, the incremental changes may be indicated to a hypervisor of an initial host of the guest by a network interface controller or other device in the computer system. The incremental changes are then copied to the target computer system. Detection and copying of incremental changes may continue until a time when all dirty pages can be copied to the target computer system.

Claims

1. A method for live migration of a guest in a computer system using device assignment, the method comprising:

copying one or more pages of the guest from an initial computer system to a target computer system, the initial computer system having one or more processors coupled to a network interface controller (NIC) through an input/output memory management unit (IOMMU);

determining, using the one or more processors, which pages have been copied;

determining, using the one or more processors, incremental changes made to the pages that have been copied, the determining including accessing page table entries in the IOMMU, wherein the page table entries were modified by the NIC; and

copying the incremental changes from the initial computer system to the target computer system.

2. The method of claim 1, wherein accessing information provided by the NIC comprises accessing a list of dirtied pages tracked by the NIC.

3. The method of claim 1, wherein accessing information provided by the NIC comprises accessing a buffer of bytes, the buffer comprising a plurality of entries with each entry corresponding to at least a portion of a page of the guest, wherein the buffer is updated by the NIC when the NIC writes to the page of the guest.

4. The method of claim 1, wherein determining the incremental changes comprises receiving information from the guest indicating the pages for which the guest provided write access.

5. The method of claim 1, wherein determining which pages have been copied comprises marking the copied one or more pages of the guest as one of not dirty and not writeable.

6. The method of claim 1, further comprising determining whether a rate at which pages are dirtied approximates a rate at which dirtied pages can be copied to the target computer system.

7. The method of claim 6, further comprising continuing to determine and copy incremental changes if the rate at which pages are dirtied does not approximate a rate at which dirtied pages can be copied.

8. The method of claim 6, further comprising copying all dirtied pages and resuming processing on the target computer device if the rate at which pages are dirtied approximates a rate at which dirtied pages can be copied.

9. An apparatus for live migration of a guest in a computer system using device assignment, the apparatus comprising:

a memory;

one or more processors in communication with the memory, the one or more processors coupled to a network interface controller (NIC) through an input/output memory management unit (IOMMU), the one or more processors performing tasks comprising;

copying one or more pages of the guest to a target computer system;

determining which pages have been copied;

determining incremental changes made to the pages that have been copied, the determining including accessing a buffer of bytes, the buffer comprising a plurality of entries with each entry corresponding to at least a portion of a page of the guest, wherein the buffer is updated by the NIC when the NIC writes to the page of the guest; and

copying the incremental changes to the target computer system.

10. The apparatus of claim 9, wherein accessing information provided by the NIC comprises accessing a list of dirtied pages tracked by the NIC.

11. The apparatus of claim 9, wherein accessing information provided by the NIC comprises accessing page table entries in the input/output memory management unit, wherein the page table entries were modified by the NIC.

12. The apparatus of claim 9, wherein determining the incremental changes comprises receiving information from the guest indicating the pages for which the guest provided write access.

13. The apparatus of claim 9, further comprising determining whether a rate at which pages are dirtied approximates a rate at which dirtied pages can be copied to the target computer system.

14. The apparatus of claim 13, further comprising copying all dirtied pages and resuming processing on the target computer system if the rate at which pages are dirtied approximates a rate at which dirtied pages can be copied.

15. A method for live migration of a guest in a computer system using device assignment, the method comprising:

copying one or more pages of the guest from an initial computer system to a target computer system, the initial computer system having one or more processors coupled to a network interface controller (NIC) through an input/output memory management unit (IOMMU);

receiving at the one or more processors information from the guest regarding which of the one or more pages it intends to give write access to the NIC;

determining, using the one or more processors, which pages have been copied;

determining, by the NIC, incremental changes made to the pages that have been copied based on the information received from the guest; and

copying, using the one or more processors, the incremental changes from the initial computer system to the target computer system.

16. A system for live migration of a guest in a computer system using device assignment, comprising:

a memory;

one or more processors in communication with the memory, the one or more processors coupled to a network interface controller (NIC) through an input/output memory management unit (IOMMU), the one or more processors performing tasks comprising;

copying one or more pages of the guest to a target computer system;

determining which pages have been copied;

determining incremental changes made to the pages that have been copied, the determining including accessing page table entries in the IOMMU, wherein the page table entries were modified by the NIC; and

copying the incremental changes to the target computer system.

17. A system for live migration of a guest in a computer system using device assignment, comprising:

a memory;

one or more processors in communication with the memory, the one or more processors coupled to a network interface controller (NIC) through an input/output memory management unit (IOMMU), the one or more processors performing tasks comprising;

copying one or more pages of the guest to a target computer system;

receiving information from the guest regarding which of the one or more pages it intends to give write access to the NIC;

determining which pages have been copied;

determining, with the NIC, incremental changes made to the pages that have been copied based on the information received from the guest; and

copying, using the one or more processors, the incremental changes to the target computer system.