Post-copy based live virtual machine migration via speculative execution and pre-paging

• US 10,481,940 B2
• Filed: 12/20/2017
• Issued: 11/19/2019
• Est. Priority Date: 06/10/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Srinivas, Venkatesh
• Primary Examiner(s): Wu, Benjamin C
• Application Number:
  US15/849,505
• Publication Number:
  US 20180136963A1
• Time in Patent Office: 699 Days
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

The method includes post copy migrating a guest virtual machine from a migration source host to a migration target host, executing the instruction stream of the guest virtual machine on the migration target host, and determining, by the migration target host, when the guest virtual machine encounters the page fault. When the guest virtual machine encounters the page fault, the method includes requesting, by the migration target host, the not-present page from the migration source host for loading on the migration target host, and emulating, on an emulator executing on or in communication with the migration target host, continued execution of the instruction stream of the guest virtual machine. The method also includes identifying, by the emulator, future page references to future pages from the emulated continued execution of the instruction stream, and requesting the future pages from the migration source host in parallel with the not-present page request.

Claims

1. A method comprising:

determining, by a migration target host, that a guest virtual machine being post-copy migrated from a migration source host to the migration target host and executed on the migration target host encounters a page fault for a page reference in an instruction stream of the guest virtual machine to a not-present page;

requesting, by the migration target host, the not-present page from the migration source host for loading on the migration target host;

emulating, on an emulator executing on or in communication with the migration target host, continued execution of the instruction stream of the guest virtual machine;

identifying, by the emulator, future page references to future pages from the emulated continued execution of the instruction stream;

examining, by the emulator, at least one instruction in the instruction stream associated with one of the identified future page references to identify at least one additional future page reference; and

requesting, by the migration target host, the future pages corresponding to the identified future page references and the at least one identified additional future page reference from the migration source host and the not-present page request for the not-present page.

2. The method of claim 1, wherein requesting the future pages corresponding to the identified future page references from the migration source host is in parallel with requesting the not-present page request for the not-present page, so as to prefetch the future pages to the migration target host while loading the not-present page on the migration target host.

3. The method of claim 1, further comprising, while emulating the continued execution of the instruction stream of the guest virtual machine, setting aside instructions that are dependent on loaded values of the not-present page, allowing the emulator to identify future page references beyond future page faults.

4. The method of claim 1, wherein the migration target host comprises:

one or more computing devices; and

one or more memory hosts in communication with the one or more computing devices, the one or more memory hosts storing pages.

5. The method of claim 1, further comprising identifying the future page references based on a relationship between the future page references and the not-present page.

6. The method of claim 5, where the relationship comprises a page navigation map between a parent page and one or more child pages.

7. The method of claim 1, further comprising ceasing emulation of the continued execution of the instruction stream by the emulator when the not-present page has been delivered to the migration target host.

8. The method of claim 7, further comprising continuing execution of the instruction stream of the guest virtual machine on the migration target host when the not-present page is resident on the migration target host.

9. The method of claim 1, wherein identifying future page references from the emulated continued execution of the instruction stream further comprises identifying a future page address of the identified future page references.

10. A migration target host comprising:

data processing hardware; and

memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising;

determining that a guest virtual machine being post-copy migrated from a migration source host to the memory hardware and executed on the data processing hardware encounters a page fault for a page reference in an instruction stream of the guest virtual machine to a not-present page;

requesting the not-present page from the migration source host for loading on the memory hardware;

emulating, on an emulator executing on the data processing hardware or in communication with the migration target host, continued execution of the instruction stream of the guest virtual machine;

identifying, by the emulator, future page references to future pages from the emulated continued execution of the instruction stream;

examining, by the emulator, at least one instruction in the instruction stream associated with one of the identified future page references to identify at least one additional future page reference; and

requesting the future pages corresponding to the identified future page references and the at least one identified additional future page reference from the migration source host and the not-present page request for the not-present page.

11. The migration target host of claim 10, wherein requesting the future pages corresponding to the identified future page references from the migration source host is in parallel with requesting the not-present page request for the not-present page, so as to prefetch the future pages to the migration target host while loading the not-present page on the migration target host.

12. The migration target host of claim 10, wherein the operations further comprise, while emulating the continued execution of the instruction stream of the guest virtual machine, setting aside instructions that are dependent on loaded values of the not-present page, allowing the emulator to identify future page references beyond future page faults.

13. The migration target host of claim 10, wherein the operations further comprise requesting the identified future page references from the migration source host as the future page references are identified.

14. The migration target host of claim 10, wherein the operations further comprise identifying the future page references based on a relationship between the future page reference and the not-present page.

15. The migration target host of claim 14, where the relationship comprises a page navigation map between a parent page and one or more child pages.

16. The migration target host of claim 10, wherein the operations further comprise ceasing emulation of the continued execution of the instruction stream by the emulator when the not-present page has been delivered to the memory hardware of the migration target host.

17. The migration target host of claim 16, wherein the operations further comprise continuing execution of the instruction stream of the guest virtual machine when the not-present page is resident on the memory hardware of the migration target host.

18. The migration target host of claim 10, wherein identifying future page references from the emulated continued execution of the instruction stream further comprises identifying a future page address of the identified future page references.