Method and System for Frequent Checkpointing

• US 20110167196A1
• Filed: 01/06/2010
• Published: 07/07/2011
• Est. Priority Date: 01/06/2010
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: VENKITACHALAM, Ganesh, SUBRAHMANYAM, Pratap, SCALES, Daniel J., NELSON, Michael
• Granted Patent: US 8,549,241 B2
• Time in Patent Office: Days
• US Class Current: 711/6
• CPC Class Codes: G06F 11/1438 Restarting or rejuvenating
• CPC Class Codes: G06F 11/1484 involving virtual machines
• CPC Class Codes: G06F 11/2023 Failover techniques
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

A virtualization platform provides fault tolerance for a primary virtual machine by continuously transmitting checkpoint information of the primary virtual machine to a collector process, such as a backup virtual machine. When implemented on a hardware platform comprising a multi-processor that supports nested page tables, the virtualization platform leverages the nested page table support to quickly identify memory pages that have been modified between checkpoints. The backup virtual machine provides feedback information to assist the virtualization platform in identifying candidate memory pages for transmitting actual modifications to the memory pages rather than the entire memory page as part of the checkpoint information. The virtualization platform further maintains a modification history data structure to identify memory pages that can be transmitted simultaneous with the execution of the primary virtual machine rather than while the primary virtual machine has been stunned.

Claims

1. A method for transmitting modifications made to a memory page of a virtual machine during a current time interval, the method comprising:

maintaining a data structure comprising an entry corresponding to the memory page that indicates a history of modifications made to the memory page in previous time intervals;

stunning the virtual machine at the end of the current time interval;

confirming that the entry in the data structure satisfies a heuristic indicating a level of absence of modifications to the memory page in previous time intervals;

resuming execution of the virtual machine; and

transmitting at least a portion of the memory page to a backup computer system while the virtual machine is executing.

2. The method of claim 1, further comprising the step of updating the data structure to indicate that the memory page was modified during the current time interval.

3. The method of claim 1, further comprising the steps of setting page protection in a page table entry of a page table that corresponds to the memory page prior to resuming execution of the virtual machine, copying a state of the memory page upon an attempt by the virtual machine to write to the memory page after resuming execution of the virtual machine, permitting the virtual machine to write to the memory page after performing the copying, and transmitting the copied state of the memory page to the backup computer system.

4. The method of claim 2, further comprising the steps of setting page protection in a page table entry of a page table that corresponds to the memory page prior to resuming execution of the virtual machine and remove the page table protection after the transmitting step.

5. The method of claim 1, further comprising the step of identifying differences between a state of the memory page at the start of the current time interval and a state of the memory page at the end of the current time interval, wherein the transmitted portion of the memory page comprises the identified differences.

6. The method of claim 1, wherein a page table entry corresponding to the memory page in a nested page table tables that maps guest physical addresses referenced by a guest operating system of the virtual machine to system physical addresses in a system memory of a computer system running the virtual machine has been marked by a processor of the computer system to indicate a successful write operation to the memory page during the current time interval.

7. The method of claim 6, wherein the heuristic measures the absence of modifications for a pre-determined number of consecutive previous time intervals.

8. A computer-readable storage medium including instructions that, when executed by a processor of a computer system, causes the processor to transmit modifications made to a memory page of a virtual machine running on the computer system during a time interval by performing the steps of:

maintaining a data structure comprising an entry corresponding to the memory page that indicates a history of modifications made to the memory page in previous time intervals;

stunning the virtual machine at the end of the current time interval;

confirming that the entry in the data structure satisfies a heuristic indicating a level of absence of modifications to the memory page in previous time intervals;

resuming execution of the virtual machine; and

transmitting at least a portion of the memory page to a backup computer system while the virtual machine is executing.

9. The computer-readable storage medium of claim 8, further including instructions that, when executed by a processor, perform the step of updating the data structure to indicate that the memory page was modified during the current time interval.

10. The computer-readable storage medium of claim 8, further including instructions that, when executed by a processor, perform the steps of setting page protection in a page table entry of a page table that corresponds to the memory page prior to resuming execution of the virtual machine, copying a state of the memory page upon an attempt by the virtual machine to write to the memory page after resuming execution of the virtual machine, permitting the virtual machine to write to the memory page after performing the copying, and transmitting the copied state of the memory page to the backup computer system.

11. The computer-readable storage medium of claim 8, further including instructions that, when executed by the processor, perform the steps of setting page protection in a page table entry of a page table that corresponds to the memory page prior to resuming execution of the virtual machine and remove the page table protection after the transmitting step.

12. The computer-readable storage medium of claim 8, further including instructions that, when executed by a processor, perform the step of identifying differences between a state of the memory page at the start of the current time interval and a state of the memory page at the end of the current time interval, wherein the transmitted portion of the memory page comprises the identified differences.

13. The computer-readable storage medium of claim 8, wherein a page table entry corresponding to the memory page in a nested page table tables that maps guest physical addresses referenced by a guest operating system of the virtual machine to system physical addresses in a system memory of the computer system running the virtual machine has been marked by the processor of the computer system to indicate a successful write operation to the memory page during the current time interval.

14. The computer-readable storage medium of claim 8, wherein the heuristic measures the absence of modifications for a pre-determined number of consecutive previous time intervals.

15. A computer system configured to periodically provide updated state information of a primary virtual machine running on the computer system to a collector process running on a backup computer system, the computer system comprising:

a system memory comprising a hypervisor component configured to instantiate the primary virtual machine; and

a multi-core processor having at least one processing unit is configured to execute the hypervisor component to transmit modifications made to a memory page of the primary virtual machine during a time interval by performing the steps of;

maintaining a data structure comprising an entry corresponding to the memory page that indicates a history of modifications made to the memory page in previous time intervals;

stunning the virtual machine at the end of the current time interval;

confirming that the entry in the data structure satisfies a heuristic indicating a level of absence of modifications to the memory page in previous time intervals;

resuming execution of the virtual machine; and

transmitting at least a portion of the memory page to a backup computer system while the virtual machine is executing.

16. The computer system of claim 15, wherein the hypervisor component further comprises instructions to perform the step of updating the data structure to indicate that the memory page was modified during the current time interval.

17. The computer system of claim 15, wherein the hypervisor component further comprises instructions to perform the steps of setting page protection in a page table entry of a page table that corresponds to the memory page prior to resuming execution of the virtual machine, copying a state of the memory page upon an attempt by the primary virtual machine to write to the memory page after resuming execution of the virtual machine, permitting the primary virtual machine to write to the memory page after performing the copying, and transmitting the copied state of the memory page to the backup computer system.

18. The computer system of claim 15, wherein the hypervisor component further comprises instructions to perform the steps of setting page protection in a page table entry of a page table that corresponds to the memory page prior to resuming execution of the primary virtual machine and remove the page table protection after the transmitting step.

19. The computer system of claim 15, wherein a page table entry corresponding to the memory page in a nested page table tables that maps guest physical addresses referenced by a guest operating system of the primary virtual machine to system physical addresses in a system memory of the computer system running the primary virtual machine has been marked by the processor of the computer system to indicate a successful write operation to the memory page during the current time interval.

20. The computer system of claim 15, wherein the heuristic measures the absence of modifications for a pre-determined number of consecutive previous time intervals.