Generating and Using Checkpoints in a Virtual Computer System

• US 20100122052A1
• Filed: 05/04/2009
• Published: 05/13/2010
• Est. Priority Date: 12/31/2003
• Status: Active Grant

• Current Assignee: VMware, Inc. (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: SUBRAHMANYAM, Pratap, WALDSPURGER, Carl A., SCALES, Daniel J., NELSON, Michael
• Granted Patent: US 7,971,015 B2
• Time in Patent Office: Days
• US Class Current: 711/162
• CPC Class Codes: F16L 53/32 using hot fluids
• CPC Class Codes: F24H 1/142 using electric energy supply
• CPC Class Codes: G06F 11/1407 Checkpointing the instruction stream
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1438 Restarting or rejuvenating
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1482 by means of middleware or OS functionality
• CPC Class Codes: G06F 11/1484 involving virtual machines
• CPC Class Codes: G06F 12/16 Protection against loss of memory contents contains no material, see G06F11/00
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

To generate a checkpoint for a virtual machine (VM), first, while the VM is still running, a copy-on-write (COW) disk file is created pointing to a parent disk file that the VM is using. Next, the VM is stopped, the VM'"'"' s memory is marked COW, the device state of the VM is saved to memory, the VM is switched to use the COW disk file, and the VM begins running again for substantially the remainder of the checkpoint generation. Next, the device state that was stored in memory and the unmodified VM memory pages are saved to a checkpoint file. Also, a copy may be made of the parent disk file for retention as part of the checkpoint, or the original parent disk file may be retained as part of the checkpoint. If a copy of the parent disk file was made, then the COW disk file may be committed to the original parent disk file.

Claims

1. A computer program product embodied in a tangible, computer-readable medium, the computer program product performing a method for generating a checkpoint for a virtual machine (VM) in a virtual computer system, the VM using a parent disk file and a set of VM memory, the method comprising:

creating a copy-on-write (COW) disk file pointing to the parent disk file in use by the VM;

stopping the VM, and while the VM is stopped;

marking the memory of the VM copy-on-write, the VM memory constituting original VM memory;

saving substantially all of the device state of the VM, including an instruction pointer, a plurality of registers and settings for one or more virtual devices, to memory; and

switching the VM to read from and write to the COW disk file instead of the parent disk file;

resuming operation of the VM, so that the VM reads from and writes to the COW disk file and so that the VM reads from, writes to, and executes code from the VM memory;

handling disk COW faults to the COW disk file;

handling memory COW faults to the original VM memory to generate copies of the original VM memory for read, write, and execution use by the VM;

saving the device state from memory to a checkpoint data store; and

saving the original VM memory to the checkpoint data store,

wherein the VM memory is a subset of a physical memory in the virtual computer system, the VM memory being a portion of the physical memory that is allocated to the VM.

2. The computer program product of claim 1, wherein the VM is still running when the COW disk file is created.

3. The computer program product of claim 1, the method further comprising copying the parent disk file after any pending disk writes complete, and using the copy of the parent disk file for the checkpoint.

4. The computer program product of claim 3, the method further comprising committing the COW disk file into the original parent disk file.

5. The computer program product of claim 4, wherein the step of committing the COW disk file into the original parent disk file comprises creating one or more new COW disk files for use by the VM while the COW disk file previously used by the VM is being committed.

6. The computer program product of claim 3, wherein the copy of the parent disk file is indicated for use for the checkpoint by adding a disk file pointer to the checkpoint file.

7. The computer program product of claim 1, wherein the steps of creating the COW disk file and handling disk COW faults are performed by a data storage device that is external to the virtual computer system.

8. The computer program product of claim 1, wherein the step of saving the original VM memory to the checkpoint data store is delayed until all pending disk reads complete.

9. The computer program product of claim 1, wherein the checkpoint data store comprises raw data stored in a data storage medium.

10. The computer program product of claim 1, wherein the checkpoint data store comprises a file stored in a data storage medium.

11. The computer program product of claim 10, wherein the data storage medium comprises a disk drive.

12. The computer program product of claim 10, wherein the data storage medium comprises memory.

13. The computer program product of claim 12, wherein the memory comprises high-speed random access memory.

14. The computer program product of claim 12, wherein the memory comprises flash memory.

15. The computer program product of claim 1, the method further comprising, after the operation of the VM is resumed, forcing memory COW faults on original VM memory that is affected by any new disk reads, prior to issuing the new disk reads.

16. The computer program product of claim 1, wherein, if there is a pending read followed by a pending write to the same disk block, the resumption of the operation of the VM is delayed until all pending disk operations complete.

17. The computer program product of claim 16, the method further comprising, if the resumption of the operation of the VM is not delayed until all pending disk operations complete, reissuing any disk reads that affect any VM memory for which a COW fault has occurred.

18. The computer program product of claim 1, wherein, for the step of handling disk COW faults to the COW disk file, if there is a pending disk write to the same COW block group for which the disk COW fault has occurred, the handling of the disk COW fault and the write that caused the disk COW fault are delayed until the pending disk write completes.

19. A computer program product embodied in a tangible, computer-readable medium, the computer program product performing a method for generating a checkpoint for a virtual machine (VM) in a virtual computer system, the VM using a virtual disk and a set of VM memory, the method comprising:

maintaining, in an unmodified state, the contents of the virtual disk at the time for which the checkpoint is generated, while allowing the VM to continue reading from and writing to the virtual disk;

saving substantially all of the device state of the VM, including an instruction pointer, a plurality of registers and settings for one or more virtual devices, at the time for which the checkpoint is generated, to a checkpoint data store; and

saving the set of VM memory, at the time for which the checkpoint is generated, to the checkpoint data store by performing the following steps;

marking the set of VM memory as copy-on-write (COW), the set of VM memory constituting original VM memory;

allowing the VM to continue reading from, writing to, and executing code from the VM memory;

responding to memory COW faults related to the VM memory by generating copies of the original VM memory for read, write and execution use by the VM; and

saving the original VM memory to the checkpoint data store,

wherein the VM memory is a subset of a physical memory in the virtual computer system, the VM memory being a portion of the physical memory that is allocated to the VM.

20. The computer program product of claim 19, wherein the execution of the VM is stopped while the VM memory is marked COW.

21. The computer program product of claim 19, wherein the virtual disk is initially mapped to a parent disk file on a physical disk and wherein the step of maintaining, in an unmodified state, the contents of the virtual disk comprises creating a copy-on-write (COW) disk file pointing to the parent disk file and mapping the virtual disk to the COW disk file.

22. The computer program product of claim 19, wherein the step of saving the device state of the VM to the checkpoint data store comprises saving the device state to memory while the VM is not executing and copying the device state from memory to the checkpoint data store after the VM has resumed execution.

23. A computer program product embodied in a tangible, computer-readable medium, the computer program product performing a method for generating a checkpoint for a virtual machine (VM) in a virtual computer system, the VM using a virtual disk and a set of VM memory, the method comprising:

maintaining, in an unmodified state, the contents of the virtual disk at the time for which the checkpoint is generated, while allowing the VM to continue reading from and writing to the virtual disk, the unmodified contents of the virtual disk constituting a checkpointed virtual disk, and the contents of the virtual disk used by the VM constituting an ongoing virtual disk;

saving substantially all of the device state of the VM, including an instruction pointer, a plurality of registers and settings for one or more virtual devices, at the time for which the checkpoint is generated, to a checkpoint data store;

saving the contents of the VM memory, at the time for which the checkpoint is generated, to the checkpoint data store, and allowing the VM to continue reading from, writing to, and executing code from the VM memory, the contents of the VM memory saved to the checkpoint data store constituting a checkpointed VM memory, and the contents of the VM memory as used by the VM constituting an ongoing VM memory; and

allowing the VM to execute, including executing code from the VM memory, during at least a part of the time during which the checkpoint is being generated, and ensuring that the results of any pending disk writes are applied to both the checkpointed virtual disk and the ongoing virtual disk, that the results of any new disk writes are applied to the ongoing virtual disk, but not to the checkpointed virtual disk, that the results of any pending disk reads are applied to both the checkpointed VM memory and the ongoing VM memory, and that the results of any new disk reads are applied to the ongoing VM memory, but not to the checkpointed VM memory

wherein the VM memory is a subset of a physical memory in the virtual computer system, the VM memory being a portion of the physical memory that is allocated to the VM.

24. The computer program product of claim 23, wherein the step of saving the contents of the VM memory to the checkpoint data store comprises:

marking the set of VM memory as copy-on-write (COW), the set of VM memory constituting original VM memory;

allowing the VM to continue reading from and writing to the VM memory;

responding to memory COW faults related to the VM memory by generating copies of the original VM memory for read and write use by the VM; and

saving the original VM memory to the checkpoint data store.

25. The computer program product of claim 24, wherein the step of ensuring that the results of any pending disk reads are applied to the ongoing VM memory comprises reissuing any pending disk reads for which the results of the read were directed to original VM memory for which a COW fault has occurred, but directing the reissued disk reads to the corresponding copies of the original VM memory instead of the original VM memory.

26. The computer program product of claim 24, wherein the step of ensuring that the results of any new disk reads are not applied to the checkpointed VM memory comprises forcing COW faults for any original VM memory that would otherwise be affected by the new disk reads.

27. The computer program product of claim 23, wherein the virtual disk is initially mapped to a parent disk file on a physical disk and wherein the step of maintaining, in an unmodified state, the contents of the virtual disk comprises creating a copy-on-write (COW) disk file pointing to the parent disk file and mapping the virtual disk to the COW disk file.

28. The computer program product of claim 27, wherein the step of ensuring that the results of any pending disk writes are applied to the ongoing virtual disk comprises, if there is a pending disk write to the same COW block group as a subsequent write for which a disk COW fault has occurred, delaying responding to the disk COW fault and delaying the subsequent write until the pending disk write completes.