Storing checkpoint file in high performance storage device for rapid virtual machine suspend and resume

• US 9,804,798 B2
• Filed: 02/11/2013
• Issued: 10/31/2017
• Est. Priority Date: 12/14/2012
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Beveridge, Daniel James
• Primary Examiner(s): KIM, HONG CHONG
• Application Number:
  US13/764,405
• Publication Number:
  US 20140173181A1
• Time in Patent Office: 1,723 Days
• Field of Search: 711134, 711135, 711144, 711145, 711130, 711 6, 714 13
• CPC Class Codes: G06F 11/14 Error detection or correction of the data by redundancy in operation G06F11/16 takes precedence
• CPC Class Codes: G06F 11/1438 Restarting or rejuvenating
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/084 with a shared cache
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 12/126 with special data handling, e.g. priority of data or instructions, handling errors or pinning
• CPC Class Codes: G06F 2009/45575 Starting, stopping, suspending or resuming virtual machine instances
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 2212/152 Virtualized environment, e.g. logically partitioned system
• CPC Class Codes: G06F 2212/217 Hybrid disk, e.g. using both magnetic and solid state storage devices
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0655 Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G06F 9/00 Arrangements for program control, e.g. control units program control for peripheral devices G06F13/10
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/4856 resumption being on a different machine, e.g. task migration, virtual machine migration G06F9/5088 takes precedence
• CPC Class Codes: G06F 9/5088 involving task migration

Abstract

A method of enabling “fast” suspend and “rapid” resume of virtual machines (VMs) employs a cache that is able to perform input/output operations at a faster rate than a storage device provisioned for the VMs. The cache may be local to a computer system that is hosting the VMs or may be shared cache commonly accessible to VMs hosted by different computer systems. The method includes the steps of saving the state of the VM to a checkpoint file stored in the cache and locking the checkpoint file so that data blocks of the checkpoint file are maintained in the cache and are not evicted, and resuming execution of the VM by reading into memory the data blocks of the checkpoint file stored in the cache.

Claims

1. A method of saving a state of a virtual machine (VM) running in a host computer to a checkpoint file, wherein the host computer includes system memory and stores files in a first storage device and a second storage device that is different from the system memory and has lower input/output latency and higher input/output throughput than the first storage device, said method comprising:

comparing a size of the VM state and amount of free space in the second storage device;

if the size of the VM state is greater than the amount of free space in the second storage device, evicting data blocks of other files stored in the second storage device to the first storage device and then saving the VM state to a checkpoint file stored in the second storage device;

if the size of the VM state is not greater than the amount of free space in the second storage device, saving the VM state to the checkpoint file stored in the second storage device without evicting data blocks of other files stored in the second storage device to the first storage device; and

locking the checkpoint file so that data blocks of the checkpoint file are maintained in the second storage device and are not evicted to the first storage device,

wherein a file system is implemented in the second storage device and the checkpoint file containing the VM state is stored in the second storage device as one of the files of the file system.

2. The method of claim 1, wherein the first storage device includes rotating disks and the second storage device does not include rotating disks.

3. The method of claim 2, wherein the first storage device is a disk-based storage array and the second storage device is a solid state drive (SSD).

4. The method of claim 3, wherein the SSD is configured in the host computer.

5. The method of claim 3, wherein the SSD is configured in a shared SSD array that is accessible by the host computer and the other computers.

6. A non-transitory computer readable medium comprising instructions for causing a host computer to carry out a method of saving a state of a virtual machine (VM) running in the host computer to a checkpoint file, wherein the host computer includes system memory and stores files in a first storage device and a second storage device that is different from the system memory and has lower input/output latency and higher input/output throughput than the first storage device, said method comprising:

comparing a size of the VM state and amount of free space in the second storage device;

if the size of the VM state is greater than the amount of free space in the second storage device, evicting data blocks of other files stored in the second storage device to the first storage device and then saving the VM state to a checkpoint file stored in the second storage device;

if the size of the VM state is not greater than the amount of free space in the second storage device, saving the VM state to the checkpoint file stored in the second storage device without evicting data blocks of other files stored in the second storage device to the first storage device; and

locking the checkpoint file so that data blocks of the checkpoint file are maintained in the second storage device and are not evicted to the first storage device,

wherein a file system is implemented in the second storage device and the checkpoint file containing the VM state is stored in the second storage device as one of the files of the file system.

7. The non-transitory computer readable medium of claim 6, wherein the first storage device includes rotating disks and the second storage device does not include rotating disks.

8. The non-transitory computer readable medium of claim 7, wherein the first storage device is a disk-based storage array and the second storage device is a solid state drive (SSD).

9. The non-transitory computer readable medium of claim 8, wherein the SSD is configured in the host computer.

10. The non-transitory computer readable medium of claim 8, wherein the SSD is configured in a shared SSD array that is accessible by the host computer and the other computers.

11. A virtualized computer system comprising:

a plurality of host computers each having system memory and one or more virtual machines (VMs) running therein;

a first storage device accessible by the host computers; and

a second storage device accessible by the host computers that is different from the system memory and has lower input/output latency and higher input/output throughput than the first storage device, wherein

each of the host computers is configured to carry out steps of saving a state of a VM running therein to a checkpoint file and resuming execution of the VM using the checkpoint file, the steps including;

comparing a size of the VM state and amount of free space in the second storage device,

if the size of the VM state is greater than the amount of free space in the second storage device, evicting data blocks of other files stored in the second storage device to the first storage device and then saving the VM state to a checkpoint file stored in the second storage device,

if the size of the VM state is not greater than the amount of free space in the second storage device, saving the VM state to the checkpoint file stored in the second storage device without evicting data blocks of other files stored in the second storage device to the first storage device, and

locking the checkpoint file so that data blocks of the checkpoint file are maintained in the second storage device and are not evicted to the first storage device,

wherein a file system is implemented in the second storage device and the checkpoint file containing the VM state is stored in the second storage device as one of the files of the file system.

12. The virtualized computer system of claim 11, wherein the first storage device includes rotating disks and the second storage device does not include rotating disks.

13. The virtualized computer system of claim 12, wherein the first storage device is a disk-based storage array and the second storage device is a solid state drive (SSD).

14. The virtualized computer system of claim 13, wherein the SSD is configured in the host computer.

15. The virtualized computer system of claim 13, wherein the SSD is configured in a shared SSD array that is accessible by the host computer and the other computers.

16. The virtualized computer system of claim 11, wherein each of the host computers has a local cache and connected to a shared cache and the steps further include:

determining whether a suspended image of the VM is stored in the shared cache;

if the suspended image of the VM is stored in the shared cache, selecting one of the host computers as a host computer for the VM and issuing a command to the selected host computer to resume execution of the VM therein; and

if the suspended image of the VM is not stored in the shared cache, issuing a command to resume execution of the VM to one of the host computers that has the local cache in which the suspended image of the VM is stored to resume execution of the VM therein.

17. The virtualized computer system of claim 16, wherein one of the host computers is selected as the host computer according to resource availability.

18. The virtualized computer system of claim 16, wherein the steps further include:

if the suspended image of the VM is stored in the shared cache, increasing available capacity of the shared cache by the size of the suspended image of the VM after the execution of the VM has been resumed; and

if the suspended image of the VM is not stored in the shared cache, increasing available capacity of the local cache by the size of the suspended image of the VM after the execution of the VM has been resumed.

19. The virtualized computer system of claim 16, wherein the steps further include:

if the suspended image of the VM is not stored in the shared cache, after execution of the VM is resumed in one of the host computers, migrating the VM to another one of the host computers for execution therein.

20. The virtualized computer system of claim 16, wherein the local caches are solid state drives (SSDs) and the shared cache is an SSD array.