Method and System for Performing Live Migration of Persistent Data of a Virtual Machine

• US 20110066597A1
• Filed: 09/14/2010
• Published: 03/17/2011
• Est. Priority Date: 09/14/2009
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: COLBERT, Osten Kit, MASHTIZADEH, Ali
• Granted Patent: US 8,478,725 B2
• Time in Patent Office: Days
• US Class Current: 707/640
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0664 at device level, e.g. emulation of a storage device or system
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

Persistent data of virtual machines are migrated between and across data stores using a method that provides for a bounded disk size, low performance overhead, arbitrarily low downtimes, atomic switch-over, and ability to build optimizations based around block level heuristics. The method includes the steps of copying the persistent data at the source data store to the destination data store, updating a bitmap data structure during the copying step to indicate which blocks of the persistent data have been modified during the copying step, identifying the blocks that have been modified during the copying step using the bitmap data structure, and copying the identified blocks to the destination data store. The method may further include the step of determining that the number of the identified blocks is below a predetermined threshold and suspending the virtual machine prior to the second copying step.

Claims

1. A method of migrating persistent data of a virtual machine from a source data store to a destination data store, the method comprising:

copying the persistent data at the source data store to the destination data store;

updating a bitmap data structure during said copying to indicate which blocks of the persistent data have been modified during said copying; and

identifying the blocks that have been modified during said copying using the bitmap data structure and copying the identified blocks to the destination data store.

2. The method of claim 1, further comprising determining that the number of the identified blocks is below a predetermined threshold and, prior to copying the identified blocks to the destination data store, suspending the virtual machine.

3. The method of claim 2, further comprising:

instantiating a new instance of the virtual machine; and

when all the persistent data of the virtual machine has been copied to the destination data store, resuming the virtual machine from the new instance of the virtual machine.

4. The method of claim 2, further comprising:

when all the persistent data of the virtual machine has been copied to the destination data store, resuming the virtual machine from the same instance that was suspended.

5. The method of claim 4, wherein said resuming includes switching a file handle of the virtual machine from a first file handle which is associated with the persistent data at the source data store to a second file handle which is associated with the persistent data at the destination data store.

6. The method of claim 1, further comprising maintaining a change block tracking (CBT) filter that monitors IO operations executed by the virtual machine and updates the bitmap data structure in accordance with the IO operations.

7. The method of claim 6, wherein the CBT filter sets the bits in the bitmap data structure corresponding to any blocks that are specified in write IO operations.

8. A non-transitory computer-readable storage medium for storing instructions that, when executed by a computer causes the computer to perform a method of migrating persistent data of a virtual machine from a source data store to a destination data store, the method comprising:

copying the persistent data at the source data store to the destination data store;

updating a bitmap data structure during said copying to indicate which blocks of the persistent data have been modified during said copying; and

identifying the blocks that have been modified during said copying using the bitmap data structure and copying the identified blocks to the destination data store.

9. The non-transitory computer-readable storage medium of claim 8, wherein the method further comprises determining that the number of the identified blocks is below a predetermined threshold and, prior to copying the identified blocks to the destination data store, suspending the virtual machine.

10. The non-transitory computer-readable storage medium of claim 9, wherein the method further comprises:

instantiating a new instance of the virtual machine; and

when all the persistent data of the virtual machine has been copied to the destination data store, resuming the virtual machine from the new instance of the virtual machine.

11. The non-transitory computer-readable storage medium of claim 9, wherein the method further comprises:

when all the persistent data of the virtual machine has been copied to the destination data store, resuming the virtual machine from the same instance that was suspended.

12. The non-transitory computer-readable storage medium of claim 11, wherein said resuming includes switching a file handle of the virtual machine from a first file handle which is associated with the persistent data at the source data store to a second file handle which is associated with the persistent data at the destination data store.

13. The non-transitory computer-readable storage medium of claim 8, wherein the method further comprises maintaining a change block tracking (CBT) filter that monitors IO operations executed by the virtual machine and updates the bitmap data structure in accordance with the IO operations.

14. The non-transitory computer-readable storage medium of claim 13, wherein the CBT filter sets the bits in the bitmap data structure corresponding to any blocks that are specified in write IO operations.

15. A computer system comprising a host including a processor and a memory, a source data store, and a destination data store, wherein the processor is configured to perform a method of migrating persistent data of a virtual machine running on the host from the source data store to the destination data store, the method including the steps of:

copying the persistent data at the source data store to the destination data store;

updating a bitmap data structure during said copying to indicate which blocks of the persistent data have been modified during said copying; and

identifying the blocks that have been modified during said copying using the bitmap data structure and copying the identified blocks to the destination data store.

16. The computer system of claim 15, further comprising:

a change block tracking (CBT) filter that monitors IO operations executed by the virtual machine; and

a bitmap data structure that is updated by the CBT filter in accordance with the IO operations.

17. The computer system of claim 16, wherein the method further comprises determining that the number of the identified blocks is below a predetermined threshold and, prior to copying the identified blocks to the destination data store, suspending the virtual machine.

18. The computer system of claim 17, wherein the method further comprises:

instantiating a new instance of the virtual machine; and

when all the persistent data of the virtual machine has been copied to the destination data store, resuming the virtual machine from the new instance of the virtual machine.

19. The computer system of claim 17, wherein the method further comprises:

when all the persistent data of the virtual machine has been copied to the destination data store, resuming the virtual machine from the same instance that was suspended.

20. The computer system of claim 19, wherein said resuming includes switching a file handle of the virtual machine from a first file handle which is associated with the persistent data at the source data store to a second file handle which is associated with the persistent data at the destination data store.

21. A method for migrating persistent data associated with a virtual machine (VM), the method comprising:

copying static persistent data from a source data store to a target data store, the static persistent data being persistent data associated with the VM that is not accessible by guest software;

copying a virtual disk associated with the VM from the source data store to the target data store, wherein the persistent data includes the virtual disk;

monitoring input and output (I/O) disk operations during the copying of the virtual disk, the monitoring including maintaining a data structure identifying each disk block written to by the VM during the copying of the virtual disk, wherein disk blocks that are written to are dirty disk blocks;

iteratively copying dirty disk blocks until a set of dirty blocks is smaller than a threshold number of disk blocks; and

after the set of dirty blocks is smaller than the threshold number of disk blocks, stunning the VM, copying remaining dirty disk blocks to the target data store, and when all the persistent data of the VM has been copied to the target data store, resuming execution of the VM using persistent data stored on the target data store.

22. The method of claim 21, further comprising:

creating a new VM using the copied static persistent data; and

modifying the copied static persistent data so that all file handles of the new VM identifying persistent data on the source data store are modified to identify corresponding persistent data on the target data store,

wherein after said stunning and before said resuming, the state of the VM is copied to the new VM, and upon said resuing, the new VM continues execution using persistent data stored on the target data store.

23. A method for migrating a disk image from a source data store to a destination data store, the method comprising:

copying the disk image from the source data store to the destination data store;

monitoring input and output (I/O) operations issued to the disk image during said copying, the monitoring including maintaining a data structure identifying blocks that are written to in accordance with the I/O operations during said copying, wherein the blocks that are written to are dirty blocks;

copying the dirty blocks as identified in the data structure from the source data store to the destination data store and, during said copying of the dirty blocks, identifying additional blocks that are written to, wherein the additional blocks that are written to constitute a new set of dirty blocks;

repeating the prior copying until a set of dirty blocks is smaller than a threshold number of blocks; and

after the set of dirty blocks is smaller than the threshold number of blocks, suspending I/O operations to the disk image, copying remaining dirty blocks to the destination data store, and resuming I/O operations to the disk image stored on the destination data store.

24. The method of claim 23, wherein the disk image is persistent data associated with a virtual machine.

25. The method of claim 23, wherein, when the I/O operations to the disk image are resumed, the disk image stored on the source data store has been completely copied over to the disk image stored on the destination data store.