VIRTUAL DISK REPLICATION USING LOG FILES

• US 20130024722A1
• Filed: 07/22/2011
• Published: 01/24/2013
• Est. Priority Date: 07/22/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kotagiri, Sriravi, Newaskar, Rahul Shrikant, Kar, Palash, Dubey, Shreesh Rajendra
• Granted Patent: US 8,689,047 B2
• Time in Patent Office: Days
• US Class Current: 714/6.1
• CPC Class Codes: G06F 11/1004 to protect a block of data words, e.g. CRC or checksum G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 2201/855 Details of asynchronous mirroring using a journal to transfer not-yet-mirrored changes
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Techniques involving replication of virtual machines at a target site are described. One representative technique includes an apparatus including a virtual machine configured to provide storage access requests targeting a virtual disk. A storage request processing module is coupled to the virtual machine to receive the storage access requests and update the virtual disk as directed by the storage access requests. A replication management module is coupled to the virtual machine to receive the storage access requests in parallel with the storage request processing module, and to store information associated with the storage access requests in a log file(s). The log file may be transferred to a destination as a recovery replica of at least a portion of the virtual disk.

Claims

1. An apparatus comprising:

a virtual machine configured to provide storage access requests targeting a virtual disk;

a storage request processing module coupled to the virtual machine to receive the storage access requests and update the virtual disk as directed by the storage access requests;

a replication management module coupled to the virtual machine to receive the storage access requests in parallel with the storage request processing module, and store information associated with the storage access requests in at least one log file; and

a transmitter configured to transfer the at least one log file to a destination as a recovery replica of at least a portion of the virtual disk.

2. The apparatus of claim 1, wherein the storage request processing module comprises:

a virtual disk request queue configured to queue the storage access requests as they arrive;

a virtual disk request processing module configured to update the virtual disk with data associated with each entry of the virtual disk request queue; and

wherein the replication management module comprises;

a log request queue configured to queue the storage access requests issued by the virtual machine contemporaneously with the queuing of the storage access requests in the virtual disk request queue; and

a log file request processing module configured to update the at least one log file with the information from the storage access requests in the log request queue.

3. The apparatus of claim 2, further comprising a virtual disk parser module including both the virtual disk request queue and the log request queue.

4. The apparatus of claim 1, further comprising a memory coupled to the replication management module to store the at least one log file.

5. The apparatus of claim 4, further comprising a storage write control module configured to initiate transfer of the at least one log file from the memory to physical storage in response to occurrence of a defined triggering event.

6. The apparatus of claim 1, wherein the log file includes a data structure comprising:

a log file header including an address of an end of the log file and a size of metadata blocks in the log file;

a plurality of metadata blocks, each including a metadata header and one or more metadata entries, wherein the metadata header includes a location of a location of a previous metadata block, and each metadata entry includes a location and length of the data associated with each metadata block; and

a plurality of data blocks reflecting the updates to the virtual disk.

7. Computer-readable media having instructions stored thereon which are executable by a computing system for performing functions comprising:

queuing write requests issued by a primary virtual machine in a first queue;

queuing the write requests issued by the virtual machine in a second queue in parallel with the queuing of the write requests in the first queue;

updating data in a virtual disk utilized by the virtual machine using the write requests from the first queue;

updating a log file using the write requests in the second queue; and

transferring the log file for use in generating replicated virtual storage accessed by a recovery virtual machine.

8. The computer-readable media of claim 7, wherein the stored instructions are executable by the computing system for performing further functions comprising:

requesting that the log file be transferred for use by the recovery virtual machine;

redirecting new write requests to a new log file;

writing write requests that were pending when the new log file was created to the log file to be transmitted;

providing a completion response when the write requests that were pending when the new log file was created are all written to the log file to be transmitted; and

enabling transfer of the log file subject to the request for transfer in response to the completion response.

9. The computer-readable media of claim 7, wherein the stored instructions are executable by the computing system for performing further functions comprising:

requesting an application-consistent snapshot of one or more components operating with the primary virtual machine;

completing write operations to the virtual disk as a result of the application-consistent snapshot; and

wherein requesting that the log file be transferred is enabled in response to completion of the write operations resulting from the application-consistent snapshot.

10. The computer-readable media of claim 7, wherein the stored instructions are executable by the computing system for performing further functions comprising:

creating a second log file at a target server in response to a migration of the primary virtual machine to the target server;

updating the second log file using duplications of the write requests that are being used to update the log file; and

utilizing the second log file in connection with a second virtual machine at the target server upon completion of the migration.

11. The computer-readable media of claim 7, wherein the executable instructions further cause the log file to be stored in memory as it be being updated.

12. The computer-readable media of claim 11, wherein the executable instructions further cause the log file to be transferred from the memory to an address of the recovery virtual machine.

13. The computer-readable media of claim 11, wherein the executable instructions further cause the log file to be transferred from the memory to physical storage in response to an occurrence of a predetermined triggering event.

14. The computer-readable media of claim 11, wherein the executable instructions further cause the log file to be transferred from the memory to physical storage in response to the memory allocated to the log file being utilized to a threshold capacity.

15. A computer-implemented method for facilitating replication of virtual machines, comprising:

receiving a log file of changes duplicating changes made to primary virtual storage of a primary virtual machine, the log file including a log file header, blocks of data that changed in the primary virtual storage, and metadata blocks to specify locations of the data in the log file;

locating a first metadata block in the log file using information from the log file header, and storing the address of the first metadata block;

locating one or more additional metadata blocks in the log file, each metadata block being located using information from its respectively preceding one of the metadata blocks in the log file;

storing the addresses of each of the one or more additional metadata blocks that are located in the log file;

locating the data identified by each of the stored metadata blocks; and

storing the located data in replicated virtual storage operable by a recovery virtual machine to replicate the primary virtual machine.

16. The computer-implemented method of claim 15, wherein:

the information from each of the metadata blocks includes an address offset relative to a base address of the replicated virtual storage, and a data length of the data identified by the respective metadata block; and

storing the located data in the replicated virtual storage comprises storing blocks of the data having the data length at their respective address offset locations in the replicated virtual storage.

17. The computer-implemented method of claim 15, wherein storing the address of the first metadata block and storing the addresses of each of the one or more additional metadata blocks comprises pushing the first and additional metadata blocks onto a stack in an order in which they will be removed to locate the data in the log file.

18. The computer-implemented method of claim 15, further comprising:

creating the log file at the primary virtual machine by duplicating write requests from the primary virtual machine into the log file in parallel with the writing of the write requests to the primary virtual storage; and

transmitting the log file to a recovery server hosting the recovery virtual machine.

19. The computer-implemented method of claim 15, further comprising:

creating one or more undo log entries in an undo log file in connection with the storing of the located data in the replicated virtual storage; and

reverting the recovery virtual machine to a point in time by applying to the virtual storage one or more of the undo log entries back to the point in time.

20. The computer-implemented method of claim 19, wherein reverting the recovery virtual machine to a point in time comprises applying the one or more of the undo log entries in reverse chronological order to the virtual storage based at least on timestamp values for each of the one or more of the undo log entries.