Host-based replication with end-to-end compression

• US 9,575,687 B2
• Filed: 01/29/2015
• Issued: 02/21/2017
• Est. Priority Date: 01/29/2015
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Desai, Asit
• Primary Examiner(s): DOAN, DUC T
• Application Number:
  US14/609,054
• Publication Number:
  US 20160224249A1
• Time in Patent Office: 754 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0664 at device level, e.g. emulation of a storage device or system
• CPC Class Codes: G06F 3/0667 at data level, e.g. file, record or object virtualisation
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0673 Single storage device

Abstract

Techniques for enabling end-to-end compression when performing host-based replication (HBR) are provided. In one set of embodiments, a hypervisor running on a host system at a primary site can intercept I/O write requests that originate from a virtual machine (VM) configured to be replicated via HBR, the I/O write requests being destined for a virtual disk (VMDK) of the VM. The hypervisor can further track VMDK file blocks that are modified by the I/O write requests and can retrieve the VMDK file blocks from a storage tier at the primary site. The hypervisor can then compress the retrieved VMDK file blocks and transmit the compressed blocks to a secondary site.

Claims

1. A method for enabling end-to-end compression when performing host-based replication (HBR), the method comprising:

intercepting, by a hypervisor running on a first host system at a primary site, I/O write requests that originate from a virtual machine (VM) configured to be replicated via HBR, the I/O write requests being destined for a virtual disk (VMDK) of the VM;

tracking, by the hypervisor, VMDK file blocks that are modified by the I/O write requests;

retrieving, by the hypervisor from a storage tier at the primary site, hash values corresponding to VMDK file blocks that have been modified during a most recent recovery point objective (RPO) interval;

comparing, by the hypervisor, the retrieved hash values with a list of hash values corresponding to VMDK file blocks already sent to a secondary site;

for each retrieved hash value that is in the list of hash values, transmitting, by the hypervisor, the retrieved hash value to the secondary site without transmitting the VMDK file block corresponding to the retrieved hash value; and

for each retrieved hash value that is not in the list of hash values;

retrieving, by the hypervisor, the VMDK file block corresponding to the retrieved hash value from the storage tier at the primary site;

compressing, by the hypervisor, the retrieved VMDK file block; and

transmitting, by the hypervisor, the compressed VMDK file block to the secondary site.

2. The method of claim 1 wherein, upon receiving the compressed VMDK file block, a second host system at the secondary site is configured to:

identify a third host system at the secondary site that is capable of writing the file block to a replica VMDK stored on a storage tier of the secondary site; and

forward the compressed VMDK file block via network file copy (NFC) to the third host system.

3. The method of claim 2 wherein a hypervisor running on the third host system is configured to:

receive the compressed VMDK file block forwarded from the second host system;

decompress the compressed VMDK file block; and

write the decompressed VMDK file block to the replica VMDK stored on the storage tier of the secondary site.

4. The method of claim 1 wherein the primary site and the secondary site are connected via a wide area network (WAN).

5. The method of claim 1 wherein, prior to compressing the retrieved VMDK file block, the hypervisor of the first host system is configured to:

determine whether the VMDK is compressed on disk; and

if the VMDK is compressed on disk, transmit the retrieved VMDK file block to the secondary site without performing any compression.

6. The method of claim 1 wherein the primary site and the secondary site are connected via an unsecure network connection, and wherein prior to transmitting the compressed VMDK file block to the secondary site, the hypervisor of the first host system is configured to:

determine whether the VMDK is encrypted on disk;

if the VMDK is encrypted on disk, transmit the compressed VMDK file block to the secondary site without performing any encryption; and

if the VMDK is not encrypted on disk;

encrypt the compressed VMDK file block; and

transmit the encrypted and compressed VMDK file block to the secondary site.

7. A non-transitory computer readable storage medium having stored thereon program code executable by a first host system at a primary site, the program code embodying a method for enabling end-to-end compression when performing host-based replication (HBR), the method comprising:

intercepting I/O write requests that originate from a virtual machine (VM) configured to be replicated via HBR, the I/O write requests being destined for a virtual disk (VMDK) of the VM;

tracking VMDK file blocks that are modified by the I/O write requests;

retrieving, from a storage tier at the primary site, hash values corresponding to VMDK file blocks that have been modified during a most recent recovery point objective (RPO) interval;

comparing the retrieved hash values with a list of hash values corresponding to VMDK file blocks already sent to a secondary site;

for each retrieved hash value that is in the list of hash values, transmitting the retrieved hash value to the secondary site without transmitting the VMDK file block corresponding to the retrieved hash value; and

for each retrieved hash value that is not in the list of hash values;

retrieving the VMDK file block corresponding to the retrieved hash value from the storage tier at the primary site;

compressing the retrieved VMDK file block; and

transmitting the compressed VMDK file block to the secondary site.

8. The non-transitory computer readable storage medium of claim 7 wherein, upon receiving the compressed VMDK file block, a second host system at the secondary site is configured to:

identify a third host system at the secondary site that is capable of writing the file block to a replica VMDK stored on a storage tier of the secondary site; and

forward the compressed VMDK file block via network file copy (NFC) to the third host system.

9. The non-transitory computer readable storage medium of claim 8 wherein a hypervisor running on the third host system is configured to:

receive the compressed VMDK file block forwarded from the second host system;

decompress the compressed VMDK file block; and

write the decompressed VMDK file block to the replica VMDK stored on the storage tier of the secondary site.

10. The non-transitory computer readable storage medium of claim 7 wherein the primary site and the secondary site are connected via a wide area network (WAN).

11. The non-transitory computer readable storage medium of claim 7 wherein, prior to compressing the retrieved VMDK file block, the first host system is configured to:

determine whether the VMDK is compressed on disk; and

if the VMDK is compressed on disk, transmit the retrieved VMDK file block to the secondary site without performing any compression.

12. The non-transitory computer readable storage medium of claim 7 wherein the primary site and the secondary site are connected via an unsecure network connection, and wherein prior to transmitting the compressed VMDK file block to the secondary site, the first host system is configured to:

determine whether the VMDK is encrypted on disk;

if the VMDK is encrypted on disk, transmit the compressed VMDK file block to the secondary site without performing any encryption; and

if the VMDK is not encrypted on disk;

encrypt the compressed VMDK file block; and

transmit the encrypted and compressed VMDK file block to the secondary site.

13. A first host system operating at a primary site in a host-based replication (HBR) workflow, the first host system comprising:

a processor; and

a hypervisor running on the processor that is configured to;

intercept I/O write requests that originate from a virtual machine (VM) configured to be replicated via HBR, the I/O write requests being destined for a virtual disk (VMDK) of the VM;

track VMDK file blocks that are modified by the I/O write requests;

retrieve, from a storage tier at the primary site, hash values corresponding to VMDK file blocks that have been modified during a most recent recovery point objective (RPO) interval;

compare the retrieved hash values with a list of hash values corresponding to VMDK file blocks already sent to a secondary site;

for each retrieved hash value that is in the list of hash values, transmit the retrieved hash value to the secondary site without transmitting the VMDK file block corresponding to the retrieved hash value; and

for each retrieved hash value that is not in the list of hash values;

retrieve the VMDK file block corresponding to the retrieved hash value from the storage tier at the primary site;

compress the retrieved VMDK file block; and

transmit the compressed VMDK file block to the secondary site.

14. The first host system of claim 13 wherein, upon receiving the compressed VMDK file block, a second host system at the secondary site is configured to:

identify a third host system at the secondary site that is capable of writing the file block to a replica VMDK stored on a storage tier of the secondary site; and

forward the compressed VMDK file block via network file copy (NFC) to the third host system.

15. The first host system of claim 14 wherein a hypervisor running on the third host system is configured to:

receive the compressed VMDK file block forwarded from the second host system;

decompress the compressed VMDK file block; and

write the decompressed VMDK file block to the replica VMDK stored on the storage tier of the secondary site.

16. The first host system of claim 13 wherein the primary site and the secondary site are connected via a wide area network (WAN).

17. The first host system of claim 13 wherein, prior to compressing the retrieved VMDK file block, the hypervisor of the first host system is configured to:

determine whether the VMDK is compressed on disk; and

if the VMDK is compressed on disk, transmit the retrieved VMDK file block to the secondary site without performing any compression.

18. The first host system of claim 13 wherein the primary site and the secondary site are connected via an unsecure network connection, and wherein prior to transmitting the compressed VMDK file block to the secondary site, the hypervisor of the first host system is configured to:

determine whether the VMDK is encrypted on disk;

if the VMDK is encrypted on disk, transmit the compressed VMDK file block to the secondary site without performing any encryption; and

if the VMDK is not encrypted on disk;

encrypt the compressed VMDK file block; and

transmit the encrypted and compressed VMDK file block to the secondary site.