Method and apparatus for replicating virtual machine images using deduplication metadata

• US 9,471,590 B2
• Filed: 02/12/2013
• Issued: 10/18/2016
• Est. Priority Date: 02/12/2013
• Status: Active Grant

• Current Assignee: Atlantis Computing, Inc. (hiveIO, Inc.)
• Original Assignee: Atlantis Computing, Inc. (hiveIO, Inc.)
• Inventors: Venkatesh, Chetan C, Dorairajan, Vinodh, Iyer, Kartikeya, Auradkar, Vikram, Parameswaran, Seshan
• Primary Examiner(s): Mahmoudi, Tony
• Assistant Examiner(s): NGUYEN, MERILYN P
• Application Number:
  US13/765,689
• Publication Number:
  US 20140229440A1
• Time in Patent Office: 1,344 Days
• Field of Search: 707/637, 707/692, 707/693
• US Class Current: 1/1
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

Techniques for fast provisioning of virtual machine images using deduplication metadata are described, including receiving a request to copy a first virtual machine to form a second virtual machine, identifying a first portion of memory comprising data for the first virtual machine; and forming the second virtual machine based on the first portion of memory comprising data for the first virtual machine, wherein forming the second virtual machine further comprises linking the second virtual machine to the first portion of memory comprising data for the first virtual machine, and implementing a second portion of memory to store data for the second virtual machine independent of the first virtual machine.

Claims

1. A method, comprising:

identifying data representing a first instance of a virtual machine image on a first storage device to be replicated to a second storage device;

copying a deduplication metadata file that describes the first instance of the virtual machine image from the first storage device to the second storage device, wherein the deduplication metadata file comprises a list of one or more virtual machine image data files for the first instance of the virtual machine image and associated reference links to data blocks of the one or more virtual machine image data files;

determining, based on metadata describing the data blocks of the one or more virtual machine image data files in the deduplication metadata file, which data representing the first instance of the virtual machine image matches data from a set of data on the second storage device;

forming a second instance of the virtual machine image on the second storage device based on the metadata description in the deduplication metadata file to replicate the first instance of the virtual machine image at the second storage device, wherein forming the second instance of the virtual machine image on the second storage device further comprises;

linking the second instance of the virtual machine image to at least a portion of the set of data on the second storage device for data representing the first instance of the virtual machine image that matches data from the set of data on the second storage device to form the second instance of the virtual machine image without transferring underlying blocks of data for the one or more virtual machine image data files from the first storage device to the second storage device.

2. The method of claim 1, wherein identifying data representing the first instance of the virtual machine image on the first storage device further comprises:

determining a block number associated with the first instance of the virtual machine image on the first storage device;

determining a hash value associated with the first instance of the virtual machine image based on the block number; and

determining the data representing the first instance of the virtual machine image on the first storage device based on the hash value.

3. The method of claim 1, wherein linking the second instance of the virtual machine image to the set of data on the second storage device further comprises:

updating a metadata file on the second storage device to include metadata associated with the second instance of the virtual machine image.

4. The method of claim 3, wherein metadata in the metadata file is associated with deduplicated data.

5. The method of claim 3, wherein the metadata file on the second storage device comprises a link counter associated with the set of data on the second storage device.

6. The method of claim 5, wherein the link counter associated with the set of data on the second storage device indicates a number of times the set of data on the second storage device is linked.

7. The method of claim 1, wherein determining whether the data representing the first instance of the virtual machine image matches a set of data on a second storage device further comprises:

determining a block number associated with the first instance of the virtual machine image on the first storage device;

determining a first hash value associated with the first instance of the virtual machine image based on the block number;

determining a second hash value associated with the set of data on the second storage device; and

determining whether the first hash value matches the second hash value.

8. The method of claim 1, wherein the first storage device is coupled to a first node of a computer cluster.

9. The method of claim 8, wherein the second storage device is coupled to a second node of the computer cluster, wherein the computer cluster comprises a virtual storage appliance that aggregates storage of data using the first node and the second node, and the second node is a mirrored backup of the first node, and wherein the second instance of the virtual machine image is formed on the second node based on the deduplication data within the aggregated storage without copying the one or more virtual machine image data files listed in the deduplication metadata file.

10. The method of claim 9, wherein the first node and second node are coupled via a cloud network.

11. A system, comprising:

a first storage device configured to store data representing a first instance of a virtual machine image;

a processor configured to

copy a deduplication metadata file that describes the first instance of the virtual machine image from the first storage device to a second storage device, wherein the deduplication metadata file comprises a list of one or more virtual machine image data files for the first instance of the virtual machine image and associated reference links to data blocks of the one or more virtual machine image data files,

determine, based on metadata describing the data blocks of the one or more virtual machine image data files in the deduplication metadata file, which data representing the first instance of the virtual machine image matches data from a set of data on the second storage device, and

form a second instance of a virtual machine image on the second storage device based on the metadata description in the deduplication metadata file to replicate the first instance of a virtual machine image at the second storage device, the processor being further configured,

link the second instance of a virtual machine image to at least a portion of the set of data on the second storage device for data representing the first instance of a virtual machine image that matches data from the set of data on the second storage device,

to form the second instance of a virtual machine image without transferring underlying blocks of data for the one or more virtual machine image data files from the first storage device to the second storage device.

12. The system of claim 11, wherein the processor is further configured to determine a block number associated with the first instance of a virtual machine image on the first storage device, to determine a hash value associated with the first instance of a virtual machine image based on the block number, and to determine the data representing the first instance of a virtual machine image on the first storage device based on the hash value.

13. The system of claim 11, wherein the processor is further configured to update a metadata file on the second storage device to include metadata associated with the second instance of a virtual machine image.

14. The system of claim 13, wherein metadata in the metadata file is associated with deduplicated data.

15. The system of claim 13, wherein the metadata file on the second storage device comprises a link counter associated with the set of data on the second storage device.

16. The system of claim 15, wherein the link counter associated with the set of data on the second storage device indicates a number of times the set of data on the second storage device is linked.

17. The system of claim 11, wherein the processor is further configured to determine a block number associated with the first instance of a virtual machine image on the first storage device, to determine a first hash value associated with the first instance of a virtual machine image based on the block number, to determine a second hash value associated with the set of data on the second storage device, and to determine whether the first hash value matches the second hash value.

18. The system of claim 11, wherein the first storage device is coupled to a first node of a computer cluster.

19. The system of claim 18, wherein the second storage device is coupled to a second node of the computer cluster, wherein the computer cluster comprises a virtual storage appliance that aggregates storage of data using the first node and the second node, and the second node is a mirrored backup of the first node, and wherein the second instance of the virtual machine is formed on the second node based on the deduplication data within the aggregated storage without copying the one or more virtual machine data files listed in the deduplication metadata file.

20. A method, comprising:

identifying data representing a first instance of a virtual machine image on a first storage device to be replicated to a second storage device;

copying a deduplication metadata file that describes the first instance of the virtual machine image from the first node to the second node, wherein the deduplication metadata file comprises a list of one or more virtual machine image data files for the first instance of the virtual machine image and associated reference links to data blocks of the one or more virtual machine image data files;

determining which data representing the first instance of the virtual machine image matches data from a set of data on the second storage device, wherein determining which data representing the first instance of a virtual machine image matches data from the set of data on the second storage device further comprises;

determining a block number associated with the first instance of the virtual machine image on the first storage device;

determining a hash value associated with the first instance of the virtual machine image based on the block number;

determining the data representing the first instance of the virtual machine image on the first storage device matches data from the set of data on the second storage device based on the hash value;

forming a second instance of the virtual machine image on the second storage device to replicate the first instance of the virtual machine image at the second storage device, wherein forming the second instance of the virtual machine image on the second storage device further comprises;

linking the second instance of the virtual machine image to at least a portion of the set of data on the second storage device for data representing the first instance of the virtual machine image that matches data from the set of data on the second storage device

to form the second instance of the virtual machine image without transferring underlying blocks of data for the one or more virtual machine data files from the first storage device to the second storage device.