De-duplicated virtual machine image transfer

• US 9,886,352 B2
• Filed: 04/27/2012
• Issued: 02/06/2018
• Est. Priority Date: 04/27/2012
• Status: Expired due to Fees

• Current Assignee: University of British Columbia
• Original Assignee: University of British Columbia
• Inventors: Al-Kiswany, Samer, Ripeanu, Matei
• Primary Examiner(s): Corrielus, Jean M
• Application Number:
  US13/824,761
• Publication Number:
  US 20140188807A1
• Time in Patent Office: 2,111 Days
• Field of Search: 707634, 707640, 707692, 711203
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0662 Virtualisation aspects
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

Technologies are generally described for a system and method effective to copy virtual machine images from a source to a destination memory. A processor may copy a first block corresponding to a first virtual machine image from an interim memory to the destination memory. The interim memory may include de-duplicated data present in the first and the second virtual machine images. The processor may identify a second block in the interim memory that corresponds to the second virtual machine image and store the second block in a buffer. The processor may identify a third block in the interim memory that corresponds to the first virtual machine image and copy the third block from the interim memory to the destination memory.

Claims

1. A method, performed by a processor configured to execute computer instructions stored in a memory, to copy a first virtual machine image and a second virtual machine image from a source memory to a destination memory, the method comprising:

copying a first block that corresponds to the first virtual machine image from an interim memory to the destination memory, wherein the interim memory includes de-duplicated data that is present in the first virtual machine image and the second virtual machine image, and wherein the interim memory is different from the source memory and the destination memory;

after copying the first block, identifying a second block in the interim memory;

in response to a determination that the identified second block corresponds to the second virtual machine image, storing the second block in a buffer;

after storing the second block, identifying a third block in the interim memory that corresponds to the first virtual machine image;

copying the third block from the interim memory to the destination memory;

in response to copying the third block from the interim memory to the destination memory, determining whether all blocks that correspond to the first virtual machine image have been copied from the interim memory to the destination memory; and

in response to a determination that all blocks that correspond to the first virtual machine image have been copied from the interim memory to the destination memory, copying the second block from the buffer to the destination memory,

wherein copying the second block, which corresponds to the second virtual machine image, from the buffer to the destination memory, after copying all blocks, which correspond to the first virtual machine image, from the interim memory to the destination memory, facilitates continual copying of blocks that correspond to the first virtual machine image, resulting in reduction in frequency of loading, spinning, and accessing the destination memory, and increase in efficiency of writing the first virtual machine image to the destination memory.

2. The method of claim 1, further comprising:

identifying duplicated data, wherein the duplicated data is present in the first and second virtual machine images that are located at the source memory; and

copying the duplicated data from the source memory to the interim memory to produce the de-duplicated data.

3. The method of claim 2, wherein copying the duplicated data from the source memory to the interim memory produces an image block map.

4. The method of claim 1, further comprising generating a map, wherein the map identifies blocks in the interim memory, and virtual machine images that correspond to the blocks in the interim memory.

5. The method of claim 1, further comprising:

generating a map, wherein the map identifies blocks in the interim memory, and virtual machine images that correspond to the blocks in the interim memory,

and

wherein the map includes;

a first file that corresponds to the first virtual machine image, and

a second file that corresponds to the second virtual machine image.

6. The method of claim 1, further comprising:

generating a map, wherein the map identifies blocks in the interim memory, and virtual machine images that correspond to the blocks in the interim memory,

wherein the map includes;

a first file that corresponds to the first virtual machine image, and

a second file that corresponds to the second virtual machine image,

wherein each of the first and second files includes one or more regions, and wherein each of the one or more regions includes a block identifier field, a region buffer number count field, a block size field, a presence field, and/or a number of regions field.

7. The method of claim 1, further comprising:

generating a map, wherein the map identifies blocks in the interim memory, and virtual machine images that correspond to the blocks in the interim memory,

wherein the map includes;

a first file that corresponds to the first virtual machine image, and

a second file that corresponds to the second virtual machine image,

wherein the first file includes two or more regions, and wherein each of the two or more regions includes an identification of consecutive blocks of the de-duplicated data that corresponds to the first virtual machine image.

8. The method of claim 1, further comprising:

generating a map, wherein the map identifies blocks in the interim memory, and virtual machine images that correspond to the blocks in the interim memory, and

wherein the map indicates that the blocks are stored in the source memory, the destination memory, and/or the buffer.

9. The method of claim 1, further comprising:

generating a map, wherein the map identifies blocks in the interim memory, and virtual machine images that correspond to the blocks in the interim memory, and

wherein the map includes a region count field that is effective to indicate a number of second blocks that are stored in the buffer.

10. The method of claim 1, further comprising:

copying blocks from the interim memory to the destination memory, until the buffer is full to a threshold level;

after the buffer is full to the threshold level, identifying a particular region, wherein a particular percentage of blocks that correspond to the particular region are stored in the buffer; and

copying the blocks that correspond to the particular region from the interim memory to the destination memory.

11. The method of claim 1, further comprising:

copying blocks from the interim memory to the destination memory, until the buffer is full to a threshold level;

after the buffer is full to the threshold level, identifying a particular region, wherein a particular percentage of blocks that correspond to the particular region are stored in the buffer;

copying the blocks that correspond to the particular region from the interim memory to the destination memory; and

copying the blocks that correspond to the particular region from the interim memory to other regions in other virtual machine images that include the blocks that correspond to the particular region.

12. The method of claim 1, further comprising:

incrementing a counter in the buffer, in response to storing the second block in the buffer;

copying blocks from the interim memory to the destination memory, until the buffer is full to a threshold level; and

after the buffer is full to the threshold level, performing;

identifying a particular region, wherein a particular percentage of blocks that correspond to the particular region are stored in the buffer;

copying the blocks that correspond to the particular region from the interim memory to the destination memory;

copying the blocks that correspond to the particular region from the interim memory to other regions in other virtual machine images that include the blocks that correspond to the particular region; and

decrementing the counter when blocks in the buffer are copied to the destination memory;

until the buffer reaches another threshold level.

13. The method of claim 1, further comprising:

copying blocks from the interim memory to the destination memory, until the buffer is full to a threshold level;

identifying a particular region, wherein a particular percentage of blocks that correspond to the particular region are stored in the buffer;

copying the blocks that correspond to the particular region from the interim memory to the destination memory;

copying the blocks that correspond to the particular region from the interim memory to other regions in other virtual machine images that include the blocks that correspond to the particular region;

identifying a partially full region that includes some blocks copied from the interim memory to the destination memory; and

copying blocks that correspond to the partially full region from the interim memory to the destination memory.

14. A system effective to copy a first virtual machine image and a second virtual machine image from a source memory to a destination memory, the system comprising:

an interim memory, different from the source memory and the destination memory; and

a processor configured to communicate with the interim memory, the source memory, and the destination memory, wherein the processor is effective to;

copy a first block that corresponds to the first virtual machine image from the interim memory to the destination memory, wherein the interim memory includes de-duplicated data present in the first and second virtual machine images;

after copying the first block, identify a second block in the interim memory;

in response to a determination that the identified second block corresponds to the second virtual machine image, store the second block in a buffer;

after the storage of the second block, identify a third block in the interim memory that corresponds to the first virtual machine image;

copy the third block from the interim memory to the destination memory;

in response to copying the third block from the interim memory to the destination memory, determine whether all blocks that correspond to the first virtual machine image have been copied from the interim memory to the destination memory; and

in response to a determination that all blocks that correspond to the first virtual machine image have been copied from the interim memory to the destination memory, copy the second block from the buffer to the destination memory,

wherein copying the second block, which corresponds to the second virtual machine image, from the buffer to the destination memory, after copying all blocks, which correspond to the first virtual machine image, from the interim memory to the destination memory, facilitates continual copying of blocks that correspond to the first virtual machine image, which results in reduction in frequency of loading, spinning, and accessing the destination memory.

15. The system of claim 14, wherein the processor is further effective to:

generate a map, wherein the map identifies blocks in the interim memory, and virtual machine images that correspond to the blocks in the interim memory, and

wherein the map indicates that the blocks are stored in the source memory, the destination memory, and/or the buffer.

16. The system of claim 14, wherein the processor is further effective to:

copy blocks from the interim memory to the destination memory, until the buffer is full to a threshold level;

after the buffer is full to the threshold level, identify a particular region, wherein a particular percentage of blocks that correspond to the particular region are stored in the buffer; and

copy the blocks that correspond to the particular region from the interim memory to the destination memory.

17. The system of claim 14, wherein the processor is further effective to:

copy blocks from the interim memory to the destination memory, until the buffer is full to a threshold level;

after the buffer is full to the threshold level, identify a particular region, wherein a particular percentage of blocks that correspond to the particular region are stored in the buffer;

copy the blocks that correspond to the particular region from the interim memory to the destination memory; and

copy the blocks that correspond to the particular region from the interim memory to other regions in other virtual machine images that include the blocks that correspond to the particular region.

18. The system of claim 14, wherein the processor is further effective to:

increment a counter in the buffer in response to storage of the second block in the buffer;

copy blocks from the interim memory to the destination memory, until the buffer is full to a threshold level; and

after the buffer is full to the threshold level, perform;

identify a particular region, wherein a particular percentage of blocks that correspond to the particular region are stored in the buffer;

copy the blocks that correspond to the particular region from the interim memory to the destination memory;

copy the blocks that correspond to the particular region from the interim memory to other regions in other virtual machine images that include the blocks that correspond to the particular region; and

decrement the counter when blocks in the buffer are copied to the destination memory;

until the buffer reaches another threshold level.

19. The system of claim 14, wherein the processor is further effective to:

copy blocks from the interim memory to the destination memory until, the buffer is full to a threshold level;

identify a particular region, wherein a particular percentage of blocks that correspond to the particular region are stored in the buffer;

copy the blocks that correspond to the particular region from the interim memory to the destination memory;

copy the blocks that correspond to the particular region from the interim memory to other regions in other virtual machine images that include the blocks that correspond to the particular region;

identify a partially full region that includes some blocks copied from the interim memory to the destination memory; and

copy blocks that correspond to the partially full region from the interim memory to the destination memory.

20. A method to copy a first virtual machine image and a second virtual machine image from a source memory to a destination memory, the method comprising:

identifying duplicated data, wherein the duplicated data is present in the first and second virtual machine images;

copying the duplicated data from the source memory to an interim memory to produce de-duplicated data and an image block map, wherein the interim memory is different from the source memory;

generating a file map based on the image block map, wherein the file map identifies blocks in the interim memory, and virtual machine images that correspond to the blocks in the interim memory;

copying, based on the file map, a first block that corresponds to the first virtual machine image from the interim memory to the destination memory, wherein the interim memory is different from the destination memory;

after copying the first block, identifying a second block in the interim memory;

in response to a determination that the identified second block corresponds to the second virtual machine image, storing the second block in a buffer;

after storing the second block, identifying a third block in the interim memory that corresponds to the first virtual machine;

copying the third block from the interim memory to the destination memory;

in response to copying the third block from the interim memory to the destination memory, determining whether all blocks that correspond to the first virtual machine image have been copied from the interim memory to the destination memory; and

in response to a determination that all blocks that correspond to the first virtual machine image have been copied from the interim memory to the destination memory, copying the second block from the buffer to the destination memory,

wherein copying the second block, which corresponds to the second virtual machine image, from the buffer to the destination memory, after copying all blocks, which correspond to the first virtual machine image, from the interim memory to the destination memory, facilitates continual copying of blocks that correspond to the first virtual machine image, resulting in reduction in frequency of loading, spinning, and accessing the destination memory, and

wherein the file map includes;

a first file that corresponds to the first virtual machine image, and

a second file that corresponds to the second virtual machine image,

wherein the first file includes two or more regions, and wherein each of the two or more regions includes;

an identification of consecutive blocks of the de-duplicated data that corresponds to the first virtual machine image,

an indication that the blocks are stored in the source memory, the destination memory, and/or the buffer, and

a region counter that is effective to indicate a number of second blocks that are stored in the buffer.

21. The method of claim 20, further comprising:

copying the blocks from the interim memory to the destination memory, until the buffer is full to a threshold level;

after the buffer is full to the threshold level, identifying a particular region, wherein a particular percentage of blocks that correspond to the particular region are stored in the buffer;

copying the blocks that correspond to the particular region from the interim memory to the destination memory; and

copying the blocks that correspond to the particular region from the interim memory to other regions in other virtual machine images that include the blocks that correspond to the particular region.

22. The method of claim 20, further comprising:

copying the blocks from the interim memory to the destination memory until the buffer is full to a threshold level;

after the buffer is full to the threshold level, identifying a particular region, wherein a particular percentage of blocks that correspond to the particular region are stored in the buffer;

copying the blocks that correspond to the particular region from the interim memory to the destination memory;

copying the blocks that correspond to the particular region from the interim memory to other regions in other virtual machine images that include the blocks that correspond to the particular region;

identifying a partially full region that includes some blocks copied from the interim memory to the destination memory; and

copying blocks that correspond to the partially full region from the interim memory to the destination memory.