Virtual machine image co-migration

• US 8,442,955 B2
• Filed: 03/30/2011
• Issued: 05/14/2013
• Est. Priority Date: 03/30/2011
• Status: Active Grant

• Current Assignee: Maplebear Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Al Kiswany, Samer, Sarkar, Prasenjit, Seaman, Mark J., Subhraveti, Dinesh K., Constantinescu, Comeliu Mihail
• Primary Examiner(s): LE, HUNG D
• Application Number:
  US13/075,623
• Publication Number:
  US 20120254131A1
• Time in Patent Office: 776 Days
• Field of Search: None
• US Class Current: 707/692
• CPC Class Codes: G06F 16/188 Virtual file systems

Abstract

Embodiments of the invention relate to co-migration in a shared pool of resources with similarity across data sets of a migrating application. The data sets are processed and profiled. Metadata is reviewed to remove duplicate elements and to distribute the processing load across available nodes. At the same time, a ranking may be assigned to select metadata to support a prioritized migration. Non-duplicate data chunks are migrated across the shared pool of resources responsive to the assigned prioritization.

Claims

1. A method comprising:

two or more data sets operating in a computer system, with one data set assigned to a single node;

pairing each node in the computer system with a specific pair node;

profiling each data set, including dividing each data set into two or more data chunks and applying a hash value to each data chunk;

assigning a hash value range to each node, and storing hash values within the assigned hash value range in memory local to the assigned node;

eliminating duplicate data chunks associated with the two or more data sets, including comparing hash values of each of two or more data sets, and for each duplicate hash value retaining a single copy of the duplicate hash value;

virtual machine co-migration, including;

profiling the data sets to be migrated, wherein the data sets are virtual machine images, including identifying data chunks required to support a boot process of a virtual machine image and prioritizing migration of the identified data chunks; and

migrating the non-duplicate data chunks of the data sets from a selected node to a corresponding pair node including eliminating duplicate data chunks between the selected node and the corresponding pair node, wherein the migration of the non-duplicate data chunks is limited to migration between paired nodes.

2. The method of claim 1, further comprising booting a virtual machine image at a destination node during migration, including intercepting an I/O request of the destination node, and if the I/O requests access of a non-migrated data chunk, migrating the non-migrated data chunk at a high priority form the source node.

3. The method of claim 1, further comprising migrating the two or more data sets and their respective virtual machine images to two or more destination nodes, with each data set being paired from an originating node to a single destination node, and building a file chunk map including assignment of a destination hash value range to the destination node, wherein the destination hash value range is equivalent to a source hash value range of the paired source node.

4. The method of claim 3, further comprising one of the destination nodes finding a local data set that is similar to the migrated data set, including the destination node comparing hash values of data chunks of the local data set with hash values of the migrated data set.

5. The method of claim 4, wherein migrating the non-duplicate data chunks includes the destination node writing a data chunk available in local data sets to a target image and migrating a data chunk not available in local data sets from the source node to the destination node.

6. The method of claim 1, further comprising load balancing hash space of data sets, including sending a hash value outside of a subject node hash value range to a node assigned to the hash value range.

7. A system comprising:

a processor;

a profile manager to profile each data set, the profile manager to divide each data set into at least two data chunks and to apply a hash value to each data chunk;

the profile manager to pair each node in the computer system with a pair node;

hash manager in communication with the profile manager, the hash manager to assign a hash range to each node and to store hash values within the range in local memory of the assigned node;

a comparison manager in communication with the hash manager, the comparison manager to compare hash values of each of the data sets, and to retain a single copy of an identified duplicate hash value, with each node to retain non-duplicate data chunks of the two or more data sets within the hash value range assigned to the node;

virtual machine co-migration, including;

profiling the data sets to be migrated, wherein the data sets are virtual machine images, including identifying data chunks required to support a boot process of a virtual machine image and prioritizing migration of the identified data chunks; and

a migration manager in communication with the comparison manager, the migration manager to migrate the non-duplicate data chunks of the data sets from a selected node to a corresponding pair node including eliminating duplicate data chunks between the selected node and the corresponding pair node, wherein the migration of the non-duplicate data chunks is limited to migration between paired nodes.

8. The system of claim 7, further comprising a boot manager in communication with the profile manager, the boot manager to boot a virtual machine image at a destination node during migration, including placement of a request to the migration manager for access to a non-migrated data chunk, and the migration manager migrating the non-migrated data chunk at an increased priority.

9. The system of claim 7, further comprising the migration manager to migrate data sets and their virtual machine images to destination nodes, each data set paired from an originating node to a single destination node, including the profile manager to build a file chunk map having an assignment of a destination hash value range to the destination node, wherein the destination hash value range is equivalent to a source hash value range of the paired source node.

10. The system of claim 7, further comprising a destination manager to write a data chunk available in local memory to a target data set and to communicate with the migration manager to migrate a data chunk not available in a local data set from the source node to the destination node.

11. The system of claim 7, further comprising the hash manager to load balance hash space of data sets, including the hash manager to send a hash value outside of a subject node hash value range to a node assigned to the hash value range.

12. A computer program product, the computer program product comprising a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising:

computer readable program code configured to pair each source node with a pair node;

computer readable program code configured to profile two or more data sets operating in a computer system, with one virtual machine image assigned to a single node;

computer readable program code configured to divide each data set into two or more data chunks and applying a hash value to each data chunk;

computer readable program code configured to assign a hash value range to each source and destination node, and storing hash values within the hash value range in memory local to the assigned node;

computer readable program code configured to eliminate duplicate data chunks associated with the data sets, including comparison of hash values of each of two or more data sets, and for each duplicate hash value retaining a single copy of the duplicate hash value;

virtual machine co-migration, including;

profiling the data sets to be migrated, wherein the data sets are virtual machine images, including identifying data chunks required to support a boot process of a virtual machine image and prioritizing migration of the identified data chunks; and

computer readable program code configured to migrate the non-duplicate data chunks from a selected source node to a corresponding pair node including eliminating duplicate data chunks between the selected source node and the corresponding pair node, wherein the migration of the non-duplicate data chunks is limited to migration between paired nodes.

13. The computer program product of claim 12, further comprising computer readable program code to prioritize migration of data chunks, including code to boot a virtual machine image at a destination node during migration and placement of a request for access to a non-migrated data chunk and migration of the non-migrated data chunk at an increased priority.

14. The compute program product of claim 12, further comprising computer readable program code to migrate virtual machines and their data sets, each data set paired from an originating node, including program code to build a file chunk map having an assignment of a hash value range to the destination node, wherein the destination hash value range is equivalent to a source hash value range of the paired source node.

15. The computer program product of claim 12, further comprising computer program code to write a data chunk available in local memory to a target data set and to migrate a data chunk not available in a local image from the source node to the destination node.

16. The computer program product of claim 12, further comprising computer program code to load balance hash space of data sets, including code to send a hash value outside of a subject node hash value range to a node assigned to the hash value range.

17. A method comprising:

in a computer system with a shared pool of resource, two or more data sets having one data set assigned to a single node;

pairing each node in the computer system with a pair node;

profiling each data set, including dividing each data set into two or more data chunks, applying a hash value to each data chunk, and prioritizing the data chunks;

assigning a hash value range to each node, and storing hash values within the hash value range of a data set in memory local to the assigned node;

eliminating duplicate data chunks in the system;

virtual machine co-migration, including;

profiling the data sets to be migrated, wherein the data sets are virtual machine images, including identifying data chunks required to support a boot process of a virtual machine image and prioritizing migration of the identified data chunks; and

migrating the non-duplicate data chunks from a selected node to a corresponding paired node responsive to the prioritization, including eliminating duplicate data chunks between the selected node and the corresponding pair node, wherein the migration of the non-duplicate data chunks is limited to migration between paired nodes.

18. A method comprising:

two or more data sets operating in a computer system, with one data set assigned to a single node;

profiling each data set, wherein the data sets are virtual machine images, the profiling including dividing each data set into two or more data chunks, applying a hash value to each data chunk, identifying data chunks required to support a boot process of a virtual machine image, and prioritizing migration of the identified data chunks;

assigning a hash value range to each node, and storing hash values within the assigned hash value range in memory local to the assigned node;

eliminating duplicate data chunks associated with the two or more data sets, including comparing hash values of each of two or more data sets, and for each duplicate hash value retaining a single copy of the duplicate hash value, with each node retaining non-duplicate data chunks within the hash value range assigned to the node;

virtual machine co-migration, including;

profiling the data sets to be migrated, wherein the data sets are virtual machine images, including identifying data chunks required to support a boot process of a virtual machine image and prioritizing migration of the identified data chunks; and

migrating the non-duplicate data chunks of the data sets, the migration including booting a virtual machine image at a destination node during migration, including intercepting an I/O request of the destination node, and if the I/O requests access of a non-migrated data chunk, migrating the non-migrated data chunk at a high priority form a source node.