Systems and methods for management of virtualization data

• US 8,725,973 B2
• Filed: 11/02/2012
• Issued: 05/13/2014
• Est. Priority Date: 09/05/2008
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Prahlad, Anand, Pawar, Rahul S., Varadharajan, Prakash, Bedadala, Pavan Kumar Reddy
• Primary Examiner(s): Lo, Kenneth
• Application Number:
  US13/667,890
• Publication Number:
  US 20130061014A1
• Time in Patent Office: 557 Days
• Field of Search: 711/6, 711/162, 711/E12.002
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: G06F 16/383 using metadata automatically derived from the content
• CPC Class Codes: G06F 2201/815 Virtual
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 9/455 Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines

Abstract

Described in detail herein is a method of copying data of one or more virtual machines being hosted by one or more non-virtual machines. The method includes receiving an indication that specifies how to perform a copy of data of one or more virtual machines hosted by one or more virtual machine hosts. The method may include determining whether the one or more virtual machines are managed by a virtual machine manager that manages or facilitates management of the virtual machines. If so, the virtual machine manager is dynamically queried to automatically determine the virtual machines that it manages or that it facilitates management of. If not, a virtual machine host is dynamically queried to automatically determine the virtual machines that it hosts. The data of each virtual machine is then copied according to the specifications of the received indication.

Claims

1. A method of managing data of one or more virtual machines hosted by a virtual machine host, wherein the method is performed by one or more computing systems, each computing system having a processor and memory, the method comprising:

receiving a request to copy data of at least one virtual machine to a storage device, wherein the at least one virtual machine stores data in at least one virtual machine disk file residing on a filesystem of the virtual machine host;

determining the at least one virtual machine disk file in which the at least one virtual machine stores data;

copying the at least one virtual machine disk file from the filesystem of the virtual machine host to the storage device;

extracting, by the one or more computing systems, information from the at least one virtual machine disk file, wherein extracting information includes extracting at least one of the following;

information that describes a virtual volume utilized by the at least one virtual machine;

information that describes a location of a virtual file allocation table utilized by the at least one virtual machine, wherein the virtual file allocation table provides for locations of virtual files within the virtual volume; and

information that describes a relationship between the at least one virtual machine disk file and another virtual machine disk file in which the at least one virtual machine stores data;

associating the extracted information with the at least one virtual machine; and

storing the extracted information in association with the at least one virtual machine.

2. The method of claim 1, further comprising:

receiving a request to copy a virtual file contained within the virtual machine disk file stored on the storage device to a requested location;

accessing the at least one virtual machine disk file stored on the storage device;

accessing the stored extracted information;

determining, based on the stored extracted information, the location of the virtual file within the at least one virtual machine disk file;

extracting the virtual file from within the at least one virtual machine disk file; and

copying the virtual file to the requested location.

3. The method of claim 1, further comprising:

receiving a request to copy a virtual volume contained within the virtual machine disk file stored on the storage device to a requested location;

accessing the at least one virtual machine disk file stored on the storage device;

accessing the stored extracted information;

determining, based on the stored extracted information, the location of the virtual volume within the at least one virtual machine disk file;

extracting the virtual volume from within the at least one virtual machine disk file; and

copying the virtual volume to the requested location.

4. The method of claim 1, further comprising:

receiving a request to copy the at least one virtual machine disk file stored on the storage device to a requested location;

accessing the at least one virtual machine disk file stored on the storage device; and

copying the at least one virtual machine disk file to the requested location.

5. The method of claim 1, wherein the virtual machine disk file is in a first format, and further comprising:

receiving a request to convert the at least one virtual machine disk file to a second format;

accessing the at least one virtual machine disk file stored on the storage device; and

converting the at least one virtual machine disk file to the second format.

6. The method of claim 1, further comprising:

determining distinct data objects within the at least one virtual machine disk file based at least in part upon the extracted information;

indexing distinct data objects within the at least one virtual machine disk file; and

storing the indexed distinct data objects in an index, wherein the index also includes data associated with at least one non-virtual machine.

7. The method of claim 1, further comprising:

determining distinct data objects within the at least one virtual machine disk file based at least in part upon the extracted information;

indexing distinct data objects within the at least one virtual machine disk file;

storing the indexed distinct data objects in an index, wherein the index also includes data associated with at least one non-virtual machine;

receiving a request to query the index; and

returning search results responsive to the request, wherein the search results include search results associated with distinct data objects of the least one virtual machine and with distinct data objects at least one non-virtual machine.

8. The method of claim 1, further comprising:

determining distinct data objects within the at least one virtual machine disk file based at least in part upon the extracted information; and

for each of the distinct data objects;

generating a substantially unique identifier for the distinct data object;

determining, based on the substantially unique identifier, if an instance of the distinct data object has already been stored on a single instance storage device; and

if an instance of the distinct data object has not already been stored on the single instance storage device, then storing the data object on the single instance storage device.

9. A computer-readable storage medium whose contents cause one or more computing systems to perform a method of managing data of one or more virtual machines hosted by a virtual machine host, wherein each of the one or more computing systems has a processor and memory, the method comprising:

receiving a request to copy data of a virtual machine to a storage device, wherein the virtual machine stores data in a virtual machine disk file residing on a filesystem of the virtual machine host;

determining the virtual machine disk file in which the virtual machine stores data;

copying the virtual machine disk file from the filesystem of the virtual machine host to the storage device;

extracting, by the one or more computing systems, information from the virtual machine disk file, wherein extracting information includes extracting information that either describes locations of data objects within the virtual machine disk file or describes a location of a data object locator data structure within the virtual machine disk file;

associating the extracted information with the virtual machine; and

storing the extracted information in association with the virtual machine.

10. The computer-readable storage medium of claim 9, wherein the method further comprises:

receiving a request to copy a data object contained within the virtual machine disk file stored on the storage device to a requested location;

accessing the virtual machine disk file stored on the storage device;

accessing the stored extracted information;

determining, based at least in part on the stored extracted information, the location of the data object within the virtual machine disk file;

extracting the data object from within the virtual machine disk file; and

copying the data object to the requested location.

11. The computer-readable storage medium of claim 9, wherein the method further comprises:

determining, based at least in part upon the extracted information, data objects within the virtual machine disk file;

indexing data objects within the virtual machine disk file; and

storing the indexed data objects in an index, wherein the index also includes data associated with at least one non-virtual machine.

12. The computer-readable storage medium of claim 9, wherein the method further comprises:

determining data objects within the virtual machine disk file based at least in part upon the extracted information; and

for at least one of the data objects;

generating a substantially unique identifier for the data object;

determining, based on the substantially unique identifier, if an instance of the data object has already been stored on a storage device; and

if an instance of the data object has not already been stored on the storage device, then storing the data object on the storage device.

13. A computing system for managing data of one or more virtual machines hosted by a virtual machine host, the computing system comprising:

means for receiving a request to copy data of a virtual machine, wherein the virtual machine stores data in a virtual machine disk file residing on a filesystem of the virtual machine host;

means for determining the virtual machine disk file in which the virtual machine stores data;

means for copying the virtual machine disk file from the filesystem of the virtual machine host to the storage device;

means for extracting information from the virtual machine disk file, wherein extracting information includes extracting information that either describes locations of data objects within the virtual machine disk file or describes a location of a data object location data structure within the virtual machine disk file; and

means for associating the extracted information with the virtual machine; and

means for storing the extracted information in association with the virtual machine.

14. The computing system of claim 13, further comprising:

means for determining, based at least in part upon the extracted information, data objects within the virtual machine disk file;

means for indexing data objects within the virtual machine disk file; and

means for storing the indexed data objects in an index, wherein the index also includes data associated with at least one non-virtual machine.

15. The computing system of claim 13, further comprising:

means for determining data objects within the virtual machine disk file based at least in part upon the extracted information;

means for generating a substantially unique identifier for at least one of the data objects;

means for determining, based on the substantially unique identifier, if an instance of the at least one data object has already been stored; and

means for storing the at least one data object if an instance of the data object has not already been stored.

16. A method of managing data of one or more virtual machines hosted by a virtual machine host, wherein the method is performed by one or more computing systems, each computing system having a processor and memory, the method comprising:

receiving a first request to copy data of at least one virtual machine to a first storage device, wherein the at least one virtual machine stores data in at least one virtual machine disk file residing on a filesystem of the virtual machine host;

determining, by the one or more computing systems, the at least one virtual machine disk file in which the at least one virtual machine stores data, wherein the at least one virtual machine disk file includes multiple blocks of data;

determining blocks within the at least one virtual machine disk file that are used by the at least one virtual machine;

for at least some of the determined blocks, generating substantially unique identifiers;

storing the generated substantially unique identifiers in a data structure;

copying at least some of the blocks of the at least one virtual machine disk file from the filesystem of the virtual machine host to the first storage device;

receiving a second request to copy data of the at least one virtual machine to a second storage device;

determining blocks within the at least one virtual machine disk file that are used by the at least one virtual machine;

for at least one of the determined blocks;

generating a substantially unique identifier;

determining from the data structure if the generated substantially unique identifier is equivalent to a substantially unique identifier in the data structure; and

when the generated substantially unique identifier is not equivalent to a substantially unique identifier in the data structure, copying the block to the second storage device.

17. The method of claim 16, wherein the data structure includes multiple items of data, and wherein the method further comprises when the generated substantially unique identifier is not equivalent to a substantially unique identifier in the data structure, updating an existing item of data in the data structure to include the generated substantially unique identifier.

18. The method of claim 16, wherein the data structure includes multiple items of data, and wherein the method further comprises when the generated substantially unique identifier is not equivalent to a substantially unique identifier in the data structure, adding an existing item of data to the data structure that includes the generated substantially unique identifier.

19. The method of claim 16, further comprising when the generated substantially unique identifier is equivalent to a substantially unique identifier in the data structure, avoiding copying the block to the second storage device.

20. A computer-readable storage medium whose contents cause one or more computing systems to perform a method of managing data of one or more virtual machines hosted by a virtual machine host, wherein each computing system having a processor and memory, the method comprising:

receiving a request to copy data of at least one virtual machine to a storage device, wherein the at least one virtual machine stores data in at least one virtual machine disk file residing on a filesystem of the virtual machine host;

determining the at least one virtual machine disk file in which the at least one virtual machine stores data;

copying the at least one virtual machine disk file from the filesystem of the virtual machine host to the storage device;

extracting, by the one or more computing systems, information from the at least one virtual machine disk file, wherein extracting information includes extracting at least one of the following;

information that describes a virtual volume utilized by the at least one virtual machine;

information that describes a location of a virtual file allocation table utilized by the at least one virtual machine, wherein the virtual file allocation table provides for locations of virtual files within the virtual volume; and

information that describes a relationship between the at least one virtual machine disk file and another virtual machine disk file in which the at least one virtual machine stores data;

associating the extracted information with the at least one virtual machine; and

storing the extracted information in association with the at least one virtual machine.

21. The computer-readable storage medium of claim 20, wherein the method further comprises:

receiving a request to copy a virtual file contained within the virtual machine disk file stored on the storage device to a requested location;

accessing the at least one virtual machine disk file stored on the storage device;

accessing the stored extracted information;

determining, based on the stored extracted information, the location of the virtual file within the at least one virtual machine disk file;

extracting the virtual file from within the at least one virtual machine disk file; and

copying the virtual file to the requested location.