Systems and methods for management of virtualization data

• US 9,740,723 B2
• Filed: 05/12/2014
• Issued: 08/22/2017
• Est. Priority Date: 09/05/2008
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Prahlad, Anand, Pawar, Rahul S., Varadharajan, Prakash, Bedadala, Pavan Kumar Reddy
• Primary Examiner(s): Rones, Charles
• Assistant Examiner(s): Li, Sidney
• Application Number:
  US14/275,381
• Publication Number:
  US 20140250093A1
• Time in Patent Office: 1,198 Days
• Field of Search: 711162
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: G06F 16/383 using metadata automatically derived from the content
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 9/455 Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines

Abstract

Described in detail herein is a method of copying data of one or more virtual machines being hosted by one or more non-virtual machines. The method includes receiving an indication that specifies how to perform a copy of data of one or more virtual machines hosted by one or more virtual machine hosts. The method may include determining whether the one or more virtual machines are managed by a virtual machine manager that manages or facilitates management of the virtual machines. If so, the virtual machine manager is dynamically queried to automatically determine the virtual machines that it manages or that it facilitates management of. If not, a virtual machine host is dynamically queried to automatically determine the virtual machines that it hosts. The data of each virtual machine is then copied according to the specifications of the received indication.

Claims

1. An automated method of managing data of one or more virtual machines managed by a virtual machine manager, wherein the method is performed by one or more physical computing systems, each computing system having a processor and memory, the method comprising:

receiving a request to copy data of one or more virtual machines to a physical storage device;

determining whether the one or more virtual machines are managed by a virtual machine manager,

wherein the determining is performed by reading a stored indication of the virtual machine manager, or by scanning a network for the virtual machine manager;

based on the determining that the one or more virtual machines are managed by the virtual machine manager, automatically accessing the virtual machine manager;

automatically requesting information from the virtual machine manager regarding virtual machines that the virtual machine manager manages;

automatically determining, by the one or more computing systems, from information provided by the virtual machine manager, the virtual machines managed by the virtual machine manager; and

for at least one of the determined virtual machines, copying the data of the virtual machine to the physical storage device.

2. The automated method of claim 1, wherein the request further specifies at least one of the following:

that a file-level copy is to be performed for at least one virtual machine managed by the virtual machine manager;

that a volume-level copy is to be performed for at least one virtual machine managed by the virtual machine manager; and

that a disk-level copy is to be performed for at least one virtual machine managed by the virtual machine manager.

3. The automated method of claim 1, wherein the request specifies that a disk-level copy is to be performed for at least one virtual machine managed by the virtual machine manager, and wherein the method further comprises:

determining a virtual machine disk file in which the at least one virtual machine stores data;

copying the virtual machine disk file from a filesystem of a virtual machine host hosting the at least one virtual machine to the storage device;

extracting information from the virtual machine disk file, wherein extracting information includes extracting information that describes a location of a virtual file allocation table utilized by the at least one virtual machine;

associating the extracted information with the at least one virtual machine; and

storing the extracted information in association with the at least one virtual machine.

4. The automated method of claim 1, wherein the request specifies that a disk-level copy is to be performed for at least one virtual machine managed by the virtual machine manager, and wherein the method further comprises:

determining at least two virtual machine disk files in which the at least one virtual machine stores data;

copying the at least two virtual machine disk files from a filesystem of a virtual machine host hosting the at least one virtual machine to the storage device;

extracting from the at least two virtual machine disk files information that describes the relationships between the virtual machine disk files;

extracting from the at least two virtual machine disk files information that describes at least one virtual volume contained within one or more of the at least two virtual machine disk files;

extracting from the at least two virtual machine disk files information that describes at least one virtual file allocation table that describes locations of virtual files contained within the at least one virtual volume; and

storing the extracted information.

5. The automated method of claim 1, further comprising, for at least one virtual machine:

determining distinct data objects within the virtual machine data;

indexing the distinct data objects; and

storing the indexed distinct data objects in an index, wherein the index also includes indexed distinct data objects associated with at least one non-virtual machine.

6. The automated method of claim 1, further comprising:

determining distinct data objects within the virtual machine data;

indexing the distinct data objects within the virtual machine data;

storing the indexed distinct data objects in an index, wherein the index also includes indexed distinct data objects associated with at least one non-virtual machine;

receiving a request to query the index; and

returning search results responsive to the request, wherein the search results include search results associated with distinct data objects of the least one virtual machine and with distinct data objects at least one non-virtual machine.

7. The automated method of claim 1, further comprising:

determining distinct data objects within the virtual machine data; and

for at least one of the distinct data objects;

generating a substantially unique identifier for the distinct data object;

determining, based on the substantially unique identifier, if an instance of the distinct data object has already been stored on a single instance storage device; and

if an instance of the distinct data object has not already been stored on the single instance storage device, then storing the data object on the single instance storage device.

8. The automated method of claim 1 wherein, for at least one virtual machine, copying the data of the virtual machine to the storage device includes determining a virtual machine disk file in which the virtual machine stores data and copying the virtual machine disk file from a filesystem of a virtual machine host hosting the virtual machine to the storage device, wherein the virtual machine is a first type of virtual machine and the virtual machine disk file is in a first format utilized by the first type of virtual machine, and wherein the method further comprises:

receiving a request to convert the virtual machine disk file to a virtual machine disk file in a second format utilized by a second type of virtual machine;

accessing the virtual machine disk file stored on the storage device;

converting the virtual machine disk file to a virtual machine disk file in the second format; and

copying the converted virtual machine disk file in the second format to a location where it may be utilized by a virtual machine of the second type.

9. The automated method of claim 1 wherein automatically determining includes automatically determining multiple virtual machines hosted by a virtual machine host, and wherein the method further comprises:

determining a threshold number of simultaneous storage operations that can be performed upon the data of the multiple virtual machines hosted by the virtual machine host; and

performing multiple storage operations upon the data of the multiple virtual machines hosted by the virtual machine host,

wherein the number of simultaneous copy operations performed upon the data of the multiple virtual machines hosted by the virtual machine host does not exceed the threshold number.

10. The automated method of claim 1, wherein automatically determining includes automatically determining multiple virtual machines, and wherein the method further comprises:

populating a first index with metadata corresponding to data of a first virtual machine, wherein the metadata includes a location of the data of the first virtual machine on the storage device;

populating a second index with metadata corresponding to the multiple virtual machines, wherein the metadata includes a reference to the first index;

receiving a request to access data of the first virtual machine on the storage device;

analyzing the metadata included in the second index;

determining the reference to the first index;

analyzing the metadata included in the first index; and

displaying metadata corresponding to the data of the first virtual machine.

11. At least one computer-readable medium, excluding transitory signals, and storing computer-executable instructions to perform an automated method of managing data of one or more virtual machines managed by a virtual machine manager, wherein the method is performed by one or more physical computing systems, each computing system having a processor and memory, the method comprising:

receiving a request to copy data of one or more virtual machines to a physical storage device;

determining whether the one or more virtual machines are managed by a virtual machine manager,

wherein the determining is performed by reading a stored indication of the virtual machine manager, or by scanning a network for the virtual machine manager;

based on the determining that the one or more virtual machines are managed by the virtual machine manager, automatically accessing the virtual machine manager;

automatically requesting information from the virtual machine manager regarding virtual machines that the virtual machine manager manages;

automatically determining, by the one or more computing systems, from information provided by the virtual machine manager, the virtual machines managed by the virtual machine manager; and

for at least one of the determined virtual machines, copying the data of the virtual machine to the physical storage device.

12. The computer-readable medium of claim 11, wherein the request specifies that a disk-level copy is to be performed for at least one virtual machine managed by the virtual machine manager, and wherein the method further comprises:

determining a virtual machine disk file in which the at least one virtual machine stores data;

copying the virtual machine disk file from a filesystem of a virtual machine host hosting the at least one virtual machine to the storage device;

extracting information from the virtual machine disk file, wherein extracting information includes extracting information that describes a location of a virtual file allocation table utilized by the at least one virtual machine;

associating the extracted information with the at least one virtual machine; and

storing the extracted information in association with the at least one virtual machine.

13. The computer-readable medium of claim 11, wherein the request specifies that a disk-level copy is to be performed for at least one virtual machine managed by the virtual machine manager, and wherein the method further comprises:

determining at least two virtual machine disk files in which the at least one virtual machine stores data;

copying the at least two virtual machine disk files from a filesystem of a virtual machine host hosting the at least one virtual machine to the storage device;

extracting from the at least two virtual machine disk files information that describes the relationships between the virtual machine disk files;

extracting from the at least two virtual machine disk files information that describes at least one virtual volume contained within one or more of the at least two virtual machine disk files;

extracting from the at least two virtual machine disk files information that describes at least one virtual file allocation table that describes locations of virtual files contained within the at least one virtual volume; and

storing the extracted information.

14. The computer-readable medium of claim 11, further comprising:

determining distinct data objects within the virtual machine data;

indexing the distinct data objects within the virtual machine data;

storing the indexed distinct data objects in an index, wherein the index also includes indexed distinct data objects associated with at least one non-virtual machine;

receiving a request to query the index; and

returning search results responsive to the request, wherein the search results include search results associated with distinct data objects of the least one virtual machine and with distinct data objects at least one non-virtual machine.

15. The computer-readable medium of claim 11, further comprising:

determining distinct data objects within the virtual machine data; and

for at least one of the distinct data objects;

generating a substantially unique identifier for the distinct data object;

determining, based on the substantially unique identifier, if an instance of the distinct data object has already been stored on a single instance storage device; and

if an instance of the distinct data object has not already been stored on the single instance storage device, then storing the data object on the single instance storage device.

16. A system configured to manage data of one or more virtual machines in a computing network, the system comprising:

at least one physical processor;

at least one physical memory device, coupled to the at least one processor, and storing instructions to be executed by the at least one processor to perform a method, the method comprising;

receiving a request to copy data of one or more virtual machines to a physical storage device;

determining whether the one or more virtual machines are managed by a virtual machine manager,

wherein the determining is performed by reading a stored indication of the virtual machine manager, or by scanning a network for the virtual machine manager;

based on the determining that the one or more virtual machines are managed by the virtual machine manager, automatically accessing the virtual machine manager;

automatically requesting information from the virtual machine manager regarding virtual machines that the virtual machine manager manages;

automatically determining, by the one or more computing systems, from information provided by the virtual machine manager, the virtual machines managed by the virtual machine manager; and

for at least one of the determined virtual machines, copying the data of the virtual machine to the physical storage device.

17. The system of claim 16, wherein the request specifies that a disk-level copy is to be performed for at least one virtual machine managed by the virtual machine manager, and wherein the method further comprises:

determining a virtual machine disk file in which the at least one virtual machine stores data;

copying the virtual machine disk file from a filesystem of a virtual machine host hosting the at least one virtual machine to the storage device;

extracting information from the virtual machine disk file, wherein extracting information includes extracting information that describes a location of a virtual file allocation table utilized by the at least one virtual machine;

associating the extracted information with the at least one virtual machine; and

storing the extracted information in association with the at least one virtual machine.

18. The system of claim 16, wherein the request specifies that a disk-level copy is to be performed for at least one virtual machine managed by the virtual machine manager, and wherein the method further comprises:

determining at least two virtual machine disk files in which the at least one virtual machine stores data;

copying the at least two virtual machine disk files from a filesystem of a virtual machine host hosting the at least one virtual machine to the storage device;

extracting from the at least two virtual machine disk files information that describes the relationships between the virtual machine disk files;

extracting from the at least two virtual machine disk files information that describes at least one virtual volume contained within one or more of the at least two virtual machine disk files;

extracting from the at least two virtual machine disk files information that describes at least one virtual file allocation table that describes locations of virtual files contained within the at least one virtual volume; and

storing the extracted information.

19. The system of claim 16, wherein the method further comprises:

determining distinct data objects within the virtual machine data;

indexing the distinct data objects within the virtual machine data;

storing the indexed distinct data objects in an index, wherein the index also includes indexed distinct data objects associated with at least one non-virtual machine;

receiving a request to query the index; and

returning search results responsive to the request, wherein the search results include search results associated with distinct data objects of the least one virtual machine and with distinct data objects at least one non-virtual machine.

20. The system of claim 16, wherein the method further comprises:

determining distinct data objects within the virtual machine data; and

for at least one of the distinct data objects;

generating a substantially unique identifier for the distinct data object;

determining, based on the substantially unique identifier, if an instance of the distinct data object has already been stored on a single instance storage device; and

if an instance of the distinct data object has not already been stored on the single instance storage device, then storing the data object on the single instance storage device.

21. The system of claim 16, wherein the request further specifies at least one of the following:

that a file-level copy is to be performed for at least one virtual machine managed by the virtual machine manager;

that a volume-level copy is to be performed for at least one virtual machine managed by the virtual machine manager; and

that a disk-level copy is to be performed for at least one virtual machine managed by the virtual machine manager.