System and method for file-based virtual machine incremental backup

• US 8,751,515 B1
• Filed: 03/30/2012
• Issued: 06/10/2014
• Est. Priority Date: 03/30/2012
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Xing, Jian, Raj, Anand
• Primary Examiner(s): VO, TRUONG V
• Application Number:
  US13/436,460
• Time in Patent Office: 802 Days
• Field of Search: 707/755
• US Class Current: 707/755
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446

Abstract

A source virtual machine (VM) disk file associated with a VM is parsed to generate metadata representing files stored in a virtual storage device of the VM, where the VM is one of VMs hosted by a VM monitor (VMM). In response to a request for an incremental backup of files stored in the source VM disk file, a file attribute of each of the files is examined based on the metadata to identify one or more files that have been modified within a predetermined past period of time. File records associated with the identified files are determined based on the metadata that have been modified within the predetermined past period of time. The identified file records are then backed up from the source VM disk file to a target VM disk file, without having to back up unmodified files in the source VM disk file.

Claims

1. A computer-implemented method, comprising:

parsing, by a backup engine, a source virtual machine (VM) disk file associated with a VM to generate metadata representing a plurality of files stored in a virtual storage device of the VM, the VM being one of one or more VMs hosted by a VM monitor (VMM) of a source storage system, wherein the backup engine operates in a backup server that is communicatively coupled to the source storage system over a network, wherein the backup engine remotely accesses the source VM disk file via an application programming interface (API) of the VMM of the source storage system over the network, wherein the backup server is configured to communicate with a plurality of source storage systems over the network and to back up data from the plurality of source storage systems;

generating, by the backup engine, a file index based on the metadata representing the source VM disk file based on the parsing of the source VM disk file;

in response to a request for an incremental backup of files stored in the source VM disk file, examining based on a file attribute of each of the files from the file index to identify one or more files that have been modified within a predetermined past period of time;

determining from the file index one or more file records associated with the identified files that have been modified within the predetermined past period of time; and

transmitting, by the backup engine, to the VMM of the source storage system via the API, information representing the identified file records to back up the identified file records from the source VM disk file to a target VM disk file of a target storage system that is remote to the source storage system over the network, without having to back up unmodified files in the source VM disk file.

2. The method of claim 1, wherein the file attribute comprises a time attribute indicating when the corresponding file was modified.

3. The method of claim 1, further comprising populating a VM disk header of the target VM disk file based on the metadata, including information of at least a portion of a VM disk header of the source VM disk file, such that the target VM disk file can be mounted as a standalone VM disk that only contains the requested subdirectory of files.

4. The method of claim 1, wherein the parsing of the source VM disk file is performed without having to mount the source VM disk file as a VM disk in a proxy server.

5. The method of claim 1, further comprising:

causing the VMM to create a snapshot of the source VM disk file, wherein the parsing of the source VM disk file is performed based on the snapshot of the source VM disk file; and

causing the VMM to erase the snapshot of the source VM disk file after the parsing of the source VM disk file is completed.

6. The method of claim 1, further comprising:

storing the file index in a file index database, wherein the file index database includes a plurality of entries, each entry including a file identifier (ID) identifying a file stored in the source VM disk file and a file record ID identifying a file record of a file system associated with the source VM disk that stores the corresponding file; and

providing an interface for querying the file index database, wherein the file records of the requested subdirectory are identified by querying the file index database via the interface.

7. The method of claim 1, further comprising invoking a deduplicated segment storage engine to back up segments of each file of the source VM disk file that have not been stored in the target VM disk file.

8. A non-transitory computer-readable storage medium having instructions stored therein, which when executed by a computer, cause the computer to perform operations, the operations comprising:

parsing, by a backup engine, a source virtual machine (VM) disk file associated with a VM to generate metadata representing a plurality of files stored in a virtual storage device of the VM, the VM being one of one or more VMs hosted by a VM monitor (VMM) of a source storage system, wherein the backup engine operates in a backup server that is communicatively coupled to the source storage system over a network, wherein the backup engine remotely accesses the source VM disk file via an application programming interface (API) of the VMM of the source storage system over the network, wherein the backup server is configured to communicate with a plurality of source storage systems over the network and to back up data from the plurality of source storage systems;

generating, by the backup engine, a file index based on the metadata representing the source VM disk file based on the parsing of the source VM disk file;

in response to a request for an incremental backup of files stored in the source VM disk file, examining based on a file attribute of each of the files from the file index to identify one or more files that have been modified within a predetermined past period of time;

determining from the file index one or more file records associated with the identified files that have been modified within the predetermined past period of time; and

transmitting, by the backup engine, to the VMM of the source storage system via the API, information representing the identified file records to back up the identified file records from the source VM disk file to a target VM disk file of a target storage system that is remote to the source storage system over the network, without having to back up unmodified files in the source VM disk file.

9. The non-transitory computer-readable storage medium of claim 8, wherein the file attribute comprises a time attribute indicating when the corresponding file was modified.

10. The non-transitory computer-readable storage medium of claim 8, wherein the operations further comprise populating a VM disk header of the target VM disk file based on the metadata, including information of at least a portion of a VM disk header of the source VM disk file, such that the target VM disk file can be mounted as a standalone VM disk that only contains the requested subdirectory of files.

11. The non-transitory computer-readable storage medium of claim 8, wherein the parsing of the source VM disk file is performed without having to mount the source VM disk file as a VM disk in a proxy server.

12. The non-transitory computer-readable storage medium of claim 8, wherein the operations further comprise:

causing the VMM to create a snapshot of the source VM disk file, wherein the parsing of the source VM disk file is performed based on the snapshot of the source VM disk file; and

causing the VMM to erase the snapshot of the source VM disk file after the parsing of the source VM disk file is completed.

13. The non-transitory computer-readable storage medium of claim 8, wherein the operations further comprise:

storing the file index in a file index database, wherein the file index database includes a plurality of entries, each entry including a file identifier (ID) identifying a file stored in the source VM disk file and a file record ID identifying a file record of a file system associated with the source VM disk that stores the corresponding file; and

providing an interface for querying the file index database, wherein the file records of the requested subdirectory are identified by querying the file index database via the interface.

14. The non-transitory computer-readable storage medium of claim 8, wherein the operations further comprise invoking a deduplicated segment storage engine to back up segments of each file of the source VM disk file that have not been stored in the target VM disk file.

15. A storage system representing a backup server, comprising:

a virtual machine (VM) disk parser to parse a source VM disk file associated with a VM to generate metadata representing a plurality of files stored in a virtual storage device of the VM, the VM being one of one or more VMs hosted by a VM monitor (VMM) of a source storage system, wherein the storage system operates as a backup server that is communicatively coupled to the source storage system over a network, wherein the VM parser remotely accesses the source VM disk file via an application programming interface (API) of the VMM of the source storage system over the network, wherein the backup server is configured to communicate with a plurality of source storage systems over the network and to back up data from the plurality of source storage systems, wherein the VM disk parser generates a file index based on the metadata representing the source VM disk file based on the parsing of the source VM disk file;

a backup engine, in response to a request for an incremental backup of files stored in the source VM disk file, to examine based on a file attribute of each of the files from the file index to identify one or more files that have been modified within a predetermined past period of time, to determine from the file index one or more file records associated with the identified files that have been modified within the predetermined past period of time, and to transmit to the VMM of the source storage system via the API information representing the identified file records to back up identified file records from the source VM disk file to a target VM disk file of a target storage system that is remote to the source storage system over the network, without having to back up unmodified files in the source VM disk file.

16. The system of claim 15, wherein the file attribute comprises a time attribute indicating when the corresponding file was modified.

17. The system of claim 15, wherein the backup engine is to populate a VM disk header of the target VM disk file based on the metadata, including information of at least a portion of a VM disk header of the source VM disk file, such that the target VM disk file can be mounted as a standalone VM disk that only contains the requested subdirectory of files.

18. The system of claim 15, wherein the parsing of the source VM disk file is performed without having to mount the source VM disk file as a VM disk in a proxy server.

19. The system of claim 15, wherein the backup engine is to

cause the VMM to create a snapshot of the source VM disk file, wherein the parsing of the source VM disk file is performed based on the snapshot of the source VM disk file, and

cause the VMM to erase the snapshot of the source VM disk file after the parsing of the source VM disk file is completed.

20. The system of claim 15, further comprising a file index database to store the file index, wherein the file index database includes a plurality of entries, each entry including a file identifier (ID) identifying a file stored in the source VM disk file and a file record ID identifying a file record of a file system associated with the source VM disk that stores the corresponding file, wherein the file index database includes an interface for querying the file index database, wherein the file records of the requested subdirectory are identified by querying the file index database via the interface.

21. The system of claim 15, wherein the backup engine is to invoke a deduplicated segment storage engine to back up segments of each file of the source VM disk file that have not been stored in the target VM disk file.