Agentless virtual disk metadata indexing

• US 10,642,790 B1
• Filed: 09/22/2017
• Issued: 05/05/2020
• Est. Priority Date: 09/22/2017
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Brenner, Adam, Starling, Jennifer
• Primary Examiner(s): Meng, Jau Shya
• Application Number:
  US15/713,040
• Time in Patent Office: 956 Days
• Field of Search: None
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/144 Query formulation
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: G06F 16/2433 Query languages
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2009/45587 Isolation or security of virtual machine instances
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

A method and system for implementing agentless virtual disk metadata indexing. Specifically, the disclosed method and system entails the parsing of data within a virtual machine and/or virtual disk snapshot to extract filesystem metadata information during data backup operations. Extraction of the filesystem metadata information is performed based on an understanding of a filesystem layout for a format in which the snapshot is defined, rather than based on the mounting and traversing of the snapshot. Furthermore, the extracted filesystem metadata information is encoded in a schema that may facilitate indexing of the information for use in a metadata searching and/or retrieval service.

Claims

1. A method for obtaining virtual disk metadata (VDM), comprising:

issuing a snapshot request comprising a virtual machine (VM) identifier;

receiving, in response to the snapshot request, a virtual disk image (VDI) for a virtual disk associated with the VM identifier;

parsing the VDI based on a filesystem layout for a format in which the VDI is received, wherein the filesystem layout is for a filesystem;

obtaining, from parsing the VDI, virtual disk data (VDD) and the VDM for the virtual disk;

generating an intermediate data object (IDO) using the VDM;

processing the IDO by building a full path for each directory and for content within each directory for the filesystem to obtain an IDO index, wherein the IDO index comprises a data structure in which the full path is stored; and

employing the IDO index to implement a VDM searching service.

2. The method of claim 1, wherein the VM identifier is associated with a VM implemented using at least the virtual disk.

3. The method of claim 1, wherein the VDI comprises the filesystem storing the VDD and the VDM, wherein the VDD comprises raw data in the filesystem, wherein the VDM comprises metadata describing the filesystem.

4. The method of claim 1, further comprising:

prior to processing of the IDO;

generating a virtual disk backup (VDB) using the VDD and the IDO; and

storing the VDB in backup storage wherefrom the IDO of the VDB is retrieved for processing.

5. The method of claim 1, wherein generating the IDO comprises encoding the VDM according to a schema, wherein the schema is based on a structured query language (SQL)ite relational database file format.

6. A system, comprising:

a virtual machine (VM) manager and backup storage;

a backup proxy node (BPN) operatively connected to the VM manager and the backup storage, and programmed to;

issue, to the VM manager, a snapshot request comprising a VM identifier;

receive, from the VM manager and in response to the snapshot request, a virtual disk image (VDI) relating to a virtual disk associated with the VM identifier;

parse the VDI based on a filesystem layout for a format in which the VDI is received, wherein the filesystem layout is for a filesystem;

obtain, from parsing the VDI, virtual disk data (VDD) and virtual disk metadata (VDM) for the virtual disk;

generate an intermediate data object (IDO) using the VDM; and

store, in the backup storage, a virtual disk backup (VDB) comprising the VDD and the IDO, and

an application programming interface (API) operatively connected to the backup storage, and programmed to;

issue, to the backup storage, a restore request comprising the VM identifier;

receive the IDO, extracted from the VDB, from the backup storage and in response to the restore request;

process the IDO by building a full path for each directory and for content within each directory for the filesystem to obtain an IDO index, wherein the IDO index comprises a data structure in which the full path is stored; and

employ the IDO index to implement a VDM searching service.

7. The system of claim 6, further comprising:

a cloud search service (CSS) comprising the API;

and

a VM host comprising the VM manager and at least one VM.

8. A non-transitory computer readable medium (CRM) comprising computer readable program code, which when executed by a computer processor, enables the computer processor to:

issue a snapshot request comprising a virtual machine (VM) identifier;

receive, in response to the snapshot request, a virtual disk image (VDI) for a virtual disk associated with the VM identifier;

parse the VDI based on a filesystem layout for a format in which the VDI is received, wherein the filesystem layout is for a filesystem; and

obtain, from parsing the VDI, virtual disk data (VDD) and the VDM for the virtual disk;

generate an intermediate data object (IDO) using the VDM;

process the IDO by building a full path for each directory and for content within each directory for the filesystem to obtain an IDO index, wherein the IDO index comprises a data structure in which the full path is stored; and

employ the IDO index to implement a VDM searching service.

9. The non-transitory CRM of claim 8, wherein the VM identifier is associated with a VM implemented using at least the virtual disk.

10. The non-transitory CRM of claim 8, wherein the VDI comprises the filesystem storing the VDD and the VDM, wherein the VDD comprises raw data in the filesystem, wherein the VDM comprises metadata describing the filesystem.

11. The non-transitory CRM of claim 8, comprising computer readable program code, which when executed by the computer processor, further enables the computer processor to:

prior to processing of the IDO;

generate a virtual disk backup (VDB) using the VDD and the IDO; and

store the VDB in backup storage wherefrom the IDO of the VDB is retrieved for processing.

12. The non-transitory CRM of claim 8, wherein generating the IDO comprises encoding the VDM according to a schema, wherein the schema is based on a structured query language (SQL)ite relational database file format.