Converged search and archival system

• US 10,133,495 B2
• Filed: 02/20/2015
• Issued: 11/20/2018
• Est. Priority Date: 11/04/2014
• Status: Active Grant

• Current Assignee: Rubrik, Inc.
• Original Assignee: Rubrik, Inc.
• Inventors: Jain, Arvind, Nithrakashyap, Arvind, Mazumdar, Soham, Madheswaran, Jayanth, Menon, Aravind
• Primary Examiner(s): Vu, Bai D
• Application Number:
  US14/628,019
• Publication Number:
  US 20160124972A1
• Time in Patent Office: 1,369 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1484 involving virtual machines
• CPC Class Codes: G06F 11/202 where processing functionality is redundant redundant communication control functionality G06F11/2005, redundant storage control functionality G06F11/2089
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/84 Mapping; Conversion
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2201/80 : Database-specific techniques
• CPC Class Codes: G06F 2201/815 : Virtual
• CPC Class Codes: G06F 2201/84 : Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0619 : in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 : De-duplication techniques
• CPC Class Codes: G06F 3/065 : Replication mechanisms
• CPC Class Codes: G06F 3/0665 : at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0685 : Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: G06F 9/45558 : Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5077 : Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: H04L 43/0817 : by checking functioning
• CPC Class Codes: H04L 61/5007 : Internet protocol [IP] addresses
• CPC Class Codes: H04L 61/5061 : Pools of addresses
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 9/3242 : involving keyed hash functions, e.g. message authentication codes [MACs], CBC-MAC or HMAC
• CPC Class Codes: H04L 9/3247 : involving digital signatures

Abstract

Methods and systems for managing, storing, and serving data within a virtualized environment are described. In some embodiments, a data management system may manage the extraction and storage of virtual machine snapshots, provide near instantaneous restoration of a virtual machine or one or more files located on the virtual machine, and enable secondary workloads to directly use the data management system as a primary storage target to read or modify past versions of data. The data management system may allow a virtual machine snapshot of a virtual machine stored within the system to be directly mounted to enable substantially instantaneous virtual machine recovery of the virtual machine.

Claims

1. A method for operating a data management system, comprising:

identifying a particular version of a file to be restored;

acquiring a virtual machine search index for a virtual machine, the virtual machine search index includes a mapping of the particular version of the file to a point in time snapshot of the virtual machine that includes the particular version of the file;

identifying a version of the virtual machine that includes the particular version of the file using the virtual machine search index;

identifying a base image for generating a portion of the version of the virtual machine;

identifying a set of incremental files for generating the portion of the version of the virtual machine;

determining a first portion of the base image that includes file system metadata, the file system metadata identifies a location of the file to be restored within the version of the virtual machine;

generating the portion of the version of the virtual machine less than the entire version of the virtual machine using the file system metadata and the set of incremental files, the generating the portion of the version of the virtual machine includes determining one or more regions within the base image that store the file to be restored and patching the one or more regions using the set of incremental files;

extracting the particular version of the file from the portion of the version of the virtual machine, the extracting the particular version of the file includes extracting the particular version of the file from the portion of the version of the virtual machine without generating a full image of the version of the virtual machine; and

outputting the particular version of the file.

2. The method of claim 1, wherein:

the one or more regions within the base image that store the file to be restored comprise a plurality of regions within the base image.

3. The method of claim 1, wherein:

the outputting the particular version of the file comprises transmitting the particular version of the file to a computing device.

4. The method of claim 1, further comprising:

acquiring a merged file corresponding with the version of the virtual machine;

the identifying the base image includes identifying the base image using the merged file; and

the identifying the set of incremental files includes identifying the set of incremental files using the merged file.

5. The method of claim 1, wherein:

the virtual machine search index includes the mapping of the particular version of the file to an earliest point in time snapshot of the virtual machine that includes the particular version of the file.

6. The method of claim 1, wherein:

the virtual machine search index includes the mapping of the particular version of the file to a latest point in time snapshot of the virtual machine that includes the particular version of the file.

7. The method of claim 1, wherein:

the virtual machine search index includes a version history for the file.

8. The method of claim 7, further comprising:

acquiring a new version of the virtual machine that includes an updated version of the file; and

updating the version history for the file to include a mapping of the updated version of the file to the new version of the virtual machine.

9. The method of claim 1, wherein:

the file system metadata includes information for locating the file within a virtual disk associated with the virtual machine.

10. The method of claim 9, wherein:

the file system metadata includes information for retrieving the file from an NTFS partition within the virtual disk.

11. The method of claim 1, wherein:

the file comprises a word processing document.

12. A data management system, comprising:

a memory configured to store a base image for generating a version of a virtual machine, a set of incremental files for generating the version of the virtual machine, and a virtual machine search index for the virtual machine; and

one or more processors in communication with the memory, the virtual machine search index includes a mapping of a particular version of a data object to a point in time snapshot of the virtual machine that includes the particular version of the data object, the one or more processors configured to identify the particular version of the data object to be restored and determine that the version of the virtual machine includes the particular version of the data object using the virtual machine search index, the one or more processors configured to determine a first portion of the base image that includes metadata for locating the data object within a virtual disk of the virtual machine, the one or more processors configured to generate a portion of the version of the virtual machine less than the entire version of the virtual machine using the metadata and the set of incremental files, the one or more processors configured to determine one or more regions within the base image that store the data object to be restored and patch the one or more regions using the set of incremental files, the one or more processors configured to extract the particular version of the data object from the portion of the version of the virtual machine without generating a full image of the version of the virtual machine and output the particular version of the data object.

13. The data management system of claim 12, wherein:

the one or more regions within the base image that store the data object to be restored comprise a plurality of regions within the base image.

14. The data management system of claim 12, wherein:

the one or more processors configured to transmit the particular version of the data object to a computing device.

15. The data management system of claim 12, wherein:

the one or more processors configured to acquire a merged file corresponding with the version of the virtual machine and identify the base image and the set of incremental files using the merged file.

16. The data management system of claim 12, wherein:

the virtual machine search index includes the mapping of the particular version of the data object to an earliest point in time snapshot of the virtual machine that includes the particular version of the data object.

17. The data management system of claim 12, wherein:

the virtual machine search index includes the mapping of the particular version of the data object to a latest point in time snapshot of the virtual machine that includes the particular version of the data object.

18. The data management system of claim 12, wherein:

the data object comprises a file and the metadata includes file system metadata for retrieving the file from a partition within the virtual disk.

19. The data management system of claim 12, wherein:

the data object comprises one of a file, a database, a spreadsheet, a word processing document, an electronic document, an image, a video, a text file, an executable file, an audio recording, an electronic message, or an email; and

the particular version of the data object comprises a point in time instance of the data object.

20. One or more storage devices containing processor readable code for programming one or more processors to perform a method for operating a data management system, the processor readable code comprising:

processor readable code configured to identify a particular version of a file to be restored;

processor readable code configured to identify a version of the virtual machine that includes the particular version of the file using a virtual machine search index, the virtual machine search index includes a mapping of the particular version of the file to a point in time snapshot of the virtual machine that includes the particular version of the file;

processor readable code configured to identify a base image for generating a portion of the version of the virtual machine;

processor readable code configured to identify a set of incremental files for generating the portion of the version of the virtual machine;

processor readable code configured to determine a first portion of the base image that includes file system metadata, the file system metadata identifies a location of the file within the version of the virtual machine;

processor readable code configured to generate the portion of the version of the virtual machine less than the entire version of the virtual machine using the file system metadata and the set of incremental files;

processor readable code configured to determine one or more regions within the base image that store the file to be restored and patch the one or more regions using the set of incremental files;

processor readable code configured to extract the particular version of the file from the portion of the version of the virtual machine without generating a full image of the version of the virtual machine; and

processor readable code configured to transfer the particular version of the file to a computing device.