MANAGEMENT OF VIRTUAL MACHINE SNAPSHOTS

• US 20160124665A1
• Filed: 02/20/2015
• Published: 05/05/2016
• Est. Priority Date: 11/04/2014
• Status: Active Grant

• Current Assignee: Rubrik, Inc.
• Original Assignee: Rubrik, Inc.
• Inventors: Jain, Arvind, Nithrakashyap, Arvind, Mazumdar, Soham, To, Kenny, Botelho, Fabiano
• Granted Patent: US 10,114,564 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1484 involving virtual machines
• CPC Class Codes: G06F 11/202 where processing functionality is redundant redundant communication control functionality G06F11/2005, redundant storage control functionality G06F11/2089
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/84 Mapping; Conversion
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2201/80 : Database-specific techniques
• CPC Class Codes: G06F 2201/815 : Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 2201/84 : Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0619 : in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 : De-duplication techniques
• CPC Class Codes: G06F 3/065 : Replication mechanisms
• CPC Class Codes: G06F 3/0665 : at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0685 : Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: G06F 9/45558 : Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5077 : Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: H04L 43/0817 : by checking functioning
• CPC Class Codes: H04L 61/5007 : Internet protocol [IP] addresses
• CPC Class Codes: H04L 61/5061 : Pools of addresses
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 9/3242 : involving keyed hash functions, e.g. message authentication codes [MACs], CBC-MAC or HMAC
• CPC Class Codes: H04L 9/3247 : involving digital signatures

Abstract

Methods and systems for managing, storing, and serving data within a virtualized environment are described. In some embodiments, a data management system may manage the extraction and storage of virtual machine snapshots, provide near instantaneous restoration of a virtual machine or one or more files located on the virtual machine, and enable secondary workloads to directly use the data management system as a primary storage target to read or modify past versions of data. The data management system may allow a virtual machine snapshot of a virtual machine stored within the system to be directly mounted to enable substantially instantaneous virtual machine recovery of the virtual machine.

Claims

1. A method for operating a data management system, comprising:

acquiring a set of files corresponding with a plurality of versions of a virtual machine;

determining a first version of the plurality of versions associated with a base file of the set of files;

determining a second version of the plurality of versions associated with a new base file to be generated, the first version of the virtual machine corresponds with a first state of the virtual machine at a first point in time, the second version of the virtual machine corresponds with a second state of the virtual machine at a second point in time subsequent to the first point in time;

generating the new base file corresponding with the second version, the generating the new base file includes patching the base file with one or more forward incremental files of the set of files, the one or more forward incremental files correspond with the second version and one or more versions of the virtual machine between the first version and the second version;

generating a set of reverse incremental files corresponding with the first version and the one or more versions of the virtual machine between the first version and the second version;

storing the new base file using a first storage device; and

storing the set of reverse incremental files using a second storage device different from the first storage device.

2. The method of claim 1, further comprising:

detecting that a threshold period of time has passed since the base file was generated; and

performing the generating the new base file and the generating the set of reverse incremental files in response to detecting that the threshold period of time has passed since the base file was generated.

3. The method of claim 1, further comprising:

detecting that a number of forward incremental files that need to be patched to the base file in order to generate the second version is greater than a threshold number of forward incremental files; and

performing the generating the new base file and the generating the set of reverse incremental files in response to detecting that the number of forward incremental files that need to be patched to generate the second version is greater than the threshold number of forward incremental files.

4. The method of claim 1, wherein:

the generating the new base file includes reading a first file of the set of files from a distributed file system, detecting a data error in the first file using a checksum comparison, and correcting the data error in the first file in response to detecting the data error.

5. The method of claim 1, further comprising:

updating a set of merged files to reference the new base file, the set of merged files corresponds with merged files for the first version, the second version, and the one or more versions of the virtual machine between the first version and the second version.

6. The method of claim 1, further comprising:

acquiring a backup policy associated with the virtual machine;

identifying a second set of reverse incremental files associated with versions of the virtual machine that are older than the first version based on the backup policy; and

transferring the second set of reverse incremental files from the second storage device to the first storage device.

7. The method of claim 1, further comprising:

acquiring a backup policy associated with the virtual machine, the backup policy specifies a maximum number of snapshots allowed for the virtual machine for a particular time period;

identifying a second set of reverse incremental files based on the backup policy; and

consolidating the second set of reverse incremental files.

8. The method of claim 1, wherein:

the first storage device has a first read speed; and

the second storage device has a second read speed that is faster than the first read speed.

9. The method of claim 1, further comprising:

detecting that a rate of change in a number of forward incremental files for the virtual machine is greater than a threshold; and

performing the generating the new base file and the generating the set of reverse incremental files in response to detecting that the rate of change in the number of forward incremental files for the virtual machine is greater than the threshold.

10. The method of claim 1, further comprising:

detecting that there is no dependency on the set of files; and

deleting the set of files in response to detecting that there is no dependency on the set of files.

11. The method of claim 10, wherein:

the detecting that there is no dependency on the set of files includes detecting that no merged files stored within the data management system include a pointer to any of the set of files.

12. A data management system, comprising:

a first storage device, the first storage device stores a base file associated with a first version of a virtual machine;

a second storage device, the second storage device stores one or more forward incremental files associated with one or more versions of the virtual machine; and

one or more processors in communication with the first storage device and the second storage device, the one or more processors determine a second version of the one or more versions associated with a new base file to be generated, the one or more forward incremental files correspond with the second version and one or more versions of the virtual machine between the first version and the second version, the one or more processors generate the new base file corresponding with the second version using the base file and the one or more forward incremental files, the one or more processors generate a set of reverse incremental files corresponding with the first version and the one or more versions of the virtual machine between the first version and the second version, the one or more processors cause the new base file to be stored using the first storage device and the set of reverse incremental files to be stored using the second storage device.

13. The data management system of claim 12, wherein:

the first version of the virtual machine corresponds with a first state of the virtual machine at a first point in time, the second version of the virtual machine corresponds with a second state of the virtual machine at a second point in time subsequent to the first point in time.

14. The data management system of claim 12, wherein:

the one or more processors detect that a threshold period of time has passed since the base file was generated, the one or more processors generate the new base file and generate the set of reverse incremental files in response to detecting that the threshold period of time has passed since the base file was generated.

15. The data management system of claim 12, wherein:

the one or more processors detect that a number of forward incremental files that need to be patched to the base file in order to generate the second version is greater than a threshold number of forward incremental files, the one or more processors generate the new base file and generate the set of reverse incremental files in response to detecting that the number of forward incremental files that need to be patched to generate the second version is greater than the threshold number of forward incremental files.

16. The data management system of claim 12, wherein:

the one or more processors acquire a backup policy associated with the virtual machine and identify a second set of reverse incremental files associated with versions of the virtual machine that are older than the first version based on the backup policy, the one or more processors transfer the second set of reverse incremental files from the second storage device to the first storage device.

17. The data management system of claim 12, wherein:

the one or more processors acquire a backup policy associated with the virtual machine, the backup policy specifies a maximum number of snapshots allowed for the virtual machine for a particular time period, the one or more processors identify a second set of reverse incremental files based on the backup policy and consolidate the second set of reverse incremental files.

18. One or more storage devices containing processor readable code for programming one or more processors to perform a method for operating a data management system, the processor readable code comprising:

processor readable code configured to determine a first version of a virtual machine, the first version is associated with a base file;

processor readable code configured to determine a second version of the virtual machine associated with a new base file to be generated, the first version of the virtual machine corresponds with a first state of the virtual machine at a first point in time, the second version of the virtual machine corresponds with a second state of the virtual machine at a second point in time subsequent to the first point in time;

processor readable code configured to generate the new base file corresponding with the second version using the base file and one or more forward incremental files, the one or more forward incremental files correspond with the second version and one or more versions of the virtual machine between the first version and the second version;

processor readable code configured to generate a set of reverse incremental files corresponding with the first version and the one or more versions of the virtual machine between the first version and the second version;

processor readable code configured to store the new base file using a first storage device; and

processor readable code configured to store the set of reverse incremental files using a second storage device different from the first storage device.

19. The one or more storage devices of claim 18, wherein the processor readable code further comprises:

processor readable code configured to detect that a number of forward incremental files that need to be patched to the base file in order to generate the second version is greater than a threshold number of forward incremental files; and

processor readable code configured to generate the new base file and the set of reverse incremental files in response to detecting that the number of forward incremental files that need to be patched to generate the second version is greater than the threshold number of forward incremental files.

20. The one or more storage devices of claim 18, wherein:

the first storage device comprises a magnetic storage device; and

the second storage device comprises a solid-state storage device.