HYBRID CLOUD DATA MANAGEMENT SYSTEM

• US 20160125059A1
• Filed: 02/20/2015
• Published: 05/05/2016
• Est. Priority Date: 11/04/2014
• Status: Abandoned Application

• Current Assignee: Rubrik, Inc.
• Original Assignee: Rubrik, Inc.
• Inventors: Jain, Arvind, Nithrakashyap, Arvind, Mazumdar, Soham, Botelho, Fabiano, Agarwal, Arpit
• Application Number:
  US14/628,031
• Publication Number:
  US 20160125059A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1484 involving virtual machines
• CPC Class Codes: G06F 11/202 where processing functionality is redundant redundant communication control functionality G06F11/2005, redundant storage control functionality G06F11/2089
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/84 Mapping; Conversion
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2201/80 : Database-specific techniques
• CPC Class Codes: G06F 2201/815 : Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 2201/84 : Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0619 : in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 : De-duplication techniques
• CPC Class Codes: G06F 3/065 : Replication mechanisms
• CPC Class Codes: G06F 3/0665 : at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0685 : Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: G06F 9/45558 : Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5077 : Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: H04L 43/0817 : by checking functioning
• CPC Class Codes: H04L 61/5007 : Internet protocol [IP] addresses
• CPC Class Codes: H04L 61/5061 : Pools of addresses
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 9/3242 : involving keyed hash functions, e.g. message authentication codes [MACs], CBC-MAC or HMAC
• CPC Class Codes: H04L 9/3247 : involving digital signatures

Abstract

Methods and systems for managing, storing, and serving data within a virtualized environment are described. In some embodiments, a data management system may manage the extraction and storage of virtual machine snapshots, provide near instantaneous restoration of a virtual machine or one or more files located on the virtual machine, and enable secondary workloads to directly use the data management system as a primary storage target to read or modify past versions of data. The data management system may allow a virtual machine snapshot of a virtual machine stored within the system to be directly mounted to enable substantially instantaneous virtual machine recovery of the virtual machine.

Claims

1. A method for operating a data management system, comprising:

acquiring a first snapshot of a first virtual machine;

storing a full image associated with the first snapshot within a first storage domain;

receiving an instruction to transfer the first snapshot to a second storage domain different from the first storage domain;

identifying a base image associated with a second virtual machine different from the first virtual machine in response to receiving the instruction;

generating a dependent base file using the full image and the base image; and

transferring the dependent base file to the second storage domain.

2. The method of claim 1, wherein:

the first storage domain includes a local storage appliance;

the second storage domain includes a remote storage appliance;

the storing a full image associated with the first snapshot includes storing the full image using the local storage appliance, the full image comprises data associated with one or more virtual disks of the first virtual machine; and

the transferring the dependent base file to the second storage domain includes transferring the dependent base file to the remote storage appliance.

3. The method of claim 1, wherein:

the second storage domain is associated with a cloud-based storage service; and

the transferring the dependent base file to the second storage domain includes transferring the dependent base file to the cloud-based storage service.

4. The method of claim 1, wherein:

the identifying a base image associated with the second virtual machine includes detecting that the second virtual machine is stored within both the first storage domain and the second storage domain and determining that the second virtual machine comprises a virtual machine within the second storage domain with the greatest amount of data similarity between a first portion of the virtual machine and a corresponding portion of the first virtual machine.

5. The method of claim 1, further comprising:

storing the base image associated with the second virtual machine within the first storage domain;

storing a copy of the base image associated with the second virtual machine within the second storage domain;

generating a copy of the full image associated with the first snapshot using the dependent base file and the copy of the base image; and

storing the copy of the full image associated with the first snapshot within the second storage domain.

6. The method of claim 5, wherein:

the first storage domain includes a local storage appliance;

the second storage domain includes a remote storage appliance;

the storing the base image associated with the second virtual machine within the first storage domain includes storing the base image using the local storage appliance;

the storing a copy of the base image associated with the second virtual machine within the second storage domain includes storing the copy of the base image using the remote storage appliance; and

the generating a copy of the full image associated with the first snapshot includes patching the dependent base file to the copy of the base image associated with the second virtual machine using the remote storage appliance.

7. The method of claim 1, further comprising:

acquiring one or more snapshots of the first virtual machine, the first snapshot corresponds with a first state of the first virtual machine at a first point in time, the one or more snapshots correspond with other states of the first virtual machine at points in time prior to the first point in time; and

storing one or more incremental files corresponding with the one or more snapshots within the first storage domain.

8. The method of claim 7, further comprising:

detecting that the first storage domain stores a number of snapshots for the first virtual machine that is greater than a maximum number of snapshots allowed for the first virtual machine within the first storage domain;

transferring a subset of the one or more incremental files to the second storage domain in response to the detecting that the first storage domain stores a number of snapshots for the first virtual machine that is greater than a maximum number of snapshots allowed for the first virtual machine within the first storage domain; and

deleting the subset of the one or more incremental files from the first storage domain.

9. The method of claim 7, further comprising:

acquiring a maximum number of snapshots for the first virtual machine within the first storage domain;

determining that the one or more incremental files associated with the one or more snapshots should be transferred to the second storage domain based on the maximum number of snapshots for the first virtual machine being exceeded;

transferring the one or more incremental files to the second storage domain; and

deleting the one or more incremental files from the first storage domain.

10. The method of claim 7, further comprising:

acquiring a maximum age for snapshots within the first storage domain;

detecting that each snapshot of the one or more snapshots exceeds the maximum age for snapshots within the first storage domain;

transferring the one or more incremental files to the second storage domain in response to detecting that each snapshot of the one or more snapshots exceeds the maximum age for snapshots within the first storage domain; and

deleting the one or more incremental files from the first storage domain.

11. The method of claim 7, further comprising:

acquiring a threshold number of snapshots for the first virtual machine within the first storage domain;

determining that the one or more incremental files associated with the one or more snapshots should be transferred to the second storage domain based on the threshold number of snapshots for the first virtual machine being exceeded; and

transferring the one or more incremental files to the second storage domain.

12. A data management system, comprising:

a memory, the memory stores a full image associated with a first snapshot of a first virtual machine, the full image comprises data associated with one or more virtual disks of the first virtual machine; and

one or more processors in communication with the memory, the one or more processors receive an instruction to transmit the first snapshot to a remote storage domain, the one or more processors identify a base image associated with a second virtual machine different from the first virtual machine in response to receiving the instruction, the one or more processors generate a dependent base file using the full image and the base image, the one or more processors transmit the dependent base file to the remote storage domain.

13. The data management system of claim 12, wherein:

the remote storage domain is associated with a cloud-based storage service.

14. The data management system of claim 12, wherein:

the one or more processors identify the base image associated with the second virtual machine by detecting that the second virtual machine is stored within the remote storage domain and determining that the second virtual machine comprises a virtual machine within the remote storage domain with the greatest amount of data similarity between a first portion of the virtual machine and a corresponding portion of the first virtual machine.

15. The data management system of claim 12, further comprising:

a remote storage appliance within the remote storage domain, the remote storage appliance stores a copy of the base image, the remote storage appliance generates a copy of the full image using the dependent base file and the copy of the base image, the remote storage appliance stores the copy of the full image within the remote storage appliance.

16. The data management system of claim 12, wherein:

the one or more processors acquire one or more snapshots of the first virtual machine, the first snapshot corresponds with a first state of the first virtual machine at a first point in time, the one or more snapshots correspond with other states of the first virtual machine at points in time prior to the first point in time; and

the memory stores one or more incremental files corresponding with the one or more snapshots of the first virtual machine.

17. The data management system of claim 16, wherein:

the one or more processors detect that the memory stores a number of snapshots for the first virtual machine that is greater than a maximum number of snapshots allowed for the first virtual machine, the one or more processors transfer a subset of the one or more incremental files to the remote storage domain in response to detecting that the memory stores more than the maximum number of snapshots allowed for the first virtual machine.

18. The data management system of claim 16, wherein:

the one or more processors acquire a maximum number of snapshots for the first virtual machine and determine that the one or more incremental files associated with the one or more snapshots should be transferred to the remote storage domain based on the maximum number of snapshots for the first virtual machine being exceeded, the one or more processors transfer the one or more incremental files to the remote storage domain.

19. The data management system of claim 16, wherein:

the one or more processors acquire a maximum age for snapshots and detect that each snapshot of the one or more snapshots exceeds the maximum age for snapshots, the one or more processors transfer the one or more incremental files to the remote storage domain in response to detecting that each snapshot of the one or more snapshots exceeds the maximum age for snapshots.

20. One or more storage devices containing processor readable code for programming one or more processors to perform a method for operating a data management system, the processor readable code comprising:

processor readable code configured to acquire a first snapshot of a first virtual machine;

processor readable code configured to store a full image associated with the first snapshot within a first storage domain, the full image comprises data associated with one or more virtual disks of the first virtual machine;

processor readable code configured to receive an instruction to transfer the first snapshot to a second storage domain different from the first storage domain;

processor readable code configured to identify a base image associated with a second virtual machine different from the first virtual machine in response to receiving the instruction;

processor readable code configured to generate a dependent base file using the full image and the base image; and

processor readable code configured to transfer the dependent base file to the second storage domain.