Virtual server cloud file system for virtual machine backup from cloud operations

• US 10,565,067 B2
• Filed: 02/21/2017
• Issued: 02/18/2020
• Est. Priority Date: 03/09/2016
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Dornemann, Henry Wallace
• Primary Examiner(s): Pham, Tuan A
• Application Number:
  US15/437,864
• Publication Number:
  US 20170262350A1
• Time in Patent Office: 1,092 Days
• Field of Search: None
• CPC Class Codes: G06F 11/14 Error detection or correction of the data by redundancy in operation G06F11/16 takes precedence
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/1658 Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45575 Starting, stopping, suspending or resuming virtual machine instances
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

Uploads of restored virtual machine (“VM”) data to cloud storage, e.g., VM restore-to-cloud operations, are performed without having to write whole restored virtual disk files to a proxy server before the virtual disk data begins uploading to cloud. Restored data blocks from a backup source are locally cached, staged for efficiency, and asynchronously uploaded to the cloud page-by-page without tapping mass storage resources on the proxy. Downloads of VM data from cloud storage, e.g., VM backup-from-cloud, are performed without having to download a virtual disk file in its entirety to the proxy server before the backup operation begins generating a backup copy. This speeds up “pulling” VM data from the cloud by pre-fetching and locally caching downloaded data blocks. The cached data blocks are processed for backup and stored page-by-page directly into a secondary copy of the cloud VM virtual-disk file without tapping mass storage resource at the proxy.

Claims

1. A method for backing up a virtual-disk file associated with a virtual machine from a cloud-based account, the method comprising:

executing on a first computing device a pseudo-disk driver that presents a file system comprising a cache storage area on the first computing device as a local mass-data-storage source for data to be backed up, wherein the first computing device comprises one or more processors and computer memory;

storing to a first page in the cache storage area, by a data agent that executes on the first computing device, a first set of data blocks downloaded from a first virtual-disk file associated with a first virtual machine hosted by a cloud-based account,

wherein the cache storage area comprises a plurality of pages, including the first page, wherein each page in the plurality of pages is sized to match a page-size configured in the cloud-based account,

wherein the cloud-based account is configured to store the first virtual-disk file in its entirety, and

wherein the cache storage area on the first computing device is insufficient in storage capacity to store the first virtual-disk file in its entirety;

from the first page in the plurality of pages in the cache storage area,

processing for backup the downloaded data blocks in the first page from the cache storage area, resulting in a second set of backed up data blocks;

storing the second set of backed up data blocks into a secondary copy of the first virtual-disk file, which is stored in a storage device apart from the first computing device;

wherein the first virtual-disk file is backed up in its entirety from the cloud-based account to the secondary copy on the storage device, based on downloading data blocks into the cache storage area on the first computing device from the cloud-based account page-by-page, without first downloading the first virtual-disk file in its entirety to the first computing device; and

if no more of the first virtual-disk file remains to be downloaded, and one or more pages in the cache storage area have not been processed into backed up pages, processing the one or more pages in the cache storage area into respective backed up pages stored into the secondary copy of the first virtual-disk file, thereby completing a backup-from-cloud operation for the first virtual machine without first downloading the first virtual-disk file in its entirety to the first computing device.

2. The method of claim 1 wherein a backup-from-cloud operation for the first virtual machine is completed by using the cache storage area without first downloading the first virtual-disk file in its entirety to the first computing device and without using mass storage resources at the first computing device.

3. The method of claim 1 wherein the data agent addresses read requests to the file system and wherein each read request is served by the pseudo-disk driver from one or more pages in the cache storage area.

4. The method of claim 1 further comprising:

when a given page in the cache storage area receives one or more data blocks downloaded from the first virtual-disk file,

updating a list maintained by the data agent to indicate that the given page is most-recently used; and

based on the list, determining by the data agent which page in the cache storage area is least-recently used.

5. The method of claim 1 further comprising:

based on tracking whether each page in the cache storage area has been processed for backup,

reusing a given page that has been processed for further storing another set of data blocks downloaded from the first virtual-disk file.

6. A computer-readable medium, excluding transitory propagating signals, storing instructions that, when executed by at least one first computing device comprising one or more processors and computer memory, cause the first computing device to perform a method for backing up a virtual-disk file associated with a virtual machine from a cloud-based account, the method comprising:

executing on a first computing device a pseudo-disk driver that presents a cache storage area on the first computing device as a local mass-data storage device;

storing to a first page in the cache storage area, by a data agent that executes on the first computing device, a first set of data blocks downloaded from a first virtual-disk file associated with a first virtual machine hosted by a cloud-based account,

wherein the cache storage area comprises a plurality of pages, including the first page, wherein each page in the plurality of pages is sized to match a page-size configured in the cloud-based account,

wherein the cloud-based account is configured to store the first virtual-disk file in its entirety, and

wherein the cache storage area on the first computing device is insufficient in storage capacity to store the first virtual-disk file in its entirety;

from the first page in the plurality of pages in the cache storage area,

processing for backup the downloaded data blocks in the first page from the cache storage area, resulting in a second set of backed up data blocks;

storing the second set of backed up data blocks into a secondary copy of the first virtual-disk file, which is stored in a storage device apart from the first computing device;

wherein the first virtual-disk file is backed up in its entirety from the cloud-based account to the secondary copy on the storage device, based on downloading data blocks into the cache storage area on the first computing device from the cloud-based account page-by-page, without first downloading the first virtual-disk file in its entirety to the first computing device; and

if no more of the first virtual-disk file remains to be downloaded, and one or more pages in the cache storage area have not been processed into backed up pages, causing the one or more pages in the cache storage area to be processed into respective backed up pages stored into the secondary copy of the first virtual-disk file, thereby completing a backup-from-cloud operation for the first virtual machine without first downloading the first virtual-disk file in its entirety to the first computing device.

7. The computer-readable medium of claim 6, wherein the secondary copy is in a secondary data format and wherein the first virtual-disk file is in a primary data format that is accessible by the first virtual machine in the cloud-based account.

8. The computer-readable medium of claim 6 wherein the first virtual-disk file is backed up in its entirety from the cloud-based account to the secondary copy on the storage device without using mass storage resources at the first computing device.

9. The computer-readable medium of claim 6, wherein the downloading of data blocks into the cache storage area is based on anticipating a next page of data blocks to be downloaded to the cache storage area before a read request is issued for the next page of data blocks.