Virtual server cloud file system for virtual machine restore to cloud operations

• US 10,592,350 B2
• Filed: 02/21/2017
• Issued: 03/17/2020
• Est. Priority Date: 03/09/2016
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Dornemann, Henry Wallace
• Primary Examiner(s): Pham, Tuan A
• Application Number:
  US15/437,841
• Publication Number:
  US 20170262347A1
• Time in Patent Office: 1,120 Days
• Field of Search: None
• CPC Class Codes: G06F 11/14 Error detection or correction of the data by redundancy in operation G06F11/16 takes precedence
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/1658 Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45575 Starting, stopping, suspending or resuming virtual machine instances
• CPC Class Codes: G06F 2201/815 Virtual
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

Uploads of restored virtual machine (“VM”) data to cloud storage, e.g., VM restore-to-cloud operations, are performed without having to write whole restored virtual disk files to a proxy server before the virtual disk data begins uploading to cloud. Restored data blocks from a backup source are locally cached, staged for efficiency, and asynchronously uploaded to the cloud page-by-page without tapping mass storage resources on the proxy. Downloads of VM data from cloud storage, e.g., VM backup-from-cloud, are performed without having to download a virtual disk file in its entirety to the proxy server before the backup operation begins generating a backup copy. This speeds up “pulling” VM data from the cloud by pre-fetching and locally caching downloaded data blocks. The cached data blocks are processed for backup and stored page-by-page directly into a secondary copy of the cloud VM virtual-disk file without tapping mass storage resource at the proxy.

Claims

1. A method for restoring a secondary copy of a virtual-disk file associated with a virtual machine to a cloud-based account, the method comprising:

executing on a first computing device a pseudo-disk driver that presents a file system comprising a cache storage area on the first computing device as a local mass-data storage device;

storing to a first page in the cache storage area, by a data agent that executes on the first computing device, a first set of data blocks restored from a secondary copy of a second virtual-disk file associated with a second virtual machine hosted by another computing device,

wherein the cache storage area comprises a plurality of pages, including the first page, wherein each page in the plurality of pages is sized to match a page-size configured in the cloud-based account,

wherein the cloud-based account is configured to store a first virtual-disk file restored in its entirety from the secondary copy of the second virtual-disk file, and

wherein the cache storage area on the first computing device is insufficient in storage capacity to store the first virtual-disk file restored in its entirety from the secondary copy of the second virtual-disk file;

from a second page in the plurality of pages in the cache storage area, wherein the second page is filled based on its page-size by data blocks restored from the secondary copy of the second virtual-disk file,

uploading the restored data blocks in the filled second page from the cache storage area to a page in the first virtual-disk file in the cloud-based account;

wherein the first virtual-disk file is restored in its entirety to the cloud-based account based on uploading, page-by-page, all restored data blocks from the cache storage area on the first computing device to the first virtual-disk file in the cloud-based account without first restoring the secondary copy of the second virtual-disk file in its entirety to the first computing device; and

if (i) no more of the secondary copy of the second virtual-disk file remains to be restored and (ii) a given page in the plurality of pages in the cache storage area is not filled by data blocks restored from the secondary copy of the second virtual-disk file,

uploading the restored data blocks in the given page from the cache storage area to a page in the first virtual-disk file in the cloud-based account.

2. The method of claim 1 wherein the first virtual-disk file, restored in its entirety to the cloud-based account, is available as primary data for a first virtual machine to execute in the cloud-based account.

3. The method of claim 1 wherein the first virtual-disk file is restored in its entirety to the cloud-based account without using physical mass-data storage in or of the first computing device, and instead using the cache storage area for staging pages to be uploaded to the first virtual-disk file in the cloud-based account.

4. The method of claim 1 further comprising:

if a given page in the plurality of pages in the cache storage area is not filled by data blocks restored from the secondary copy of the second virtual-disk file,

awaiting more restored data blocks to be stored therein by the data agent.

5. The method of claim 1 further comprising:

if (i) more of the secondary copy of the second virtual-disk file remains to be restored, and (ii) a given page, determined to be a least-recently used page, in the plurality of pages in the cache storage area is not filled by data blocks restored from the secondary copy of the second virtual-disk file, and (iii) the cache storage area is running below a threshold on storage space,

uploading the restored data blocks in the least-used page from the cache storage area to a third page in the first virtual-disk file in the cloud-based account; and

when more data blocks are restored belonging to the third page in the first virtual-disk file in the cloud-based account,

(a) downloading the restored data blocks from the third page in the first virtual-disk file in the cloud-based account to a second page in the plurality of pages in the cache storage area on the first computing device, and

(b) storing to the second page, by the data agent, a second set of data blocks restored from the secondary copy of the second virtual-disk file.

6. The method of claim 1 further comprising:

tracking, by the data agent, based on page-size, whether each page in the plurality of pages in the cache storage area is filled by data blocks restored from the secondary copy of the second virtual-disk file.

7. The method of claim 1 further comprising:

when a given page in the plurality of pages in the cache storage area receives one or more data blocks restored from the secondary copy of the second virtual-disk file,

updating a list maintained by the data agent to indicate that the given page is most-recently used.

8. The method of claim 1 further comprising:

when a given page in the plurality of pages in the cache storage area receives one or more data blocks restored from the secondary copy of the second virtual-disk file,

updating a list maintained by the data agent to indicate that the given page is most-recently used; and

based on the list, determining by the data agent which page in the plurality of pages in the cache storage area is least-recently used.

9. The method of claim 1 further comprising:

based on tracking, by the data agent, when each page in the plurality of pages in the cache storage area receives one or more data blocks restored from the secondary copy of the second virtual-disk file,

determining by the data agent which page in the plurality of pages in the cache storage area is least-recently used.

10. The method of claim 1 further comprising:

executing a first virtual machine in the cloud-based account using the restored first virtual-disk file as primary data for the first virtual machine.

11. The method of claim 1 further comprising:

creating a mount point on the first computing device for the cache storage area presented by the pseudo-disk driver as the local mass-data storage device,

wherein a restore operation for the secondary copy of the second virtual-disk file treats the local mass-data storage device as a destination for the first virtual-disk file restored from the secondary copy of the second virtual-disk file, thereby causing the data agent to store restored data blocks to the cache storage area.

12. A computer-readable medium, excluding transitory propagating signals, storing instructions that, when executed by a first computing device comprising one or more processors and computer memory, cause the first computing device to perform operations comprising:

executing a pseudo-disk driver that presents a cache storage area on the first computing device as a local mass-data storage device;

storing to a first page in the cache storage area a first set of data blocks restored from a secondary copy of a second virtual-disk file associated with a second virtual machine hosted by another computing device,

wherein the cache storage area comprises a plurality of pages, including the first page, wherein each page in the plurality of pages is sized to match a page-size configured in a cloud-based account,

wherein the cloud-based account is configured to store a first virtual-disk file restored in its entirety from the secondary copy of the second virtual-disk file, and

wherein the cache storage area on the first computing device is insufficient in storage capacity to store the first virtual-disk file restored in its entirety from the secondary copy of the second virtual-disk file;

from a second page in the plurality of pages in the cache storage area, wherein the second page is filled based on its page-size by data blocks restored from the secondary copy of the second virtual-disk file,

uploading the restored data blocks in the filled second page from the cache storage area to a page in the first virtual-disk file in the cloud-based account; and

wherein the first virtual-disk file is restored in its entirety to the cloud-based account (a) based on uploading, page-by-page, all restored data blocks from the cache storage area on the first computing device to the first virtual-disk file in the cloud-based account without first restoring the secondary copy of the second virtual-disk file in its entirety to the first computing device, and (b) without using physical mass-data storage in or of the first computing device, and instead using the cache storage area for staging pages to be uploaded to the first virtual-disk file in the cloud-based account; and

if (A) no more of the secondary copy of the second virtual-disk file remains to be restored and (B) a given page in the plurality of pages in the cache storage area is not filled by data blocks restored from the secondary copy of the second virtual-disk file,

uploading the restored data blocks in the given page from the cache storage area to a page in the first virtual-disk file in the cloud-based account.

13. The computer-readable medium of claim 12 wherein the first virtual-disk file, restored in its entirety to the cloud-based account, is available as primary data for a first virtual machine to execute in the cloud-based account.

14. The computer-readable medium of claim 12 wherein the operations further comprise:

if (i) more of the secondary copy of the second virtual-disk file remains to be restored, and (ii) the given page in the plurality of pages in the cache storage area is not filled by data blocks restored from the secondary copy of the second virtual-disk file,

awaiting more restored data blocks to be stored therein by a data agent until the given page is filled, before uploading the restored data blocks in the given page from the cache storage area to a page in the first virtual-disk file in the cloud-based account.

15. The computer-readable medium of claim 12, wherein a data agent addresses write requests to a file system comprising the cache storage area and wherein each write request is intercepted by the pseudo-disk driver and written thereby into one or more pages in the cache storage area.