Assignment of proxies for virtual-machine secondary copy operations including streaming backup job

• US 10,169,067 B2
• Filed: 12/13/2016
• Issued: 01/01/2019
• Est. Priority Date: 06/19/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Dornemann, Henry Wallace
• Primary Examiner(s): Kim, Sisley
• Application Number:
  US15/376,922
• Publication Number:
  US 20170090974A1
• Time in Patent Office: 749 Days
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54

Abstract

A comprehensive approach to streaming backups for virtual machines (“VMs”) in a storage management system comprises improvements to the assignment of data agent proxies for VM secondary copy operations. New considerations in performing a VM streaming backup job include without limitation: determining and enforcing a system-wide per-proxy limit of concurrent data streams; generating an ordered priority list of the VMs to be backed up as a basis for choosing which proxies will back up the respective VM, though the illustrative system may not strictly adhere to the priority list based on further considerations; identifying a next available proxy based on data stream utilization at the proxy; and dynamically re-generating the priority list and re-evaluating considerations if some VMs become “stranded” due to a failure to be backed up. Secondary copy operations are distributed to proxies in ways that improve the chances of successfully completing VM streaming backups.

Claims

1. A computer-readable medium, excluding transitory propagating signals, storing instructions that, when executed by a computing device having one or more processors and non-transitory computer-readable memory, cause the computing device to perform a method comprising:

identifying, by a first data agent executing on the computing device, one or more proxies in a storage management system that are eligible to back up a given virtual machine in a first set of virtual machines in the storage management system,

wherein any one proxy among the one or more proxies is one of;

(a) a first virtual machine that executes on a first computing device, wherein the first virtual machine executes a second data agent for virtual-machine backup, and

(b) a second computing device that executes a second data agent for virtual-machine backup;

wherein the identifying comprises;

(i) determining (A) a set of candidate proxies for backing up the given virtual machine, and (B) a mode of access available to each respective candidate proxy for accessing the given virtual machine'"'"'s data as a source for backup,

wherein the mode of access has a predefined tier of preference,

wherein the determining is based on analyzing, by the first data agent, data from a database that is associated with a storage manager component that manages the storage management system, and

wherein the storage manager component designates the first data agent as a coordinator data agent for a first backup job for the first set of virtual machines,

(ii) classifying each candidate proxy in the set of candidate proxies based on the predefined tier of preference for the respective candidate proxy'"'"'s mode of access to the given virtual machine'"'"'s data as the source for backup, and

(iii) defining one or more candidate proxies that are classified in a highest tier of preference as being eligible to back up the given virtual machine; and

wherein if the defining results in the given virtual machine being stranded without an eligible proxy, subsequently defining one or more candidate proxies, which are classified in a next highest tier of preference that is less than the highest tier of preference, as being eligible to back up the given virtual machine.

2. The computer-readable medium of claim 1, wherein a first mode of access wherein a candidate proxy has direct access to the given virtual machine'"'"'s data via a storage area network has a higher tier of preference than a second mode of access wherein the candidate proxy is a virtual machine that executes on the same computing device as the given virtual machine to be backed up and has local access to the given virtual machine'"'"'s data from the same computing device.

3. The computer-readable medium of claim 2, wherein the second mode of access has a higher tier of preference than a third mode of access wherein the candidate proxy is a virtual machine that executes on a different computing device from the given virtual machine to be backed up and has networked access to the given virtual machine'"'"'s data.

4. The computer-readable medium of claim 2, wherein the second mode of access has a higher tier of preference than a fourth mode of access wherein the candidate proxy has networked access to the given virtual machine'"'"'s data as a network block device.

5. A method for generating a priority list of a first set of virtual machines to be backed up in a storage management system, the method comprising:

associating each virtual machine in the first set of virtual machines with one or more proxies from a set of candidate proxies in the storage management system,

wherein the associating includes determining which of the set of candidate proxies are eligible to back up a respective virtual machine, and

wherein any one candidate proxy among the set of candidate proxies is one of;

(a) a first virtual machine that executes on a first computing device, wherein the first virtual machine executes a second data agent for virtual-machine backup, and

(b) a second computing device that executes a second data agent for virtual-machine backup, and

wherein if a given virtual machine in the first set is not associated with at least one eligible proxy having a highest tier of preference for a transport mode between the at least one eligible proxy and the given virtual machine'"'"'s data, associating the given virtual machine with one or more candidate proxies having a next highest tier of preference that is less than the highest tier of preference;

ordering the first set of virtual machines into a priority list in order of increasing number of associated eligible proxies for each respective virtual machine;

while executing a first backup job, assigning to a first virtual machine in the first set, a first eligible proxy based on the priority list, wherein the second data agent executing on the assigned first eligible proxy participates in a backup of the first virtual machine during the first backup job; and

wherein the associating, the ordering, and the assigning are performed by a first data agent designated a coordinator data agent for the first backup job, and wherein a storage manager that manages the storage management system designates the first data agent to be the coordinator data agent.

6. The method of claim 5 wherein the ordering into the priority list comprises breaking a tie among a plurality of virtual machines in the first set by ordering the virtual machines in the plurality in decreasing order of a total amount of storage allocated to each respective virtual machine.

7. The method of claim 5 wherein a given virtual machine has a higher priority in the priority list when one of:

the given virtual machine is connected to a storage area network and when the given virtual machine has more data to back up than other virtual machines in the first set of virtual machines.

8. The method of claim 5 wherein the first backup job is configured to perform secondary copy operations for the first set of virtual machines in the storage management system, and

wherein the coordinator data agent executes on a first computing device having one or more processors and non-transitory computer-readable memory which is designated a first proxy.

9. The method of claim 8

wherein any given eligible proxy executes a data agent that is suitable for backing up data associated with one or more virtual machines that the given proxy is eligible to back up.

10. A method for generating a priority list of a first set of virtual machines to be backed up in a storage management system, the method comprising:

designating, by a storage manager that manages a data storage management system, a first virtual-server data agent as a coordinator data agent for a first backup job of the first set of virtual machines;

associating, by the coordinator data agent, each virtual machine in the first set of virtual machines with one or more proxies from a set of candidate proxies in the storage management system,

wherein the associating includes determining which of the set of candidate proxies are eligible to back up a respective virtual machine, and

wherein the associating results in each virtual machine in the first set being associated with at least one eligible proxy, and

wherein if a given virtual machine in the first set is not associated with at least one eligible proxy having a highest tier of preference for a transport mode between the at least one eligible proxy and the given virtual machine'"'"'s data, associating the given virtual machine with one or more candidate proxies having a next highest tier of preference that is less than the highest tier of preference;

ordering, by the coordinator data agent, the first set of virtual machines into a priority list in order of increasing number of associated eligible proxies for each respective virtual machine; and

assigning, by the coordinator data agent, based on the priority list, a first eligible proxy to a first virtual machine in the first set of virtual machines, wherein the assigned first eligible proxy is to perform one or more storage operations for the first virtual machine during the first backup job.

11. The method of claim 10 wherein the ordering into the priority list comprises breaking a tie among a plurality of virtual machines in the first set by ordering the virtual machines in the plurality in decreasing order of a total amount of storage allocated to each respective virtual machine.

12. The method of claim 10 wherein a given virtual machine has a higher priority in the priority list when the given virtual machine is one of:

(a) connected to a storage area network (SAN), and (b) has more data to back up than other virtual machines in the first set of virtual machines.

13. The method of claim 10 wherein the coordinator data agent executes on a first computing device having one or more processors and non-transitory computer-readable memory which is designated a first proxy.

14. The method of claim 10 wherein the coordinator data agent executes on a virtual machine hosted by a first computing device having one or more processors and non-transitory computer-readable memory which is designated a first proxy.

15. The method of claim 10 wherein any given eligible proxy executes a data agent that is suitable for backing up data associated with one or more virtual machines that the given proxy is eligible to back up.