Assignment of data agent proxies for executing virtual-machine secondary copy operations including streaming backup jobs

• US 10,148,780 B2
• Filed: 07/14/2017
• Issued: 12/04/2018
• Est. Priority Date: 06/19/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Dornemann, Henry Wallace
• Primary Examiner(s): Kim, Sisley
• Application Number:
  US15/650,503
• Publication Number:
  US 20170318111A1
• Time in Patent Office: 508 Days
• CPC Class Codes: H04L 41/042 comprising distributed management centres cooperatively managing the network
• CPC Class Codes: H04L 41/0856 by backing up or archiving configuration information
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54

Abstract

A comprehensive approach to streaming backups for virtual machines (“VMs”) in a storage management system comprises improvements to the assignment of data agent proxies for VM secondary copy operations. New considerations in performing a VM streaming backup job include without limitation: determining and enforcing a system-wide per-proxy limit of concurrent data streams; generating an ordered priority list of the VMs to be backed up as a basis for choosing which proxies will back up the respective VM, though the illustrative system may not strictly adhere to the priority list based on further considerations; identifying a next available proxy based on data stream utilization at the proxy; and dynamically re-generating the priority list and re-evaluating considerations if some VMs become “stranded” due to a failure to be backed up. Secondary copy operations are distributed to proxies in ways that improve the chances of successfully completing VM streaming backups.

Claims

1. A method for generating an ordered priority list of a first set of virtual machines that are to be backed up in a storage management system, the method comprising:

associating each virtual machine in the first set of virtual machines with one or more proxies from a set of candidate proxies in the storage management system,

wherein the associating includes determining which one or more candidate proxies from the set of candidate proxies are eligible to back up a respective virtual machine;

ordering the first set of virtual machines into a priority list in an order of increasing number of associated eligible proxies for each respective virtual machine;

while executing a first backup job of the first set of virtual machines according to the order of the priority list, assigning a first eligible proxy to each respective virtual machine in the first set for backing up the respective virtual machine;

wherein a given proxy is one of;

(a) a physical computing device comprising one or more processors and computer-readable memory, and (b) a virtual machine that executes on a physical computing device comprising one or more processors and computer-readable memory, and further wherein the given proxy executes a respective data agent suitable to back up one or more virtual machines in the first set;

designating, by a storage manager component of the storage management system, a first data agent to act as coordinator of the first backup job,

wherein the first backup job comprises secondary copy operations for the first set of virtual machines in the storage management system, and

wherein the coordinator data agent executes on a first proxy from the set of candidate proxies; and

wherein the associating and the ordering are performed by the coordinator data agent.

2. The method of claim 1, wherein a given virtual machine is given a higher priority in the order of the priority list if the given virtual machine has a larger amount of data to back up as compared to other virtual machines in the priority list.

3. The method of claim 1, wherein the method further comprises:

designating, by the storage manager, a second data agent to act as controller in the first backup job, wherein the controller data agent executes on a second proxy from the set of candidate proxies.

4. The method of claim 1, wherein the determining which one or more candidate proxies from the set of candidate proxies are eligible to back up a given virtual machine in the first set comprises:

determining a mode of access available to each respective candidate proxy for accessing the given virtual machine'"'"'s data as a source for backup, wherein the mode of access has a predefined tier of preference.

5. The method of claim 4, wherein the determining which one or more candidate proxies from the set of candidate proxies are eligible to back up a given virtual machine in the first set further comprises:

classifying the respective candidate proxy according to the predefined tier of preference for the respective mode of access.

6. The method of claim 4, wherein the determining which one or more candidate proxies from the set of candidate proxies are eligible to back up a given virtual machine in the first set further comprises:

defining the respective candidate proxy as being eligible to back up the given virtual machine if the respective candidate proxy has a highest tier of preference for the mode of access to the given virtual machine'"'"'s data as compared to other candidate proxies.

7. The method of claim 1, wherein a respective candidate proxy that has direct access to a given virtual machine'"'"'s data via a storage area network (SAN) is given a first predefined tier of preference;

wherein a respective candidate proxy that has local access to a given virtual machine'"'"'s data from a computing device that also hosts the given virtual machine is given a second predefined tier of preference that is lower than the first predefined tier of preference; and

wherein a respective candidate proxy that has networked access to a computing device that hosts a given virtual machine is given a third highest predefined tier of preference that is lower than the second predefined tier of preference.

8. The method of claim 7, wherein a respective candidate proxy that has a highest predefined tier of preference for the mode of access to a given virtual machine'"'"'s data as compared to other candidate proxies is determined to be eligible to back up the given virtual machine.

9. A method for generating an ordered priority list of a first set of virtual machines that are to be backed up in a storage management system, the method comprising:

identifying which one or more proxies from a set of candidate proxies in the storage management system are eligible to back up each respective virtual machine in the first set of virtual machines,

wherein a given proxy is one of;

(a) a physical computing device comprising one or more processors and computer-readable memory, and (b) a virtual machine that executes on a physical computing device comprising one or more processors and computer-readable memory, and

further wherein the given proxy executes a respective data agent suitable to back up one or more virtual machines in the first set;

ordering the first set of virtual machines into a priority list in an order of increasing number of eligible proxies for each respective virtual machine;

executing a first backup job of the first set of virtual machines according to the order of the priority list,

wherein each respective virtual machine in the first set is assigned a first eligible proxy for performing secondary copy operations for the respective virtual machine in the backup job; and

wherein the identifying and the ordering are performed by a data agent designated as a coordinator for the first backup job, and

wherein the data agent executes on one of the candidate proxies from the set of candidate proxies in the storage management system, which is not an eligible proxy for any virtual machine in the first set of virtual machines.

10. The method of claim 9, wherein the ordering into the priority list comprises breaking a tie among a plurality of virtual machines in the first set by ordering the plurality of virtual machines in decreasing order of a total amount of storage allocated to each respective virtual machine.

11. The method of claim 9, wherein a given virtual machine in the priority list is given a higher priority if the given virtual machine has a larger amount of data to back up as compared to other virtual machines in the first set of virtual machines.

12. A method for generating an ordered priority list of a first set of virtual machines that are to be backed up in a storage management system, the method comprising:

identifying which one or more proxies from a set of candidate proxies in the storage management system are eligible to back up each respective virtual machine in the first set of virtual machines,

wherein a given proxy is one of;

(a) a physical computing device comprising one or more processors and computer-readable memory, and (b) a virtual machine that executes on a physical computing device comprising one or more processors and computer-readable memory, and

further wherein the given proxy executes a respective data agent suitable to back up one or more virtual machines in the first set;

ordering the first set of virtual machines into a priority list in an order of increasing number of eligible proxies for each respective virtual machine;

executing a first backup job of the first set of virtual machines according to the order of the priority list,

wherein each respective virtual machine in the first set is assigned a first eligible proxy for performing secondary copy operations for the respective virtual machine in the backup job; and

wherein the identifying and the ordering are performed by a data agent designated as a coordinator for the first backup job, and

wherein the data agent executes on one of the candidate proxies from the set of candidate proxies in the storage management system, which is also an eligible proxy for at least one virtual machine in the first set of virtual machines.

13. The method of claim 12, wherein a respective candidate proxy is eligible to back up a given virtual machine if the respective candidate proxy has a highest tier of preference for a mode of access to the given virtual machine'"'"'s data as compared to other candidate proxies.

14. The method of claim 13, wherein a respective candidate proxy that has direct access to a given virtual machine'"'"'s data via a storage area network (SAN) is given a first tier of preference;

wherein a respective candidate proxy that has local access to a given virtual machine'"'"'s data from a computing device that also hosts the given virtual machine is given a second tier of preference which is lower than the first tier of preference; and

wherein a respective candidate proxy that has networked access to a computing device that also hosts a given virtual machine is given a third tier of preference which is lower than the second tier of preference.