Assignment of proxies for virtual-machine secondary copy operations including streaming backup jobs

• US 9,563,514 B2
• Filed: 06/19/2015
• Issued: 02/07/2017
• Est. Priority Date: 06/19/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Dornemann, Henry Wallace
• Primary Examiner(s): Kim, Sisley
• Application Number:
  US14/745,130
• Publication Number:
  US 20160371153A1
• Time in Patent Office: 599 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2201/815 Virtual
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54

Abstract

A comprehensive approach to streaming backups for virtual machines (“VMs”) in a storage management system comprises improvements to the assignment of data agent proxies for VM secondary copy operations. New considerations in performing a VM streaming backup job include without limitation: determining and enforcing a system-wide per-proxy limit of concurrent data streams; generating an ordered priority list of the VMs to be backed up as a basis for choosing which proxies will back up the respective VM, though the illustrative system may not strictly adhere to the priority list based on further considerations; identifying a next available proxy based on data stream utilization at the proxy; and dynamically re-generating the priority list and re-evaluating considerations if some VMs become “stranded” due to a failure to be backed up. Secondary copy operations are distributed to proxies in ways that improve the chances of successfully completing VM streaming backups.

Claims

1. A method comprising:

identifying one or more proxies in a storage management system that are eligible to back up a given virtual machine in a first set of virtual machines in the storage management system,

wherein any one proxy among the one or more proxies represents one of;

(a) a first virtual machine that executes on a first computing device, wherein the first virtual machine executes a data agent for virtual-machine backup, and

(b) a second computing device that executes a data agent for virtual-machine backup;

wherein the identifying comprises;

(i) determining (A) a set of candidate proxies that are candidates to back up the given virtual machine, and (B) a mode of access available to each respective candidate proxy for accessing the given virtual machine'"'"'s data as a source for backup,

wherein the mode of access has a predefined tier of preference;

(ii) classifying each candidate proxy in the set of candidate proxies based on the predefined tier of preference for the respective candidate proxy'"'"'s mode of access to the given virtual machine'"'"'s data as the source for backup; and

(iii) defining one or more candidate proxies that are classified as having the highest-tier of preference as being eligible to back up the given virtual machine; and

wherein the identifying is performed by a first data agent for virtual-machine backup that operates as a component of the storage management system.

2. The method of claim 1, wherein the determining is based on analyzing, by the first data agent, data from a database which is associated with a storage manager component that manages the storage management system.

3. The method of claim 1, wherein the determining is based on analyzing, by the first data agent, data from a database that is associated with a storage manager component that manages the storage management system, and

wherein the storage manager designates the first data agent to act as a coordinator for a first backup job for the first set of virtual machines.

4. The method of claim 1, wherein a first mode of access wherein a candidate proxy has direct access to the given virtual machine'"'"'s data via a storage area network has a higher tier of preference than a second mode of access wherein the candidate proxy is a virtual machine that executes on the same computing device as the given virtual machine to be backed up and has local access to the given virtual machine'"'"'s data from the same computing device.

5. The method of claim 4, wherein the second mode of access has a higher tier of preference than a third mode of access wherein the candidate proxy is a virtual machine that executes on a different computing device from the given virtual machine to be backed up and has networked access to the given virtual machine'"'"'s data.

6. The method of claim 4, wherein the second mode of access has a higher tier of preference than a fourth mode of access wherein the candidate proxy has networked access to the given virtual machine'"'"'s data as a network block device.

7. The method of claim 1, wherein the first data agent, which performs the identifying, executes on a host computing device in the storage management system, wherein the host computing device is one of the first computing device, the second computing device, and a third computing device;

and

wherein each of the first computing device, the second computing device, and the third computing device comprises one or more processors and non-transitory computer-readable memory for executing the first data agent.

8. The method of claim 1, wherein the first data agent performs the identifying by executing on one of:

the first virtual machine which executes on the first computing device, the second computing device, and a third computing device that is distinct from the first computing device and the second computing device; and

wherein each of the first computing device, the second computing device, and the third computing device comprises one or more processors and non-transitory computer-readable memory for executing the first data agent.

9. A method for attempting to back up a first set of virtual machines in a storage management system, the method comprising:

identifying, by a coordinator data agent that operates as a component of the storage management system, a plurality of proxies for collectively backing up the set of virtual machines in the storage management system,

wherein any one proxy among the plurality of proxies represents one of;

(a) a first virtual machine that executes on a first computing device having one or more processors and non-transitory computer-readable memory, wherein the first virtual machine executes a first virtual-server data agent for virtual-machine backup, and

(b) a second computing device having one or more processors and non-transitory computer-readable memory, wherein the second computing device executes the first virtual-server data agent for virtual-machine backup;

identifying, by the coordinator data agent, a first proxy among the plurality of proxies, wherein the first proxy is available for backup using a first data stream which terminates at the first proxy;

identifying, by the coordinator data agent, a second virtual machine on a priority list that identifies a set of virtual machines awaiting backup,

wherein the priority list orders the set of virtual machines in increasing order of the number of proxies that are eligible to back up each respective virtual machine;

if the first proxy is eligible to back up the second virtual machine and a score of the second virtual machine'"'"'s data store is under a predefined threshold, causing, by the coordinator data agent, a backup process for the second virtual machine to be launched at the first proxy,

wherein the first proxy backs up, at least in part, the second virtual machine using the first data stream, and

wherein the first data stream originates at a first storage device comprising the second virtual machine'"'"'s data store;

if the first proxy is eligible to back up the second virtual machine and the score of the second virtual machine'"'"'s data store is not under the predefined threshold, identifying a third virtual machine on the priority list; and

if the first proxy is eligible to back up the third virtual machine and a score of the third virtual machine'"'"'s data store is under the predefined threshold, causing, by the coordinator data agent, a backup process for the third virtual machine to be launched at the first proxy,

wherein the first proxy backs up, at least in part, the third virtual machine using the first data stream, and

wherein the first data stream originates at a second storage device comprising the third virtual machine'"'"'s data store.

10. The method of claim 9 wherein the set of virtual machines awaiting backup is associated with a backup job in the storage management system, and further comprising:

requesting, by the first proxy, a second data stream to be used for the backup job; and

allowing, by the coordinator data agent, the second data stream to be used by the first proxy if a total number of data streams in concurrent use for the backup job does not exceed a predefined threshold.

11. A method for attempting to back up a first set of virtual machines in a storage management system, the method comprising:

identifying, by a coordinator data agent that operates as a component of the storage management system, a plurality of proxies for collectively backing up the set of virtual machines in the storage management system,

wherein any one proxy among the plurality of proxies represents one of;

(a) a first virtual machine that executes on a first computing device having one or more processors and non-transitory computer-readable memory, wherein the first virtual machine executes a first virtual-server data agent for virtual-machine backup, and

(b) a second computing device having one or more processors and non-transitory computer-readable memory, wherein the second computing device executes the first virtual-server data agent for virtual-machine backup;

identifying, by the coordinator data agent, a first proxy among the plurality of proxies, wherein the first proxy is available for backup using a first data stream which terminates at the first proxy;

identifying, by the coordinator data agent, a second virtual machine on a priority list that identifies a set of virtual machines awaiting backup,

wherein the priority list orders the set of virtual machines in increasing order of the number of proxies that are eligible to back up each respective virtual machine;

if the first proxy is eligible to back up the second virtual machine and a score of the second virtual machine'"'"'s data store is under a predefined threshold, causing, by the coordinator data agent, a backup process for the second virtual machine to be launched at the first proxy,

wherein the first proxy backs up, at least in part, the second virtual machine using the first data stream, and

wherein the first data stream originates at a first storage device comprising the second virtual machine'"'"'s data store;

if the first proxy is not eligible to back up the second virtual machine, identifying a third virtual machine on the priority list; and

if the first proxy is eligible to back up the third virtual machine and a score of the third virtual machine'"'"'s data store is under the predefined threshold, causing, by the coordinator data agent, a backup process for the third virtual machine to be launched at the first proxy,

wherein the first proxy backs up, at least in part, the third virtual machine using the first data stream, and

wherein the first data stream originates at a second storage device comprising the third virtual machine'"'"'s data store.

12. A method for attempting to back up a first set of virtual machines in a storage management system, the method comprising:

identifying, by a coordinator data agent that operates as a component of the storage management system, a plurality of proxies for collectively backing up the set of virtual machines in the storage management system,

wherein any one proxy among the plurality of proxies represents one of;

(a) a first virtual machine that executes on a first computing device having one or more processors and non-transitory computer-readable memory, wherein the first virtual machine executes a first virtual-server data agent for virtual-machine backup, and

(b) a second computing device having one or more processors and non-transitory computer-readable memory, wherein the second computing device executes the first virtual-server data agent for virtual-machine backup;

identifying, by the coordinator data agent, a first proxy among the plurality of proxies, wherein the first proxy is available for backup using a first data stream which terminates at the first proxy;

identifying, by the coordinator data agent, a second virtual machine on a priority list that identifies a set of virtual machines awaiting backup,

wherein the priority list orders the set of virtual machines in increasing order of the number of proxies that are eligible to back up each respective virtual machine;

if the first proxy is eligible to back up the second virtual machine and a score of the second virtual machine'"'"'s data store is under a predefined threshold, causing, by the coordinator data agent, a backup process for the second virtual machine to be launched at the first proxy,

wherein the first proxy backs up, at least in part, the second virtual machine using the first data stream, and

wherein the first data stream originates at a first storage device comprising the second virtual machine'"'"'s data store;

wherein the set of virtual machines awaiting backup is associated with a backup job in the storage management system; and

further comprising;

if after traversing the priority list identifying the set of virtual machines, the coordinator data agent determines that a subset of the set of virtual machines is still awaiting backup;

(i) pending the backup job, by the coordinator data agent;

(ii) restarting the backup job, by the coordinator data agent, based only on the subset of virtual machines that are still awaiting backup, wherein the restarted backup job comprises;

generating, by the coordinator data agent, a revised priority list identifying the virtual machines in the subset of virtual machines in order of increasing number of associated eligible proxies for each respective virtual machine, and

identifying, by the coordinator data agent, a revised plurality of proxies for collectively backing up the subset of virtual machines.

13. The method of claim 12 wherein the restarted backup job further comprises:

coordinating, by the coordinator data agent, an attempt to back up the subset of virtual machines using the revised plurality of proxies.

14. A method for attempting to back up a first set of virtual machines in a storage management system, the method comprising:

identifying, by a coordinator data agent that operates as a component of the storage management system, a plurality of proxies for collectively backing up the set of virtual machines in the storage management system,

wherein any one proxy among the plurality of proxies represents one of;

(a) a first virtual machine that executes on a first computing device having one or more processors and non-transitory computer-readable memory, wherein the first virtual machine executes a first virtual-server data agent for virtual-machine backup, and

(b) a second computing device having one or more processors and non-transitory computer-readable memory, wherein the second computing device executes the first virtual-server data agent for virtual-machine backup;

identifying, by the coordinator data agent, a first proxy among the plurality of proxies, wherein the first proxy is available for backup using a first data stream which terminates at the first proxy;

identifying, by the coordinator data agent, a second virtual machine on a priority list that identifies a set of virtual machines awaiting backup,

wherein the priority list orders the set of virtual machines in increasing order of the number of proxies that are eligible to back up each respective virtual machine;

if the first proxy is eligible to back up the second virtual machine and a score of the second virtual machine'"'"'s data store is under a predefined threshold, causing, by the coordinator data agent, a backup process for the second virtual machine to be launched at the first proxy,

wherein the first proxy backs up, at least in part, the second virtual machine using the first data stream, and

wherein the first data stream originates at a first storage device comprising the second virtual machine'"'"'s data store;

wherein the identifying of the first proxy which is available for backup comprises choosing the first proxy from a set of candidate proxies based on the first proxy having the lowest data stream utilization score among the set of candidate proxies; and

wherein the data stream utilization score for the first proxy is based on a ratio of (i) a number of data streams currently active at the first proxy to (ii) a maximum limit of data streams assignable to the first proxy, and wherein any data stream considered in the data stream utilization score terminates at the first proxy.

15. A computer-readable medium, excluding transitory propagating signals, storing instructions that, when executed by at least one computing device having one or more processors and non-transitory computer-readable memory, cause the computing device to perform a method comprising:

defining a set of candidate proxies for a virtual-machine backup job in a storage management system,

wherein any one proxy in the set of candidate proxies represents one of;

(a) a first virtual machine that executes on a first computing device having one or more processors and non-transitory computer-readable memory, wherein the first virtual machine executes a first virtual-server data agent for virtual-machine backup, and

(b) a second computing device having one or more processors and non-transitory computer-readable memory, wherein the second computing device executes the first virtual-server data agent;

evaluating the set of candidate proxies, wherein the evaluating for a given candidate proxy in the set of candidate proxies comprises;

determining a maximum limit of data streams assignable to the given candidate proxy, based on operational properties of the given candidate proxy,

identifying a number of data streams that are currently active at the given candidate proxy, including any data streams in the virtual-machine backup job and any data streams for other jobs in the storage management system,

computing a data stream utilization score for the given candidate proxy, based on a ratio of (i) the number of data streams currently active at the given candidate proxy to (ii) the maximum limit of data streams assignable to the given candidate proxy,

wherein any data stream considered in the evaluating operation terminates at the given candidate proxy; and

assigning a new data stream to a first proxy in the set of candidate proxies that has the lowest data stream utilization score among the set of candidate proxies.

16. The computer-readable medium claim 15, wherein, in the course of performing operations for one or more backup jobs in the storage management system including for the virtual-machine backup job, the first proxy may concurrently use no more than the respective maximum limit of data streams assignable to the first proxy.

17. The computer-readable medium claim 15, wherein the operational properties of the first proxy comprise at least one of:

a processing capacity of the first proxy, and an amount of random-access memory of the first proxy; and

wherein the maximum limit of data streams assignable to the first proxy is based on the at least one of;

the processing capacity of the first proxy, and the amount of random-access memory of the first proxy.

18. The computer-readable medium claim 15, wherein the maximum limit of data streams assignable to the first proxy positively correlates to at least one of:

the processing capacity of the first proxy, and the amount of random-access memory of the first proxy.

19. The computer-readable medium claim 15 wherein the method further comprises:

identifying a second virtual machine on a priority list that identifies a set of virtual machines awaiting backup in the virtual-machine backup job; and

causing a backup process for the second virtual machine to be launched at the first proxy using the new data stream.