Assignment of data agent proxies for executing virtual-machine secondary copy operations including streaming backup jobs

• US 10,084,873 B2
• Filed: 06/19/2015
• Issued: 09/25/2018
• Est. Priority Date: 06/19/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Dornemann, Henry Wallace
• Primary Examiner(s): Dailey, Thomas J
• Application Number:
  US14/744,952
• Publication Number:
  US 20160373291A1
• Time in Patent Office: 1,194 Days
• Field of Search: None
• CPC Class Codes: H04L 41/042 comprising distributed management centres cooperatively managing the network
• CPC Class Codes: H04L 41/0856 by backing up or archiving configuration information
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54

Abstract

A comprehensive approach to streaming backups for virtual machines (“VMs”) in a storage management system comprises improvements to the assignment of data agent proxies for VM secondary copy operations. New considerations in performing a VM streaming backup job include without limitation: determining and enforcing a system-wide per-proxy limit of concurrent data streams; generating an ordered priority list of the VMs to be backed up as a basis for choosing which proxies will back up the respective VM, though the illustrative system may not strictly adhere to the priority list based on further considerations; identifying a next available proxy based on data stream utilization at the proxy; and dynamically re-generating the priority list and re-evaluating considerations if some VMs become “stranded” due to a failure to be backed up. Secondary copy operations are distributed to proxies in ways that improve the chances of successfully completing VM streaming backups.

Claims

1. A method for determining a per-proxy limit of concurrent data streams allowed for backup operations in a storage management system, the method comprising:

designating, by a storage manager component of the storage management system, a first data agent to act as coordinator of a first backup job,

wherein the first backup job is configured to perform secondary copy operations for a first set of virtual machines in the storage management system,

wherein the coordinator data agent executes on a first virtual machine, and wherein the first virtual machine is designated a first proxy, and

wherein the first virtual machine executes on a first computing device having one or more processors and non-transitory computer-readable memory;

designating, by the storage manager, a second data agent to act as controller in the first backup job,

wherein the controller data agent executes on a second computing device having one or more processors and non-transitory computer-readable memory, and wherein the second computing device is designated a second proxy; and

determining, by the coordinator data agent, a respective maximum limit of concurrent data streams assignable by the coordinator data agent to each of the first proxy and the second proxy for performing one or more backup jobs in the storage management system, including the first backup job,

wherein for a given proxy the respective maximum limit of concurrent data streams is based on a processing capacity and an amount of random-access memory of the given proxy.

2. The method of claim 1 further comprising:

assigning, by the coordinator data agent, a first data stream to the first proxy in the course of performing the first backup job,

wherein the first data stream originates at a first storage device comprising virtual-machine data being backed up in the first backup job and terminates at the coordinator data agent that executes on the first proxy; and

wherein the first proxy concurrently uses no more than the maximum limit of concurrent data streams determined by the coordinator data agent for the first proxy.

3. The method of claim 1, wherein the coordinator data agent and the controller data agent are virtual-server data agents, which are configured for backup of virtual machines in the storage management system.

4. The method of claim 1, wherein the first set of virtual machines comprises a first subset of virtual machines that execute on a first host computing device, and a second subset of virtual machines that execute on a second host computing device.

5. The method of claim 4, wherein the first proxy which hosts the coordinator data agent executes on the same computing device as the first subset of virtual machines, such that the first computing device coincides with the first host computing device.

6. The method of claim 1 further comprising:

collecting, by the coordinator data agent, job-related information for executing the first backup job; and

wherein the job-related information for executing the first backup job comprises one or more of;

identities of the first set of virtual machines to be backed up, one or more operational properties of each virtual machine in the first set, information about one or more storage devices storing virtual-machine data to be backed up, and a maximum number of concurrent data streams allowed to be used concurrently while performing the first backup job.

7. The method of claim 1, wherein

for a given proxy the respective maximum limit of concurrent data streams is further based on a number of processors that the given proxy comprises.

8. A computer-readable medium, excluding transitory propagating signals, storing instructions that, when executed by a computing device, cause the computing device to perform a method for managing backup operations in a storage management system, the method comprising:

collecting, by a designated coordinator for a first backup job in the storage management system, information about operational properties, comprising a respective processing capacity and a respective amount of random-access memory, of a first proxy and a second proxy,

wherein the coordinator operates as the first proxy,

wherein a designated controller operates as the second proxy, and

wherein the first backup job is associated with one or more secondary copy operations for a first set of virtual machines; and

determining, by the coordinator, a respective maximum limit of concurrent data streams assignable by the coordinator to each of the first proxy and the second proxy for performing one or more backup jobs including the first backup job,

wherein for a given proxy the respective maximum limit of concurrent data streams is based on the operational properties of the given proxy.

9. The computer-readable medium of claim 8 wherein the coordinator is a data agent that executes on a first computing device having one or more processors and non-transitory computer-readable memory, and wherein the controller is a data agent that executes on a first virtual machine, and wherein the first virtual machine executes on a second computing device having one or more processors and non-transitory computer-readable memory.

10. The computer-readable medium of claim 8 wherein

for a given proxy the operational properties further include how many processors the given proxy comprises.

11. A computer-readable medium, excluding transitory propagating signals, storing instructions that, when executed by a computing device having one or more processors and non-transitory computer-readable memory, cause the computing device to perform a method comprising:

establishing inter-process communications between a coordinator component of a storage management system and one or more controller components of the storage management system in regard to one or more backup jobs to be performed in the storage management system;

collecting information about the one or more backup jobs to be performed;

further collecting information about operational properties of one or more proxy components of the storage management system that are candidates for executing at least part of the one or more backup jobs to be performed,

wherein for the given candidate proxy the operational properties include a processing capacity and an amount of random-access memory of the given candidate proxy; and

determining a respective maximum limit of concurrent data streams assignable by the coordinator component to each candidate proxy for transferring data in the one or more backup jobs to be performed,

wherein for a given candidate proxy the respective maximum limit of concurrent data streams is based on the operational properties collected about the given candidate proxy.

12. The computer-readable medium of claim 11, wherein each proxy may concurrently use no more than the respective maximum limit of concurrent data streams while performing the one or more backup jobs in the storage management system.

13. The computer-readable medium of claim 11, wherein the information about a backup job comprises one or more of:

identities of virtual machines to be backed up in the backup job, operational properties of virtual machines to be backed up in the backup job, and a number of concurrent data streams allowed for performing the backup job.

14. The computer-readable medium of claim 11, wherein for the given candidate proxy the operational properties further include how many processors the given proxy comprises.

15. The computer-readable medium of claim 11, wherein one of the candidate proxies hosts the coordinator component.