AUTOMATION OF DATA STORAGE ACTIVITIES

• US 20130254402A1
• Filed: 03/06/2013
• Published: 09/26/2013
• Est. Priority Date: 03/23/2012
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Vibhor, Anand, Karandikar, Amey Vijaykumar
• Granted Patent: US 9,292,815 B2
• Time in Patent Office: Days
• US Class Current: 709/226
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: G06Q 10/06316 Sequencing of tasks or work
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A system receives data storage workflow activities that include computer-executable instructions for carrying out data storage workflow in a network data storage system. Once the workflow is received, the system deploys the workflow to one or more workflow engines that can execute the various data storage activities related to the workflow. Prior to executing a data storage activity, the system can determine which workflow engine to use based on an allocation scheme.

Claims

1. A method for allocating the execution of data storage workflow activities within a networked data storage system, the method comprising:

receiving a data storage workflow suite comprising executable instructions for carrying out a data storage workflow in a networked data storage system, the data storage workflow including a plurality of data storage activities, the data storage system comprising;

one or more client computing devices;

secondary storage; and

one or more agent modules configured to manage storage operations between client computing devices and the secondary storage.

deploying the workflow suite to at least two workflow engines, wherein each workflow engine executes on a separate one of at least two computing devices;

based on an allocation scheme, selecting, by one or more processors, a first workflow engine of the at least two workflow engines on which to execute at least one current activity of the plurality of data storage activities, wherein at least a second of the at least two workflow engines is not selected.

2. The method of claim 1, wherein the at least one current activity comprises a secondary copy operation in which production data associated with at least one of the one or more client computing devices is copied to the secondary storage to create a secondary copy of the production data.

3. The method of claim 1, wherein said at least two workflow engines on which the workflow suite is deployed are included in a pool of workflow engines, and wherein said deploying further comprises selecting the at least two workflow engines for deployment of the workflow suite, wherein one or more other workflow engines in the pool are not selected.

4. The method of claim 3, further comprising maintaining a data store comprising:

a listing of workflows that have been deployed to one or more of the workflow engines in the pool; and

for each workflow in the listing, an indication of which of the one or more workflow engines the workflow has been deployed to, wherein the data store is consulted as part of said selecting.

5. The method of claim 1, wherein said selecting is based on one or more of computing resources associated with the at least two workflow engines and geographic proximity of the at least two workflow engines to one or more components in the networked data storage system.

6. The method of claim 1, further comprising obtaining information related to activity levels associated with the at least two workflow engines, wherein the allocation scheme and said selecting is at least in part based on the obtained activity levels.

7. The method of claim 6, wherein the obtained information comprises information related to one or more of a total number of workflow suites deployed on the at least two workflow engines, a total number of processes active on the at least two workflow engines, and utilization of the at least two workflow engines.

8. The method of claim 1, wherein the allocation scheme is based on at least one of computing resources associated with the at least two workflow engines, physical proximity of the at least two workflow engines to other data storage system components, and communication pathway speeds between the at least two workflow engines and other data storage system components.

9. The method of claim 1, further comprising:

providing a graphical user interface including one or more icons representative of different pre-configured workflow tasks;

generating the workflow suite at least in part based on user manipulation of the one or more icons via the graphical user interface.

10. A networked data storage system comprising:

a storage manager including a workflow manager module executing on a first computing device and configured to generate a data storage workflow suite comprising executable instructions for carrying out a data storage workflow in a networked data storage system, the data storage workflow including a plurality of data storage activities; and

at least two workflow engines each executing on a separate one of at least two second computing devices, wherein

the storage manager deploys the workflow suite to each of the at least two workflow engines, and

based on an allocation scheme, the storage manager selects a first workflow engine of the at least two workflow engines on which to execute at least one current activity of the plurality of data storage activities, wherein at least a second of the at least two workflow engines is not selected.

11. The networked data storage system of claim 10, wherein the at least one current activity comprises a secondary copy operation in which production data associated with at least one of the one or more client computing devices is copied to the secondary storage to create a secondary copy of the production data.

12. The networked data storage system of claim 10, wherein said at least two workflow engines on which the workflow suite is deployed are included in a pool of workflow engines, and wherein the storage manager selects the at least two workflow engines for deployment of the workflow suite, wherein one or more other workflow engines in the pool are not selected.

13. The networked data storage system of claim 12, further comprising a data store, the data store including:

a listing of workflows that have been deployed to one or more of the workflow engines in the pool; and

for each workflow in the listing, an indication of which of the one or more workflow engines the workflow has been deployed to, wherein the data store is consulted as part of said selecting.

14. The networked data storage system of claim 10, wherein said storage manager selects the at least two workflow engines for deployment of the workflow based on one or more of computing resources associated with the at least two workflow engines and geographic proximity of the at least two workflow engines to one or more components in the networked data storage system.

15. The networked data storage system of claim 10, wherein the storage manager obtains information related to activity levels associated with the at least two workflow engines selects the first workflow engine at least in part based on the obtained activity levels.

16. The networked data storage system of claim 15, wherein the obtained information comprises information related to one or more of a total number of workflow suites deployed on the at least two workflow engines, a total number of processes active on the at least two workflow engines, and utilization of the at least two workflow engines.

17. The networked data storage system of claim 10, wherein the allocation scheme is based on at least one of computing resources associated with the at least two workflow engines, physical proximity of the at least two workflow engines to other data storage system components, and communication pathway speeds between the at least two workflow engines and other components in the network data storage system.

18. The networked data storage system of claim 10, further comprising:

a graphical user interface including one or more icons representative of different pre-configured workflow tasks;

wherein the workflow suite is generated at least in part based on user manipulation of the one or more icons via the graphical user interface.

19. A method for allocating the execution of data storage workflow activities within a networked data storage system, the method comprising:

receiving, by a workflow manager module executing on a first computing device, a data storage workflow suite comprising executable instructions for carrying out a data storage workflow in a networked data storage system, the data storage workflow including a plurality of data storage activities, the data storage system comprising;

a storage manager;

one or more client computing devices;

one or more storage devices; and

one or more media agent modules configured, in response to instructions from the storage manager, to manage data storage operations between the client computing devices and the one or more storage devices;

deploying, by the workflow manager module, the workflow suite to at least two workflow engines that each execute on a separate one of at least two second computing devices, the second computing devices remote from the first computing device;

querying activity levels associated with the at least two workflow engines;

based at least in part on the results of said querying, selecting a first workflow engine of the at least two workflow engines on which to execute at least one current activity of the plurality of data storage activities, the current activity comprising a secondary copy operation in which a first media agent of the one or more media agents manages the copying of production data associated with a first client of the one or more client computing devices to at least one of the one or more storage devices to create a secondary copy of the production data; and

in response to the monitored activity levels, instructing the first workflow engine to initiate the current activity.

20. A networked data storage system for allocating the execution of data storage workflow activities within a, the method comprising:

a storage manager including a workflow manager module executing on a first computing device and configured to generate a data storage workflow suite comprising executable instructions for carrying out a data storage workflow in a networked data storage system, the data storage workflow including a plurality of data storage activities;

one or more client computing devices;

one or more storage devices;

one or more media agent modules configured, in response to instructions from the storage manager, to manage data storage operations between the client computing devices and the one or more storage devices; and

at least two workflow engines each executing on a separate one of at least two second computing devices and each comprising the data storage workflow suite received from the storage manager, wherein the second computing devices is remote from the first computing device, wherein

the storage manager queries activity levels associated with the at least two workflow engines, and

based at least in part on the results of said querying, the storage manager selects a first workflow engine of the at least two workflow engines on which to execute at least one current activity of the plurality of data storage activities, the current activity comprising a secondary copy operation in which a first media agent of the one or more media agents manages the copying of production data associated with a first client of the one or more client computing devices to at least one of the one or more storage devices to create a secondary copy of the production data; and

in response to the monitored activity levels, instructing the first workflow engine to initiate the current activity.