Automation of data storage activities

• US 9,292,815 B2
• Filed: 03/06/2013
• Issued: 03/22/2016
• Est. Priority Date: 03/23/2012
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Vibhor, Anand, Karandikar, Amey Vijaykumar
• Primary Examiner(s): Chan, Wing F
• Assistant Examiner(s): KRISHNAN, RAJESHWARI
• Application Number:
  US13/787,152
• Publication Number:
  US 20130254402A1
• Time in Patent Office: 1,112 Days
• Field of Search: 709/226, 709/219, 705/7.26, 705/7.27, 705/7.21, 705/301, 718/102, 718/103, 718/104, 707/624, 715/734, 715/736, 715/769
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: G06Q 10/06316 Sequencing of tasks or work
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A system receives data storage workflow activities that include computer-executable instructions for carrying out data storage workflow in a network data storage system. Once the workflow is received, the system deploys the workflow to one or more workflow engines that can execute the various data storage activities related to the workflow. Prior to executing a data storage activity, the system can determine which workflow engine to use based on an allocation scheme.

Claims

1. A method for allocating execution of a data storage workflow suite within a networked data storage system comprising one or more client computing devices having one or more processors and one or more secondary storage devices, the method comprising:

causing a computing device to display a plurality of data storage display objects, wherein the plurality of data storage display objects are associated with a plurality of data storage activities, each of the plurality of data storage activities comprising computer executable instructions for implementing a particular data storage activity;

receiving from the computing device relationship data between the plurality of data storage display objects, wherein the relationship data indicates an order in which the data storage activities are to be performed;

generating a data storage workflow suite based at least in part on the received relationship data, the data storage workflow suite comprising executable instructions for carrying out the plurality of data storage activities in the networked data storage system, the plurality of data storage activities of the data storage workflow suite including at least a backup storage activity in which primary data blocks forming at least a portion of one or more first files residing in primary storage are copied and the copies are stored in secondary storage, and a restore storage activity in which secondary data blocks residing in the secondary storage are restored to primary storage as one or more second files;

deploying the data storage workflow suite to at least two workflow engines, wherein each workflow engine executes on a separate one of at least two computing devices;

receiving an instruction to initiate the data storage workflow suite;

based at least in part on the instruction to initiate the data storage workflow suite and based at least in part on an allocation scheme, selecting, by one or more processors, a first workflow engine of the at least two workflow engines on which to execute a first data storage activity comprising the backup storage activity for a first client computing device of the one or more client computing devices, wherein at least a second of the at least two workflow engines is not selected, and wherein executing the backup storage activity for the first client computing device comprises;

identifying a set of primary data blocks that do not reside in the secondary storage, wherein the set of primary data blocks form at least a portion of one or more first files residing on a primary data store associated with the first client computing device,

generating a copy of each of the set of primary data blocks, and

storing the copy of each of the set of primary data blocks in the secondary storage; and

following the backup storage activity and based at least in part on the relationship data of the data storage workflow suite, identifying a second data storage activity of the data storage workflow suite to execute.

2. The method of claim 1, wherein said at least two workflow engines on which the data storage workflow suite is deployed are included in a pool of workflow engines, and wherein said deploying further comprises selecting the at least two workflow engines for deployment of the data storage workflow suite, wherein one or more other workflow engines in the pool are not selected.

3. The method of claim 2, further comprising maintaining a data store comprising:

a listing of workflows that have been deployed to one or more of the workflow engines in the pool; and

for each workflow in the listing, an indication of which of the one or more workflow engines the workflow has been deployed to, wherein the data store is consulted as part of said selecting.

4. The method of claim 1, wherein said selecting is based on one or more of computing resources associated with the at least two workflow engines and geographic proximity of the at least two workflow engines to one or more components in the networked data storage system.

5. The method of claim 1, further comprising obtaining information related to activity levels associated with the at least two workflow engines, wherein the allocation scheme and said selecting is at least in part based on the activity levels.

6. The method of claim 5, wherein the obtained information comprises information related to one or more of a total number of data storage workflow suites deployed on the at least two workflow engines, a total number of processes active on the at least two workflow engines, and utilization of the at least two workflow engines.

7. The method of claim 1, wherein the allocation scheme is based on at least one of computing resources associated with the at least two workflow engines, physical proximity of the at least two workflow engines to other data storage system components, and communication pathway speeds between the at least two workflow engines and other data storage system components.

8. A networked data storage system, comprising:

one or more first processors;

a storage manager including a workflow manager module executing on the one or more first processors and configured to;

cause a computing device to display a plurality of data storage display objects, wherein the plurality of data storage display objects are associated with a plurality of data storage activities, each of the plurality of data storage activities comprising computer executable instructions for implementing a particular data storage activity;

receive from the computing device relationship data between the plurality of data storage display objects, wherein the relationship data indicates an order in which the data storage activities are to be performed; and

generate a data storage workflow suite based at least in part on the received relationship data, the data storage workflow suite comprising executable instructions for carrying out the plurality of data storage activities in a networked data storage system, the plurality of data storage activities of the data storage workflow suite including at least a backup storage activity in which primary data blocks forming at least a portion of one or more first files residing in primary storage are copied and the copies are stored in secondary storage, and a restore storage activity in which secondary data blocks residing in the secondary storage are restored to the primary storage as one or more second files;

one or more second processors; and

at least two workflow engines each executing on a separate one of the one or more second processors, wherein the storage manager is further configured to;

deploy the data storage workflow suite to each of the at least two workflow engines;

based at least in part on an allocation scheme, select a first workflow engine of the at least two workflow engines on which to execute a first data storage activity comprising the backup storage activity for a first client computing device of one or more client computing devices, wherein at least a second of the at least two workflow engines is not selected, and wherein to execute the backup storage activity for the first client computing device the first workflow engine is configured to;

identify a set of primary data blocks that do not reside in the secondary storage, wherein the set of primary data blocks form at least a portion of the one or more first files residing on a primary data store associated with the first client computing device,

generate a copy of each of the set of primary data blocks, and

store the copy of each of the set of primary data blocks in the secondary storage; and

following the backup storage activity and based at least in part on the relationship data of the data storage workflow suite, identify a second data storage activity of the data storage workflow suite to execute.

9. The networked data storage system of claim 8, wherein said at least two workflow engines on which the data storage workflow suite is deployed are included in a pool of workflow engines, and wherein the storage manager selects the at least two workflow engines for deployment of the data storage workflow suite, wherein one or more other workflow engines in the pool are not selected.

10. The networked data storage system of claim 9, further comprising a data store, the data store including:

a listing of workflows that have been deployed to one or more of the workflow engines in the pool; and

for each workflow in the listing, an indication of which of the one or more workflow engines the workflow has been deployed to, wherein the data store is consulted as part of said selecting.

11. The networked data storage system of claim 8, wherein said storage manager selects the at least two workflow engines for deployment of the workflow based on one or more of computing resources associated with the at least two workflow engines and geographic proximity of the at least two workflow engines to one or more components in the networked data storage system.

12. The networked data storage system of claim 8, wherein the storage manager obtains information related to activity levels associated with the at least two workflow engines and selects the first workflow engine at least in part based on the activity levels.

13. The networked data storage system of claim 12, wherein the obtained information comprises information related to one or more of a total number of data storage workflow suites deployed on the at least two workflow engines, a total number of processes active on the at least two workflow engines, and utilization of the at least two workflow engines.

14. The networked data storage system of claim 8, wherein the allocation scheme is based on at least one of computing resources associated with the at least two workflow engines, physical proximity of the at least two workflow engines to other data storage system components, and communication pathway speeds between the at least two workflow engines and other components in the network data storage system.