Load balancing across multiple data paths

• US 10,310,950 B2
• Filed: 08/21/2017
• Issued: 06/04/2019
• Est. Priority Date: 05/09/2014
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Kochunni, Jaidev O., Liu, Chong, Vijayan, Manoj Kumar, Kottomtharayil, Rajiv
• Primary Examiner(s): Gillis, Brian J.
• Assistant Examiner(s): Ling, Chhian (Amy)
• Application Number:
  US15/682,085
• Publication Number:
  US 20180011767A1
• Time in Patent Office: 652 Days
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 11/349 for interfaces, buses
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0626 Reducing size or complexity of storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Multiple data paths may be available to a data management system for transferring data between a primary storage device and a secondary storage device. The data management system may be able to gain operational advantages by performing load balancing across the multiple data paths. The system may use application layer characteristics of the data for transferring from a primary storage to a backup storage during data backup operation, and correspondingly from a secondary or backup storage system to a primary storage system during restoration.

Claims

1. At least one non-transitory computer-readable medium storing operations, which when executed by at least one data processor, retrieve data from a secondary data storage device, wherein the data has been copied from primary data storage to the secondary storage device, the operations comprising:

allocating a job for retrieving the data from the secondary data storage,

wherein the secondary data storage comprises multiple storage locations, and wherein each storage location stores at least one of multiple subsets of the data, and

wherein each unit of data to be retrieved from the secondary data storage device includes information identifying a primary data storage location from which the unit of data was copied; and

dividing the job into multiple data streams based on application-level characteristics identified to the data of each subset,

wherein the multiple streams include;

at least one stream associated with data identified with one or more application-level characteristics, and

at least one stream associated with data identified with an application-level characteristic of control data, index data, or log data, and

wherein each data path is associated with a corresponding and different storage location from the multiple storage locations of the secondary data storage.

2. The computer-readable medium of claim 1, further including:

assigning the data path for restoring one of the subset of the data stored at the corresponding storage location; and

restoring the multiple subsets of data,

wherein at least two subsets of data are restored using two different data paths.

3. The computer-readable medium of claim 1, further including:

providing, to a media agent, the information identifying the primary data storage location and a destination storage location.

4. The computer-readable medium of claim 1, further including:

receiving, for the job, a storage policy that specifies a rule for assigning a media agent and a destination storage location; and

allocating resources to the job according to the storage policy, wherein the storage policy further specifies a de-duplication rule for the job, wherein the rule specifies at least one of a directory to be used for de-duplication and a hash function to be used for de-duplication.

5. The computer-readable medium of claim 1, further including:

receiving, for the job, a storage policy that specifies a rule for assigning a media agent and a destination storage location; and

allocating resources to the job according to the storage policy.

6. The computer-readable medium of claim 1, wherein the at least two different streams comprise a first stream configured to transfer data over a first host bus adapter and a second stream configured to transfer data over a second host bus adapter that uses a different bus technology than the first host bus adapter.

7. At least one non-transitory computer-readable medium carrying instructions, which when executed by at least one data processor, perform operations for copying data from primary data storage to secondary data storage, the operations comprising:

identifying first and second subsets of the data to be copied from the primary data storage based on an application level characteristic different for the first subset of the data and the second subset of the data,

wherein the first subset of the data includes data identified as user data, and

wherein the second subset of the data includes data identified as control data, index data or log data;

dividing a job for transferring the data to the secondary data storage into multiple streams of data,

wherein each stream has a corresponding data path for carrying data to the secondary data storage, and

wherein the secondary data storage comprises multiple storage locations;

maintaining information identifying a primary data storage location and a destination data storage location for each unit of data copied to the secondary data storage; and

transferring the data to be copied using one or more data streams for the first subset of data and one or more different data streams for the second subset of data.

8. The computer-readable medium of claim 7, further including:

receiving, from a media agent, the information identifying the primary data storage location and the corresponding destination data storage location.

9. The computer-readable medium of claim 7, further including:

receiving, for the job, a storage policy that specifies a rule for assigning a media agent and the corresponding destination storage location; and

allocating resources to the job according to the storage policy, wherein the storage policy further specifies a de-duplication rule for the job.

10. The computer-readable medium of claim 7, further including:

receiving, for the job, a storage policy that specifies a rule for assigning a media agent and the corresponding destination storage location; and

allocating resources to the job according to the storage policy.

11. The computer-readable medium of claim 7, further comprising:

maintaining, for the primary data storage location, information identifying the first subset of data and a first group of destination storage locations at which units making up the first subset of data are stored, and the second subset of data and a second group of destination storage locations at which units making up the second subset of data are stored.

12. The computer-readable medium of claim 7, wherein the at least two different data streams comprise first data streams configured to transfer data over a first host bus adapter and second data streams configured to transfer data over a second host bus adapter that uses a different bus technology than the first host bus adapter.

13. The computer-readable medium of claim 7, wherein the primary data storage location comprises a source file name.

14. A computer-executable method, which when executed by at least one data processor, performs a method for copying data from primary data storage to target data storage, the method comprising:

initiating load balancing copy operations for the data from the primary data storage to target storage devices,

wherein the load balancing copy operations are performed using multiple different system resources,

wherein the load balancing copy operations are performed on different target storage devices,

wherein one type of data to be copied is generated by a user or an application, and

wherein the one type of data is to be accessed more frequently or is more critical as compared to other type of data that is to be accessed less frequently or is more tolerant of errors;

reserving at least one of the system resources for performing the load balancing copy operations across multiple data paths; and

causing the load balancing copy operations to be performed for the data from the primary data storage to the target storage devices.

15. The method of claim 14, wherein the other type of data is database data, wherein the one type of data includes log files and index tables for the database data, wherein the log files and index tables are copied to a first data storage device that is faster, more accurate or provides a greater number of read/write cycles over its lifetime as compared to a second data storage device, while the database data is copied to the second data storage device.

16. The method of claim 14, wherein the one type of data includes encrypted data or financial data that is more sensitive to individual bit errors, and the other type of data includes bulk media files, wherein the one type of data is copied using a first host bus adapter and first storage drive having a lower probability of read/write errors or a longer shelf life as compared to a second host bus adapter and a second storage drive for copying the bulk media files.

17. The method of claim 14, wherein the load balancing copy operations provide balanced use of system resources, including host bus adapters and data storage devices having different characteristics such as read/write speeds, read/write life cycles and error tolerances.

18. The method of claim 14, wherein the load balancing copy operations allow a backup storage system to use an application level characteristic associated with data in data backup and data restoration operations with respect to different data storage devices within the system.