Methods and system of pooling secondary storage devices

• US 10,191,675 B2
• Filed: 11/16/2016
• Issued: 01/29/2019
• Est. Priority Date: 11/05/2004
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Kottomtharayil, Rajiv, Chen, Ho-Chi
• Primary Examiner(s): Rigol, Yaima
• Application Number:
  US15/353,546
• Publication Number:
  US 20170060458A1
• Time in Patent Office: 804 Days
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 12/00 Accessing, addressing or allocating within memory systems or architectures digital input from, or digital output to record carriers, e.g. to disk storage units, G06F3/06
• CPC Class Codes: G06F 12/02 Addressing or allocation; Relocation program address sequencing G06F9/00; arrangements for selecting an address in a digital store G11C8/00
• CPC Class Codes: G06F 2201/815 Virtual
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 3/0686 Libraries, e.g. tape libraries, jukebox
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 41/40 using virtualisation of network functions or resources, e.g. SDN or NFV entities
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A system and method are provided for pooling storage devices in a virtual library for performing a storage operation. A storage management device determines a storage characteristic of a plurality of storage devices with respect to performing a storage operation. Based on a storage characteristic relating to performing the storage operation, the storage management device associates at least two storage devices in a virtual library. The storage management device may continuously monitor the virtual library and detect a change in storage characteristics of the storage devices. When changes in storage characteristics are detected, the storage management device may change associations of the storage device in the virtual library.

Claims

1. A method for grouping storage devices in a virtual library comprising:

storing with computer hardware comprising one or more computer processors, primary data on a plurality of primary storage devices in a primary storage system and storing at least one or more secondary copies of the primary data on one or more secondary storage devices in a secondary storage system;

creating a virtual library that is associated with a logical network pathway, the virtual library comprising a first configuration of a first group of the secondary storage devices, the first group of the secondary storage devices comprise at least first and second secondary storage devices, wherein the first and second secondary storage devices are accessible with the logical network pathway associated with the virtual library;

storing at least first and second storage policies in a storage policy data structure, the first storage policy comprising at least a first set of future scheduled secondary storage operations and the second storage policy comprising at least a second set of future scheduled secondary storage operations;

automatically determining when the first set of future scheduled secondary storage operations and a when the second set of future scheduled secondary storage operations will occur;

automatically determining the first group of the secondary storage devices in the virtual library have insufficient capacity for the first set future scheduled secondary storage operations;

automatically determining availability of a third secondary storage device based at least in part on the amount of capacity needed for the future scheduled secondary storage operations, and based at least in part on whether the first set of future scheduled storage operations on the third secondary storage device will conflict with the second set of future scheduled operations associated with the second storage policy; and

automatically associating, without the need of user intervention, the third secondary storage device with the virtual library wherein the third secondary storage device is accessible with the logical network pathway associated with the virtual library, the virtual library comprising a second configuration of a second group of secondary storage devices.

2. The method of claim 1, further comprising disassociating the first secondary storage device with the second group of secondary storage devices.

3. The method of claim 2 wherein disassociating the first secondary storage device from the second group of secondary storage devices comprises updating an index with data indicating a disassociation.

4. The method of claim 1, further comprising:

detecting a change in the characteristic of the first secondary storage device that causes the first secondary storage device to not satisfy a storage characteristic threshold;

identifying that a fourth secondary storage device has a storage characteristic satisfying the storage characteristic threshold;

disassociating the first secondary storage device; and

associating the fourth secondary physical storage device with the second group of secondary storage devices.

5. The method of claim 1 further comprising receiving a first request for data and a second request for data stored on the secondary storage devices and automatically determining that the first request is associated with data stored on the first configuration of the virtual library and the second request is associated with data stored on the second configuration of the virtual library.

6. The method of claim 1 wherein determining the availability of the third secondary storage device is based at least in part on the likelihood of network congestion associated with the future scheduled secondary storage operations.

7. The method of claim 1 wherein determining the availability of the third secondary storage device is based at least in part on capacity of the third secondary storage device.

8. The method of claim 1 wherein the third secondary storage device continues to be associated with a third group of secondary storage devices.

9. The method of claim 1 wherein the future scheduled secondary storage operations are stored in an index associated with at least one media management component.

10. The method of claim 1 wherein the future scheduled secondary storage operations are stored in association with a storage policy.

11. A system that groups storage devices in a virtual library comprising:

a primary storage system comprising a plurality of primary storage devices that store primary data;

a secondary storage system comprising one or more secondary storage devices, the secondary storage system stores at least one or more secondary copies of the primary data;

a virtual library that is associated with a logical network pathway, the virtual library comprising a first configuration of a first group of the secondary storage devices, the first group of the secondary storage devices comprise at least first and second secondary storage devices, wherein the first and second secondary storage devices are accessible with the logical network pathway associated with the virtual library;

at least first and second storage policies stored in a storage policy data structure, the first storage policy comprising at least a first set of future scheduled secondary storage operations and the second storage policy comprising at least a second set of future scheduled secondary storage operations;

a media management component comprising at least computer hardware, the media management component automatically determines when the first set of future scheduled secondary storage operations and when the second set of future scheduled secondary storage operations will occur;

the media management component automatically determines the first group of the secondary storage devices in the virtual library have insufficient capacity for the first set future scheduled secondary storage operations;

the media management component automatically determines availability of a third secondary storage device based at least in part on the amount of capacity needed for the future scheduled secondary storage operations, and based at least in part on whether the first set of future scheduled storage operations on the third secondary storage device will conflict with the second set of future scheduled operations associated with the second storage policy; and

the media management component automatically associates, without the need of user intervention, the third secondary storage device with the virtual library wherein the third secondary storage device is accessible with the logical network pathway associated with the virtual library, the virtual library comprising a second configuration of a second group of secondary storage devices.

12. The system of claim 11, wherein the at least one media management component disassociates the first secondary storage device with the second group of secondary storage devices.

13. The system of claim 12 wherein disassociating the first secondary storage device from the second group of secondary storage devices comprises updating an index with data indicating a disassociation.

14. The system of claim 11 wherein the media management component:

detects a change in the characteristic of the first secondary storage device that causes the first secondary storage device to not satisfy a storage characteristic threshold;

identifies that a fourth secondary storage device has a storage characteristic satisfying the storage characteristic threshold;

disassociates the first secondary storage device from the second group of secondary storage devices; and

associates the fourth secondary storage device with the second group of storage devices.

15. The system of claim 11 wherein the media management component receives a first request for data and a second request for data stored on the secondary storage devices and automatically determines that the first request is associated with data stored on a first configuration of the virtual library and the second request is associated with data stored on the second configuration of the virtual library.

16. The system of claim 11 wherein determining the availability of the third secondary storage device is based at least in part on the likelihood of network congestion associated with the future scheduled storage operations.

17. The system of claim 11 wherein determining the availability of the third secondary storage device is based at least in part on capacity of the third secondary storage device.

18. The system of claim 11 wherein the third secondary storage device continues to be associated with a third group of secondary storage devices.

19. The system of claim 11 wherein the future scheduled storage operations are stored in an index associated with the at least one media management component.

20. The system of claim 11 wherein the future scheduled storage operations are stored in association with a storage policy.