Methods and system of pooling storage devices

• US 7,809,914 B2
• Filed: 11/07/2005
• Issued: 10/05/2010
• Est. Priority Date: 11/05/2004
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Chen, Ho-Chi, Kottomtharayil, Rajiv
• Primary Examiner(s): Shah; Sanjiv
• Assistant Examiner(s): Campos; Yaima
• Application Number:
  US11/269,514
• Publication Number:
  US 20060224852A1
• Time in Patent Office: 1,793 Days
• Field of Search: 711/154, 711161-162, 711170-173, 711203-210
• US Class Current: 711/170
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 12/00 Accessing, addressing or allocating within memory systems or architectures digital input from, or digital output to record carriers, e.g. to disk storage units, G06F3/06
• CPC Class Codes: G06F 12/02 Addressing or allocation; Relocation program address sequencing G06F9/00; arrangements for selecting an address in a digital store G11C8/00
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 3/0686 Libraries, e.g. tape libraries, jukebox
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 41/40 using virtualisation of network functions or resources, e.g. SDN or NFV entities
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A system and method are provided for pooling storage devices in a virtual library for performing a storage operation. A storage management device determines a storage characteristic of a plurality of storage devices with respect to performing a storage operation. Based on a storage characteristic relating to performing the storage operation, the storage management device associates at least two storage devices in a virtual library. The storage management device may continuously monitor the virtual library and detect a change in storage characteristics of the storage devices. When changes in storage characteristics are detected, the storage management device may change associations of the storage device in the virtual library.

Claims

1. A method for performing a storage operation using at least two physical storage devices, the method comprising:

receiving a request to perform a first storage operation to store first data, the first storage operation having at least a first storage characteristic associated therewith;

identifying a first virtual library with dynamically allocated physical storage devices that are allocated based on characteristics associated with different storage operations, wherein the first virtual library comprises a first configuration with at least first and second physical storage devices, wherein a single logical network pathway is associated with the first configuration of the first virtual library such that the first and second physical storage devices appear as a single device;

performing the first storage operation using the first virtual library;

storing the first configuration of the first virtual library;

receiving a request to perform a second storage operation to store second data, the second storage operation having at least a second storage characteristic associated therewith that is different than the first storage characteristic, wherein the second storage operation requires more storage than exists in the first configuration of the first virtual library;

automatically determining that the second storage operation requires more storage than exists in the first and second physical storage devices associated with the first configuration of the first virtual library;

automatically detecting with one or more computer processors, at least a third physical storage device associated with a second virtual library, wherein the third physical storage is adequate to satisfy both the second storage characteristic associated with the second storage operation for the first virtual library and one or more future scheduled storage operations associated with the second virtual library, and wherein detection of the third storage device is based at least in part on the likelihood of congestion in one or more of the future scheduled storage operations;

automatically allocating with one or more computer processors the third physical storage device with both the first virtual library and the second virtual library without the need of user intervention, wherein the first virtual library comprises a second configuration with at least the first physical storage device, the second physical storage device and the third physical storage device, and wherein the single logical network pathway is associated with the second configuration of the first virtual library such that the first, second and third physical storage devices of the second configuration appear to be the single device, and wherein the second configuration of the first virtual library and the first configuration of the first virtual library share the single logical network pathway;

performing the second storage operation using the second configuration comprising third storage device in association with the first virtual library;

performing one or more of the future scheduled storage operations using the third storage device in association with second virtual library while the third storage device remains in association with the first virtual library;

storing the second configuration of the first virtual library;

receiving a first request for data via the single logical network pathway at the first virtual library wherein the first request is for data associated with the first configuration of the first virtual library;

receiving a second request for data via the single logical network pathway at the first virtual library wherein the second request is for data associated with the second configuration of the first virtual library; and

automatically determining the location of the data associated with the first and second requests by accessing the stored first and second configurations of the first virtual library.

2. The method of claim 1, wherein creating the first configuration of the first virtual library further comprises storing data indicating an association of the first and second physical storage device with the first virtual library in an index.

3. The method of claim 1, further comprising:

detecting a fourth physical storage device among a plurality of physical storage devices, the fourth physical storage device satisfying a storage characteristic associated with a third storage operation;

disassociating the first physical storage device with the first virtual library;

associating the fourth physical storage device with the first virtual library wherein the first virtual library comprises a third configuration with at least the second physical storage device, the third physical storage device and the fourth physical storage device;

performing the fourth storage operation; and

storing the third configuration of the first virtual library.

4. The method of claim 3, wherein the third physical storage device comprises a different type of storage media than the first physical storage device.

5. The method of claim 1, further comprising:

receiving a first value of a characteristic of the first physical storage device, the first physical storage device being associated in the first virtual library;

detecting a change in the characteristic of the first physical storage device, wherein the change in the characteristic of the first physical storage device causes the first physical storage device to not satisfy a storage characteristic threshold;

identifying that the third physical storage device has a storage characteristic satisfying the storage characteristic threshold;

disassociating the first physical storage device from the first virtual library; and

associating the third physical storage device in the first virtual library.

6. The method of claim 5 wherein detecting a change in the characteristic of the first physical storage device comprises examining the first physical storage device prior to receiving the request to perform the storage operation.

7. The method of claim 5 wherein

disassociating the first physical storage device from the first virtual library comprises updating an index with data indicating a disassociation of the first physical storage device with the first virtual library;

and

wherein associating the third physical storage device in the first virtual library comprises updating the index with data indicating an association of the third physical storage device with the first virtual library.

8. The method of claim 1, wherein the storage characteristic comprises a security requirement.

9. The method of claim 1, wherein the storage characteristic comprises a frequency interval for performing the requested storage operation.

10. The method of claim 1, wherein the storage characteristic comprises a media type for performing the requested storage operation.

11. A method for storing data using a virtual library, the method comprising:

receiving a request to perform a first data storage operation;

identifying a first virtual library comprising at least two physical storage devices that satisfy a storage characteristic associated with the first data storage operation and that are capable of storing data associated with the first data storage operation and wherein the first virtual library comprises a first configuration with the at least two physical storage devices;

performing the first storage operation using the first configuration of the first virtual library;

storing the first configuration of the first virtual library;

receiving a request to perform a second storage operation;

automatically determining that the second storage operation requires more storage than exists in the first and second physical storage devices associated with the first configuration of the first virtual library;

automatically detecting, among a plurality of physical storage devices, at least a third physical storage device associated with a second virtual library, wherein the third physical storage is adequate to satisfy both the second storage operation for the first virtual library and one or more future scheduled storage operations associated with the second virtual library, and wherein detection of the third storage device is based at least in part on the likelihood of congestion in one or more of the future scheduled storage operations;

automatically associating the third physical storage device with both the first virtual library and the second virtual library wherein the first virtual library includes at least three physical storage devices capable of performing the second data storage operation and wherein the first virtual library comprises a second configuration;

performing the second data storage operation using the second configuration of the first virtual library; and

storing the second configuration of the first virtual library.

12. The method of claim 11 further comprising:

detecting that a given one of the at least two physical storage devices becomes incapable of performing the data storage operation;

disassociating the given one of the at least two physical storage devices with the first virtual library; and

associating another physical storage device capable of performing the storage operation with the first virtual library.

13. The method of claim 12 wherein disassociating the given one of the at least two physical storage devices with the first virtual library comprises storing data in an index indicating a disassociation of the given one of the at least two physical storage devices with the first virtual library and storing data in an index indicating a previous configuration of the first virtual library.

14. The method of claim 12 wherein associating a third physical storage device capable of performing the storage operation with the first virtual library comprises storing data in an index indicating an association of the third physical storage device with the first virtual library and storing data in an index indicating a current configuration of the first virtual library.

15. The method of claim 12 further comprising consulting the index to identify the current configuration of the first virtual library for performing the storage operation.

16. A computer readable storage medium including computer executable code for a method for performing a storage operation using at least two physical storage devices, the method comprising:

receiving with one or more processors a request to perform a first storage operation;

determining a storage characteristic related to performing the storage operation;

identifying a first virtual library comprising a first configuration with at least a first physical storage device and a second physical storage device;

automatically determining that the first and second physical storage devices associated with the first configuration of the first virtual library do not have adequate storage capacity to perform the storage operation;

automatically modifying the first virtual library to include at least a third physical storage device associated with a second virtual library, wherein the third physical storage device is adequate to satisfy both the data storage operation for the first virtual library and one or more future scheduled storage operations associated with the second virtual library, and wherein selection of the third storage device is based at least in part on the likelihood of congestion in one or more of the future scheduled storage operations, and wherein modifying the first virtual library comprises associating a second configuration with the first virtual library that comprises at least the first, second and third physical storage devices; and

performing the storage operation using the second configuration of the first virtual library.