System and method for performing integrated storage operations

• US 7,979,389 B2
• Filed: 06/04/2010
• Issued: 07/12/2011
• Est. Priority Date: 11/13/2003
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: May, Andreas, Ngo, David, Prahlad, Anand, Zhou, Lixin
• Primary Examiner(s): Mofiz; Apu M
• Assistant Examiner(s): Bibbee; Jared M
• Application Number:
  US12/794,478
• Publication Number:
  US 20100287141A1
• Time in Patent Office: 403 Days
• Field of Search: 707/640, 707/649, 707/654, 707/609, 707/705
• US Class Current: 707/609
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: Y10S 707/99931 Database or file accessing

Abstract

The present invention relates to a method for performing integrated storage operations on an information store. The present invention comprises identifying a plurality disparate types of storage operations stored in a policy option table. A first storage operation is performed according to a first set of storage criteria stored in the policy option table and a second operation, disparate from the first storage operation, is performed according to a second set of storage criteria stored in the policy option table.

Claims

1. A system for performing data storage operations on at least one information store, the system comprising:

a processor;

means for performing a first type of storage operation according to a first set of storage preferences,

wherein performing the first type of storage operation includes copying at least a first portion of the information store to one or more storage devices and generating first metadata that describes at least the copied first portion stored on the one or more storage devices, wherein the first metadata has a first format associated with the first type of storage operation;

means for performing a second type of storage operation, different from the first storage operation, according to a second set of storage preferences,

wherein performing the second type of storage operation includes copying at least a second portion of the information store to one or more storage devices and generating second metadata that describes at least the copied second portion stored on the one or more storage devices,

wherein the second metadata has a second format associated with the second type of storage operation, and wherein the second format of metadata is different from the first format of metadata;

means for converting metadata, associated with storage operations from multiple information stores employing incompatible formats, into a normalized format;

wherein the means for converting converts the first metadata having the first format associated with the first type of storage operation, and the second metadata having the second format associated with the second type of storage operation, into the normalized metadata format, and

wherein records or data structures for the first and second sets of storage preferences store an indication of a type of data processing operation to be performed; and

,

wherein the means for performing the first and second types of storage operations include means for coordinating actions to perform at least one of;

fully copying data in the at least one information store,

performing snapshots of data in the at least one information store,

performing serverless copy of data in the at least one information store,

archiving data in the at least one information store,

migrating data in the at least one information store, and

recovering data in the at least one information store.

2. The system of claim 1, further comprising:

means for managing different types of storage operations according to incompatible formats associated with multiple information stores employing incompatible formats; and

means, communicating between the means for managing and the information store, for performing local, low-level read and write calls to the information store, at least a cluster level, and under control of the means for managing.

3. The system of claim 1 wherein the records or data structures comprise two of:

records or data objects for at least three of snapshot operations, full copy operations, archive operations, and serverless copy operations;

records or data objects regarding a time interval over which the system should wait after a start of a given data processing operation before starting a next operation;

target host fields or data objects identifying hosts, or identifying network locations for hosts, for which data processing operation, are to be performed;

target volume identifier fields or data objects identifying volumes of the hosts for which data processing operations are to be performed; and

destination volume identifier fields or data objects identifying destination volumes where results of processing operations are to be stored.

4. The system of claim 1, further comprising:

means for managing different types of storage operations according to diverse and incompatible formats based at least in part on the normalized format.

5. The system of claim 1, wherein:

the means for performing the first and second types of storage operations implement, via a proxy host or intelligent network device, different types of storage operations according to diverse and incompatible formats associated with multiple information stores using the incompatible formats.

6. A system for operating within a network, wherein the network couples multiple information stores with at least one storage device, the system comprising:

a processor;

a first data agent, wherein the first data agent operates with a first information store, wherein the first information store employs a first format associated with at least a first data storage operation, and

wherein the first format includes first metadata, and wherein the first metadata describes data copied from the first information store under the first data storage operation;

a second data agent, wherein the second data agent operates with a second information store, and wherein the second information store employs a second format associated with at least a second data storage operation;

wherein the first and second information stores differ,

wherein the first and second formats differ, and

wherein the first and second storage operations differ, and

wherein the second format includes second metadata, and wherein the second metadata describes data copied from the second information store under the second data storage operation; and

,

a data protection agent communicating with the first and second data agents and with the at least one storage device, wherein the data protection agent manages and controls different types of storage operations for data from the first and second information stores to the at least one storage device, via the first and second data agents, despite the first and second different formats and the first and second different storage operations,

wherein the data protection agent employs a normalized metadata format,

wherein the first metadata and the second metadata are converted to the normalized metadata format, and,

wherein the first metadata and the second metadata employ a format that differs from the normalized metadata format.

7. The system of claim 6 wherein the data protection agent is further configured to:

generating first metadata which describes at least a copied first data portion stored on the first information store, wherein the first metadata has the first format associated with the first type of storage operation;

generating second metadata which describes at least a copied second data portion stored on the second information store, wherein the second metadata has the second format associated with the second type of storage operation, and wherein the second format is incompatible with the first format; and

converting the first metadata having the first format associated with the first type of storage operation and the second metadata having the second format associated with the second type of storage operation into a common metadata format.

8. The system of claim 6 wherein first and second data agents are executed in association with a host computer, wherein the host computer is coupled to the first and second information stores and the network, and wherein the system further comprises a proxy host, employing the data protection agent, to perform storage operations to relieve the host computer of processing functions required in storage operations.

9. The system of claim 6, further comprising multiple media agents coupled between the network and multiple storage devices, wherein each media agent is configured to transfer data from at least one of the first and second data agents, as directed by a storage manager associated with the data protection agent.

10. The system of claim 6 wherein the data protection agent is further configured to execute a copy or snapshot of at least one of the first and second information stores, and process the copy or snapshot on a proxy host computer or intelligent network device.

11. The system of claim 6 wherein the data protection agent is further configured to employ a file allocation table (FAT) of a target volume associated with one of multiple storage devices to create a snapshot or full copy as an image level copy, wherein the image level copy permits user access to stored data at a file and folder level.

12. The system of claim 6 wherein the data protection agent is further configured to perform local, low-level read and write calls at least a cluster level, under control of the data protection agent.

13. The system of claim 6 wherein the data protection agent is further configured to generate metadata in a common format based on metadata received from the first and second data agents, wherein the common metadata format is configured to permit managing of different kinds of storage operations into a unified view of storage operations performed on the first and second information stores, and permit comparisons among the common metadata.

14. The system of claim 6, further comprising a storage manager, communicatively coupled with the data protection agent, wherein the storage manager includes at least one of an index cache, a replication volume table, and a policy option table.

15. A non-transitory computer-readable medium storing instructions that, when executed by at least one computer, performs a method for performing integrated storage operations on an information store, the method comprising:

initiating a first type of storage operation to be performed on an information store, wherein initiating the first type of storage operation includes selecting the first type of storage operation based on a first set of storage preferences;

performing the first type of storage operation on the information store, wherein performing the first type of storage operation includes copying at least a first portion of the information store to one or more storage devices and generating first metadata

wherein the first metadata describes at least the copied first portion stored on the one or more storage devices, and

wherein the first metadata has a first format associated with the first type of storage operation;

initiating a second type of storage operation disparate from the first type of storage operation to be performed on the information store, wherein initiating the second type of storage operation includes selecting the second type of storage operation based on a second set of storage preferences,

wherein the second type of storage operation differs from the first type of storage operation; and

performing the second type of storage operation on the information store, wherein performing the second type of storage operation includes copying at least a second portion of the information store to one or more storage devices and generating second metadata which describes at least the copied second portion stored on the one or more storage devices,

wherein the second metadata has a second format associated with the second type of storage operation, and

wherein the second format of metadata is different from the first format of metadata;

converting the first metadata having a first format associated with the first type of storage operation, and the second metadata having a second format associated with the second type of storage operation, into a normalized metadata format.

16. The non-transitory computer-readable medium of claim 15, wherein the method includes identifying a plurality of disparate types of storage operations stored in a policy option table, wherein each type of storage operation copies at least a portion of the information store to one or more storage devices.

17. The non-transitory computer-readable medium of claim 15, wherein the first type of storage operation is one of a full copy, snapshot, and serverless copy, and

wherein the second type of storage operation is a different one of a full copy, snapshot or serverless copy.

18. The non-transitory computer-readable medium of claim 15 wherein the first metadata includes the location of at least the copied first portion on the one or more storage devices and the second metadata includes the location of at least the copied second portion stored on the one or more storage devices.

19. The non-transitory computer-readable medium of claim 15 wherein the first type of storage operation is a full copy operation and the second type of storage operation is a snapshot operation.

20. A non-transitory computer-readable medium of copying at least a portion of an information store, the method comprising:

performing a first type of storage operation to copy at least a first portion of an information store to one or more storage devices;

generating first metadata which describes at least the copied first portion stored on the one or more storage devices,

wherein the first metadata has a first format associated with the first type of storage operation;

performing a second type of storage operation, disparate from the first type of storage operation, to copy at least a second portion of the information store to one or more storage devices;

generating second metadata which describes at least the copied second portion stored on the one or more storage devices,

wherein the second metadata has a second format associated with the second type of storage operation, and,

wherein the second format is incompatible with the first format; and

converting the first metadata having a first format associated with the first type of storage operation, and the second metadata having the second format associated with the second type of storage operation, into a normalized metadata format.

21. The non-transitory computer-readable medium of claim 20, further comprising:

receiving a request to restore a portion of the information store; and

utilizing the normalized metadata to restore the requested portion of the information store from the one or more storage devices.

22. The non-transitory computer-readable medium of claim 20, further comprising:

receiving a request to restore a single file from the information store;

utilizing the normalized metadata to restore the requested single file the information store from the one or more storage devices.

23. The non-transitory computer-readable medium of claim 20, wherein the first type of storage operation and the second type of storage operation are selected after reviewing a policy option table.