Systems and methods for classifying and transferring information in a storage network

• US 8,051,095 B2
• Filed: 01/28/2010
• Issued: 11/01/2011
• Est. Priority Date: 11/28/2005
• Status: Expired due to Fees

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Brockway, Brian, Schwartz, Jeremy A., Ngo, David, Prahlad, Anand, Muller, Marcus S.
• Primary Examiner(s): Trujillo; James
• Assistant Examiner(s): MOSER, BRUCE M
• Application Number:
  US12/695,203
• Publication Number:
  US 20100131461A1
• Time in Patent Office: 642 Days
• Field of Search: 707/10, 707/999.01, 707/705, 707/713, 707/770, 707/776, 707/968, 707/825
• US Class Current: 707/770
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1466 to make the backup process non-disruptive
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/285 : Clustering or classification
• CPC Class Codes: G06F 2201/80 : Database-specific techniques
• CPC Class Codes: G06F 3/0604 : Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 : by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0649 : Lifecycle management
• CPC Class Codes: G06F 3/0685 : Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: Y10S 707/99942 : Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99948 : Application of database or data structure, e.g. distributed, multimedia, or image
• CPC Class Codes: Y10S 707/99955 : Archiving or backup

Abstract

Systems and methods for data classification to facilitate and improve data management within an enterprise are described. The disclosed systems and methods evaluate and define data management operations based on data characteristics rather than data location, among other things. Also provided are methods for generating a data structure of metadata that describes system data and storage operations. This data structure may be consulted to determine changes in system data rather than scanning the data files themselves.

Claims

1. A computer-implemented method for accessing data through a distributed metabase, comprising:

receiving a request to access data stored on a storage device;

selecting at least one of multiple distributed metabases containing metadata related to the data stored on the storage device, wherein the metadata includes metadata other than file system metadata and other than metadata identifying logical locations of the data, and wherein the multiple metabases store metadata related to the data stored on the storage device, and wherein the selecting includes—

determining characteristics of the multiple distributed metabases,

evaluating the characteristics against one or more criteria, and

selecting the at least one metabase based on the evaluation;

analyzing the metadata stored at the at least one metabase to identify a set of data, wherein the set of data is less than the data stored on the storage device;

based on the analysis of the metadata, associating one or more data classifications with a subset of data, wherein—

the subset of data is less than the set of data, and

the one or more data classifications describe characteristics of at least some of the data stored on the storage device; and

,

accessing the subset of data based on the metadata contained by the at least one selected metabase and the associated one or more data classifications.

2. The computer-implemented method of claim 1 wherein selecting comprises selecting one of the multiple metabases based on a level of activity of each metabase, wherein the level is past, current, or projected.

3. The computer-implemented method of claim 1 wherein selecting comprises determining a content type of the data and selecting a metabase based on the content type.

4. The computer-implemented method of claim 1 wherein selecting further comprises:

selecting a first metabase;

determining whether the first metabase is available; and

selecting a second metabase when the first metabase is not available.

5. The computer-implemented method of claim 1 wherein selecting comprises determining which of the multiple metabases contains the most recent metadata related to the data.

6. The computer-implemented method of claim 1 wherein selecting comprises determining which of the multiple metabases contains metadata associated with a most recent copy of the data.

7. The computer-implemented method of claim 1 wherein selecting at least one of multiple metabases comprises consulting a metabase index to identify a set of data without accessing the data storage device.

8. The computer-implemented method of claim 1 wherein the request comprises a query to find identified data, and the selected metabase performs the query without accessing the data storage device.

9. The computer-implemented method of claim 1 wherein the selecting includes monitoring the metabases.

10. The computer-implemented method of claim 1 wherein the selecting includes evaluating the characteristics against one or more criteria relating to availability of the metabases.

11. The computer-implemented method of claim 1, further comprising:

receiving a data management request, wherein the data management request includes one or more criteria and comprises a request to—

identify the subset of data based on the one or more data classifications, and

perform a data storage operation on the subset of data; and

,

identifying the subset of data that is associated with one or more data classifications that satisfy the one or more criteria; and

performing the data storage operation requested by the data management request on the subset of data.

12. The computer-implemented method of claim 1, wherein the metadata of at least one of the multiple metabases is a copy of metadata in another of the multiple metabases.

13. A computer-implemented method for accessing data through a distributed metabase, the method comprising:

receiving a request to access data stored on a storage device;

selecting at least one of multiple distributed metabases based on at least one characteristic of the multiple metabases,

wherein the metabases contain metadata associated with the data stored on the storage device,

wherein the metadata includes metadata other than file system metadata and other than metadata identifying logical locations of the data,

wherein the metadata facilitates performance of data storage operations on the data, and

wherein the characteristic of each of the multiple metabases includes a past, current, or projected level of activity associated with each of the metabases;

analyzing the metadata stored at the at least one metabase to identify a set of data, wherein the set of data is less than the data stored on the storage device;

based on the analysis of the metadata, associating one or more data classifications with a subset of data, wherein—

the subset of data is less than the set of data, and

the one or more data classifications describe characteristics of at least some of the data stored on the storage device; and

,

accessing the subset of data based on the metadata contained by the at least one selected metabase and the associated one or more data classifications.

14. The computer-implemented method of claim 13 wherein the selecting includes:

determining the characteristics of the metabases; and

evaluating the characteristics against one or more criteria.

15. The computer-implemented method of claim 13 wherein the selecting further comprises determining which of the multiple metabases contains the most recent metadata related to the data.

16. The computer-implemented method of claim 13 wherein the selecting further comprises determining which of the multiple metabases contains metadata associated with a most recent copy of the data.

17. The computer-implemented method of claim 13, wherein the metadata of at least one of the multiple metabases is a copy of metadata in another of the multiple metabases.

18. A method for accessing data based on metadata stored in a network, the method comprising:

receiving multiple requests to access data stored on at least one of multiple storage devices accessible via the network;

selecting at least one of multiple distributed metabases containing metadata related to data stored on the multiple storage devices,

wherein the metadata includes metadata other than file system metadata and other than metadata identifying logical locations of the data, and

wherein the classifications include information used to facilitate performance of a data storage operation on the data;

sending at least one of the multiple requests to the at least one selected metabase,

wherein sending the at least one request includes not sending at least some of the multiple requests to the multiple storage devices; and

analyzing the metadata stored at the at least one selected metabase to identify a set of data, wherein the set of data is less than the data stored on the storage device;

based on the analysis of the metadata, associating one or more data classifications with a subset of data, wherein—

the subset of data is less than the set of data, and

the one or more data classifications describe characteristics of at least some of the data stored on the storage device; and

,

accessing the subset of data based on the metadata contained by the at least one selected metabase and the associated one or more data classifications.

19. A non-transitory computer-readable medium storing instruction, which when performed by a data processing device, perform a method for accessing data based on metadata stored in a network, the method comprising:

receiving multiple requests to access data stored on one or more storage devices accessible via the network;

selecting at least one of multiple distributed metabases containing metadata related to the data stored on the one or more storage devices,

wherein the metadata includes metadata other than file system metadata and other than metadata identifying logical locations of the data, and

wherein the classifications include information used to facilitate performance of a data storage operation on the data;

sending at least one of the multiple requests to the at least one selected metabase; and

processing the metadata stored at the at least one selected metabase to identify a set of data, wherein the set of data is less than the data stored on the storage device;

based on the processing of the metadata, associating one or more data classifications with a subset of data, wherein—

the subset of data is less than the set of data, and

the one or more data classifications describe characteristics of at least some of the data stored on the storage device; and

,

accessing the subset of data based on the metadata contained by the at least one selected metabase and the associated one or more data classifications.

20. The computer-readable medium of claim 19 wherein the selecting includes:

determining the characteristics of the multiple metabases; and

evaluating the characteristics against one or more criteria.

21. The computer-readable medium of claim 19 wherein the selecting further comprises determining which of the multiple metabases contains the most recent metadata related to the data.

22. The computer-readable medium of claim 19 wherein the selecting further comprises determining which of the multiple metabases contains metadata associated with a most recent copy of the data.