SYSTEMS AND METHODS FOR USING METADATA TO ENHANCE STORAGE OPERATIONS

• US 20070192385A1
• Filed: 11/28/2006
• Published: 08/16/2007
• Est. Priority Date: 11/28/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Brockway, Brian, Ngo, David, Prahlad, Anand, Muller, Marcus, Schwartz, Jeremy
• Granted Patent: US 8,271,548 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1466 to make the backup process non-disruptive
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/285 : Clustering or classification
• CPC Class Codes: G06F 2201/80 : Database-specific techniques
• CPC Class Codes: G06F 3/0604 : Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 : by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0649 : Lifecycle management
• CPC Class Codes: G06F 3/0685 : Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: Y10S 707/99942 : Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99948 : Application of database or data structure, e.g. distributed, multimedia, or image
• CPC Class Codes: Y10S 707/99955 : Archiving or backup

Abstract

Systems and methods for managing electronic data are disclosed. Various data management operations can be performed based on a metabase formed from metadata. Such metadata can be identified from an index of data interactions generated by a journaling module, and obtained from their associated data objects stored in one or more storage devices. In various embodiments, such processing of the index and storing of the metadata can facilitate, for example, enhanced data management operations, enhanced data identification operations, enhanced storage operations, data classification for organizing and storing the metadata, cataloging of metadata for the stored metadata, and/or user interfaces for managing data. In various embodiments, the metabase can be configured in different ways. For example, the metabase can be stored separately from the data objects so as to allow obtaining of information about the data objects without accessing the data objects or a data structure used by a file system.

Claims

1. A method for managing electronic data in a storage system, the method comprising:

storing a plurality of data objects in a storage device;

receiving a log having a plurality of entries identifying data interactions between at least one application and one or more of the plurality of data objects, the log entries being populated by a journaling application;

updating, based on the log entries, metadata stored in a metabase;

accessing a selection criteria regarding performing at least one storage operation on at least one of the plurality of data objects based on the metadata; and

identifying at least a first data object of the plurality of data objects based on the metadata that satisfies the selection criteria.

2. The method of claim 1, wherein the metadata in the metabase are stored separate from entire content of each of the plurality of data objects.

3. The method of claim 1, wherein the identifying is performed without accessing the plurality of data objects on the storage device.

4. The method of claim 1, wherein the updating comprises:

processing the log to select at least one data interaction that modifies the metadata of at least one of the plurality of data objects; and

updating the metadata in the metabase based on the selected at least one data interaction.

5. The method of claim 4, wherein the identifying is performed without accessing a data structure associated with storage of the plurality of data objects on the storage device.

6. The method of claim 5, wherein the identifying is performed without accessing a file access table (FAT) associated with storage of the plurality of data objects on the storage device.

7. The method of claim 1, additionally comprising performing the at least one storage operation on the first data object.

8. The method of claim 7, wherein the at least one storage operation comprises a copy operation.

9. The method of claim 7, wherein the at least one storage operation comprises a snapshot operation.

10. The method of claim 9, wherein the at least one storage operation comprises a hierarchical storage management (HSM) operation.

11. The method of claim 7, wherein performing the at least one storage operation comprises copying the first data object to a second storage device.

12. The method of claim 11, additionally comprising updating records of a second metabase associated with the second storage device to include the metadata of the first data object.

13. The method of claim 1, wherein the selection criteria comprises a user-defined policy.

14. The method of claim 13, wherein the selection criteria identifies metadata relating to at least one of a read operation, a write operation, and an access operation associated with the data object.

15. The method of claim 14, wherein the access operation comprises at least one of web-page access, electronic gaming interaction, chat-room access, instant messaging, communication interaction, or multimedia interaction.

16. The method of claim 15, wherein the selection criteria involving the access operation allows at least one of identifying, capturing, classifying, or tracking of the access operation.

17. The method of claim 1, further comprising accessing one or more of the metadata associated with the corresponding data objects one or more times to update the metabase.

18. A system for managing electronic data stored on a storage device, the system comprising:

a selection criteria for performing a storage operation on at least one of a plurality of data objects on a storage device, the selection criteria being dependent upon metadata, other than location data, associated with the plurality of data objects;

a metabase having a plurality of records storing the metadata associated with the plurality of data objects;

a classification module configured to process a log file generated by a journal program, the log file comprising a list of interactions between at least one application and the plurality of data objects, the classification module being further configured to update the metabase based on interactions identified in the log file; and

a management module configured to access the metabase to identify at least a first data object that satisfies the selection criteria.

19. The system of claim 18, wherein the metadata in the metabase are stored separate from entire content of the plurality of data objects.

20. The system of claim 18, wherein the classification module is configured to update the metabase based on interactions that modify metadata of at least a portion of the plurality of data objects.

21. The system of claim 18, wherein the management module accesses the metabase to identify the first data object without accessing either the plurality of data objects or a data structure used in storage of the plurality of data objects on the storage device.

22. The system of claim 18, wherein the management module is configured to copy the first data object from the storage device to a second storage device.

23. The system of claim 22, wherein the second storage device comprises a storage device configured to retain the copy of the first data object.

24. The system of claim 22, wherein the plurality of records on the metabase store metadata associated with data objects on the second storage device.

25. The system of claim 22, further comprising a second metabase having a second plurality of records storing metadata associated with data objects on the second storage device.

26. The system of claim 22, wherein the selection criteria comprises a data migration policy.

27. A system for performing storage operations on electronic data, the system comprising:

means for storing a plurality of data objects;

means for identifying at least one storage operation to be performed on one or more data objects based on the metadata associated with the plurality of data objects;

means for storing the metadata;

means for processing a journal file comprising a list of interactions between at least one application and one or more of the plurality of data objects, the processing means being further configured to update the means for storing the metadata based on interactions identified by the journal file; and

means for performing a storage operation on the at least one data object, based on accessing the metadata in the means for storing the metadata.

28. The system of claim 27, wherein the interactions identified by the journal file comprised interactions that modify metadata of at least a portion of the plurality of data objects.

29. The system of claim 27, wherein accessing of the metadata in the means for storing the metadata is performed without accessing the plurality of data objects on the means for storing the data objects.

30. The system of claim 27, wherein the at least one application is managed by a first processor and the processing means is managed by a second processor.