Metabase for facilitating data classification

• US 8,285,685 B2
• Filed: 06/23/2010
• Issued: 10/09/2012
• Est. Priority Date: 11/28/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Prahlad, Anand, Schwartz, Jeremy Alan, Ngo, David, Brockway, Brian, Muller, Marcus S.
• Primary Examiner(s): Badawi, Sherief
• Application Number:
  US12/822,043
• Publication Number:
  US 20110016163A1
• Time in Patent Office: 839 Days
• Field of Search: 707/672, 707/741, 707/762, 707/763, 707/739, 707/803, 707/812, 707/665, 707/667, 707/668, 707/653, 707/662, 707/666, 711/162
• US Class Current: 707/665
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1466 to make the backup process non-disruptive
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/285 : Clustering or classification
• CPC Class Codes: G06F 2201/80 : Database-specific techniques
• CPC Class Codes: G06F 3/0604 : Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 : by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0649 : Lifecycle management
• CPC Class Codes: G06F 3/0685 : Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: Y10S 707/99942 : Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99948 : Application of database or data structure, e.g. distributed, multimedia, or image
• CPC Class Codes: Y10S 707/99955 : Archiving or backup

Abstract

Systems and methods for managing electronic data are disclosed. Various data management operations can be performed based on a metabase formed from metadata. Such metadata can be identified from an index of data interactions generated by a journaling module, and obtained from their associated data objects stored in one or more storage devices. In various embodiments, such processing of the index and storing of the metadata can facilitate, for example, enhanced data management operations, enhanced data identification operations, enhanced storage operations, data classification for organizing and storing the metadata, cataloging of metadata for the stored metadata, and/or user interfaces for managing data. In various embodiments, the metabase can be configured in different ways. For example, the metabase can be stored separately from the data objects so as to allow obtaining of information about the data objects without accessing the data objects or a data structure used by a file system.

Claims

1. A method for managing electronic data in one or more storage devices, the method comprising:

obtaining metadata from a plurality of data objects associated with one or more data interactions, wherein the data objects are stored in a file system on at least a first storage device wherein the metadata comprises aging information that identifies when data objects are to be migrated;

storing the metadata in a first metabase wherein the first metabase is stored separately from the file system;

migrating a plurality of the data objects from the first storage device to a second storage device based on a first policy, wherein the first policy determines which data objects to migrate based on aging information in the metadata and further migrating at least portion of the associated metadata in the first metabase to a second metabase, wherein the second metabase is associated with the second storage device and wherein the second metabase is stored separately from the first metabase and separately from the file system;

removing from the first storage device the plurality of data objects migrated to the secondary storage device;

deleting from the first metabase, metadata about the plurality of data objects migrated to the secondary storage device based on a second storage policy that identifies when to delete the metadata from the first metabase, based on diminished storage capacity associated with the first metabase; and

obtaining metadata about the plurality of data objects migrated to the secondary device from the second metabase without accessing the first metabase, or the data objects on the secondary storage device via the file system.

2. The method of claim 1, wherein the data object comprises a data file.

3. The method of claim 1, wherein the first metabase does not store the entire contents of the data objects.

4. The method of claim 1, wherein the first metabase is stored on a storage device different than the at least one primary storage device.

5. The method of claim 1, wherein the first metabase is configured to allow obtaining of information about the plurality of data objects without accessing a data structure used by a file system.

6. The method of claim 1, wherein the data interactions are controlled by at least a first processor and the first metabase is controlled by a second processor.

7. The method of claim 1, further comprising monitoring the data interactions with an application separate from applications associated with the data interactions.

8. The method of claim 1, additionally comprising compiling an index of data interactions.

9. The method of claim 1, further comprising moving data objects from the primary storage device to the secondary storage device based on a data migration policy stored in the first metabase.

10. A system for managing electronic data in one or more storage devices, the system comprising:

a first metabase that comprises metadata obtained from a plurality of data objects associated with one or more data interactions, wherein the data objects are stored in a file system on at least a first storage device and wherein the first metabase is stored separately from the file system, wherein the metadata comprises aging information that identifies when data objects are to be migrated;

a first processing module executing in one or more computer processors that is configured to migrate a plurality of the data objects from the first storage device to a second storage device based on a first policy, wherein the first policy determines which data objects to migrate based on aging information in the metabase and further migrate at least portion of the associated metadata in the first metabase to a second metabase,

wherein the second metabase is stored separately from the first metabase and separately from the file system and wherein references to the metadata about the data objects migrated to the secondary storage device are retained in the first metabase such that information about the data objects migrated to the secondary device can be located, and

wherein the first processing module is further configured to direct the removal from the first storage device, the plurality of data objects;

wherein the first processing module is further configured to direct the deletion from the first metabase, metadata about the plurality of data objects migrated to the secondary storage device based on a second storage policy that identifies when to delete the metadata from the first metabase, based on diminished storage capacity associated with the first metabase; and

a second processing module executing in one or more computer processors that is configured to obtain metadata about the data objects migrated to the secondary device from the second metabase without accessing the first metabase, or the data objects on the secondary storage device via the file system.

11. The system of claim 10, wherein the data object comprises a data file.

12. The system of claim 10, wherein the first metabase does not store the entire contents of the data objects.

13. The system of claim 10, wherein the first metabase is stored on a storage device different than the at least one primary storage device.

14. The system of claim 10, wherein the first metabase is configured to allow obtaining of information about the plurality of data objects without accessing a data structure used by a file system.

15. The system of claim 10, further comprising a monitoring module configured to monitor the data interactions with an application separate from applications associated with the data interactions.

16. The system of claim 10 further comprising an index of data interactions.

17. The system of claim 10, wherein data objects are moved from the primary storage device to the secondary storage device based on a data migration policy stored in the first metabase.