Data classification systems and methods for organizing a metabase

• US 20070198612A1
• Filed: 11/28/2006
• Published: 08/23/2007
• Est. Priority Date: 11/28/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Brockway, Brian, Ngo, David, Prahlad, Anand, Muller, Marcus, Schwartz, Jeremy
• Granted Patent: US 7,849,059 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1466 to make the backup process non-disruptive
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/285 : Clustering or classification
• CPC Class Codes: G06F 2201/80 : Database-specific techniques
• CPC Class Codes: G06F 3/0604 : Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 : by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0649 : Lifecycle management
• CPC Class Codes: G06F 3/0685 : Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: Y10S 707/99942 : Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99948 : Application of database or data structure, e.g. distributed, multimedia, or image
• CPC Class Codes: Y10S 707/99955 : Archiving or backup

Abstract

Systems and methods for managing electronic data are disclosed. Various data management operations can be performed based on a metabase formed from metadata. Such metadata can be identified from an index of data interactions generated by a journaling module, and obtained from their associated data objects stored in one or more storage devices. In various embodiments, such processing of the index and storing of the metadata can facilitate, for example, enhanced data management operations, enhanced data identification operations, enhanced storage operations, data classification for organizing and storing the metadata, cataloging of metadata for the stored metadata, and/or user interfaces for managing data. In various embodiments, the metabase can be configured in different ways. For example, the metabase can be stored separately from the data objects so as to allow obtaining of information about the data objects without accessing the data objects or a data structure used by a file system.

Claims

1. A method of classifying data in a storage system, the method comprising:

processing an index of data interactions between at least one application and one or more data objects on at least one storage device, the index being populated by a journaling module;

selecting an entry from the index corresponding to a first data interaction involving a first data object;

obtaining from the selected entry information indicative of metadata associated with the first data object; and

updating a metabase with the information obtained from the selected entry, the metabase having a plurality of records storing metadata associated with the data objects.

2. The method of claim 1, wherein the selected entry information comprises information indicative of modifications to metadata associated with the first data object.

3. The method of claim 1, wherein the metadata in the metabase is stored separately from the entire contents of the data objects.

4. The method of claim 1, further comprising accessing one or more of the metadata associated with the corresponding data objects again one or more times to update the metabase.

5. The method of claim 1, wherein said updating comprises:

determining whether the selected entry in the index of data interactions has an existing record in the metabase;

if no record exists corresponding to the selected entry, creating a new record in the metabase; and

updating the existing record or the new record with at least a part of the information obtained from the selected entry.

6. The method of claim 1, wherein said selecting comprises determining whether the entry is a new entry in the index of data interactions.

7. The method of claim 6, wherein the entry is considered to be new if a time stamp of the entry is later than a time at which a previous entry was analyzed.

8. The method of claim 6, wherein the entry is considered to be new based on an identifier of the entry.

9. The method of claim 8, wherein the identifier comprises an update sequence number that identifies the entry in the index of data interactions.

10. The method of claim 1, further comprising initially populating the metabase by accessing the data objects so as to access available metadata associated with the data objects.

11. The method of claim 10, additionally comprising:

quiescing the data interactions associated with the at least one storage device; and

performing said populating during said quiescing.

12. The method of claim 11, wherein said populating is performed during operation of the at least one storage device.

13. The method of claim 12, additionally comprising queuing the data interactions generated during said populating to allow capture of the data interactions during the accessing process.

14. The method of claim 1, additionally comprising receiving input regarding a user-defined tag, wherein said obtaining information is based at least in part on said user-defined tag.

15. The method of claim 1, wherein said updating further comprises determining which of a plurality of metabases comprises records storing metadata associated with the first data object.

16. A system for managing electronic data in a storage network, the system comprising:

a data classification module configured to;

scan entries in a journal file of data interactions between at least one application and one or more data objects stored on at least one storage device;

select an entry from the index corresponding to a first data interaction involving a first data object;

classify one or more properties of the data object based on information obtained from the selected entry; and

update the one or more classified properties of the data object in a metabase.

17. The system of claim 16, wherein the journal file is populated by a monitoring module.

18. The system of claim 16, wherein the data classification module is further configured to access the one or more data objects one or more times to update the metabase.

19. The system of claim 16, wherein the properties of the data objects are stored in the metabase separately from entire content of the data objects.

20. The system of claim 16, wherein the information obtained from the selected entry is indicative of modifications to metadata of the first data object resulting from the first data interaction.

21. The system of claim 20, wherein the metadata comprises at least one of:

a data owner, a last modified time, a last accessed time, a data object size and an application type.

22. The system of claim 16, wherein the data classification module is further configured to classify the one or more properties of the data object based on user-defined tags.

23. The system of claim 16, wherein the data classification module is further configured to periodically scan the entries in the index.

24. The system of claim 23, wherein the data classification module is further configured to allow analysis of the one or more properties of the data objects based on a selected criteria without accessing the data objects.

25. A system for managing electronic data, the system comprising:

means for storing a list of data object interactions between at least one application and one or more data objects;

means for storing metadata associated with the one or more data objects; and

means for identifying from the list of data object interactions a first data object interaction involving metadata modifications to a first data object, said identifying means being further configured to update a record in said means for storing metadata associated with the metadata of the first data object.

26. The system of claim 25, wherein said means for storing is generated by a journaling application.

27. The system of claim 25, wherein said means for storing metadata comprises means for storing the metadata separate from the entire content of each of the data objects.

28. The system of claim 25, wherein said means for identifying is further configured to initially populate said means for storing metadata by accessing the data objects so as to access available metadata associated with the data objects.

29. The system of claim 25, wherein said means for identifying is configured to process the list of data object interactions by accessing an identifier of each entry in the list.