Systems and methods for using metadata to enhance data identification operations

• US 8,131,725 B2
• Filed: 09/20/2010
• Issued: 03/06/2012
• Est. Priority Date: 11/28/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Prahlad, Anand, Schwartz, Jeremy Alan, Ngo, David, Brockway, Brian, Muller, Marcus S.
• Primary Examiner(s): Trujillo, James
• Assistant Examiner(s): MOSER, BRUCE M
• Application Number:
  US12/886,487
• Publication Number:
  US 20110078146A1
• Time in Patent Office: 533 Days
• Field of Search: 707/2, 707/102, 707/104.1, 707/999.002, 707/999.102, 707/999.107, 707/736, 707/739, 707/741, 707/803, 707/812, 707/825
• US Class Current: 707/739
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1466 to make the backup process non-disruptive
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/285 : Clustering or classification
• CPC Class Codes: G06F 2201/80 : Database-specific techniques
• CPC Class Codes: G06F 3/0604 : Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 : by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0649 : Lifecycle management
• CPC Class Codes: G06F 3/0685 : Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: Y10S 707/99942 : Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99948 : Application of database or data structure, e.g. distributed, multimedia, or image
• CPC Class Codes: Y10S 707/99955 : Archiving or backup

Abstract

Systems and methods for managing electronic data are disclosed. Various data management operations can be performed based on a metabase formed from metadata. Such metadata can be identified from an index of data interactions generated by a journaling module, and obtained from their associated data objects stored in one or more storage devices. In various embodiments, such processing of the index and storing of the metadata can facilitate, for example, enhanced data management operations, enhanced data identification operations, enhanced storage operations, data classification for organizing and storing the metadata, cataloging of metadata for the stored metadata, and/or user interfaces for managing data. In various embodiments, the metabase can be configured in different ways. For example, the metabase can be stored separately from the data objects so as to allow obtaining of information about the data objects without accessing the data objects or a data structure used by a file system.

Claims

1. A method of identifying data to store in a metabase, the method comprising:

displaying with one or more computer processors a user interface that allows a user to input a user-defined tag expression, wherein the user-defined tag expression comprises information associated with data interactions the user desires to track;

tagging entries in an index of data interactions that record modifications to data objects, wherein tagged entries are associated with data interactions that meet the user-defined tag expression, wherein the tagging associates a tag identifier with the tagged entries;

accessing a first data object associated with a first tagged entry, to obtain first metadata about the first data object that is not stored in the index of data interactions, wherein the first data object is stored separately from the index of data interactions;

obtaining second metadata from the index of data operations, wherein the second metadata comprises at least the tag identifier associated with the first tagged entry;

storing in a metabase the first metadata in association with the second metadata such that the metabase associates the tag identifier with the first metadata obtained from the first data object, wherein the metabase is stored separately from the file system containing the first data object; and

in response to a user request for information about data interactions associated with the user-defined tag expression, accessing the metabase to determine data interactions that meet the user-defined tag expression without accessing the plurality of data objects in the file system.

2. The method of claim 1, wherein the first and second metadata in the metabase is stored separately from the entire contents of the data objects.

3. The method of claim 1, further comprising accessing at least one of the first or second metadata associated with the data objects one or more times to update the metabase.

4. The method of claim 1, wherein said updating comprises:

determining whether a selected entry in the index of data interactions has an existing record in the metabase;

if no record exists corresponding to the selected entry, creating a new record in the metabase; and

updating the existing record or the new record with at least a part of the information obtained from the selected entry.

5. The method of claim 1, further comprising determining whether the tagged entry is a new entry in the index of data interactions.

6. The method of claim 5, wherein the tagged entry is considered to be new if a time stamp of the tagged entry is later than a time at which a previous tagged entry was analyzed.

7. The method of claim 5, wherein the tagged entry is considered to be new based on an identifier of the tagged entry.

8. The method of claim 7, wherein the identifier comprises an update sequence number that identifies the tagged entry in the index of data interactions.

9. The method of claim 1, further comprising populating the metabase by accessing the data objects so as to access available first metadata associated with the data objects.

10. A system for managing electronic data in a storage network, the system comprising:

one or more computer processors executing a user interface, wherein the user interface is configured to receive user input comprising a user-defined tag expression that comprises information associated with data interactions the user desires to track;

one or more computer processors executing a data classification module, wherein the data classification module is configured to;

tag entries in an index of data interactions, wherein tagged entries are associated with data interactions that meet the user-defined tag expression, wherein the data classification module associates a tag identifier with the entries;

access a first data object associated with the first tagged entry to obtain first metadata about the first data object that is not stored in the index of data interactions, wherein the first data object is stored separately from the index of data interactions;

obtain second metadata from the index of data operations, wherein the second metadata comprises at least the tag identifier associated with the first tagged entry;

store in a metabase the first metadata in association with the second metadata such that the metabase associates the tag identifier with the first metadata obtained from the first data object, wherein the metabase is stored separately from the file system containing the first data object;

in response to a user request for information about data interactions associated with the user-defined tag expression, the data classification module is configured to access the metabase to determine data interactions that meet the user-defined tag expression without accessing the plurality of data objects in the file system.

11. The system of claim 10, wherein the index is populated by a journal module.

12. The system of claim 10, wherein the data classification module is further configured to access the one or more data objects one or more times to update the metabase.

13. The system of claim 10, wherein the first and second metadata in the metabase is stored separately from the entire contents of the data objects.

14. The system of claim 10, wherein the data classification module is further configured to:

determine whether a selected entry in the index of data interactions has an existing record in the metabase;

if no record exists corresponding to the selected entry, create a new record in the metabase; and

update the existing record or the new record with at least a part of the information obtained from the selected entry.

15. The system of claim 10, wherein said the data classification module is further configured determine whether the tagged entry is a new entry in the index of data interactions.

16. The system of claim 15, wherein the tagged entry is considered to be new if a time stamp of the tagged entry is later than a time at which a previous entry was analyzed.

17. The system of claim 15, wherein the tagged entry is considered to be new based on an identifier of the tagged entry.

18. The system of claim 17, wherein the identifier comprises an update sequence number that identifies the tagged entry in the index of data interactions.

19. The system of claim 10, the data classification module is further configured to populate the metabase by accessing the data objects so as to access available first metadata associated with the data objects.