Systems and methods for using metadata to enhance data identification operations

• US 7,801,864 B2
• Filed: 11/28/2006
• Issued: 09/21/2010
• Est. Priority Date: 11/28/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Brockway, Brian, Ngo, David, Schwartz, Jeremy Alan, Prahlad, Anand, Muller, Marcus S.
• Primary Examiner(s): Trujillo; James
• Assistant Examiner(s): Moser; Bruce M
• Application Number:
  US11/564,174
• Publication Number:
  US 20070192360A1
• Time in Patent Office: 1,393 Days
• Field of Search: 707/104.1, 707/103.R, 707/202, 707/200, 707/999.103, 707/999.104, 707/999.2, 707/999.202, 707/667, 707/741, 707/812
• US Class Current: 707/667
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1466 to make the backup process non-disruptive
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/285 : Clustering or classification
• CPC Class Codes: G06F 2201/80 : Database-specific techniques
• CPC Class Codes: G06F 3/0604 : Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 : by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0649 : Lifecycle management
• CPC Class Codes: G06F 3/0685 : Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: Y10S 707/99942 : Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99948 : Application of database or data structure, e.g. distributed, multimedia, or image
• CPC Class Codes: Y10S 707/99955 : Archiving or backup

Abstract

Systems and methods for managing electronic data are disclosed. Various data management operations can be performed based on a metabase formed from metadata. Such metadata can be identified from an index of data interactions generated by a journaling module, and obtained from their associated data objects stored in one or more storage devices. In various embodiments, such processing of the index and storing of the metadata can facilitate, for example, enhanced data management operations, enhanced data identification operations, enhanced storage operations, data classification for organizing and storing the metadata, cataloging of metadata for the stored metadata, and/or user interfaces for managing data. In various embodiments, the metabase can be configured in different ways. For example, the metabase can be stored separately from the data objects so as to allow obtaining of information about the data objects without accessing the data objects or a data structure used by a file system.

Claims

1. A method for performing data identification operations involving electronic data, the method comprising:

receiving an index generated by a journal application, the index identifying data interactions made by at least one other application to one or more data objects on a storage device;

accessing with one or more computer processors the data interactions in the index to obtain first metadata associated with the one or more data objects, wherein the first metadata is generated by the at least one other application or a file system associated with the storage of the one or more data objects;

receiving second metadata associated with the one or more data objects, wherein the second metadata is not generated by the at least one other application or the file system;

updating a metabase, based on the index with the first and second metadata, wherein the metabase is in addition to the file system;

receiving selected criteria identifying one or more properties of at least one data object;

accessing the first and second metadata in the metabase according to the selected criteria without accessing the file system associated with the data objects; and

identifying, based on accessing either the first or second metadata in the metabase, the at least one data object that satisfies the selected criteria.

2. The method of claim 1, wherein the metadata in the metabase are stored separate from the entire content of each of the data objects.

3. The method of claim 1, wherein accessing of the metabase is performed without accessing the data objects on the storage device.

4. The method of claim 1, further comprising accessing one or more of the metadata associated with the corresponding data objects again one or more times to update the metabase.

5. The method of claim 1, additionally comprising receiving the selected criteria from a user.

6. The method of claim 5, additionally comprising providing a graphical user interface for prompting the selected criteria.

7. The method of claim 1, wherein the selected criteria comprises identification and tracking of the data interactions on a user or group basis.

8. The method of claim 7, wherein the data interactions comprise at least one of a read operation, a write operation and an access operation.

9. The method of claim 8, wherein the access operation comprises at least one of web-page access, electronic gaming interaction, chat-room access, instant messaging, communication interaction and multimedia interaction.

10. The method of claim 1, wherein the accessing comprises accessing a plurality of metabases.

11. The method of claim 10, wherein the at least one storage device comprises a plurality of storage devices and each of the plurality of metabases is associated with one of the plurality of storage devices.

12. The method of claim 10, additionally comprising accessing a metadata index to determine which of the plurality of metabases is likely to contain information relating to the selected criteria.

13. The method of claim 1, wherein the identifying comprises providing a storage device path of the at least one data object.

14. The method of claim 1, wherein the identifying comprises providing a file reference number of the at least one data object.

15. A system for performing data identification operations involving electronic data, the system comprising:

at least one storage device configured to store a plurality of data objects having first metadata and content associated therewith;

a journal file maintained by a monitoring module, the journal file identifying data operations directed to one or more of the plurality of data objects by at least one application;

one or more computer processors that obtain the first metadata associated with the one or more of the plurality of data objects identified in the journal file, wherein the first metadata is generated by the at least one application or a file system associated with the storage of the one or more of the plurality of data objects;

one or more of the computer processors configured to obtain second metadata associated with the one or more of the plurality of data objects identified in the journal file, wherein the second metadata is not generated by the at least one other application or the file system;

at least one metabase comprising a plurality of records storing the first and second metadata associated with one or more of the plurality of data objects, the plurality of records being updated based on the first and second metadata obtained by the one or more computer processors, wherein the metabase is in addition to the file system; and

one or more of the computer processors configured to receive a selected criteria and to access the metadata in the at least one metabase without accessing the file system associated with the data objects to identify based on the metadata in the metabase, at least one data object of the plurality of data objects that satisfies the selected criteria.

16. The system of claim 15, wherein the at least one metabase does not store the entire content of each of the plurality of data objects.

17. The system of claim 15, wherein the access of the at least one metabase is performed without accessing the plurality of data objects on the at least one storage device.

18. The system of claim 15, further comprising a classification module configured to update the plurality of records on the at least one metabase, the classification module being located outside a data flow path between the at least one application and the plurality of data objects.

19. The system of claim 15, wherein the processor comprises first and second processors, wherein the first processor is configured to access the at least one metabase and the second processor is configured handle the data operations associated with the plurality of data objects on the at least one storage device.

20. The system of claim 19, wherein the first processor is associated with a first computing device and the second processor is associated with a second computing device.

21. The system of claim 15, wherein the at least one storage device comprises multiple storage devices and the at least one metabase comprises multiple metabases, each of the multiple metabases being associated with one of the multiple storage devices.

22. The system of claim 21, wherein the multiple storage devices comprise a primary storage device having production data and a secondary storage device having one or more copies of at least some of the production data.

23. The system of claim 22, wherein the secondary storage device has at least some additional data other than the one or more copies of the at least some of the production data.

24. The system of claim 22, wherein the processor is further configured to access each of the multiple metabases.

25. A system for performing data identification operations involving electronic data, the system comprising:

means for storing a plurality of data objects;

means for identifying with one or more computer processors data operations directed to one or more of the data objects by at least one application, the means for identifying being generated by a journaling module;

means for accessing with one or more computer processors the data interactions to obtain first metadata associated with the one or more data objects, wherein the first metadata is generated by the at least one application or a file system associated with the storage of the one or more data objects;

means for receiving second metadata associated with the one or more data objects, wherein the second metadata is not generated by the at least one other application or the file system;

means for storing metadata associated with the data objects, wherein the means for storing metadata is updated with the first and second metadata based on selected ones of the data operations identified by said means for identifying and wherein the means for storing metadata is in addition to the file system;

means for receiving at least one selected criteria identifying one or more properties of at least one data object; and

means for accessing the first and second metadata in the means for storing metadata based on at least one selected criteria to identify, based on either the first or second metadata in the metabase, at least one data object corresponding to the selected criteria without accessing the file system associated with the data objects located on the means for storing the data objects.

26. The system of claim 25, wherein said selected ones of the data operations result in modification of the metadata of at least one of the data objects.

27. The system of claim 25, wherein said identifying of the at least one data object is performed without accessing the data objects on said first storing means.