METABASE FOR FACILITATING DATA CLASSIFICATION

• US 20070179995A1
• Filed: 11/28/2006
• Published: 08/02/2007
• Est. Priority Date: 11/28/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Brockway, Brian, Ngo, David, Prahlad, Anand, Muller, Marcus, Schwartz, Jeremy
• Granted Patent: US 7,747,579 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1466 to make the backup process non-disruptive
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/285 : Clustering or classification
• CPC Class Codes: G06F 2201/80 : Database-specific techniques
• CPC Class Codes: G06F 3/0604 : Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 : by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0649 : Lifecycle management
• CPC Class Codes: G06F 3/0685 : Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: Y10S 707/99942 : Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99948 : Application of database or data structure, e.g. distributed, multimedia, or image
• CPC Class Codes: Y10S 707/99955 : Archiving or backup

Abstract

Systems and methods for managing electronic data are disclosed. Various data management operations can be performed based on a metabase formed from metadata. Such metadata can be identified from an index of data interactions generated by a journaling module, and obtained from their associated data objects stored in one or more storage devices. In various embodiments, such processing of the index and storing of the metadata can facilitate, for example, enhanced data management operations, enhanced data identification operations, enhanced storage operations, data classification for organizing and storing the metadata, cataloging of metadata for the stored metadata, and/or user interfaces for managing data. In various embodiments, the metabase can be configured in different ways. For example, the metabase can be stored separately from the data objects so as to allow obtaining of information about the data objects without accessing the data objects or a data structure used by a file system.

Claims

1. A method for managing electronic data in one or more storage devices, the method comprising:

receiving from a journaling module an index of data interactions made by one or more applications to one or more data objects on at least one storage device;

processing the index to identify at least one data interaction with at least one of the data objects;

obtaining the metadata associated with the at least one data interaction; and

storing the metadata associated with the at least one data interaction in a metabase.

2. The method of claim 1, wherein the data object comprises a data file.

3. The method of claim 1, wherein the metabase does not store the entire contents of the data objects.

4. The method of claim 1, wherein the metabase is stored separately from the data objects.

5. The method of claim 1, wherein the metabase is stored on a storage device different than the at least one storage device.

6. The method of claim 1, wherein the metabase is configured to allow obtaining of information about the plurality of data objects without accessing a data structure used by a file system.

7. The method of claim 1, wherein the data interactions are controlled by at least a first processor and the metabase is controlled by a second processor.

8. The method of claim 1, further comprising accessing the metadata one or more times to update the metabase.

9. The method of claim 1, wherein the journaling module comprises an application separate from the applications associated with the data interactions.

10. The method of claim 1, additionally comprising compiling the index of data interactions.

11. The method of claim 1, wherein said storing comprises updating the metadata stored in the metabase.

12. The method of claim 1, additionally comprising:

receiving a selection criteria; and

accessing the metabase to identify which of the plurality of data objects satisfies the selection criteria without accessing the plurality of data objects on the at least one storage device.

13. The method of claim 12, wherein the selection criteria identifies data objects subject to a data migration policy.

14. The method of claim 1, additionally comprising initially populating the metabase by traversing the plurality of data objects to access available metadata associated with the plurality of data objects.

15. A system for managing electronic data in a network environment, the system comprising:

at least one storage device configured to store one or more data objects;

a journal file populated by a monitoring module, the journal file comprising a list of data interactions between at least one application and at least of one of the data objects;

a classification module configured to process the journal file to identify at least one data interaction with at least one of the data objects, the classification module being further configured to obtain the metadata associated with the data object; and

a metabase comprising a plurality of records, at least one record storing the metadata associated with the data object.

16. The system of claim 15, wherein the at least one data object interaction comprises a modification of the at least one data object.

17. The system of claim 15, wherein the classification module is further configured to update the plurality of records in the metabase based at least in part on the metadata obtained from the data object.

18. The system of claim 15, wherein for each listed data interaction in the journal file, the journal file comprises at least one of an update sequence number and a change journal reason code.

19. The system of claim 15, wherein the classification module is further configured to initially populate the metabase by traversing the at least one storage device to access available metadata associated with the data objects.

20. The system of claim 15, further comprising a first processor and a second processor, wherein operations associated with the monitoring module are substantially controlled by the first processor and operations associated with the metabase are substantially controlled by the second processor.

21. The system of claim 15, wherein the journal file is associated with a first computing device and the metabase is associated with a second computing device.

22. The system of claim 15, wherein the metabase comprises a centralized metabase in communication with a plurality of computing devices.

23. The system of claim 22, wherein each of the plurality of computing devices has its own journal file.

24. The system of claim 23, wherein at least some of the plurality of records includes an identifier that indicates which computing device, and where on that computing device, the metadata is located to allow comparison with a selection criteria without accessing the data objects.

25. The system of claim 15, wherein the classification module is located outside a data flow path between the at least one application and the data objects on the at least one storage device.

26. The system of claim 15, wherein the at least one storage device comprises a first storage device and a second storage device.

27. The system of claim 26, further comprising a management module configured to access the metabase to determine which of the plurality of data objects on the first storage device is subject to a data transfer policy.

28. The system of claim 27, wherein the management module performs the determination without accessing the plurality of data objects on the first storage device.

29. The system of claim 27, wherein the first storage device comprises a primary storage device and the second storage device comprises a secondary storage device.

30. The system of claim 27, wherein the management module is configured to copy a first data object of the plurality of data objects on the first storage device to the second storage device.

31. The system of claim 30, wherein the metabase comprises first and second metabases, the first metabase having records storing metadata associated with the data objects on the first storage device and the second metabase having records storing metadata associated with the data objects on the second storage device.

32. The system of claim 31, wherein the metabase is updated according to locations of the data objects.

33. A system for managing electronic data in a network environment, the system comprising:

means for receiving from a journaling module an index of data interactions between at least one application and one or more data objects;

means for processing the index to identify at least one data interaction with at least one data object;

means for obtaining metadata associated with the at least one data object; and

means for storing the metadata, wherein the metadata in said means for storing is separate from the at least one data object.

34. The system of claim 33, wherein said means for processing is located outside a data flow path between the at least one application and the plurality of data objects.

35. The system of claim 34, wherein the journaling module is located in said data flow path.