Systems and methods for using metadata to enhance data identification operations

• US 9,098,542 B2
• Filed: 05/07/2014
• Issued: 08/04/2015
• Est. Priority Date: 11/28/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Prahlad, Anand, Schwartz, Jeremy Alan, Ngo, David, Brockway, Brian, Muller, Marcus S.
• Primary Examiner(s): MOSER, BRUCE M
• Application Number:
  US14/271,756
• Publication Number:
  US 20140358935A1
• Time in Patent Office: 454 Days
• Field of Search: 707/999.003, 707/999.006, 707/999.102, 707/999.107, 707/667, 707/713, 707/718, 707/739, 707/741, 707/755, 707/758, 707/770, 707/771, 707/776
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1466 to make the backup process non-disruptive
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/285 : Clustering or classification
• CPC Class Codes: G06F 2201/80 : Database-specific techniques
• CPC Class Codes: G06F 3/0604 : Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 : by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0649 : Lifecycle management
• CPC Class Codes: G06F 3/0685 : Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: Y10S 707/99942 : Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99948 : Application of database or data structure, e.g. distributed, multimedia, or image
• CPC Class Codes: Y10S 707/99955 : Archiving or backup

Abstract

Systems and methods for managing electronic data are disclosed. Various data management operations can be performed based on a metabase formed from metadata. Such metadata can be identified from an index of data interactions generated by a journaling module, and obtained from their associated data objects stored in one or more storage devices. In various embodiments, such processing of the index and storing of the metadata can facilitate, for example, enhanced data management operations, enhanced data identification operations, enhanced storage operations, data classification for organizing and storing the metadata, cataloging of metadata for the stored metadata, and/or user interfaces for managing data. In various embodiments, the metabase can be configured in different ways. For example, the metabase can be stored separately from the data objects so as to allow obtaining of information about the data objects without accessing the data objects or a data structure used by a file system.

Claims

1. A method for performing data identification operations involving electronic data, the method comprising:

receiving an index generated by a journal application, the index identifying data interactions made by at least one other application to data objects stored on one or more primary storage devices;

updating, based on the index, first metadata stored in a first metabase, wherein the updating comprises accessing the data objects to obtain the first metadata about the data objects;

receiving a query comprising selection criteria associated with moving data from the one or more primary storage devices to one or more secondary storage devices;

identifying with the first metabase, one or more of the data objects which meet the selection criteria without accessing the data objects stored on the primary storage device;

copying the one or more of the data objects identified by the first metabase to the one or more secondary storage devices; and

storing second metadata in a second metabase, the second metadata comprising information about the one or more data objects copied to the one or more secondary storage devices.

2. The method of claim 1 wherein the first and second metabases are stored separately from the data objects.

3. The method of claim 1 further comprising deleting the first metadata from the first metabase associated with the one or more data objects copied to the secondary storage.

4. The method of claim 1 wherein updating the first metabase comprises:

determining whether a selected entry in the index of data interactions has an existing record in the first metabase;

if no record exists corresponding to the selected entry, creating a new record in the first metabase; and

updating the existing record or the new record with at least a part of the information obtained from the selected entry.

5. The method of claim 1 wherein the selection criteria comprises a permission level.

6. The method of claim 1 wherein the first metadata further comprises at least one of:

a last modified time, a last accessed time, a data object size and an application type.

7. The method of claim 1 wherein the first metadata is associated with email and the index of data interactions is generated by a journaling program.

8. The method of claim 7 wherein the first metadata comprises at least one of the group consisting of:

to/from information for email, creation date, last accessed time, and application type.

9. The method of claim 1 wherein the one or more data objects copied to secondary storage are deleted from the first metabase based on the second metadata in the second metabase.

10. The method of claim 1 wherein the one or more storage devices associated with secondary memory are selected to store the data objects transferred to secondary storage based on the first metabase in the first metabase.

11. A system for performing data identification operations involving electronic data, the system comprising:

a computer processor;

a monitor agent comprising at least computer hardware, the monitor agent receives data generated by a journal application, the index identifying data interactions made by at least one other application to data objects stored on a primary storage device;

a first metabase, wherein the monitor agent updates, based on the index, first metadata stored in the first metabase, wherein the updates comprise accessing the data objects top obtain the first metadata about the data objects; and

a storage manager comprising at least computer hardware, the storage manager receives selection criteria associated with moving data from the one or more primary storage devices to one or more secondary storage devices,

wherein the storage manager identifies with the first metabase, one or more data objects which meet the selection criteria without accessing the data objects stored on the one or more primary storage devices;

wherein the storage manager directs the copying of the one or more data objects identified by the first metabase to the one or more secondary storage devices;

wherein the storage manager directs storage of second metadata in a second metabase, the second metadata comprising information about the one or more data objects copied to the one or more secondary storage devices.

12. The system of claim 11 wherein the first and second metabases are stored separately from the data objects.

13. The system of claim 11 wherein the storage manager directs deletion of the first metadata from the first metabase associated with the one or more data objects copied to the secondary storage.

14. The system of claim 11 wherein the monitor agents updates the first metadata by:

determining whether a selected entry in the index of data interactions has an existing record in the first metabase;

if no record exists corresponding to the selected entry, creating a new record in the first metabase; and

updating the existing record or the new record with at least a part of the information obtained from the selected entry.

15. The system of claim 11 wherein the selection criteria comprises a permission level.

16. The system of claim 11 wherein the first metadata further comprises at least one of:

a last modified time, a last accessed time, a data object size and an application type.

17. The system of claim 11 wherein the first metadata is associated with email and the index of data interactions is generated by a journaling program.

18. The system of claim 17 wherein the first metadata comprises at least one of the group consisting of:

to/from information for email, creation date, last accessed time, and application type.

19. The system of claim 11 wherein the one or more data objects copied to secondary storage are deleted from the first metabase based on the second metadata in the second metabase.

20. The system of claim 11 wherein the one or more storage devices associated with secondary memory are selected to store the data objects transferred to secondary storage based on the first metabase in the first metabase.