SYSTEMS AND METHODS FOR CLASSIFYING AND TRANSFERRING INFORMATION IN A STORAGE NETWORK

• US 20070185925A1
• Filed: 11/28/2006
• Published: 08/09/2007
• Est. Priority Date: 11/28/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Brockway, Brian, Ngo, David, Prahlad, Anand, Muller, Marcus, Schwartz, Jeremy
• Granted Patent: US 7,668,884 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1466 to make the backup process non-disruptive
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/285 : Clustering or classification
• CPC Class Codes: G06F 2201/80 : Database-specific techniques
• CPC Class Codes: G06F 3/0604 : Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 : by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0649 : Lifecycle management
• CPC Class Codes: G06F 3/0685 : Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: Y10S 707/99942 : Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99948 : Application of database or data structure, e.g. distributed, multimedia, or image
• CPC Class Codes: Y10S 707/99955 : Archiving or backup

Abstract

Systems and methods for data classification to facilitate and improve data management within an enterprise are described. The disclosed systems and methods evaluate and define data management operations based on data characteristics rather than data location, among other things. Also provided are methods for generating a data structure of metadata that describes system data and storage operations. This data structure may be consulted to determine changes in system data rather than scanning the data files themselves.

Claims

1. A method of classifying data generated within a computer system, wherein the computer system includes a data storage device, the method comprising:

intercepting the generated data, wherein the intercepted data is to be stored at the data storage device;

generating a log entry within a log, wherein the log contains entries associated with the intercepted data;

analyzing entries within the log to identify entries that satisfy at least one data management criterion;

storing the intercepted data generated within the computer system on the data storage device; and

adding metadata to a metabase, wherein the metadata includes information that describes the intercepted data associated with the identified log entries and refers to the data stored on the data storage device.

2. The method of claim 1 further comprising accessing the data storage device to retrieve additional information related to the intercepted data, and adding the retrieved additional information to the metabase.

3. The method of claim 1 wherein intercepting comprises trapping the generated data and wherein the data is generated by an application program.

4. The method of claim 1 wherein analyzing entries within the log comprises parsing the log entries into a predetermined format that includes a file reference number.

5. The method of claim 1 wherein the storage criterion identifies a particular user and wherein analyzing entries identifies entries related to intercepted data that has been accessed by the identified user and classifies the intercepted data by characteristics associated with the user.

6. The method of claim 1 wherein the storage criterion identifies a department within an organization and wherein analyzing entries identifies entries related to data that has been accessed by a member of the identified department and classifies the data based on the identified department.

7. The method of claim 1 wherein the storage criterion identifies a pattern of access and wherein analyzing entries identifies entries related to data that has been accessed in accordance with the identified pattern and classifies the data based on the identified pattern.

8. The method of claim 1 wherein adding metadata comprises associating one or more predefined classifications with the data.

9. The method of claim 1 wherein adding metadata comprises associating one or more user-defined classifications with the data.

10. The method of claim 1 wherein adding metadata comprises associating one or more classifications with the data based on the similarity of the data to previously classified data and storing the metadata based on the classifications.

11. A system for classifying data stored within a computer system, wherein the computer system includes an application program configured to generate program data to be employed by the computer system, the system comprising:

a monitor agent configured to process at least part of the program data to create at least one log entry related to the program data;

a data collection agent configured to analyze the at least one log entry and select at least part of the program data associated with the at least one log entry;

a data classification agent configured to identify characteristics of and assign classifications to the selected program data, wherein a classification associates the selected program data with other program data having similar characteristics; and

,

a metabase configured to store metadata information about the selected program data, wherein the metadata contains at least the assigned classifications of the selected program data, and wherein the metabase is configured to receive requests from the computer system to perform data management operations based on the assigned classifications.

12. The system of claim 11 wherein the monitor agent processes program data by capturing a request to store the program data.

13. The system of claim 11 wherein the monitor agent processes program data by traversing a file system containing the program data.

14. The system of claim 11 wherein the monitor agent is further configured to identify changes in the program data.

15. The system of claim 11 wherein the data classification agent identifies characteristics of the selected program data that are not stored with the selected program data.

16. The system of claim 11 wherein the data classification agent identifies characteristics including an aging criteria.

17. The system of claim 11 wherein the data classification agent identifies characteristics including a state of protection that identifies how recently an archive copy of the data has been made.

18. The system of claim 11 wherein the data classification agent identifies characteristics including a pattern of use.

19. The system of claim 11 wherein the data collection agent captures additional information about the selected program data as metadata describing the selected program data and wherein the metadata includes at least one of the following types of information:

an owner, a last modified time, a size, an application that generated the selected program data, a user that generated the selected program data, header information, a creation date, a file type, a last accessed time, an application type, a location, a frequency of change, a business unit, usage trends associated with the selected program data, or aging information.

20. The system of claim 11 wherein the data collection agent captures additional information about the selected program data if changes are detected in the program data.

21. The system of claim 11 wherein the monitor agent is further configured to create a log entry when at least part of the program data changes, and the monitor agent is a filter driver.

22. The system of claim 11 wherein the monitor agent is further configured to create a log entry when at least part of the program data changes, and the monitor agent is Microsoft Change Journal.

23. The system of claim 11 wherein the monitor agent is further configured to create a log entry when at least part of the program data changes, and the monitor agent is a separate update agent.

24. The system of claim 11 wherein the monitor agent is further configured to create a log entry when at least part of the program data changes.

25. The system of claim 11 further comprising a change journal database configured to store log entries created by the monitor agent.

26. The system of claim 11 wherein the metabase is further configured to perform the data management operations without traversing a data store where the program data is stored.

27. A data storage system, comprising:

a processor for executing instructions;

an input/output component coupled to the processor, wherein the input/output component receives a set of data to be stored; and

,

a data storage device coupled to the processor,

wherein the processor is configured to;

analyze a data object associated with the set of data,

determine one or more categorizations for the data object, wherein the categorizations identify one or more characteristics of the data object that are not stored with the data object itself,

facilitate storage of the set of data in the data storage device, and

perform a storage operation associated with the set of data with respect to the data storage device, based on a rule determined at least in part from the one or more categorizations for the data object, and not based solely on the location where the block of data is to be stored.

28. The system of claim 27 wherein the rule is predetermined.

29. The system of claim 27 wherein the rule is calculated based on the one or more categorizations.

30. A system for classifying data stored within a computing device, the system comprising:

a means for storing data;

a means for collecting stored data comprising a data collection agent configured to identify data objects within the data store and obtain information about identified data objects;

a means for classifying data comprising a data classification agent for assigning classifications to identified data objects, wherein the classifications are related to one or more applications that may use the data object; and

,

a means for storing metadata comprising a metabase configured to store information about identified data objects and their classifications.

31. The system of claim 30 wherein the means for collecting stored data further comprises a means for journaling changes.

32. The system of claim 30 further comprising a means for administering the configuration of data movement to and from the means for storing data.

33. A computer-readable medium containing instructions for classifying data stored in a data storage system, by a method comprising:

receiving a file to be stored in a data storage device;

analyzing the file to determine characteristics associated with the file;

determining one or more of multiple categorizations for the file based on the determined characteristics, wherein the categorizations identify trends associated with the file that are related to trends identified about other files; and

,

facilitating a storage operation on the file with respect to the data storage device, based at least in part on the one or more categorizations for the file.

34. The computer-readable medium of claim 33 wherein the system is part of a network and the processor is coupled to the data storage device through the network.

35. The computer-readable medium of claim 33 wherein the input/output device is a port for receiving the file from a network.

36. The computer-readable medium of claim 33 wherein the processor executes an agent specialized to manipulate data related to an application that is associated with the file.