Systems and methods for probabilistic data classification

• US 9,740,764 B2
• Filed: 12/14/2015
• Issued: 08/22/2017
• Est. Priority Date: 01/30/2008
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Lunde, Norman R.
• Primary Examiner(s): WILSON, KIMBERLY LOVEL
• Application Number:
  US14/968,719
• Publication Number:
  US 20160171079A1
• Time in Patent Office: 617 Days
• Field of Search: 707738
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/90 Details of database functions independent of the retrieved data types
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

A system for performing data classification operations. In one embodiment, the system comprises a file system configured to store a plurality of computer files and a scanning agent configured to traverse the file system and compile data regarding the attributes and content of the plurality of computer files. The system also comprises an index configured to store the data regarding attributes and content of the plurality of computer files and a file classifier configured to analyze the data regarding the attributes and content of the plurality of computer files and to classify the plurality of computer files into one or more categories based on the data regarding the attributes and content of the plurality of computer files. Results of the file classification operations can be used to set appropriate security permissions on files which include sensitive information or to control the way that a file is backed up or the schedule according to which it is archived.

Claims

1. A computer system comprising:

a file system configured to store electronic files;

a plurality of file system scanning agents configured to access the electronic files, the file system scanning agents comprising computer hardware with one or more processors and configured to compile, based on the electronic files, index data usable for classifying the electronic files, and transmit the index data over a network to be stored in one or more indexes stored separately from the file system; and

a file classification server configured to, without directly accessing the content of the electronic files, access the index data previously compiled by the plurality of file system scanning agents from the one or more indexes stored separately from the file system, and classify the electronic files based on the index data, wherein classifying the electronic files comprises assigning one or more labels to the electronic files based at least in part on a set of user-defined rules and the index data previously compiled by the plurality of file system scanning agents, and wherein the file classification server is further configured to determine a probability that one or more of the electronic files should be classified as members of a category, determine that the probability is within a threshold amount from a probability threshold for classifying the one or more of the electronic files as the members of the category, and mark the one or more of the electronic files as being questionable members of the category.

2. The system of claim 1, wherein the file system comprises one or more data storage devices coupled to a plurality of client computers via a Storage Area Network (SAN), a Network Attached Storage (NAS) unit, or some combination of the two.

3. The system of claim 1, wherein the index data comprises data indicating file size, name, path, type, or date of creation or modification of the electronic files.

4. The system of claim 1, wherein the index data comprises data indicating at least one classification category that the one or more of the electronic files have been identified as being the members of.

5. The system of claim 4, wherein the system is configured to alter security access restrictions of the one or more of the electronic files based upon the at least one classification category.

6. The system of claim 4, wherein the system is configured to alter a data backup schedule of the one or more of the electronic files based upon the at least one classification category.

7. The system of claim 4, wherein the system is configured to alter a data migration plan of the one or more of the electronic files based upon the at least one classification category.

8. The system of claim 1, wherein the one or more indexes include a file content index, the plurality of file system scanning agents configured to store, for each electronic file, a list of keywords in the electronic file and a frequency count for each keyword in the file content index.

9. The system of claim 1, wherein the content comprises a keyword present in an electronic file.

10. A method comprising:

with a plurality of file system scanning agents,

accessing electronic files stored in a file system;

compiling index data usable for classifying the electronic files; and

transmitting the index data over a network to be stored in one or more indexes stored separately from the file system; and

with a file classification server separate from the plurality of file system scanning agents,

classifying the electronic files without directly accessing the content of the electronic files by assigning one or more labels to the electronic files based at least in part on a set of user-defined rules and the index data previously compiled by the plurality of file system scanning agents;

determining a probability that one or more of the electronic files should be classified as members of a category;

determining that the probability is within a threshold amount from a probability threshold for classifying the one or more of the electronic files as the members of the category; and

marking the one or more of the electronic files as being questionable members of the category.

11. The method of claim 10, wherein the index data comprises data indicating file size, name, path, type, or date of creation or modification of the electronic files.

12. The method of claim 10, wherein the index data comprises data indicating at least one classification category that the one or more of the electronic files have been identified as being the members of.

13. The system of claim 12, wherein the system is configured to alter security access restrictions of the one or more of the electronic files based upon the at least one classification category.

14. The system of claim 12, wherein the system is configured to alter a data backup schedule of the one or more of the electronic files based upon the at least one classification category.

15. The system of claim 12, wherein the system is configured to alter a data migration plan of the one or more of the electronic files based upon the at least one classification category.

16. The method of claim 10, wherein the one or more indexes include a file content index, the method further comprising, with the plurality of file system scanning agents, storing, for each electronic file, a list of keywords in the electronic file and a frequency count for each keyword in the file content index.

17. A computer system comprising:

means for accessing electronic files stored in a file system;

means for compiling index data usable for classifying the electronic files;

means for transmitting the index data over a network to be stored in one or more indexes stored separately from the file system;

means for classifying the electronic files without directly accessing the content of the electronic files by assigning one or more labels to the electronic files based at least in part on a set of user-defined rules and the index data previously compiled and transmitted to the one or more indexes;

means for determining a probability that one or more of the electronic files should be classified as members of a category;

means for determining that the probability is within a threshold amount from a probability threshold for classifying the one or more of the electronic files as the members of the category; and

means for marking the one or more of the electronic files as being questionable members of the category.

18. The system of claim 17, wherein the means for classifying comprises a Naï

ve Bayes classifier.