System and method for data mining and security policy management

• US 8,635,706 B2
• Filed: 03/16/2012
• Issued: 01/21/2014
• Est. Priority Date: 07/10/2008
• Status: Active Grant

• Current Assignee: McAfee, LLC
• Original Assignee: McAfee, Inc. (McAfee, LLC)
• Inventors: Liu, Weimin
• Primary Examiner(s): LEMMA, SAMSON B
• Application Number:
  US13/422,791
• Publication Number:
  US 20120180137A1
• Time in Patent Office: 676 Days
• Field of Search: 726/1, 726/26, 713/165, 713/167, 713/176, 713/180, 713/193, 707/741, 707/E17.002, 707/E17.014
• US Class Current: 726/26
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

A system and method to generate and maintain controlled growth DAG are described. The controlled growth DAG conveys information about objects captured by a capture system.

Claims

1. A method, comprising:

capturing, by an electronic computing device, one or more registered documents;

classifying the one or more registered documents with one or more classifications using one or more of the six mechanisms;

(1) content signature, (2) grammar analysis, (3) statistical analysis, (4) file classification, (5) document biometrics, and (6) concept maps;

capturing, by electronic computing device, an object traversing a network environment;

classifying the object with one or more classifications using the one or more of the six mechanisms;

generating a plurality of signatures of the object;

comparing the plurality of signatures with previously stored signatures in a signatures database, wherein parts of the signatures database is excluded from said comparing step based on the one or more classifications of the object and/or the one or more classifications of the registered documents; and

identifying that at least a portion of the plurality of signatures match a portion of the previously stored signatures;

prohibiting delivery of the object;

identifying one of the registered documents associated with the portion of the previously stored signatures; and

sending an alert to respective owner of the registered documents about the object.

2. The method of claim 1, wherein the portion of the plurality of signatures corresponds to a first content in the object, wherein the portion of the previously stored signatures corresponds to a second content in the registered documents, and wherein if the first content is the same as the second content, then a match is declared between the portion of the plurality of signatures and the portion of the previously stored signatures.

3. The method of claim 1, wherein each of the previously stored signatures includes hashes over at least one selection from a group consisting of embedded content, indices, headers, footers, formatting information, and font utilization of the associated registered document.

4. The method of claim 1, wherein the alert includes a request for permission from the respective owner to deliver the object to its intended destination.

5. The method of claim 1, wherein the generating the plurality of signatures, comprises:

extracting and decoding a content in the object;

normalizing a text included in the content, wherein the normalizing comprises;

comparing the text to a plurality of patterns according to a pattern priority that is generated from a hierarchy of restrictiveness, with a more restrictive pattern having a higher priority than a less restrictive pattern; and

removing delimiters in the text according to matching patterns in the plurality of patterns to create a plurality of delimited items;

tokenizing the plurality of delimited items into a list of tokens; and

calculating hashes of the tokens.

6. The method of claim 5, wherein the tokenizing includes converting the delimited items into corresponding tokens of fixed size according to a hash function.

7. The method of claim 5, wherein the calculating hashes of the tokens comprises:

selecting M tokens from the list of tokens, wherein M is a positive integer;

selecting N tokens from the M tokens, wherein N is a positive integer that is less than or equal to M;

calculating a hash of the N tokens;

skipping P tokens of the list of tokens, wherein P is a non-zero integer that can be less than M, equal to M, or greater than M;

determining whether a number of remaining tokens to be hashed in the list of tokens is less than M;

if the number of remaining tokens is less than M, stopping the calculating; and

if the number of remaining tokens is greater than or equal to M, continuing the calculating.

8. The method of claim 1, further comprising:

storing a plurality of objects in a storage medium;

creating a keyword index and a metadata index having respective entries corresponding to a content in the plurality of objects; and

querying at least one of the keyword index and the metadata index to find a specific object.

9. A non-transitory machine-readable storage medium including program code which, when executed by a processor, causes the processor to perform a method, the method comprising:

capturing, by an electronic computing device, one or more registered documents;

classifying the one or more registered documents with one or more classifications using one or more of the six mechanisms;

(1) content signature, (2) grammar analysis, (3) statistical analysis, (4) file classification, (5) document biometrics, and (6) concept maps;

capturing, by the electronic computing device, an object traversing a network environment;

classifying the object with one or more classifications using the one or more of the six mechanisms;

generating a plurality of signatures of the object;

comparing the plurality of signatures with previously stored signatures in a signatures database, wherein parts of the signatures database is excluded from said comparing step based on the one or more classifications of the object and/or the one or more classifications of the registered documents; and

identifying that at least a portion of the plurality of signatures match a portion of the previously stored signatures;

prohibiting delivery of the object;

identifying one of the registered documents associated with the portion of the previously stored signatures; and

sending an alert to respective owner of the registered documents about the object.

10. The non-transitory machine-readable storage medium of claim 9, wherein the portion of the plurality of signatures corresponds to a first content in the object, wherein the portion of the previously stored signatures corresponds to a second content in the registered documents, and wherein if the first content is the same as the second content, then a match is declared between the portion of the plurality of signatures and the portion of the previously stored signatures.

11. The non-transitory machine-readable storage medium of claim 9, wherein the alert includes a request for permission from the respective owner to deliver the object to its intended destination.

12. The non-transitory machine-readable storage medium of claim 9, wherein the generating the plurality of signatures comprises:

extracting and decoding a content in the object;

normalizing a text included in the content, wherein the normalizing comprises;

comparing the text to a plurality of patterns according to a pattern priority that is generated from a hierarchy of restrictiveness, with a more restrictive pattern having a higher priority than a less restrictive pattern; and

removing delimiters in the text according to matching patterns in the plurality of patterns to create a plurality of delimited items;

tokenizing the plurality of delimited items into a list of tokens; and

calculating hashes of the tokens.

13. The non-transitory machine-readable storage medium of claim 12, wherein the calculating hashes of the tokens comprises:

selecting M tokens from the list of tokens, wherein M is a positive integer;

selecting N tokens from the M tokens, wherein N is a positive integer that is less than or equal to M;

calculating a hash of the N tokens;

skipping P tokens of the list of tokens, wherein P is a non-zero integer that can be less than M, equal to M, or greater than M;

determining whether a number of remaining tokens to be hashed in the list of tokens is less than M;

if the number of remaining tokens is less than M, stopping the calculating; and

if the number of remaining tokens is greater than or equal to M, continuing the calculating.

14. The non-transitory machine-readable storage medium of claim 9, wherein the method further comprises:

storing a plurality of objects in a storage medium;

creating a keyword index and a metadata index having respective entries corresponding to a content in the plurality of objects; and

querying at least one of the keyword index and the metadata index to find a specific object.

15. An apparatus, comprising:

a memory for storing data; and

a processor operable to execute instructions associated with the data, wherein the apparatus is configured for;

capturing, by an electronic computing device, one or more registered documents;

classifying the one or more registered documents with one or more classifications using one or more of the six mechanisms;

(1) content signature, (2) grammar analysis, (3) statistical analysis, (4) file classification, (5) document biometrics, and (6) concept maps;

capturing, by the electronic computing device, an object traversing a network environment;

classifying the object with one or more classifications using the one or more of the six mechanisms;

generating a plurality of signatures of the object;

comparing the plurality of signatures with previously stored signatures in a signatures database, wherein parts of the signatures database is excluded from said comparing step based on the one or more classifications of the object and/or the one or more classifications of the registered documents; and

identifying that at least a portion of the plurality of signatures match a portion of the previously stored signatures;

prohibiting delivery of the object;

identifying one of the registered documents associated with the portion of the previously stored signatures; and

sending an alert to respective owner of the registered documents about the object.

16. The apparatus of claim 15, wherein the portion of the plurality of signatures corresponds to a first content in the object, wherein the portion of the previously stored signatures corresponds to a second content in the registered documents, and wherein if the first content is the same as the second content, then a match is declared between the portion of the plurality of signatures and the portion of the previously stored signatures.

17. The apparatus of claim 15, wherein the alert includes a request for permission from the respective owner to deliver the object to its intended destination.

18. The apparatus of claim 15, wherein the generating the plurality of signatures comprises:

extracting and decoding a content in the object;

normalizing a text included in the content, wherein the normalizing comprises;

comparing the text to a plurality of patterns according to a pattern priority that is generated from a hierarchy of restrictiveness, with a more restrictive pattern having a higher priority than a less restrictive pattern; and

removing delimiters in the text according to matching patterns in the plurality of patterns to create a plurality of delimited items;

tokenizing the plurality of delimited items into a list of tokens; and

calculating hashes of the tokens.

19. The apparatus of claim 18, wherein the calculating hashes of the tokens comprises:

selecting M tokens from the list of tokens, wherein M is a positive integer;

selecting N tokens from the M tokens, wherein N is a positive integer that is less than or equal to M;

calculating a hash of the N tokens;

skipping P tokens of the list of tokens, wherein P is a non-zero integer that can be less than M, equal to M, or greater than M;

determining whether a number of remaining tokens to be hashed in the list of tokens is less than M;

if the number of remaining tokens is less than M, stopping the calculating; and

if the number of remaining tokens is greater than or equal to M, continuing the calculating.

20. The apparatus of claim 15, further configured for:

storing a plurality of objects in a storage medium;

creating a keyword index and a metadata index having respective entries corresponding to a content in the plurality of objects; and

querying at least one of the keyword index and the metadata index to find a specific object.