SYSTEMS AND METHODS FOR EFFICIENT DETECTION OF FINGERPRINTED DATA AND INFORMATION

• US 20110035805A1
• Filed: 05/24/2010
• Published: 02/10/2011
• Est. Priority Date: 05/26/2009
• Status: Active Grant

• Current Assignee: Forcepoint LLC (Francisco Partners Management LLC)
• Original Assignee: Forcepoint LLC (Francisco Partners Management LLC)
• Inventors: Lazarov, David, Barkan, Roy, Menaker, Yevgeny, Troyansky, Lidror
• Granted Patent: US 9,130,972 B2
• Time in Patent Office: Days
• US Class Current: 726/26
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 21/10 Protecting distributed programs or content, e.g. vending or licensing of copyrighted material protection in video systems or pay television H04N7/16 ; Digital rights management [DRM]
• CPC Class Codes: G06F 21/16 Program or content traceability, e.g. by watermarking
• CPC Class Codes: G06F 21/606 by securing the transmission between two devices or processes
• CPC Class Codes: G06F 2221/2101 Auditing as a secondary aspect
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 63/12 Applying verification of the received information cryptographic mechanisms or cryptographic arrangements for data integrity or data verification H04L9/32

Abstract

The disclosed embodiments provide systems, methods, and apparatus for efficient detection of fingerprinted content and relate generally to the field of information (or data) leak prevention. Particularly, a compact and efficient repository of fingerprint ingredients is used to analyze content and determine the content'"'"'s similarity to previously fingerprinted content. Some embodiments employ probabilistic indications regarding the existence of fingerprint ingredients in the repository.

Claims

1. A system for compacting fingerprints of electronic content, the system comprising:

a compact representation module having a processor, the processor configured to hash the contents of the electronic content to create an electronic fingerprint of the content.

2. The system of claim 1 wherein the processor of the compact representation module is configured to perform multiple hashes of the electronic content and generate multiple fingerprints of the electronic content.

3. The system of claim 1 further comprising a compact repository for storing the fingerprints of the electronic content.

4. The system of claim 3 further comprising an analysis module having a processor and configured to hash new content and generate a fingerprint for the new content.

5. The system of claim 4 further comprising a match estimator for comparing fingerprints stored in the compact repository with fingerprints of the new content and to determine whether fingerprints match based upon the probability that a match exists within the compact repository.

6. The system of claim 5 further comprising a decision module configured to apply a transmission policy on the new content based upon the probability that a match exists between the fingerprint of the new content and a fingerprint stored in the compact repository.

7. A system for applying a transmission policy on electronic content stored in an electronic network, the system comprising:

a scanning engine having a processor and configured to identify electronic content;

a fingerprinting module in electronic communication with the scanning engine and configured to hash the electronic content in order to generate a compact fingerprint of the electronic content identified by the scanning engine;

at least one compact fingerprint repository in electronic communication with the fingerprinting module and configured to store fingerprints generated by the fingerprinting module;

a content processor in electronic communication with the electronic network, the content processor configured to hash new electronic content of the electronic network;

a match estimator in electronic communication with the fingerprint repository and the content processor, the match estimator configured to determine probabilistic matches between the fingerprint of the new electronic content and a fingerprint stored in the fingerprint repository; and

a decision module in electronic communication with the match estimator and the electronic network, the decision module configured to apply the transmission policy to the new electronic content based upon an output from the match estimator.

8. The system of claim 7 further comprising a fingerprint repository in electronic communication with the fingerprinting module, the fingerprinting module being configured to generate a hash of the electronic content and to store it in the fingerprint repository.

9. The system of claim 7 wherein the match estimator is configured to determine a match based upon a distance between matched hashes.

10. A method of transmitting electronic content over an electronic network, the method comprising the steps:

identifying electronic content on the electronic network with a scanning engine;

fingerprinting the electronic content with at least one hash to generate a compact fingerprint of the electronic content;

storing at least one compact fingerprint in a compact fingerprint repository;

fingerprinting new electronic content with at least one hash, the new electronic content to be transmitted over the electronic network;

comparing the fingerprint of the new electronic content with fingerprints stored in the compact fingerprint repository;

determining whether to transmit the new electronic content over the electronic network in response to the comparison of the fingerprints.

11. The method of claim 10 further comprising the step of applying a transmission policy to the new electronic content in response to determining whether the new electronic content should be transmitted.

12. The method of claim 10 wherein the step of determining whether the new electronic content should be transmitted comprises determining matches between fingerprints stored in the compact fingerprint repository and the fingerprint of the new electronic content.

13. The method of claim 12 wherein the step of determining matches comprises determining the distances between matches of the fingerprints.

14. A system for compacting fingerprints of an electronic content, the system comprising:

one or more processors, wherein;

at least one of the one or more processors is configured to receive at least a portion of a first electronic fingerprint; and

at least one of the one or more processors is configured to flag one or more addresses in an array, the one or more addresses associated with the at least a portion of the first electronic fingerprint.

15. The system of claim 14, wherein a hash function determines the associations of addresses to the at least a portion of the first fingerprint.

16. The system of claim 15, wherein the hash function is configured such that at least one address is associated with two or more fingerprints.

17. The system of claim 14, further comprising a compact repository, the compact repository comprising an array of addresses, the addresses which are associated with a second fingerprint of a second electronic content being flagged.

18. The system of claim 17, wherein the array having addresses associated with the first fingerprint of the electronic content is stored in the compact repository.

19. The system of claim 18, wherein the array of addresses associated with the first fingerprint and the array of addresses associated with the second fingerprint have no addresses associated with both the first and second fingerprint.

20. The system of claim 17, wherein at least one of the one or more processors is configured to compare the flagged addresses associated with the first fingerprint and the flagged addresses associated with the second fingerprint, and to determine a probability that the fingerprints match.

21. The system of claim 20, wherein at least one of the one or more processors is configured to apply a transmission policy to the electronic content based upon the probability that a match exists between the fingerprint of the electronic content and the fingerprints represented in the compact repository.

22. A system for applying a transmission policy on electronic content stored in an electronic network, the system comprising:

a scanning engine processor configured to identify electronic content;

a hashing processor configured to receive at least a portion of an electronic fingerprint of the electronic content identified by the scanning engine processor, the hashing processor further configured to flag a plurality of addresses associated with the at least a portion of an electronic fingerprint in an array, the hashing processor in electrical communication with the scanning engine processor;

at least one compact fingerprint repository in electrical communication with the hashing processor and configured to store the array associated with the at least a portion of an electronic fingerprint;

a content processor in electrical communication with the electronic network, the content processor configured to generate fingerprints of new electronic content of the electronic network;

a match estimation processor in electrical communication with the fingerprint repository and the content processor, the match estimation processor configured to determine probabilistic matches between the fingerprint of the new electronic content and the fingerprints stored in the fingerprint repository; and

a decision processor in electrical communication with the match estimation processor and the electronic network, the decision processor configured to apply the transmission policy to the new electronic content based upon an output from the match estimation processor.

23. The system of claim 22, wherein at least two of the scanning engine processor, hashing processor, content processor, match estimation processor, and decision processor, comprise a single processor.

24. The system of claim 22 further comprising a non-compact fingerprint repository in electrical communication with the hashing processor.

25. The system of claim 22 wherein the match estimation processor is further configured to determine a match based upon a distance between two fingerprints.

28. The method of claim 25, wherein determining whether to transmit the new electronic content over the electronic network depends at least in part on the comparing the new fingerprint of the electronic content with fingerprints stored in the compact fingerprint repository.

26. A method of transmitting electronic content over an electronic network comprising:

performing the following on one or more electronic processors;

identifying electronic content on the electronic network;

generating a new fingerprint of the electronic content;

comparing the new fingerprint of the electronic content with fingerprints stored in a compact fingerprint repository, comprising determining the one or more addresses flagged by new fingerprint when applied to a hash function and comparing these addresses with the one or more addresses flagged by the fingerprint of the repository when applied to the hash function; and

determining whether to transmit the new electronic content over the electronic network in response to the comparison of the fingerprints.

27. The method of claim 26, wherein determining whether to transmit the new electronic content comprises applying a transmission policy to the new electronic content.

29. The method of claim 27, wherein the step of determining matches comprises determining the distances between matches of the fingerprints.

30. A computer-readable medium comprising code configured to cause one or more processors to perform the following:

receiving at least a portion of a first electronic fingerprint;

flagging one or more addresses in an array, the one or more addresses associated with the at least a portion of the first electronic fingerprint;

storing the array;

receiving at least a portion of a second electronic fingerprint;

determining if an address associated with the at least a portion of the second electronic fingerprint is the same as an address associated with the first electronic fingerprint.

31. The computer-readable medium of claim 30, wherein the first electronic fingerprint is associated with a first electronic document and the second electronic fingerprint is associated with a second electronic document.

32. The computer-readable medium of claim 31, wherein the one or more processors are further configured to determine a probability that the first electronic document and the second electronic document are the same based on at least the determining if an addresses associated with the at least a portion of the second electronic fingerprint is the same as an address associated with the first electronic fingerprint.

33. The computer-readable medium of claim 30, wherein the array is stored in a compact repository.