Systems and methods for efficient detection of fingerprinted data and information

• US 9,692,762 B2
• Filed: 09/04/2015
• Issued: 06/27/2017
• Est. Priority Date: 05/26/2009
• Status: Active Grant

• Current Assignee: Forcepoint LLC
• Original Assignee: Websense, Llc (Forcepoint LLC)
• Inventors: Troyansky, Lidror, Barkan, Roy, Lazarov, David, Menaker, Yevgeny
• Primary Examiner(s): HENDERSON, ESTHER BENOIT
• Application Number:
  US14/845,816
• Publication Number:
  US 20150381626A1
• Time in Patent Office: 662 Days
• Field of Search: None
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 21/10 Protecting distributed programs or content, e.g. vending or licensing of copyrighted material protection in video systems or pay television H04N7/16 ; Digital rights management [DRM]
• CPC Class Codes: G06F 21/16 Program or content traceability, e.g. by watermarking
• CPC Class Codes: G06F 21/606 by securing the transmission between two devices or processes
• CPC Class Codes: G06F 2221/2101 Auditing as a secondary aspect
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 63/12 Applying verification of the received information cryptographic mechanisms or cryptographic arrangements for data integrity or data verification H04L9/32

Abstract

The disclosed embodiments provide systems, methods, and apparatus for efficient detection of fingerprinted content and relate generally to the field of information (or data) leak prevention. Particularly, a compact and efficient repository of fingerprint ingredients is used to analyze content and determine the content'"'"'s similarity to previously fingerprinted content. Some embodiments employ probabilistic indications regarding the existence of fingerprint ingredients in the repository.

Claims

1. A system for managing transmission of a plurality of electronic documents over a network, the system comprising:

a compact fingerprint repository storing a repository of compact fingerprints, the compact fingerprints based on hashes of electronic documents to which a transmission policy corresponding to the compact fingerprint is applied, the hashes of each of the compact fingerprints including a subset of a full set of hashes, the subset including only hashes that are evenly divisible by one or more specified numbers, the subset including fewer hashes than the full set of hashes;

one or more hardware electronic processors configured to;

determine probabilistic matches between fingerprints of an electronic document identified on the network and the compact fingerprints stored in the compact fingerprint repository, and

determine whether to transmit the identified electronic document over the electronic network based, at least in part, on a number of matching fingerprints.

2. The system of claim 1, wherein the one or more hardware electronic processors are further configured to:

identify a compact fingerprint matching the fingerprints of the electronic document,

identify a policy associated with the identified compact fingerprint, and

determine whether to transmit the identified electronic document based on the identified policy.

3. The system of claim 1, wherein the one or more hardware electronic processors are configured to determine a probabilistic match based on distances between portions of content of the electronic document corresponding to hashes matching a compact fingerprint in the repository.

4. The system of claim 3, wherein the one or more hardware electronic processors are configured to determine no match exists between a compact fingerprint and the electronic document in response to the compact fingerprint failing to represent a fingerprint of the electronic document.

5. The system of claim 1, wherein the one or more hardware electronic processors are configured to determine a first hash of the electronic document using a first hash function and determine a second hash of the electronic document using a second hash function.

6. The system of claim 5, wherein the one or more hardware electronic processors are configured to determine whether the first and second hashes are represented by a compact fingerprint in the repository, and determine the compact fingerprint does not match the electronic document if either of the hashes is not represented by the compact fingerprint.

7. A method of transmitting electronic content over a network, the method comprising:

identifying electronic content on the network;

generating hashes of the electronic content identified on the network;

selecting a subset of the generated hashes, the selected including only hashes that are evenly divisible by one or more specified numbers, the subset including fewer hashes than the generated hashes; and

determining whether to transmit the electronic content identified on the network over the network, based at least in part on a transmission policy corresponding to a compact fingerprint stored in a compact fingerprint repository matching the selected hashes.

8. The method of claim 7, wherein each compact fingerprint in the compact fingerprint repository is based on hashes of electronic documents to which the corresponding transmission policy is applied.

9. The method of claim 7, further comprising:

identifying a compact fingerprint matching the fingerprints of the electronic document,

identifying a policy corresponding to the identified compact fingerprint; and

determining whether to transmit the identified electronic document based on the identified policy.

10. The method of claim 7, further comprising determining a probabilistic match based on distances between portions of content of the electronic document corresponding to hashes matching a compact fingerprint in the repository.

11. The method of claim 9, further comprising determining no match exists between a compact fingerprint and the electronic document in response to the compact fingerprint failing to represent a fingerprint of the electronic document.

12. The method of claim 7, further comprising determining a first hash of the electronic document using a first hash function and determining a second hash of the electronic document using a second hash function.

13. The method of claim 12, further comprising:

determining whether the first and second hashes are represented by a compact fingerprint in the repository; and

determining the compact fingerprint does not match the electronic document in response to either of the hashes not being represented by the compact fingerprint.

14. A non-transitory computer-readable medium comprising instructions that when executed cause one or more processors to perform a method of transmitting electronic content over a network, the method comprising:

identifying electronic content on the network;

generating hashes of the electronic content identified on the network;

selecting a subset of the generated hashes, the selected including only hashes that are evenly divisible by one or more specified numbers, the subset including fewer hashes than the generated hashes; and

determining whether to transmit the electronic content identified on the network over the network, based at least in part on a transmission policy corresponding to a compact fingerprint stored in a compact fingerprint repository matching the selected hashes.

15. The non-transitory computer-readable medium of claim 14, wherein each compact fingerprint in the compact fingerprint repository is based on hashes of electronic documents to which the corresponding transmission policy is applied.

16. The non-transitory computer-readable medium of claim 14, the method further comprising:

identifying a compact fingerprint matching the fingerprints of the electronic document;

identifying a policy corresponding to the compact fingerprint; and

determining whether to transmit the identified electronic document based on the identified policy.

17. The non-transitory computer-readable medium of claim 14, the method further comprising determining a probabilistic match based on distances between portions of content of the electronic document corresponding to hashes matching a compact fingerprint in the repository.

18. The non-transitory computer-readable medium of claim 17, the method further comprising determining no match exists between a compact fingerprint and the electronic document in response to the compact fingerprint failing to represent a fingerprint of the electronic document.

19. The non-transitory computer-readable medium of claim 14, the method further comprising determining a first hash of the electronic document using a first hash function and determining a second hash of the electronic document using a second hash function.

20. The transitory computer-readable medium of claim 19, the method further comprising:

determining whether the first and second hashes are represented by a compact fingerprint in the repository; and

determining the compact fingerprint does not match the electronic document in response to either of the hashes not being represented by the compact fingerprint.