System and method for using file hashes to track data leakage and document propagation in a network

• US 9,367,707 B2
• Filed: 02/23/2012
• Issued: 06/14/2016
• Est. Priority Date: 02/23/2012
• Status: Active Grant

• Current Assignee: Tenable, Inc. (Tenable Holdings, Inc.)
• Original Assignee: Tenable Network Security Incorporated (Tenable Holdings, Inc.)
• Inventors: Gula, Ron, Ranum, Marcus
• Primary Examiner(s): CHOUDHURY, AZIZUL Q
• Application Number:
  US13/403,108
• Publication Number:
  US 20130227714A1
• Time in Patent Office: 1,573 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0864 using pseudo-associative means, e.g. set-associative or hashing
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: G06F 16/152 using file content signatures, e.g. hash values
• CPC Class Codes: G06F 21/64 Protecting data integrity, e.g. using checksums, certificates or signatures
• CPC Class Codes: H04L 63/0236 Filtering by address, protocol, port number or service, e.g. IP-address or URL
• CPC Class Codes: H04L 63/0876 based on the identity of the terminal or configuration, e.g. MAC address, hardware or software configuration or device fingerprint
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00

Abstract

The system and method described herein may use file hashes to track data leakage and document propagation in a network. For example, file systems associated with known reference systems and various user devices may be compared to classify the user devices into various groups based on differences between the respective file systems, identify files unique to the various groups, and detect potential data leakage or document propagation if user devices classified in certain groups include any files that are unique to other groups. Additionally, various algorithms may track locations, movements, changes, and other events that relate to normal or typical activity in the network, which may be used to generate statistics that can be compared to subsequent activities that occur in the network to detect potentially anomalous activity that may represent potential data leakage or document propagation.

Claims

1. A system for using file hashes to track data leakage and document propagation in a network, comprising:

one or more physical processors programmed to execute computer program instructions which, when executed, cause the physical processors to;

obtain a set of hashes that are associated with files of a user device of a set of user devices, and a reference set of hashes that are associated with files of a reference system, wherein the reference system is limited to files authorized to be on all devices of the set of user devices;

determine an additional subset of hashes included in the set of hashes and not included in the reference set of hashes based on a comparison between the set of hashes and the reference set of hashes;

classify the user device into a group based on the additional subset of hashes comprising a hash that is the same as a hash associated with a file of at least another user device classified into the group;

predict that the file associated with the same hash is exclusive for the group to which the user device is classified;

scan one or more other user devices not classified into the group to determine what files are on the other user devices;

generate an alert indicating unauthorized file access, wherein the alert is generated responsive to the scan indicating that the other user devices contain the file predicted to be exclusive for the group to which the user device is classified; and

deliver the alert to a user.

2. The system of claim 1, wherein the physical processors are further caused to:

obtain a set of names or paths that are associated with the files of the user device;

obtain a reference set of names or paths associated with the files of the reference system; and

determine an additional subset of names or paths included in the set of names or paths and not included in the reference set of names or paths based on a comparison between the set of names or paths and the reference set of names or paths,

wherein classifying the user device into the group is further based on the additional subset of names or paths, and

wherein predicting the file associated with the same hash as being exclusive for the group to which the user device is classified is further based on the additional subset of names or paths.

3. The system of claim 1, wherein the physical processors are further caused to:

obtain a set of names and paths that are associated with the files of the user device;

obtain a reference set of names and paths associated with the files of the reference system; and

determine an additional subset of names and paths included in the set of names and paths and not included in the reference set of names and paths based on a comparison between the set of names and paths and the reference set of names and paths,

wherein classifying the user device into the group is further based on the additional subset of names and paths, and

wherein predicting the file associated with the same hash as being exclusive for the group to which the user device is classified is further based on the additional subset of names and paths.

4. The system of claim 1, wherein the physical processors are further caused to:

observe traffic associated with the network to detect activity that changed, copied, moved, or accessed one or more files on user devices of the network; and

generate, based on the detected activity, an audit trail associated with the files that were changed, copied, moved, or accessed, wherein the audit trail includes information to describe (i) one or more of the user devices at which the detected activity was observed, (ii) one or more users that owned the files associated with the detected activity, (iii) times when the users owned the files associated with the detected activity, and (iv) the detected activity.

5. The system of claim 1, wherein the physical processors are further caused to:

obtain, based on the scan, hashes associated with files that are on the other user devices that are not classified into the group to which the user device is classified; and

generate the alert responsive to a determination that at least one of the hashes obtained based on the scan matches the same hash to which the file predicted to be exclusive for the group is associated.

6. The system of claim 1, wherein the physical processors are further caused to:

generate the alert responsive to a determination that one or more files of user devices of the network have different owners on more than one of the user devices.

7. The system of claim 1, wherein the physical processors are further caused to:

identify one or more files or file sets having auditing or security significance, wherein one or more of user devices of the network are designated to store the files or file sets having the auditing or security significance; and

generate the alert responsive to a determination that the files or file sets have been copied or moved off of the designated user devices.

8. The system of claim 1, wherein the physical processors are further caused to:

generate statistics that describe normal activities that one or more users perform to interact with files of user devices of the network; and

generate the alert responsive to a determination that interactions with the files of the user devices deviate from the statistics describing the normal activities that the users perform to interact with the files of the user devices.

9. The system of claim 1, wherein the physical processors are further caused to:

generate statistics that describe normal activities that one or more users perform to interact with one or more directories or folders that contain files of user devices of the network; and

generate the alert responsive to a determination that interactions with the directories or folders deviate from the statistics describing the normal activities that the users perform to interact with the directories or folders.

10. The system of claim 1, wherein the set of hashes associated with the files of the user device comprises one or more of checksums, complete cryptographic hashes, or partial cryptographic hashes.

11. A method for using file hashes to track data leakage and document propagation in a network, the method being implemented on a computer system that includes one or more physical processors executing computer program instructions which, when executed, perform the method, the method comprising:

obtaining, by the physical processors, a set of hashes that are associated with files of a user device of a set of user devices, and a reference set of hashes that are associated with files of a reference system, wherein the reference system is limited to files authorized to be on all devices of the set of user devices;

determining, by the physical processors, an additional subset of hashes included in the set of hashes and not included in the reference set of hashes based on a comparison between the set of hashes and the reference set of hashes;

classifying, by the physical processors, the user device into a group based on the additional subset of hashes comprising a hash that is the same as a hash associated with a file of at least another user device classified into the group;

predicting, by the physical processors, that the file associated with the same hash is exclusive for the group to which the user device is classified;

scanning, by the physical processors, one or more other user devices not classified into the group to determine what files are on the other user devices;

generating, by the physical processors, an alert indicating unauthorized file access responsive to the scan indicating that the other user devices contain the file predicted to be exclusive for the group to which the user device is classified; and

delivering, by the physical processors, the alert to a user.

12. The method of claim 11, further comprising:

obtaining, by the physical processors, a set of names or paths that are associated with the files of the user device;

obtaining, by the physical processors, a reference set of names or paths associated with the files of the reference system; and

determining, by the physical processors, an additional subset of names or paths included in the set of names or paths and not included in the reference set of names or paths based on a comparison between the set of names or paths and the reference set of names or paths;

wherein classifying the user device into the group is further based on the additional subset of names or paths, and

wherein predicting the file associated with the same hash as being exclusive for the group to which the user device is classified is further based on the additional subset of names or paths.

13. The method of claim 11, further comprising:

obtaining, by the physical processors, a set of names and paths that are associated with the files of the user device;

obtaining, by the physical processors, a reference set of names and paths associated with the files of the reference system; and

determining, by the physical processors, an additional subset of names and paths included in the set of names and paths and not included in the reference set of names and paths based on a comparison between the set of names and paths and the reference set of names and paths,

wherein classifying the user device into the group is further based on the additional subset of names and paths, and

wherein predicting one or more of the files of the user device as being exclusive for the group to which the user device is classified is further based on the additional subset of names and paths.

14. The method of claim 11, further comprising:

observing, by the physical processors, traffic associated with the network to detect activity that changed, copied, moved, or accessed one or more files on user devices of the network; and

generating, by the physical processors, based on the detected activity, an audit trail associated with the files that were changed, copied, moved, or accessed, wherein the audit trail includes information to describe (i) one or more of the user devices at which the detected activity was observed, (ii) one or more users that owned the files associated with the detected activity, (iii) times when the users owned the files associated with the detected activity, and (iv) the detected activity.

15. The method of claim 11, further comprising:

obtaining, by the physical processors, based on the scan, hashes associated with files that are on the other user devices that are not classified into the group to which the user device is classified; and

generating, by the computer system, the alert responsive to a determination that at least one of the hashes obtained based on the scan matches the same hash to which the file predicted to be exclusive for the group is associated.

16. The method of claim 11, further comprising:

generating, by the physical processors, the alert responsive to a determination that one or more files of user devices on the network have different owners on more than one of the user devices.

17. The method of claim 11, further comprising:

identifying, by the physical processors, one or more files or file sets having auditing or security significance, wherein one or more user devices of the network are designated to store the files or file sets having the auditing or security significance; and

generating, by the physical processors, the alert responsive to a determination that the files or file sets have been copied or moved off of the designated user devices.

18. The method of claim 11, further comprising:

generating, by the physical processors, statistics, to describe normal activities that one or more users perform to interact with files of user devices of the network; and

generating, by the physical processors, the alert responsive to a determination that interactions with the files of the user devices deviate from the statistics describing the normal activities that the users perform to interact with the files of the user devices.

19. The method of claim 11, further comprising:

generating, by the physical processors, statistics that describe normal activities that one or more users perform to interact with one or more directories or folders that contain files of user devices of the network; and

generating, by the physical processors, the alert responsive to a determination that interactions with the directories or folders deviate from the statistics describing the normal activities that the users perform to interact with the directories or folders.

20. The method of claim 11, wherein the sets of hashes associated with the files of the user devices comprises one or more of checksums, complete cryptographic hashes, or partial cryptographic hashes.