Lateral movement detection for network security analysis

• US 10,015,177 B2
• Filed: 10/30/2015
• Issued: 07/03/2018
• Est. Priority Date: 08/31/2015
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Muddu, Sudhakar, Tryfonas, Christos, Lam, Fumei, Apostolopoulos, Georgios
• Primary Examiner(s): Hirl, Joseph P
• Assistant Examiner(s): GYORFI, THOMAS A
• Application Number:
  US14/929,196
• Publication Number:
  US 20170063911A1
• Time in Patent Office: 977 Days
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 10/225 based on a marking or identifier characterising the area
• CPC Class Codes: H04L 2463/121 Timestamp cryptographic mechanisms or cryptographic arrangements involving time stamps H04L9/3297
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/145 involving simulating, designing, planning or modelling of a network
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 43/00 : Arrangements for monitoring or testing data switching networks
• CPC Class Codes: H04L 43/045 : for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/062 : related to network traffic
• CPC Class Codes: H04L 43/08 : Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 63/06 : for supporting key management in a packet data network cryptographic mechanisms or cryptographic arrangements for key management H04L9/08
• CPC Class Codes: H04L 63/1408 : by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1416 : Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 : Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1433 : Vulnerability analysis
• CPC Class Codes: H04L 63/1441 : Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/20 : for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H05K 999/99 : dummy group

Abstract

A security platform employs a variety techniques and mechanisms to detect security related anomalies and threats in a computer network environment. The security platform is “big data” driven and employs machine learning to perform security analytics. The security platform performs user/entity behavioral analytics (UEBA) to detect the security related anomalies and threats, regardless of whether such anomalies/threats were previously known. The security platform can include both real-time and batch paths/modes for detecting anomalies and threats. By visually presenting analytical results scored with risk ratings and supporting evidence, the security platform enables network security administrators to respond to a detected anomaly or threat, and to take action promptly.

Claims

1. A method comprising:

receiving, by a computer system, first event data indicative of computer network activity of a plurality of users and network devices in a computer network;

generating, by the computer system, classification metadata for each of the network devices and users, based on the first event data, to indicate relevance in a network security context of each of the users and network devices;

identifying, by the computer system, usage relationships between one or more of the users and one or more of the network devices, based on first event data;

assigning, by the computer system, usage similarity scores to the network devices based on the identified usage relationships, the usage similarity scores being indicative of which of the network devices have been used by the same or similar group of users;

receiving, by the computer system, second event data indicative of computer network activity of a particular user of the plurality of users; and

detecting, by the computer system and in response to the second event data, an anomaly indicative that the particular user has interacted with a particular network device with which the particular user does not normally interact, based on the usage similarity scores and the classification metadata.

2. The method of claim 1, further comprising:

retrieving a graph data structure that records anomalies in the network and the relationships between the anomalies and the users and network devices;

identifying a security threat based on the detected anomaly by identifying, in the graph data structure, a relationship path in the data structure from the particular user to a network device designated as a critical resource of the network, the relationship path including users and network devices interlinked by anomalies; and

reporting the security threat to an administrator of the network.

3. The method of claim 1, further comprising:

retrieving a graph data structure that records anomalies in the network and the relationships between the anomalies and the users and network devices, the graph data structure including nodes interconnected by edges, the nodes representing the entities, the edges representing usage relationships between the entities;

identifying a security threat based on the detected anomaly by identifying, in the graph data structure, a relationship path from the particular user through one or more anomalies in the data structure to a network device designated as a critical resource of the network; and

reporting the security threat to an administrator of the network.

4. The method of claim 1, further comprising:

determining that the detected anomaly is indicative of a suspicious lateral movement of the particular user in the network.

5. The method of claim 1, wherein the relationships between the users and the network devices include login events indicative of the users logging into the network devices.

6. The method of claim 1, wherein said assigning usage similarity scores is performed such that any first set of network devices that are accessed by the same or similar group of users are assigned usage similarity scores that are closer in value to each other than usage similarity scores of any second set of network devices that are not accessed by the same or similar group of users.

7. The method of claim 1, wherein said assigning usage similarity scores comprises:

assigning usage similarity scores to the network devices based on login events indicative of users accessing the network devices, such that a first group of network devices that have similar groups of login users have usage similarity scores that are closer in value to each other than usage similarity scores of network devices that do not have similar groups of login users.

8. The method of claim 1, wherein said detecting an anomaly comprises:

detecting an anomaly in response to detecting that a particular user has interacted with a particular network device when a usage similarity score of the particular network device fails to satisfy a specific closeness criterion relative to usage similarity scores of network devices with which the particular user usually interacts.

9. The method of claim 1, wherein the classification metadata for a particular user includes metadata indicative of the particular user being at least one of a regular user, an administrative user, or an automated user.

10. The method of claim 1, wherein the classification metadata for a particular network device includes metadata indicative of the particular network device being at least one of a workstation, a server, or a printer.

11. The method of claim 1, wherein the relationships between the users and the network devices form a bipartite graph including a first set of the users and a second set of the network devices.

12. The method of claim 1, wherein the relationships between the users and the network devices represent a time series of events in which the users have interacted with the network devices.

13. The method of claim 1, wherein said assigning usage similarity scores comprises:

forming a graph having nodes representing the users and the network devices and edges representing the relationships between the users and the network devices;

distributing weight values with a probability percentage along edges between nodes representing the users and the network devices; and

repeating the distributing along edges until the weight values at nodes representing the network devices converge; and

assigning usage similarity scores to the network devices based on the converged weight values.

14. The method of claim 1, wherein said assigning usage similarity scores comprises:

forming a bipartite graph having nodes representing the users and the network devices and edges representing the relationships between the users and the network devices;

assigning an initial weight value to a node of the bipartite graph;

keeping a percentage of the initial weight value at the node and equally distributing a remainder of the initial weight value from the node along edges of the node to other nodes;

at each node, repeating a process until a weight value at each node converges, the process including;

summing weight values at each node of the bipartite graph, and

keeping the percentage of a previously distributed weight value at the node and equally distributing a remainder of the weight value from the node along edges of the node to other nodes; and

assigning usage similarity scores to the network devices based on the converged weight values at the nodes representing the network devices.

15. The method of claim 1, wherein said assigning usage similarity scores comprises:

assigning usage similarity scores to the network devices based on relationships between the users and the network devices, such that particular network devices having multiple exclusive users interacting with only the particular network devices tend to have usage similarity scores that have differences less than a threshold value.

16. The method of claim 1, wherein said assigning usage similarity scores comprises:

assigning usage similarity scores to the network devices based on relationships between the users and the network devices, such that particular network devices having only a single shared user interacting with all of the particular network devices have different usage similarity scores.

17. The method of claim 1, wherein said detecting an anomaly comprises:

detecting that the particular user has interacted with the particular network device;

calculating an anomaly score for the particular user based on a difference between a usage similarity score of the particular network device and a statistical measure of usage similarity scores of other network devices with which the user has interacted; and

detecting the anomaly in response to detecting that the anomaly score exceeds an anomaly threshold value.

18. The method of claim 1, wherein said detecting an anomaly comprises:

determining an access profile of the particular user including network devices with which the particular user interacts and that have usage similarity scores that satisfy a specific closeness criterion;

in response to the particular user interacting with a particular network device, calculating an anomaly score for the particular user based on a difference between a usage similarity score of the particular network device and an average of usage similarity scores of network devices in the access profile of the particular user; and

detecting the anomaly in response to detecting that the usage similarity score difference indicates that the particular user has interacted with the particular network device outside of the access profile of the particular user.

19. The method of claim 1, wherein said detecting an anomaly comprises:

determining an access profile of the particular user including network devices with which the particular user interacts and that have usage similarity scores that satisfy a specific closeness criterion; and

wherein the access profile of the particular user further includes information of events indicative that the particular user has logged into a network device, validated credential of a network device, or accessed a network object stored on a network device.

20. The method of claim 1, wherein said detecting an anomaly comprises:

determining an access profile of the particular user including network devices with which the particular user interacts and that have usage similarity scores that satisfy a specific closeness criterion; and

wherein the access profile of the particular user further includes information of events indicative that the particular user has succeeded logging in to a network device, failed logging in to a network device, succeeded validating credential of a network device, failed validating credential of a network device, succeeded accessing a network object stored on a network device, or failed accessing a network object stored on a network device.

21. The method of claim 1, wherein said detecting an anomaly comprises:

identifying a first group including network devices with which the particular user interacts and that have usage similarity scores that satisfy a specific closeness criterion;

identifying a second group including a new network device with which the particular user interacts; and

detecting an out-of-profile anomaly in response to determining that the particular user is the only user who has interacted with both the first and second groups.

22. The method of claim 1, further comprising:

identifying a security threat based on the detected anomaly by identifying a relationship path from the particular user through one or more anomalies to a network device designated as a critical resource of the network, the critical resource being a server responsible for handling security authentication requests for the network.

23. The method of claim 1, further comprising:

identifying a security threat based on the detected anomaly by identifying a relationship path from the particular user through one or more anomalies to a network device designated as a critical resource of the network, the security threat indicative that an unauthorized user has misappropriated a credential of the particular user to enter the network and breached one or more network devices along the relationship path to access the critical resource.

24. The method of claim 1, wherein the first event data is machine data.

25. The method of claim 1, wherein the first event data is timestamped machine data.

26. The method of claim 1, wherein said generating classification metadata or said assigning usage similarity scores are performed by a machine learning model.

27. The method of claim 1, wherein said generating classification metadata or said assigning usage similarity scores are performed in real time as the first event data are received.

28. The method of claim 1, wherein said generating classification metadata or said assigning usage similarity scores are performed in batch mode based on the first event data being retrieved from a persistent storage facility.

29. A computing device comprising:

a processor; and

a memory storing instructions that, when executed by the processor, cause the computing device to perform a process including;

receiving first event data indicative of computer network activity of a plurality of users and network devices in a computer network;

generating classification metadata for each of the network devices and users, based on the first event data, to indicate relevance in a network security context of each of the users and network devices;

identifying usage relationships between one or more of the users and one or more of the network devices, based on first event data;

assigning usage similarity scores to the network devices based on the identified usage relationships, the usage similarity scores being indicative of which of the network devices have been used by the same or similar group of users;

receiving second event data indicative of computer network activity of a particular user of the plurality of users; and

in response to the second event data, detecting an anomaly indicative that the particular user has interacted with a particular network device with which the particular user does not normally interact, based on the usage similarity scores and the classification metadata.

30. A non-transitory machine readable storage medium storing instructions, execution of which in a machine causes the machine to perform a process including:

receiving, by the machine, first event data indicative of computer network activity of a plurality of users and network devices in a computer network;

generating, by the machine, classification metadata for each of the network devices and the users, based on the first event data, to indicate relevance in a network security context of each of the users and the network devices;

identifying, by the machine, usage relationships between one or more of the users and one or more of the network devices, based on first event data;

assigning, by the machine, usage similarity scores to the network devices based on the identified usage relationships, the usage similarity scores being indicative of which of the network devices have been used by the same or similar group of users;

receiving, by the machine, second event data indicative of computer network activity of a particular user of the plurality of users; and

detecting, by the machine and in response to the second event data, an anomaly indicative that the particular user has interacted with a particular network device with which the particular user does not normally interact, based on the usage similarity scores and the classification metadata.