Malware communications detection

• US 10,389,738 B2
• Filed: 10/30/2015
• Issued: 08/20/2019
• Est. Priority Date: 08/31/2015
• Status: Active Grant

• Current Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Original Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Inventors: Muddu, Sudhakar, Tryfonas, Christos, Zadeh, Joseph Auguste, Bond, Alexander Beebe, Athalye, Ashwin
• Primary Examiner(s): Dada, Beemnet W
• Application Number:
  US14/929,183
• Publication Number:
  US 20170063888A1
• Time in Patent Office: 1,390 Days
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 10/225 based on a marking or identifier characterising the area
• CPC Class Codes: H04L 2463/121 Timestamp
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/145 involving simulating, designing, planning or modelling of a network
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 43/00 : Arrangements for monitoring or testing data switching networks
• CPC Class Codes: H04L 43/045 : for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/062 : related to network traffic
• CPC Class Codes: H04L 43/08 : Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 63/06 : for supporting key management in a packet data network cryptographic mechanisms or cryptographic arrangements for key management H04L9/08
• CPC Class Codes: H04L 63/1408 : by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1416 : Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 : Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1433 : Vulnerability analysis
• CPC Class Codes: H04L 63/1441 : Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/20 : for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

A security platform employs a variety techniques and mechanisms to detect security related anomalies and threats in a computer network environment. The security platform is “big data” driven and employs machine learning to perform security analytics. The security platform performs user/entity behavioral analytics (UEBA) to detect the security related anomalies and threats, regardless of whether such anomalies/threats were previously known. The security platform can include both real-time and batch paths/modes for detecting anomalies and threats. By visually presenting analytical results scored with risk ratings and supporting evidence, the security platform enables network security administrators to respond to a detected anomaly or threat, and to take action promptly.

Claims

1. A method comprising:

receiving, by a computer system, event data associated with a communication between an internal entity within a computer network and an external entity outside the computer network;

generating, by the computer system, a plurality of entity-specific feature scores by processing the event data, wherein each of the entity-specific features scores is representative of a quantified evaluation of a level of risk associated with a particular entity, each of the feature scores generated by a different one of a plurality of different types of analyses of the event data, the particular entity being the internal entity or the external entity;

wherein the plurality of entity-specific feature scores include;

a first entity-specific feature score based on a lexical analysis of an identifier associated with the particular entity; and

a second entity-specific feature score based on an analysis of the timing or sequencing of communications by the particular entity;

generating, by the computer system, an entity profile associated with the particular entity, the entity profile including the plurality of entity-specific feature scores;

generating, by the computer system, an anomaly score based on the entity profile; and

detecting, by the computer system, an anomaly if the anomaly score satisfies a specified criterion.

2. The method of claim 1, wherein the detected anomaly is indicative of malware within the computer network.

3. The method of claim 1, wherein the lexical analysis of the identifier associated with the particular entity includes any of

an analysis of the sequencing of characters in a domain name associated with the particular entity;

or

an n-gram analysis of the domain name associated with the particular entity.

4. The method of claim 1, wherein the plurality of different types of analyses of event data include analysis of one or more of the following:

sequencing of characters in a domain name associated with the particular entity;

timing of communications associated with the particular entity;

sequencing of communications associated with the particular entity;

data transmission statistics associated with the particular entity;

referral strings associated with the particular entity;

or

entities within the computer network associated with the particular entity.

5. The method of claim 1, wherein generating the first entity-specific feature score includes:

analyzing a sequencing of characters in a domain name associated with the particular entity; and

assigning the first entity-specific feature score based on the analysis;

wherein the first entity-specific feature score is indicative of a level of confidence that the domain name is machine-generated.

6. The method of claim 1, wherein generating the first entity-specific feature score includes:

performing an n-gram analysis on a domain name associated with the particular entity; and

assigning the first entity-specific feature score based on the n-gram analysis;

wherein the first entity-specific feature score is indicative of a level of confidence that the domain name is machine-generated.

7. The method of claim 1, wherein generating the second entity-specific feature score includes:

analyzing the timing of communications by the particular entity over a time period; and

assigning the second entity-specific feature score based on the analysis;

wherein the second entity-specific feature score is indicative of a level of confidence that the communications are machine-generated.

8. The method of claim 1, wherein generating the second entity-specific feature score includes:

analyzing the timing of communications by the particular entity over a time period, the analyzing including calculating one or more of;

a variance of interval times between communications and a periodicity of communications; and

assigning the second entity-specific feature score based on the analysis;

wherein the second entity-specific feature score is indicative of a level of confidence that the communications are machine-generated.

9. The method of claim 1, wherein generating the second entity-specific feature score includes:

analyzing the sequencing of communications by the particular entity over a time period; and

assigning the second entity-specific feature score based on the analysis;

wherein the second entity-specific feature score is indicative of a level of confidence that the communications are associated with an exploit chain.

10. The method of claim 1, wherein generating the plurality of entity-specific feature scores includes:

analyzing the data transmission statistics associated with the particular entity over a time period; and

assigning an entity-specific feature score based on the analysis;

wherein the entity-specific feature score is indicative of a level of confidence that the particular entity is associated with a command and control infrastructure external to the computer network.

11. The method of claim 1, wherein generating the plurality of entity-specific feature scores includes:

processing the event data through a plurality of machine-learning models, each of the plurality of machine-learning models including;

model processing logic defining a process for assigning an entity-specific feature score based on the event data; and

a model state defining a set of parameters for applying the model processing logic; and

assigning the plurality of entity-specific features scores based on the processing of the event data;

wherein the entity profile is represented as a feature vector including the plurality of entity-specific feature scores.

12. The method of claim 1, wherein generating the anomaly score includes:

processing the entity profile through an anomaly model, the anomaly model including;

model processing logic defining a process for assigning an anomaly score based on the plurality of entity-specific feature scores of the entity profile; and

a model state defining a set of parameters for applying the model processing logic; and

assigning the anomaly score based on the processing of the entity profile.

13. The method of claim 1, wherein generating the anomaly score includes calculating a weighted combination of the plurality of entity-specific feature scores of the entity profile.

14. The method of claim 1, wherein generating the anomaly score includes:

determining a volume of event data used to generate the entity profile;

processing the entity profile through;

a first anomaly model if the volume of event data is at or above a threshold volume;

or

a second anomaly model if the volume of event data is below the threshold volume; and

assigning the anomaly score based on the processing of the entity profile.

15. The method of claim 1, wherein generating the anomaly score includes:

processing the entity profile according to a plurality of machine-learning models;

assigning a plurality of intermediate anomaly scores, each of the plurality of intermediate anomaly scores based on processing of the entity profile according to one of the plurality of machine-learning models;

processing the plurality of intermediate anomaly scores according to an ensemble-learning model; and

assigning the anomaly score based on processing the plurality of intermediate anomaly scores.

16. The method of claim 1, wherein generating the anomaly score includes:

comparing the identifier associated with the particular entity with entries in an external data source external to the computer network; and

assigning the anomaly score indicating a confidence level that the identifier matches a particular entry in the external data source based on the comparing.

17. The method of claim 1, further comprising:

annotating, by the computer system, the detected anomaly with data from an external data source external to the computer network.

18. The method of claim 1, further comprising:

outputting, via a user interface, an indication of the detected anomaly to a user.

19. The method of claim 1, further comprising:

outputting, via a user interface, an incident response output based on the entity profile, the incident response output including;

the identifier associated with the particular entity;

the plurality of entity-specific feature scores; and

a recommended response based on the plurality of entity-specific feature scores.

20. The method of claim 1, further comprising:

incorporating the detected anomaly into a network security graph, the network security graph including;

a plurality of nodes representing entities associated with the computer network, the entities including the particular entity; and

a plurality of edges, each of the plurality of edges linking two of the plurality of nodes and representing an association between the entities represented by the nodes;

wherein the detected anomaly is incorporated as a node linked to the particular entity by an edge.

21. The method of claim 1, wherein detecting the anomaly is performed in real time as the event data are received.

22. The method of claim 1, wherein detecting the anomaly is performed using Apache Storm or Apache Spark Streaming as a processing engine.

23. The method of claim 1, wherein receiving the event data includes adaptively filtering the event data according to a dynamic whitelist.

24. The method of claim 1, wherein receiving the event data includes receiving event data from a plurality of entities associated with the computer network via an extract, transform, and load (ETL) pipeline.

25. The method of claim 1, wherein the event data are timestamped machine data.

26. The method of claim 1, wherein the event data include one or more of:

domain name system (DNS) generated log data, firewall generated log data, or proxy generated log data.

27. The method of claim 1, further comprising storing the detected anomaly in an anomaly graph data structure that includes:

a plurality of nodes representing entities associated with the computer network, the entities including users and/or devices; and

a plurality of edges, each of the plurality of edges representing an anomaly linking two of the plurality of nodes.

28. The method of claim 1, wherein the the identifier associated with the particular entity includes one or more of:

a domain name, a uniform resource locator (URL), uniform resource identifier (URI), an Internet Protocol (IP) address, a unique identifier (UID), a device identification, or a user identification.

29. A system comprising:

a processor; and

a memory unit having instructions stored thereon, which when executed by the processor cause the system to;

receive event data associated with a communication between an internal entity within a computer network and an external entity outside the computer network;

generate a plurality of entity-specific feature scores by processing the event data, wherein each of the entity-specific features scores is representative of a quantified evaluation of a level of risk associated with a particular entity, each of the feature scores generated by a different one of a plurality of different types of analyses of the event data, the particular entity being the internal entity or the external entity;

wherein the plurality of entity-specific feature scores include;

a first entity-specific feature score based on a lexical analysis of an identifier associated with the particular entity; and

a second entity-specific feature score based on an analysis of the timing or sequencing of communications by the particular entity;

generate an entity profile associated with the particular entity, the entity profile including the plurality of entity-specific feature scores;

generate an anomaly score based on the entity profile; and

detect an anomaly if the anomaly score satisfies a specified criterion.

30. A non-transient computer readable medium containing instructions for causing a computer system to:

receive event data associated with a communication between an internal entity within a computer network and an external entity outside the computer network;

generate a plurality of entity-specific feature scores by processing the event data, wherein each of the entity-specific features scores is representative of a quantified evaluation of a level of risk associated with a particular entity, each of the feature scores generated by a different one of a plurality of different types of analyses of the event data, the particular entity being the internal entity or the external entity;

wherein the plurality of entity-specific feature scores include;

a first entity-specific feature score based on a lexical analysis of an identifier associated with the particular entity; and

a second entity-specific feature score based on an analysis of the timing or sequencing of communications by the particular entity;

generate an entity profile associated with the particular entity, the entity profile including the plurality of entity-specific feature scores;

generate an anomaly score based on the entity profile; and

detect an anomaly if the anomaly score satisfies a specified criterion.