Machine-generated traffic detection (beaconing)

• US 10,069,849 B2
• Filed: 10/30/2015
• Issued: 09/04/2018
• Est. Priority Date: 08/31/2015
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Muddu, Sudhakar, Tryfonas, Christos, Iliofotou, Marios
• Primary Examiner(s): DADA, BEEMNET W
• Application Number:
  US14/929,184
• Publication Number:
  US 20170063889A1
• Time in Patent Office: 1,040 Days
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 10/225 based on a marking or identifier characterising the area
• CPC Class Codes: H04L 2463/121 Timestamp cryptographic mechanisms or cryptographic arrangements involving time stamps H04L9/3297
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/145 involving simulating, designing, planning or modelling of a network
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 43/00 : Arrangements for monitoring or testing data switching networks
• CPC Class Codes: H04L 43/045 : for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/062 : related to network traffic
• CPC Class Codes: H04L 43/08 : Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 63/06 : for supporting key management in a packet data network cryptographic mechanisms or cryptographic arrangements for key management H04L9/08
• CPC Class Codes: H04L 63/1408 : by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1416 : Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 : Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1433 : Vulnerability analysis
• CPC Class Codes: H04L 63/1441 : Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/20 : for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H05K 999/99 : dummy group

Abstract

A security platform employs a variety techniques and mechanisms to detect security related anomalies and threats in a computer network environment. The security platform is “big data” driven and employs machine learning to perform security analytics. The security platform performs user/entity behavioral analytics (UEBA) to detect the security related anomalies and threats, regardless of whether such anomalies/threats were previously known. The security platform can include both real-time and batch paths/modes for detecting anomalies and threats. By visually presenting analytical results scored with risk ratings and supporting evidence, the security platform enables network security administrators to respond to a detected anomaly or threat, and to take action promptly.

Claims

1. A method comprising:

detecting, by a computer system, outgoing traffic from a computer device, the outgoing traffic being indicative of a plurality of connection requests and being user-generated traffic or machine generated traffic;

forming, by the computer system, a plurality of groups of the traffic indicative of the plurality of connection requests;

determining, by the computer system, whether a particular group of the groups corresponds to user-generated traffic or machine generated traffic based on a periodicity of traffic indicative of connection requests determined as a function of a timing between traffic indicative of connection requests in the particular group and a plurality of connection parameters associated with the traffic indicative of connection requests in the particular group; and

responsive to a determination that the particular group is machine generated traffic, determining, by the computer system, whether the particular group of traffic indicative of connection requests represents an anomaly based on a frequency at which the particular group of traffic indicative of connection requests has occurred for the outgoing traffic.

2. The method of claim 1, wherein forming the groups of the traffic indicative of the plurality of connection requests is done as part of execution of a machine learning model executing at the computer system.

3. The method of claim 1, wherein determining whether a particular group corresponds to user-generated traffic or machine generated traffic is performed as part of execution of a machine learning model executing at the computer system.

4. The method of claim 1, wherein determining whether the particular group represents an anomaly is performed as part of execution of a machine learning model executing at the computer system.

5. The method of claim 1, wherein detecting the outgoing traffic from the computer device includes detecting the outgoing traffic as IP traffic.

6. The method of claim 1, wherein detecting the outgoing traffic from the computer device includes detecting the outgoing traffic as web traffic.

7. The method of claim 1, wherein forming the plurality of groups of the traffic indicative of the plurality of connection requests includes:

establishing a first group of the groups upon receiving a first traffic indicative of a connection request of the connections, and

adding traffic indicative of connection requests that are received within a particular period from the first traffic indicative of a connection request to the first group.

8. The method of claim 1, wherein determining whether the particular group of the traffic indicative of connection requests is user-generated traffic or machine generated traffic occurs in real time.

9. The method of claim 1, wherein determining whether the particular group is user-generated traffic or machine generated traffic includes:

determining whether traffic indicative of connection requests in the particular group are to a destination that is included in a list of acceptable destinations, and

excluding the particular group from the determining of whether the particular group is user-generated traffic or machine generated traffic when the destination is included in the list.

10. The method of claim 1, wherein determining whether the particular group is user-generated traffic or machine generated traffic includes:

determining whether traffic indicative of connection requests in the particular group are likely to be user-generated traffic, and

excluding the particular group from the determining of whether the particular group represents an anomaly when the traffic indicative of connection requests in the particular group are likely to be user-generated traffic.

11. The method of claim 1, wherein determining whether the particular group is user-generated traffic or machine generated traffic includes:

determining that the particular group is likely to be machine-generated traffic when the timing between the traffic indicative of connection requests in the particular group satisfies a specified criterion for periodicity.

12. The method of claim 1, wherein determining whether the particular group is user-generated traffic or machine generated traffic includes:

determining whether the particular group is user-generated traffic or machine generated traffic as a function of a number of destinations of the traffic indicative of connection requests within a particular period in the particular group.

13. The method of claim 1, wherein determining whether the particular group is user-generated traffic or machine generated traffic includes:

determining that the particular group is likely to be machine-generated traffic when a number of destinations of the traffic indicative of connection requests within a particular period is less than a particular threshold number of destinations.

14. The method of claim 1, wherein determining whether the particular group is user-generated traffic or machine generated traffic includes:

determining based on the connection parameters including at least one of a plurality of IP addresses of the traffic indicative of connection requests in the particular group, a number of web objects requested by the traffic indicative of connection requests in the particular group or a number of ports of the traffic indicative of connection requests in the particular group.

15. The method of claim 1, wherein determining whether the particular group is user-generated traffic or machine generated traffic includes:

determining that the particular group is likely to be user-generated traffic when at least one of a diversity of IP addresses of the traffic indicative of connection requests in the particular group exceeds a first threshold, a number of web objects requested by the traffic indicative of connection requests in the particular group exceeds a second threshold or a number of ports of the connections exceeds a third threshold; and

otherwise determining that the particular group is likely to be machine-generated traffic.

16. The method of claim 1, wherein determining whether the particular group is user-generated traffic or machine generated traffic is done based on a diversity of IP addresses of the traffic indicative of connection requests in the particular group.

17. The method of claim 1, wherein determining whether the particular group is user-generated traffic or machine generated traffic includes:

determining that the particular group is likely to be machine-generated traffic when a diversity of IP addresses of the traffic indicative of connection requests in the particular group is less than a particular threshold.

18. The method of claim 1, wherein determining whether the particular group is user-generated traffic or machine generated traffic is performed as a function of a number of web objects requested by the traffic indicative of connection requests in the particular group.

19. The method of claim 1, wherein determining whether the particular group is user-generated traffic or machine generated traffic includes:

determining that the particular group is likely to be machine-generated traffic when a number of web objects requested by the traffic indicative of connection requests in the particular group is less than a particular threshold.

20. The method of claim 1, wherein determining whether the particular group is user-generated traffic or machine generated traffic is performed as a function of a number of ports of the traffic indicative of connection requests in the particular group.

21. The method of claim 1, wherein determining whether the particular group is user-generated traffic or machine generated traffic includes:

determining that the particular group is likely to be machine-generated traffic when a number of ports of connections is less than a particular threshold.

22. The method of claim 1, wherein determining whether the particular group of traffic indicative of connection requests represents an anomaly includes:

determining that the particular group is an anomaly when the particular group or a group that is determined to be similar to the particular group occurs a threshold number of times in the outgoing traffic and satisfies a specified criterion for the frequency.

23. The method of claim 1, wherein determining whether the particular group of traffic indicative of connection requests represents an anomaly includes:

determining that the particular group is an anomaly when the particular group or a group that is determined to be similar to the particular group occurs (a) a first threshold number of times in the outgoing traffic and satisfies a specified criterion for the frequency or (b) a second threshold number of times if specified criterion for the frequency is not satisfied, the first threshold number being different from the second threshold number.

24. The method of claim 1, wherein determining whether the particular group of traffic indicative of connection requests represents an anomaly includes:

determining whether the particular group is similar to a group of traffic indicative of connection requests that is identified as likely to be an anomaly based on at least one of an IP address of a destination of traffic indicative of a connection request in the particular group, a URI of the destination, or a type of a web request.

25. The method of claim 1, wherein determining whether the particular group of traffic indicative of connection requests represents an anomaly includes:

identifying, based on a plurality of group parameters, a group type that contains a set of groups similar to the particular group, the set of groups being identified as likely to be an anomaly, the group type being one of a plurality of group types, and

determining the particular group as an anomaly based on a number of times and a particular frequency at which the set of groups occurred in the outgoing traffic.

26. The method of claim 1, wherein determining whether the particular group of traffic indicative of connection requests represents an anomaly includes:

comparing the particular group with a plurality of group types stored in a memory cache of the computer system based on a plurality of group parameters, the group types including groups of traffic indicative of connection requests identified as likely to be an anomaly,

identifying a first group type of the group types that is similar to the particular group, and

determining the particular group as an anomaly when groups in the first group type occurred at least a particular threshold number of times satisfying a specified criterion for the frequency.

27. A non-transitory computer-readable storage medium storing computer-readable instructions, the instructions comprising:

instructions for detecting, by a computer system, outgoing traffic from a computer device, the outgoing traffic being indicative of a plurality of connection requests and being user-generated traffic or machine generated traffic;

instructions for forming, by the computer system, a plurality of groups of the traffic indicative of the plurality of connection requests;

instructions for determining, by the computer system, whether a particular group of the groups corresponds to user generated traffic or machine generated traffic based on a periodicity of traffic indicative of connection requests determined as a function of a timing between traffic indicative of connection requests in the particular group and a plurality of connection parameters associated with the traffic indicative of connection requests in the particular group; and

instructions for determining, by the computer system and responsive to a determination that the particular group is machine generated traffic, whether the particular group of traffic indicative of connection requests represents an anomaly based on a frequency at which the particular group of traffic indicative of connection requests has occurred for the outgoing traffic.

28. The computer-readable storage medium of claim 27, wherein the instructions for determining whether the particular group is user-generated traffic or machine generated traffic include:

instructions for determining that the particular group is likely to be machine-generated traffic when the timing between the traffic indicative of connection requests in the particular group is periodic.

29. The computer-readable storage medium of claim 27, wherein the instructions for determining whether the particular group is user-generated traffic or machine generated traffic further include:

instructions for determining that the particular group is likely to be user-generated traffic when at least one of a diversity of IP addresses of the traffic indicative of connection requests in the particular group exceeds a first threshold, a number of web objects requested by the traffic indicative of connection requests in the particular group exceeds a second threshold or a number of ports of the connections exceeds a third threshold else determining that the particular group is likely to be machine-generated traffic.

30. A system, comprising:

a processor;

a memory having instructions executable by the processor to cause the system to;

detect outgoing traffic from a computer device, the outgoing traffic being indicative of a plurality of connection requests and being user-generated traffic or machine generated traffic;

form a plurality of groups of the traffic indicative of the plurality of connection requests;

determine whether a particular group of the groups corresponds to user-generated traffic or machine generated traffic based on a periodicity of traffic indicative of connection requests determined as a function of a timing between traffic indicative of connection requests in the particular group and a plurality of connection parameters associated with the traffic indicative of connection requests in the particular group; and

determine, responsive to a determination that the particular group is machine generated traffic, whether the particular group of traffic indicative of connection requests represents an anomaly based on a frequency at which the particular group of traffic indicative of connection requests has occurred for the outgoing traffic.