Anomaly detection based on connection requests in network traffic

• US 10,587,633 B2
• Filed: 07/31/2018
• Issued: 03/10/2020
• Est. Priority Date: 08/31/2015
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Muddu, Sudhakar, Tryfonas, Christos, Iliofotou, Marios
• Primary Examiner(s): Dada, Beemnet W
• Application Number:
  US16/050,368
• Publication Number:
  US 20180367551A1
• Time in Patent Office: 588 Days
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 10/225 based on a marking or identifier characterising the area
• CPC Class Codes: H04L 2463/121 Timestamp cryptographic mechanisms or cryptographic arrangements involving time stamps H04L9/3297
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/145 involving simulating, designing, planning or modelling of a network
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 43/00 : Arrangements for monitoring or testing data switching networks
• CPC Class Codes: H04L 43/045 : for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/062 : related to network traffic
• CPC Class Codes: H04L 43/08 : Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 63/06 : for supporting key management in a packet data network cryptographic mechanisms or cryptographic arrangements for key management H04L9/08
• CPC Class Codes: H04L 63/1408 : by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1416 : Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 : Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1433 : Vulnerability analysis
• CPC Class Codes: H04L 63/1441 : Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/20 : for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

The disclosed embodiments include a method performed by a computer system. The method includes forming groups of traffic, where each group includes a subset of detected connection requests. The method further includes determining a periodicity of connection requests for each group, identifying a particular group based on whether the periodicity of connection requests of the particular group satisfies a periodicity criterion, determining a frequency of the particular group in the traffic, and identifying the particular group as an anomaly based on whether the frequency of the particular group satisfies a frequency criterion.

Claims

1. A method comprising:

forming, by a computer system, groups of traffic, traffic forming the groups of traffic including connection requests;

determining, by the computer system, a periodicity of a set of connection requests included in each group of the groups of traffic;

identifying, by the computer system, a particular group of the groups of traffic based on whether the periodicity of a particular set of connection requests in the particular group satisfies a periodicity criterion, wherein the periodicity criterion includes a timing of a regular occurrence between requests in a set of connection requests in a group from the groups of traffic;

determining, by the computer system, a frequency of a set of the groups of traffic, the set of groups of traffic including the particular group; and

identifying, by the computer system, the particular group as corresponding to an anomaly based on whether the frequency of the particular group satisfies a frequency criterion.

2. The method of claim 1 further comprising, prior to forming the groups of traffic:

detecting, by the computer system, the connection requests in the traffic, wherein the connection requests are communicated by a computer device.

3. The method of claim 1, further comprising:

identifying, by the computer system, a second group of the groups of traffic that satisfies the periodicity criterion as corresponding to machine-generated traffic, wherein identifying the particular group is further based on the second group corresponding to machine-generated traffic.

4. The method of claim 1, further comprising:

identifying, by the computer system, a second group of the groups of traffic that does not satisfy the periodicity criterion as corresponding to user-generated traffic, wherein identifying the particular group is further based on the second group corresponding to user-generated traffic.

5. The method of claim 1, wherein identifying the particular group is further based on:

determining, by the computer system, whether each group of the groups of traffic corresponds to either machine-generated traffic or user-generated traffic based on whether the group satisfies the periodicity criterion.

6. The method of claim 1, further comprising:

determining, by the computer system, whether each group of the groups of traffic corresponds to either machine-generated traffic or user-generated traffic in accordance with a machine learning process.

7. The method of claim 1, further comprising:

determining, by the computer system, whether each group of the groups of traffic corresponds to either machine-generated traffic or user-generated traffic in real-time or near real-time.

8. The method of claim 1 further comprising, prior to forming a plurality of groups of traffic:

detecting, by the computer system, that the traffic is Internet Protocol (IP) traffic.

9. The method of claim 1 further comprising, prior to forming the groups of traffic:

detecting, by the computer system, that the traffic is web traffic.

10. The method of claim 1, wherein forming the plurality of groups further comprises:

establishing, by the computer system, a first group of the groups of traffic upon receiving a first connection request; and

adding any connection request obtained within a period following the first connection request to the first group.

11. The method of claim 1, further comprising:

determining, by the computer system, whether a connection request included in a first group of the groups of traffic indicates a destination included in a list of acceptable destinations; and

excluding the first group determining whether the first group corresponds to machine-generated traffic or user-generated traffic.

12. The method of claim 1, further comprising:

determining, by the computer system, whether a connection request in a first group of the groups of traffic is likely user-generated traffic; and

excluding the first group from determining whether the first group is machine-generated traffic.

13. The method of claim 1, further comprising:

determining, by the computer system, whether a first group of the groups of traffic corresponds to either machine-generated traffic or user-generated traffic based on a destination associated traffic included in the first group, wherein the destination is detected within a period associated with the first group.

14. The method of claim 1, further comprising:

determining, by the computer system, whether a first group of the groups of traffic corresponds to either machine-generated traffic or user-generated traffic based on whether a quantity of destinations associated with the first group satisfies a threshold.

15. The method of claim 1, further comprising:

determining, by the computer system, whether a first group of the groups of traffic corresponds to either machine-generated traffic or user-generated traffic based on whether the first group has connection parameters that includes at least one of a plurality of IP addresses, a number of web object requests, or a number of port requests.

16. The method of claim 1, further comprising:

determining, by the computer system, that a first group of the groups of traffic is likely user-generated traffic when a number of connection parameters of the first group exceeds a threshold, wherein the connection parameters are diverse IP addresses, web object requests, or port requests; and

otherwise determining that a group is likely machine-generated traffic.

17. The method of claim 1, further comprising:

determining, by the computer system, whether a first group corresponds to either machine-generated traffic or user-generated traffic based on a diversity of IP addresses associated with a first set of connection requests included in the first group.

18. The method of claim 1, further comprising:

determining, by the computer system, that a first group of the groups of traffic likely corresponds to machine-generated traffic when a diversity of IP addresses associated with a first set of connection requests included in the first group is less than a threshold.

19. The method of claim 1, further comprising:

determining, by the computer system, that a first group of the groups of traffic likely corresponds to machine-generated traffic based on a web object request associated with the first group.

20. The method of claim 1, further comprising:

determining, by the computer system, that a first group of the groups of traffic likely corresponds to machine-generated traffic when a quantity of web object requests associated with the first group is less than a threshold.

21. The method of claim 1, further comprising:

determining, by the computer system, that a first group of the groups of traffic likely corresponds to machine-generated traffic based on port requests associated with the first group.

22. The method of claim 1, further comprising:

determining, by the computer system, that a first group of the groups of traffic likely corresponds to machine-generated traffic when a quantity of port requests associated with the first group is less than a threshold quantity.

23. The method of claim 1, wherein identifying the particular group as corresponding to an anomaly is further based on:

determining, by the computer system, that the particular group occurs a threshold number of times that satisfies the frequency criterion.

24. The method of claim 1, wherein identifying the particular group as corresponding to an anomaly is further based on:

determining, by the computer system, that the particular group (a) satisfies the frequency criterion and occurs a first threshold number of times in the traffic or (b) does not satisfy the frequency criterion and occurs a second threshold number of times in the traffic, the first threshold being different from the second threshold.

25. The method of claim 1, wherein identifying the particular group as corresponding to an anomaly is further based on:

determining, by the computer system, that is similar to second group of the groups of traffic, wherein the second group is identified as a likely anomaly based on at least one of a destination IP address of a connection request included in the second group, a destination URI of a connection request in the second group, or a type of a web request of a connection request in the second group.

26. The method of claim 1, wherein identifying the particular group as corresponding to an anomaly is further based on:

determining, by the computer system, that a type parameter of the particular group matches a type parameter of a set of groups of the groups of traffic identified as likely an anomaly.

27. The method of claim 1, further comprising:

comparing a type associated with the particular group with group types stored in a memory cache of the computer system, wherein the type is based on group parameters, the group types being indicative of connection requests identified as likely anomalies;

identifying a first group type of the group types that is similar to the type of associated with the particular group; and

determining, by the computer system, that the particular group is an anomaly when the groups of traffic include one or more groups having the first group type and a quantity of the one or more groups is greater than or equal to a threshold number of times and the one or more groups satisfying the frequency criterion.

28. A non-transitory computer-readable storage medium storing computer-readable instructions, the instructions comprising:

instructions for forming, by a computer system, groups of traffic, traffic forming the groups of traffic including connection requests;

instructions for determining, by the computer system, a periodicity of a set of connection requests included in each of group of the groups of traffic;

instructions for identifying, by the computer system, a particular group of the groups of traffic based on whether the periodicity of a particular set of connection requests of the particular group satisfies a periodicity criterion, wherein the periodicity criterion includes a timing of a regular occurrence between requests in a set of connection requests in a group from the groups of traffic;

instructions for determining, by the computer system, a frequency of a set of the groups of traffic, the set of groups of traffic including the particular group;

instructions for identifying, by the computer system, the particular group as corresponding to an anomaly based on whether the frequency of the particular group satisfies a frequency criterion.

29. A system, comprising:

a processor; and

a memory having instructions executable by the processor to cause the system to;

form groups of traffic, traffic forming the groups of traffic including connection requests;

determine a periodicity of a set of connection requests included in each group of the groups of traffic;

identify a particular group of the groups of traffic based on whether the periodicity of a particular set of connection requests in the particular group satisfies a periodicity criterion, wherein the periodicity criterion includes a timing of a regular occurrence between requests in a set of connection requests in a group from the groups of traffic;

determine a frequency of a set of the groups of traffic, the set of groups of traffic including the particular group; and

identify the particular group as corresponding to an anomaly based on whether the frequency of the particular group satisfies a frequency criterion.

30. The system of claim 29 being further caused to, prior to forming the groups of traffic:

detecting, by the computer system, the connection requests in the traffic, wherein the connection requests are communicated by a computer device.