Network anomaly detection

• US 10,735,448 B2
• Filed: 08/22/2018
• Issued: 08/04/2020
• Est. Priority Date: 06/26/2015
• Status: Active Grant

• Current Assignee: Palantir Technologies Incorporated
• Original Assignee: Palantir Technologies Incorporated
• Inventors: Kesin, Maxim, Jones, Samuel
• Primary Examiner(s): Tran, Ellen
• Application Number:
  US16/109,379
• Publication Number:
  US 20190007441A1
• Time in Patent Office: 713 Days
• Field of Search: 726 23
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: H04L 2463/143 Denial of service attacks involving systematic or selective dropping of packets
• CPC Class Codes: H04L 61/5007 Internet protocol [IP] addresses
• CPC Class Codes: H04L 63/083 using passwords cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226
• CPC Class Codes: H04L 63/12 Applying verification of the received information cryptographic mechanisms or cryptographic arrangements for data integrity or data verification H04L9/32
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

A security system detects anomalous activity in a network. The system logs user activity, which can include ports used, compares users to find similar users, sorts similar users into cohorts, and compares new user activity to logged behavior of the cohort. The comparison can include a divergence calculation. Origins of user activity can also be used to determine anomalous network activity. The hostname, username, IP address, and timestamp can be used to calculate aggregate scores and convoluted scores.

Claims

1. A computer-implemented method for detecting an anomalous activity in a network, the method being implemented by one or more computer readable storage devices configured to store computer executable instructions, and by one or more hardware computer processors in communication with the one or more computer readable storage devices configured to execute the computer executable instructions, the method comprising:

logging, to the one or more computer readable storage devices, user activity for a plurality of users in the network;

sorting the plurality of users into a plurality of cohorts;

detecting a new activity by a first user of the plurality of users, wherein the first user is sorted into a first cohort of the plurality of cohorts;

determining a geographic region from which the new activity originated;

determining attack origin distribution data, wherein the attack origin distribution data includes statistical information of network attacks originating in a plurality of countries, and wherein the attack origin distribution data further includes at least a first probability that network attacks originate from the geographic region;

determining network activity origin distribution data, wherein the network activity origin distribution data is based on an analysis of origins of network activity over a period of time, and wherein the network activity origin distribution data further includes at least a second probability of network activity originating from the geographic region, wherein the network activity includes both malicious and non-malicious traffic;

determining a third probability of a network attack;

generating a statistical probability that the new activity is the network attack based at least in part on a combination of the first probability, the second probability, and the third probability; and

generating an indicator of a potential anomaly for display based at least in part on the statistical probability.

2. The method of claim 1, wherein determining the statistical probability that the new activity is the network attack is further based, at least in part, on attack origin distribution data indicating geographic origin distribution statistics about a plurality of geographic regions where known network attacks have originated.

3. The method of claim 2, further comprising:

receiving the attack origin distribution data for the plurality of countries; and

interpolating attack origin distribution data for a country not in the plurality of countries.

4. The method of claim 1, further comprising:

comparing the geographic region to geographic origins of logged activity of the first user to generate a second comparison result; and

comparing the geographic region of the new activity to geographic origins of logged activity of the first cohort to generate a third comparison result;

wherein generating the indicator of the potential anomaly is further based, at least in part, on the second comparison result and the third comparison result.

5. The method of claim 1, wherein the statistical probability exceeds a threshold probability, the method further comprising:

taking one or more initial network security measures in response to the statistical probability exceeding the threshold;

receiving a user confirmation that the new activity is anomalous; and

in response to receiving the user confirmation, taking one or more additional network security measures that are different from the one or more initial network security measures.

6. The method of claim 1, wherein the second probability of the network activity originating from the geographic region is a fraction of the user activity from users of the first cohort that originated from the geographic region.

7. The computer-implemented method of claim 1, wherein the combination of the first probability, the second probability, and the third probability is a result of a mathematical operation including the first probability, the second probability, and the third probability.

8. The computer-implemented method of claim 1, wherein the combination of the first probability, the second probability, and the third probability includes the first probability divided by the second probability.

9. A computer system comprising:

one or more computer readable storage devices configured to store computer executable instructions; and

one or more hardware computer processors configured to execute the computer executable instructions in order to cause the computer system to;

log, to one or more computer readable storage devices, user activity for a plurality of users in a network;

sort the plurality of users into a plurality of cohorts;

detect a new activity by a first user of the plurality of users, wherein the first user is sorted into a first cohort of the plurality of cohorts;

determine a geographic region from which the new activity originated;

determine attack origin distribution data, wherein the attack origin distribution data includes statistical information of network attacks originating in a plurality of countries, and wherein the attack origin distribution data further includes at least a first probability that network attacks originate from the geographic region; and

determine network activity origin distribution data, wherein the network activity origin distribution data is based on an analysis of origins of network activity over a period of time, and wherein the network activity origin distribution data further includes at least a second probability that network activity through the network originated from the geographic region, wherein the network activity includes both malicious and non-malicious traffic;

determine a third probability of a network attack;

generate, a statistical probability that the new activity is the network attack based at least in part on a combination of the first probability, the second probability, and the third probability; and

generate an indicator of a potential anomaly for display based at least in part on the statistical probability.

10. The computer system of claim 9, wherein determining the statistical probability that the new activity is the network attack is further based, at least in part, on attack origin distribution data indicating geographic origin distribution statistics about a plurality of geographic regions where known network attacks have originated.

11. The computer system of claim 10, wherein the one or more hardware processors are further configured to cause the computer system to:

receive the attack origin distribution data for the plurality of countries; and

interpolate attack origin distribution data for a country not in the plurality of countries.

12. The computer system of claim 9, wherein the one or more hardware processors are further configured to cause the computer system to:

compare the geographic region to geographic origins of logged activity of the first user to generate a second comparison result; and

compare the geographic region to geographic origins of logged activity of the first cohort to generate a third comparison result;

wherein generating the indicator of the potential anomaly is further based, at least in part, on the second comparison result and the third comparison result.

13. The computer system of claim 9, wherein the statistical probability exceeds a threshold probability, and wherein the one or more hardware processors are further configured to cause the computer system to:

take one or more initial network security measures in response to the statistical probability exceeding the threshold;

receive a user confirmation that the new activity is anomalous; and

in response to receiving the user confirmation, take one or more additional network security measures that are different from the one or more initial network security measures.

14. The computer system of claim 9, wherein the second probability of the network activity originating from the geographic region is a fraction of the user activity from users of the first cohort that originated from the geographic region.

15. A method for detecting anomalous network activity, the method comprising:

logging user activity for a plurality of users through a network;

sorting the plurality of users into a plurality of cohorts;

detecting a new activity by a first user of a first plurality of users, wherein the first user is sorted into a first cohort of the plurality of cohorts;

determining attack origin distribution data, wherein the attack origin distribution data includes statistical information of network attacks originating in a plurality of countries, and wherein the attack origin distribution data further includes at least a first probability of network attacks originate from the geographic region;

determining network activity origin distribution data, wherein the network activity origin distribution data is based on an analysis of origins of network activity over a period of time, and wherein the network activity origin distribution data further includes at least a second probability of network activity originating from the geographic region, wherein the network activity includes both malicious and non-malicious traffic;

determining a third probability of a network attack;

generating a statistical probability that the new activity is the network attack based at least in part on a combination of the first probability, the second probability, and the third probability; and

generating an indicator of a potential anomaly for display based at least in part on the statistical probability.

16. The method of claim 15, wherein determining the statistical probability that the new activity is the network attack is further based, at least in part, on attack origin distribution data indicating geographic origin distribution statistics of network activity that are known to be network attacks.

17. The method of claim 16, further comprising:

receiving the attack origin distribution data for the plurality of countries; and

interpolating attack origin distribution data for a country not in the plurality of countries.

18. The method of claim 15, further comprising:

comparing the geographic region to geographic origins of logged activity of the first user to generate a second comparison result; and

comparing the geographic region to geographic origins of logged activity of the first cohort to generate a second comparison result;

wherein generating the indicator of the potential anomaly is further based, at least in part, on the second comparison result and a third comparison result.

19. The method of claim 15, wherein the statistical probability exceeds a threshold probability, the method further comprising:

taking one or more initial network security measures in response to the statistical probability exceeding the threshold;

receiving a user confirmation that the new activity is anomalous; and

in response to receiving the user confirmation, taking one or more additional network security measures that are different from the one or more initial network security measures.