Anomaly detection using adaptive behavioral profiles

• US 9,544,321 B2
• Filed: 07/28/2015
• Issued: 01/10/2017
• Est. Priority Date: 01/30/2015
• Status: Active Grant

• Current Assignee: Securonix, Inc.
• Original Assignee: Securonix, Inc.
• Inventors: Baikalov, Igor A., Gulati, Tanuj, Nayyar, Sachin, Shenoy, Anjaneya, Patwardhan, Ganpatrao H.
• Primary Examiner(s): Patel, Haresh N
• Application Number:
  US14/811,732
• Publication Number:
  US 20160226901A1
• Time in Patent Office: 532 Days
• Field of Search: 726 20- 25
• US Class Current: 1/1
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1433 Vulnerability analysis

Abstract

Anomalous activities in a computer network are detected using adaptive behavioral profiles that are created by measuring at a plurality of points and over a period of time observables corresponding to behavioral indicators related to an activity. Normal kernel distributions are created about each point, and the behavioral profiles are created automatically by combining the distributions using the measured values and a Gaussian kernel density estimation process that estimates values between measurement points. Behavioral profiles are adapted periodically using data aging to de-emphasize older data in favor of current data. The process creates behavioral profiles without regard to the data distribution. An anomaly probability profile is created as a normalized inverse of the behavioral profile, and is used to determine the probability that a behavior indicator is indicative of a threat. The anomaly detection process has a low false positive rate.

Claims

1. A method of automated detection of anomalous activities in a computer network of an organization comprising:

measuring, at a plurality of points, values of observables corresponding to behavioral indicators related to an activity over a predetermined period of time;

forming distributions of estimated values about said plurality of points based upon said measured values of observables at said points;

creating a behavioral profile for each of said behavioral indicators over a range of points by combining said distributions and said measured values using a kernel density estimation process, wherein said creating a behavioral profile using the kernel density estimation process comprises selecting a kernel bandwidth based upon the type of data being measured;

forming an anomaly probability based upon a normalized inverse of said behavioral profile;

determining a probability that a behavioral indicator that deviates from said behavioral profile for said behavioral indicator by more than a predetermined amount is an anomaly by comparing said behavioral indicator to said anomaly probability; and

identifying that said activity is an anomaly when said determined probability exceeds a predetermined threshold.

2. The method of claim 1 further comprising aggregating in groups entities associated said activity that have similar functions, and wherein said measuring comprises measuring observables associated with said grouped entities.

3. The method of claim 2 wherein said aggregating comprises identifying as being the same entity entities that have multiple identities or aliases.

4. The method of claim 1, wherein said forming distributions of estimated values comprises forming kernels of distributed values about said points.

5. The method of claim 1, wherein creating said behavioral profile comprises summing said distributions, and wherein said kernel density estimation process comprises using a Gaussian kernel density function to estimate values between measurement points to create said behavioral profile.

6. The method of claim 5, wherein said anomaly probability is created by a Lorentz function of said Gaussian kernel density function.

7. The method of claim 1, wherein selecting a kernel bandwidth comprises selecting for data having an unbounded range a bandwidth that incrementally increases with the value of a measurement point, and selecting for data with a bounded range a bandwidth that is constant and based upon the actual data distribution.

8. The method of claim 1 further comprising periodically adapting said behavioral profile using aging to deemphasize older data, said aging comprising reducing values of a previous behavioral profile by a predetermined decay factor before updating the behavioral profile with current data.

9. The method of claim 8 further comprising using measured observable values that have a probability less than said predetermined threshold to update said behavioral profile, and characterizing as outliers for threat analysis values having a probability greater than said predetermined value.

10. A non-transitory computer readable medium embodying executable instructions for controlling a computer to perform automated detection of anomalous activities in a computer network comprising;

measuring, at a plurality of points, values of observables corresponding to behavioral indicators related to an activity over a predetermined period of time, wherein said creating a behavioral profile using the kernel density estimation process comprises selecting a kernel bandwidth based upon the type of data being measured;

forming distributions of estimated values about said plurality of points based upon said measured values of observables at said points;

creating a behavioral profile for each of said behavioral indicators over a range of points by combining said distributions and said measured values using a kernel density estimation process forming an anomaly probability based upon a normalized inverse of said behavioral profile;

determining a probability that a behavioral indicator that deviates from said behavioral profile for said behavioral indicator by more than a predetermined amount is an anomaly by comparing said behavioral indicator to said anomaly probability; and

identifying that said activity is an anomaly when said determined probability exceeds a predetermined threshold.

11. The non-transitory medium of claim 10 further comprising aggregating in groups entities associated said activity that have similar functions, and wherein said measuring comprises measuring observables associated with said grouped entities.

12. The non-transitory medium of claim 10, wherein said forming distributions of estimated values comprises forming kernels of distributed values about said points.

13. The non-transitory medium of claim 10, wherein creating said behavioral profile comprises summing said distributions, and wherein said kernel density estimation process comprises using a Gaussian kernel density function to estimate values between measurement points to create said behavioral profile.

14. The non-transitory medium of claim 13, wherein said anomaly probability is created by a Lorentz function of said Gaussian kernel density function.

15. The non-transitory medium of claim 10, wherein selecting a kernel bandwidth comprises selecting for data having an unbounded range a bandwidth that incrementally increases with the value of a measurement point, and selecting for data with a bounded range a bandwidth that is constant and based upon the actual data distribution.

16. The non-transitory medium of claim 10 further comprising periodically adapting said behavioral profile using aging to deemphasize older data, said aging comprising reducing values of a previous behavioral profile by a predetermined decay factor before updating the behavioral profile with current data.

17. The non-transitory medium of claim 10 further comprising using measured observable values that have a probability less than said predetermined threshold to update said behavioral profile, and characterizing as outliers for threat analysis values having a probability greater than said predetermined value.