System, method, and computer program for detection of anomalous user network activity based on multiple data sources

• US 10,645,109 B1
• Filed: 03/29/2018
• Issued: 05/05/2020
• Est. Priority Date: 03/31/2017
• Status: Active Grant

• Current Assignee: Exabeam, Inc.
• Original Assignee: Exabeam, Inc.
• Inventors: Lin, Derek, Hu, Qiaona, Mihovilovic, Domingo, Gil, Sylvain, Steiman, Barry
• Primary Examiner(s): Pyzocha, Michael
• Application Number:
  US15/940,673
• Time in Patent Office: 768 Days
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 21/554 involving event detection and direct action
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 5/047 Pattern matching networks; Rete networks
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

The present disclosure relates a system, method, and computer program for detecting anomalous user network activity based on multiple data sources. The system extracts user event data for n days from multiple data sources to create a baseline behavior model that reflects the user'"'"'s daily volume and type of IT events. In creating the model, the system addresses data heterogeneity in multi-source logs by categorizing raw events into meta events. Thus, baseline behavior model captures the user'"'"'s daily meta-event pattern and volume of IT meta events over n days. The model is created using a dimension reduction technique. The system detects any anomalous pattern and volume changes in a user'"'"'s IT behavior on day n by comparing user meta-event activity on day n to the baseline behavior model. A score normalization scheme allows identification of a global threshold to flag current anomalous activity in the user population.

Claims

1. A method, performed by a computer system, for detecting anomalous IT pattern and volume event behavior for a user during a period of time based on multiple data sources, the method comprising:

creating a baseline behavior model P that captures a user'"'"'s daily pattern and volume of IT meta events over n days based on multiple data sources, wherein creating the baseline behavior model comprises;

receiving raw event logs from multiple data sources for a period of n days from days 0 to n−

1;

categorizing raw event logs into meta events using an event taxonomy;

for each of the days 0 to n−

1, creating a vector with a weighted count of each unique meta event observed that day;

creating a matrix, M, with the vectors for days 0 to n−

1; and

modeling the data in the matrix (M) from day 0 to day n−

1 using a dimension reduction technique to create the resulting baseline behavior model P;

determining whether there are anomalous pattern and volume changes in a user'"'"'s IT behavior on day n using the baseline behavior model P, wherein the determining step comprises;

creating a vector, f_n, with a weighted count of each unique meta event observed on day n;

scoring the activity vector f_n by measuring the magnitude of its reconstruction error as the difference between f_n and f_nPP^T;

normalizing the reconstruction error; and

comparing the normalized reconstruction error to an anomaly threshold;

in response to the normalized reconstruction error satisfying the anomaly threshold, concluding that the user'"'"'s meta event behavior on day n is anomalous and elevating a risk assessment associated with the user'"'"'s IT activities on day n; and

in response to the normalized reconstruction error not satisfying the anomaly threshold, updating the baseline behavior model with the user'"'"'s meta event activity from day n.

2. The method of claim 1, wherein the dimension reduction technique is Principal Component Analysis (PCA).

3. The method of claim 2, wherein the baseline behavior model P is the top K eigenvectors of M^TM.

4. The method of claim 3, wherein normalizing the reconstruction error comprises dividing the reconstruction error by the sum of all eigenvalues in the baseline behavior model P.

5. The method of claim 1, wherein elevating the risk assessment comprises adding points to a risk score for the user'"'"'s logon session.

6. The method of claim 1, wherein the events include a plurality of the following:

log-on events, account-creation events, account-deletion events, account-password-change events, and events relates to access of machines, documents, and applications.

7. The method of claim 1, wherein the threshold is set so that the normalized reconstruction error is above the threshold for between 0.4% and 0.6% of users in the network.

8. A non-transitory computer-readable medium comprising a computer program, that, when executed by a computer system, enables the computer system to perform the following method for detecting anomalous IT pattern and volume event behavior for a user during a period of time based on multiple data sources, the method comprising:

creating a baseline behavior model P that captures a user'"'"'s daily pattern and volume of IT meta events over n days based on multiple data sources, wherein creating the baseline behavior model comprises;

receiving raw event logs from multiple data sources for a period of n days from days 0 to n−

1;

categorizing raw event logs into meta events using an event taxonomy;

for each of the days 0 to n−

1, creating a vector with a weighted count of each unique meta event observed that day;

creating a matrix, M, with the vectors for days 0 to n−

1; and

modeling the data in the matrix (M) from day 0 to day n−

1 using a dimension reduction technique to create the resulting baseline behavior model P;

determining whether there are anomalous pattern and volume changes in a user'"'"'s IT behavior on day n using the baseline behavior model P, wherein the determining step comprises;

creating a vector, f_n, with a weighted count of each unique meta event observed on day n;

scoring the activity vector f_n by measuring the magnitude of its reconstruction error as the difference between f_n and f_nPP^T;

normalizing the reconstruction error; and

comparing the normalized reconstruction error to an anomaly threshold;

in response to the normalized reconstruction error satisfying the anomaly threshold, concluding that the user'"'"'s meta event behavior on day n is anomalous and elevating a risk assessment associated with the user'"'"'s IT activities on day n; and

in response to the normalized reconstruction error not satisfying the anomaly threshold, updating the baseline behavior model with the user'"'"'s meta event activity from day n.

9. The non-transitory computer-readable medium of claim 8, wherein the dimension reduction technique is Principal Component Analysis (PCA).

10. The non-transitory computer-readable medium of claim 9, wherein the baseline behavior model P is the top K eigenvectors of M^TM.

11. The non-transitory computer-readable medium of claim 10, wherein normalizing the reconstruction error comprises dividing the reconstruction error by the sum of all eigenvalues in the baseline behavior model P.

12. The non-transitory computer-readable medium of claim 8, wherein elevating the risk assessment comprises adding points to a risk score for the user'"'"'s logon session.

13. The non-transitory computer-readable medium of claim 8, wherein the events include a plurality of the following:

log-on events, account-creation events, account-deletion events, account-password-change events, and events relates to access of machines, documents, and applications.

14. The non-transitory computer-readable medium of claim 8, wherein the threshold is set so that the normalized reconstruction error is above the threshold for between 0.4% and 0.6% of users in the network.

15. A computer system for detecting anomalous IT pattern and volume event behavior for a user during a period of time based on multiple data sources the system comprising:

one or more processors;

one or more memory units coupled to the one or more processors, wherein the one or more memory units store instructions that, when executed by the one or more processors, cause the system to perform the operations of;

creating a baseline behavior model P that captures a user'"'"'s daily pattern and volume of IT meta events over n days based on multiple data sources, wherein creating the baseline behavior model comprises;

receiving raw event logs from multiple data sources for a period of n days from days 0 to n−

1;

categorizing raw event logs into meta events using an event taxonomy;

for each of the days 0 to n−

1, creating a vector with a weighted count of each unique meta event observed that day;

creating a matrix, M, with the vectors for days 0 to n−

1; and

modeling the data in the matrix (M) from day 0 to day n−

1 using a dimension reduction technique to create the resulting baseline behavior model P;

determining whether there are anomalous pattern and volume changes in a user'"'"'s IT behavior on day n using the baseline behavior model P, wherein the determining step comprises;

creating a vector, f_n, with a weighted count of each unique meta event observed on day n;

scoring the activity vector f_n by measuring the magnitude of its reconstruction error as the difference between f_n and f_nPP^T;

normalizing the reconstruction error; and

comparing the normalized reconstruction error to an anomaly threshold;

in response to the normalized reconstruction error satisfying the anomaly threshold, concluding that the user'"'"'s meta event behavior on day n is anomalous and elevating a risk assessment associated with the user'"'"'s IT activities on day n; and

in response to the normalized reconstruction error not satisfying the anomaly threshold, updating the baseline behavior model with the user'"'"'s meta event activity from day n.

16. The system of claim 15, wherein the dimension reduction technique is Principal Component Analysis (PCA).

17. The system of claim 16, wherein the baseline behavior model P is the top K eigenvectors of M^TM.

18. The system of claim 17, wherein normalizing the reconstruction error comprises dividing the reconstruction error by the sum of all eigenvalues in the baseline behavior model P.

19. The system of claim 15, wherein elevating the risk assessment comprises adding points to a risk score for the user'"'"'s logon session.

20. The system of claim 15, wherein the events include a plurality of the following:

log-on events, account-creation events, account-deletion events, account-password-change events, and events relates to access of machines, documents, and applications.

21. The method of claim 15, wherein the threshold is set so that the normalized reconstruction error is above the threshold for between 0.4% and 0.6% of users in the network.

22. A non-transitory computer-readable medium comprising a computer program, that, when executed by a computer system, enables the computer system to perform the following method for detecting anomalous IT pattern and volume event behavior for a user during a period of time based on multiple data sources, the method comprising:

creating a baseline behavior model P that captures a user'"'"'s periodic pattern and volume of IT meta events over n periods based on multiple data sources, wherein creating the baseline behavior model comprises;

receiving raw event logs from multiple data sources for n periods from periods 0 to n−

1;

categorizing raw event logs into meta events using an event taxonomy;

for each of the periods 0 to n−

1, creating a vector with a weighted count of each unique meta event observed during that period;

creating a matrix, M, with the vectors for periods 0 to n−

1; and

modeling the data in the matrix (M) from day 0 to day n−

1 using a dimension reduction technique to create the resulting baseline behavior model P;

determining whether there are anomalous pattern and volume changes in a user'"'"'s IT behavior during period n using the baseline behavior model P, wherein the determining step comprises;

creating a vector, f_n, with a weighted count of each unique meta event observed during period n;

scoring the activity vector f_n by measuring the magnitude of its reconstruction error as the difference between f_n and f_nPP^T;

normalizing the reconstruction error; and

comparing the normalized reconstruction error to an anomaly threshold;

in response to the normalized reconstruction error satisfying the anomaly threshold, concluding that the user'"'"'s meta event behavior during period n is anomalous and elevating a risk assessment associated with the user'"'"'s IT activities during period n; and

in response to the normalized reconstruction error not satisfying the anomaly threshold, updating the baseline behavior model with the user'"'"'s meta event activity from period n.

23. The non-transitory computer-readable medium of claim 22, wherein the dimension reduction technique is Principal Component Analysis (PCA).

24. The non-transitory computer-readable medium of claim 23, wherein the baseline behavior model P is the top K eigenvectors of M^TM.

25. The non-transitory computer-readable medium of claim 24, wherein normalizing the reconstruction error comprises dividing the reconstruction error by the sum of all eigenvalues in the baseline behavior model P.

26. The non-transitory computer-readable medium of claim 22, wherein elevating the risk assessment comprises adding points to a risk score for the user'"'"'s logon session.

27. The non-transitory computer-readable medium of claim 22, wherein the events include a plurality of the following:

log-on events, account-creation events, account-deletion events, account-password-change events, and events relates to access of machines, documents, and applications.

28. The non-transitory computer-readable medium of claim 22, wherein the threshold is set so that the normalized reconstruction error is above the threshold for between 0.4% and 0.6% of users in the network.