DETECTION OF ANOMALOUS EVENTS

• US 20150161394A1
• Filed: 12/11/2013
• Published: 06/11/2015
• Est. Priority Date: 12/11/2013
• Status: Active Grant

• Current Assignee: UT-Battelle LLC
• Original Assignee: UT-Battelle LLC
• Inventors: Ferragut, Erik M., Laska, Jason A., Bridges, Robert A.
• Granted Patent: US 9,361,463 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 21/577 Assessing vulnerabilities and evaluating computer system security
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 9/3273 for mutual authentication network architectures or network communication protocols for achieving mutual authentication in a packet data network H04L63/0869

Abstract

A system is described for receiving a stream of events and scoring the events based on anomalousness and maliciousness (or other classification). The system can include a plurality of anomaly detectors that together implement an algorithm to identify low-probability events and detect atypical traffic patterns. The anomaly detector provides for comparability of disparate sources of data (e.g., network flow data and firewall logs.) Additionally, the anomaly detector allows for regulatability, meaning that the algorithm can be user configurable to adjust a number of false alerts. The anomaly detector can be used for a variety of probability density functions, including normal Gaussian distributions, irregular distributions, as well as functions associated with continuous or discrete variables.

Claims

1. A method of detecting anomalous events, comprising:

receiving a first log file including a first plurality of events from a first data source;

receiving a second log file including a second plurality of events from a second data source that is a different type than the first data source;

using the first log file, generating a first anomaly score, the generation being derived from an area associated with a probability density function of the first log file;

using the second log file, generating a second anomaly score, the generation being derived from an area associated with a probability density function of the second log file; and

comparing the first and second anomaly scores.

2. The method of claim 1, wherein generating the first anomaly score includes calculating the anomaly score using a function P_f(f(X)≦

f(x)), wherein f(X) is related to a probability of an occurrence of an event, f(x) is a current event being analyzed, and P_f is a probability determination.

3. The method of claim 1, wherein generating the first anomaly score includes using the formula A_f(x)=−

log_bPf(f(X)≦

f(x)) where b>

1.

4. The method of claim 1, wherein α

function used to calculate the anomaly score is tunable through user input.

5. The method of claim 1, wherein generating the first anomaly score includes using the function P_f(A_f(x)≧

α

)≦

b<sup>−

α</sup>

wherein α

is a tunable parameter to change a number of false alerts and b is any number >

1.

6. The method of claim 1, wherein the first and second log files include disparate event types.

7. The method of claim 1, wherein each of the first and second log files are streaming data.

8. The method of claim 1, wherein comparing the first and second anomaly scores includes generating a linear combination of the anomaly scores using weighted inputs.

9. A computer-readable storage having instructions thereon for executing a method of detecting anomalous events, the method comprising:

receiving a plurality of input network events from disparate network sources; and

calculating multiple anomaly scores for each of the plurality of input network events using a function formed at least in part by the expression P_f(f(X)≦

f(x)), wherein f(X) is related to a probability of an occurrence of an event, f(x) is a current event being analyzed, and P_f is a probability determination.

10. The computer-readable storage of claim 9, wherein generating the anomaly scores includes using the formula A_f(x)=−

log_bPf(f(X)≦

f(x)) where b>

1.

11. The computer-readable storage of claim 9, wherein the function used to calculate the anomaly scores is tunable.

12. The computer-readable storage of claim 9, wherein generating the anomaly scores includes using the function P_f(A_f(x)≧

α

)≦

b<sup>−

α</sup>

wherein α

is a tunable parameter to change a number of false alerts and b is any number >

1.

13. The computer-readable storage of claim 9, wherein receiving the plurality of input network events includes receiving multiple log files that include disparate event types.

14. The computer-readable storage of claim 9, wherein receiving the plurality of input network events includes receiving the input network events as streaming data.

15. The computer-readable storage of claim 9, further including comparing the anomaly scores, wherein the comparing of anomaly scores includes generating a linear combination of the anomaly scores using weighted inputs.

16. The computer-readable storage of claim 9, wherein the network sources are selected from a list including one or more of the following:

a firewall, a workstation, a DNS server, and a web server.

17. The computer-readable storage of claim 9, wherein the network events include a timestamp, a source IP address, a destination IP address, a destination port, a protocol and a message code.

18. A system for detecting anomalous events, comprising:

a first anomaly detector for receiving a first log file;

a second anomaly detector for receiving a second log file;

wherein the first and second anomaly detectors calculate anomaly scores for the respective first and second log files, the anomaly detectors using a function formed at least in part by the expression P_f(f(X)≦

f(x)), wherein f(X) is related to a probability of an occurrence of an event, f(x) is a current event being analyzed, and P_f is a probability determination; and

a comparator coupled to the anomaly detectors for comparing the anomaly scores.

19. The system of claim 18, wherein the first log file and the second log file are sourced from devices having different platforms.

20. The system of claim 18, wherein the first and second anomaly detectors are tunable to change a number of false alerts.