Systems and methods for classifying malicious network events

• US 9,306,962 B1
• Filed: 07/24/2014
• Issued: 04/05/2016
• Est. Priority Date: 07/25/2013
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Niddel Corp. (Verizon Communications Inc.)
• Inventors: Pinto, Alexandre De Melo Correia
• Primary Examiner(s): Siddiqi, Mohammad A
• Application Number:
  US14/340,397
• Time in Patent Office: 621 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 21/34 involving the use of external additional devices, e.g. dongles or smart cards
• CPC Class Codes: G06F 21/35 communicating wirelessly
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

A system for classifying events on a computer network includes an event clustering engine for receiving event and log data related to identifiable actors from a security information and event management (SIEM) or log management module and selecting behavioral groupings of the event and log data. An affinity-based feature generation module assigns a value to each identifiable actor based on occurrences within predetermined time intervals of the identifiable actors having the selected behavioral grouping. A time-based weighting decay module applies a time decaying function to the assigned values for each identifiable actor. A feature engineering storage module stores information relating to the identifiable actors and their associated time-decayed values. A machine learning module generates a prediction model based on information received from the event clustering engine and the time-based weighting decay module, and the prediction model is utilized by a prediction engine on a computer to predict and classify received event and log data as malicious or non-malicious.

Claims

1. A system for classifying events on a computer network comprising:

a feature engineering module on a host computer having a non-transitory memory and a processor, the feature engineering module comprising;

an event clustering engine hosted on the non-transitory memory, wherein the event clustering engine receives event and log data related to identifiable actors from a security information and event management (SIEM) or log management module and selects behavioral groupings of the event and log data;

an affinity-based feature generation module hosted on the non-transitory memory and in communication with the event clustering engine, wherein the affinity-based feature generation module assigns a value to each identifiable actor based on occurrences within predetermined time intervals of the identifiable actors having the selected behavioral grouping, wherein the value is assigned to each identifiable actor for each event within the event and log data;

a time-based weighting decay module hosted on the non-transitory memory and in communication with the affinity-based feature generation module, wherein the time-based weighting decay module applies a time decaying function to the assigned values for each identifiable actor at a reference time, wherein the time decaying function is weighted based on an occurrence time of an event within the event and log data relative to the reference time; and

a feature engineering storage module hosted on the non-transitory memory and in communication with the time-based weighting decay module, wherein information relating to the identifiable actors and their associated time-decayed values are stored on the feature engineering storage module; and

a prediction model generated by a machine learning module on a host computer based on information received from the event clustering engine and the time-based weighting decay module, wherein the prediction model is utilized by a prediction engine on a computer to predict and classify received event and log data as malicious or non-malicious for each event within the event and log data.

2. The system of claim 1, wherein the identifiable actors are identifiable by at least one of:

an IP address, a domain name, a user name and a user identifier that can be uniquely identified according to the context in which the event itself is represented.

3. The system of claim 1, wherein the identifiable actors are related to other identifiable actors by a proximity measurement or a group membership.

4. The system of claim 1, wherein the event clustering engine is further configured to receive event and log data from an independent sensor.

5. The system of claim 1, wherein the behavioral groupings selected by the event clustering engine relate to one or more of:

whether an IP address associated with an identifiable actor was blocked when trying to access a specific network port;

whether an IP address associated with an identifiable actor was found to be engaging in port scanning activity or has triggered detection signatures in an intrusion detection system;

whether a user, associated with an identifiable actor by a username or other identifier on an organization-wide user director or on an internet application, is found to be denied access on a specific application, or to be included or removed from a specific access role.

6. The system of claim 1, wherein the affinity-based feature generation module is further configured to assign an aggregate value to each feature grouping of identifiable actors belonging to them based on proximity or similarity clusters of the actors in the selected behavioral groupings, wherein the proximity or similarity clusters are based on a grouping identifier of the identifiable actor, wherein the grouping identifier is received external from the event and log data.

7. The system of claim 1, wherein the feature engineering module and the machine learning module are on the same host computer, and the prediction engine is on a user computer.

8. The system of claim 1, wherein the feature engineering module, the machine learning module and the prediction engine are on the same computer.

9. The system of claim 1, wherein the event clustering engine is further configured to receive data identifying known malicious or non-malicious actors from at least one of a ground truth repository and external intelligence sources.

10. The system of claim 1, wherein the machine learning module generates a prediction model based on an equal number of malicious and non-malicious events.

11. The system of claim 1, wherein the prediction engine provides information relating to the classified event and log data to a monitoring computer.

12. A computer-implemented method of classifying events on a computer network comprising:

receiving by an event clustering engine event and log data related to identifiable actors from a security information and event management (SIEM) or log management module;

selecting behavioral groupings of the event and log data;

assigning, by an affinity-based feature generation module, a value to each identifiable actor for each event within the event and log data based on occurrences within predetermined time intervals of the identifiable actors having the selected behavioral grouping;

applying, by a time-based weighting decay module, a time decaying function to the assigned values for each identifiable actor at a reference time, and weighting the time decay function based on an occurrence time of an event within the event and log data relative to the reference time;

storing information relating to the identifiable actors and their associated time-decayed values;

generating, by a machine learning module, a prediction model based on information received from the event clustering engine and the time-based weighting decay module; and

predicting and classifying by a computer, based on the prediction model, received event and log data as malicious or non-malicious for each event within the event and log data.

13. The method of claim 12, wherein the identifiable actors are identifiable by at least one of:

an IP address, a domain name, a user name and a user identifier that can be uniquely identified according to the context in which the event itself is represented.

14. The method of claim 12, further comprising:

associating identifiable actors with other identifiable actors by a proximity measurement or a group membership.

15. The method of claim 12, further comprising:

receiving by the event clustering engine event and log data from an independent sensor.

16. The method of claim 12, wherein the behavioral groupings selected by the event clustering engine relate to one or more of:

whether an IP address associated with an identifiable actor was blocked when trying to access a specific network port;

whether an IP address associated with an identifiable actor was found to be engaging in port scanning activity or has triggered detection signatures in an intrusion detection system;

whether a user, associated with an identifiable actor by a username or other identifier on an organization-wide user director or on an internet application, is found to be denied access on a specific application, or to be included or removed from a specific access role.

17. The method of claim 12, further comprising:

assigning an aggregate value to each feature grouping of identifiable actors belonging to them based on proximity or similarity clusters of the actors in the selected behavioral groupings, wherein the proximity or similarity clusters are based on a grouping identifier of the identifiable actor, wherein the grouping identifier is received external from the event and log data.

18. The method of claim 12, further comprising:

receiving by the event clustering engine data identifying known malicious or non-malicious actors from at least one of a ground truth repository and external intelligence sources.

19. The method of claim 12, further comprising:

providing information relating to the classified event and log data to a monitoring computer.

20. A non-transitory computer readable medium containing instructions for providing a process of classifying events on a computer network enabled at least in part on a processor of a computerized device, the instructions, which when executed by the processor, performing the steps of:

receiving by an event clustering engine event and log data related to identifiable actors from a security information and event management (SIEM) or log management module;

selecting behavioral groupings of the event and log data;

assigning, by an affinity-based feature generation module, a value to each identifiable actor for each event within the event and log data based on occurrences within predetermined time intervals of the identifiable actors having the selected behavioral grouping;

applying, by a time-based weighting decay module, a time decaying function to the assigned values for each identifiable actor at a reference time, and weighting the time decay function based on an occurrence time of an event within the event and log data relative to the reference time;

storing information relating to the identifiable actors and their associated time-decayed values;

generating, by a machine learning module, a prediction model based on information received from the event clustering engine and the time-based weighting decay module; and

predicting and classifying, based on the prediction model, received event and log data as malicious or non-malicious for each event within the event and log data.