Capturing correlations between activity and non-activity attributes using N-grams

• US 9,569,614 B2
• Filed: 05/03/2016
• Issued: 02/14/2017
• Est. Priority Date: 06/17/2015
• Status: Active Grant

• Current Assignee: Crowdstrike, Inc. (CrowdStrike Holdings Incorporated)
• Original Assignee: International Business Machines Corporation
• Inventors: Pieczul, Olgierd S.
• Primary Examiner(s): Rashid, Harunur
• Assistant Examiner(s): TAYLOR, SAKINAH W
• Application Number:
  US15/145,105
• Publication Number:
  US 20160371486A1
• Time in Patent Office: 287 Days
• Field of Search: 726/23
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 21/554 involving event detection and direct action
• CPC Class Codes: G06F 21/56 Computer malware detection or handling, e.g. anti-virus arrangements
• CPC Class Codes: G06F 2221/034 Test or assess a computer or a system
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

Identifying correlations between events recorded in a computer system log, the recorded events are generated by a plurality of processes executing on the computer. A system log is partitioned into a plurality of segments, each segment associated with a characteristic found in an event, each segment including one or more events having a same characteristic value. A plurality of attributes of the events in a segment are selected. The attributes selected do not describe an action of the event. One or more distinct n-grams are generated, each distinct n-gram including the selected attributes from successive events within the segment. A distinct n-gram is distinct from all other generated n-grams. A correlation is identified for each first selected attribute of each successive event of an n-gram with all other second selected attributes from each successive event of the n-gram, and the correlations are recorded for each first selected attribute.

Claims

1. A method for identifying correlations between events recorded in a system log of a computer, the recorded events generated by a plurality of processes executing on the computer, the method comprising:

partitioning, by the computer, a system log into a plurality of segments, each segment associated with a characteristic found in an event, each segment including one or more events having a same characteristic value;

selecting, by the computer, a plurality of attributes of the one or more events in a segment, wherein the plurality of attributes do not describe an action of the event;

generating, by the computer, one or more distinct n-grams, each distinct n-gram including the selected attributes from successive events within the segment, wherein a distinct n-gram is distinct from all other generated n-grams;

identifying, by the computer, a correlation for each first selected attribute of each of the successive events of an n-gram with all other second selected attributes from each of the successive events of the n-gram;

generating, by the computer, a correlation metric as a function of the number of correlated first selected attributes and the total number of selected attributes of each of the successive events of the n-gram, wherein generating the correlation metric includes;

incrementing, by the computer, a count of n-gram instances in which the first selected attribute of each of the successive events of the n-gram correlates with one of the second selected attributes of each of the successive events of the n-gram; and

dividing, by the computer, the count by a total number of possible correlations between the first selected attributes and the second selected attributes; and

recording, by the computer, the correlations for each first selected attribute.

2. The method according to claim 1, wherein a correlations is one of:

always, never, or sometimes.

3. The method according to claim 1, wherein a correlation is a numeric value between zero and one representing a proportion of n-gram instances out of a total number of n-gram instances for which the first selected attributes correlate with a second selected attribute.

4. The method according to claim 1, further comprising:

identifying, by the computer, a correlation for each first selected attribute of each of the successive events of one of the one or more n-grams with all other third selected attributes of each event of a different non-overlapping n-gram.