Capturing correlations between activity and non-activity attributes using N-grams

• US 9,591,014 B2
• Filed: 06/17/2015
• Issued: 03/07/2017
• Est. Priority Date: 06/17/2015
• Status: Expired due to Fees

• Current Assignee: Crowdstrike, Inc. (CrowdStrike Holdings Incorporated)
• Original Assignee: International Business Machines Corporation
• Inventors: Pieczul, Olgierd S.
• Primary Examiner(s): Rashid, Harunur
• Assistant Examiner(s): TAYLOR, SAKINAH W
• Application Number:
  US14/741,484
• Publication Number:
  US 20160373472A1
• Time in Patent Office: 629 Days
• Field of Search: 726/23
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 21/554 involving event detection and direct action
• CPC Class Codes: G06F 21/56 Computer malware detection or handling, e.g. anti-virus arrangements
• CPC Class Codes: G06F 2221/034 Test or assess a computer or a system
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

Identifying correlations between events recorded in a computer system log, the recorded events are generated by a plurality of processes executing on the computer. A system log is partitioned into a plurality of segments, each segment associated with a characteristic found in an event, each segment including one or more events having a same characteristic value. A plurality of attributes of the events in a segment are selected. The attributes selected do not describe an action of the event. One or more distinct n-grams are generated, each distinct n-gram including the selected attributes from successive events within the segment. A distinct n-gram is distinct from all other generated n-grams. A correlation is identified for each first selected attribute of each successive event of an n-gram with all other second selected attributes from each successive event of the n-gram, and the correlations are recorded for each first selected attribute.

Claims

1. A computer program product for identifying correlations between events recorded in a system log of a computer, the recorded events generated by a plurality of processes executing on the computer, the computer program product comprising one or more non-transitory computer readable storage medium and program instructions stored on at least one of the one or more non-transitory computer readable storage medium, the program instructions comprising:

program instructions to partition, by the computer, a system log into a plurality of segments, each segment associated with a characteristic found in an event, each segment including one or more events having a same characteristic value;

program instructions to select, by the computer, a plurality of attributes of the one or more events in a segment, wherein the plurality of attributes do not describe an action of the event;

program instructions to generate, by the computer, one or more distinct n-grams, each distinct n-gram including the selected attributes from successive events within the segment, wherein a distinct n-gram is distinct from all other generated n-grams;

program instructions to identify, by the computer, a correlation for each first selected attribute of each of the successive events of an n-gram with all other second selected attributes from each of the successive events of the n-gram;

program instructions to generate, by the computer, a correlation metric as a function of the number of correlated first selected attributes and the total number of selected attributes of each of the successive events of the n-gram, wherein the program instructions to generate the correlation metric include;

program instructions to increment, by the computer, a count of n-gram instances in which the first selected attribute of each of the successive events of the n-gram correlates with one of the second selected attributes of each of the successive events of the n-gram; and

program instructions to divide, by the computer, the count by a total number of possible correlations between the first selected attributes and the second selected attributes; and

program instructions to record, by the computer, the correlations for each first selected attribute.

2. The computer program product according to claim 1, wherein a correlations is one of:

always, never, or sometimes.

3. The computer program product according to claim 1, wherein a correlation is a numeric value between zero and one representing a proportion of n-gram instances out of a total number of n-gram instances for which the first selected attributes correlate with a second selected attribute.

4. The computer program product according to claim 1, further comprising:

program instructions to identify, by the computer, a correlation for each first selected attribute of each of the successive events of one of the one or more n-grams with all other third selected attributes of each event of a different non-overlapping n-gram.

5. A computer system for identifying correlations between events recorded in a system log of a computer, the recorded events generated by a plurality of processes executing on the computer, the computer system comprising one or more processors, one or more computer readable memories, one or more non-transitory computer readable storage medium, and program instructions stored on at least one of the one or more non-transitory computer readable storage medium for execution by at least one of the one or more processors via at least one of the one or more memories, the program instructions comprising:

program instructions to partition, by the computer, a system log into a plurality of segments, each segment associated with a characteristic found in an event, each segment including one or more events having a same characteristic value;

program instructions to select, by the computer, a plurality of attributes of the one or more events in a segment, wherein the plurality of attributes do not describe an action of the event;

program instructions to generate, by the computer, one or more distinct n-grams, each distinct n-gram including the selected attributes from successive events within the segment, wherein a distinct n-gram is distinct from all other generated n-grams;

program instructions to identify, by the computer, a correlation for each first selected attribute of each of the successive events of an n-gram with all other second selected attributes from each of the successive events of the n-gram;

program instructions to generate, by the computer, a correlation metric as a function of the number of correlated first selected attributes and the total number of selected attributes of each of the successive events of the n-gram, wherein the program instructions to generate the correlation metric include;

program instructions to increment, by the computer, a count of n-gram instances in which the first selected attribute of each of the successive events of the n-gram correlates with one of the second selected attributes of each of the successive events of the n-gram; and

program instructions to divide, by the computer, the count by a total number of possible correlations between the first selected attributes and the second selected attributes; and

program instructions to record, by the computer, the correlations for each first selected attribute.

6. The computer system according to claim 5, wherein a correlations is one of:

always, never, or sometimes.

7. The computer system according to claim 5, wherein a correlation is a numeric value between zero and one representing a proportion of n-gram instances out of a total number of n-gram instances for which the first selected attributes correlate with a second selected attribute.

8. The computer system according to claim 5, further comprising:

program instructions to identify, by the computer, a correlation for each first selected attribute of each of the successive events of one of the one or more n-grams with all other third selected attributes of each event of a different non-overlapping n-gram.