Event correlation

• US 9,697,100 B2
• Filed: 03/10/2014
• Issued: 07/04/2017
• Est. Priority Date: 03/10/2014
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Puri, Colin, Kurth, Scott, Yelisetty, Srinivas
• Primary Examiner(s): Mofiz, Apu
• Assistant Examiner(s): Holland, Sheryl
• Application Number:
  US14/203,036
• Publication Number:
  US 20150254158A1
• Time in Patent Office: 1,212 Days
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/3072 where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
• CPC Class Codes: G06F 11/3089 Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents software debugging using additional hardware using a specific debug interface G06F11/3656; performance evaluation by tracing or monitoring G06F11/3466
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 2201/86 Event-based monitoring

Abstract

Event correlation may include identifying a feature set for each log file of a plurality of log files, and extracting the feature set for each event of a plurality of events in each log file of the plurality of log files. Event correlation may further include determining a plurality of trace event pairs linkage strength values for an event from a first log file of the plurality of log files and a plurality of events from a second log file of the plurality of log files. The trace event pairs linkage strength values may represent an overlap of the feature set for the event from the first log file and the feature set for each of the plurality of events from the second log file.

Claims

1. An event correlation system comprising:

at least one processor;

a feature identification module, executed by the at least one processor, to identify a feature set for each log file of a plurality of log files;

a feature extraction module, executed by the at least one processor, to extract the feature set for each event of a plurality of events in each log file of the plurality of log files;

a trace event pairs linkage strength determination module, executed by the at least one processor, to determine a plurality of trace event pairs linkage strength values for at least one event from a first log file of the plurality of log files and a plurality of events from a second log file of the plurality of log files, wherein

the plurality of trace event pairs linkage strength values represent an overlap of the feature set for the at least one event from the first log file and the feature set for each of the plurality of events from the second log file, wherein each linkage strength value increases as the overlap of the feature set increases; and

a trace event pairs link time strength determination module, executed by the at least one processor, to determine trace event pairs link time strength values between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files,

wherein

the trace event pairs link time strength values represent a strength of time difference between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files,

the trace event pairs link time strength values are based on a time difference between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files, and a highest absolute difference of all timestamp pairs between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files,

an event correlation between the at least one event from the first log file of the plurality of log files and at least one event of the plurality of events from the second log file of the plurality of log files is identified based on the plurality of trace event pairs linkage strength values and the trace event pairs link time strength values,

the at least one event of the plurality of events from the second log file of the plurality of log files represents an anomaly associated with the second log file of the plurality of log files, and

the anomaly associated with the second log file of the plurality of log files is related to the at least one event from the first log file of the plurality of log files.

2. The event correlation system according to claim 1, wherein the trace event pairs linkage strength values are based on an intersection of the feature set for the at least one event from the first log file of the plurality of log files and the feature set for each of the plurality of events from the second log file of the plurality of log files, and a union of the feature set for the at least one event from the first log file of the plurality of log files and the feature set for each of the plurality of events from the second log file of the plurality of log files.

3. The event correlation system according to claim 1, further comprising:

a timestamp determination module, executed by the at least one processor, to determine a timestamp for each event of the plurality of events in each log file of the plurality of log files; and

a trace event pairs time lapse determination module, executed by the at least one processor, to use the timestamps associated with the at least one event from the first log file of the plurality of log files and the plurality of events from the second log file of the plurality of log files to determine a time difference between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files.

4. The event correlation system according to claim 1, wherein the trace event pairs link time strength values are modified by application of a kernel function to scale the trace event pairs link time strength values.

5. The event correlation system according to claim 1, further comprising:

a trace event pairs link score determination module, executed by the at least one processor, to determine trace event pairs link score values based on the trace event pairs linkage strength values and the trace event pairs link time strength values, wherein the trace event pairs link score values are based on a time difference between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files, and a maximum trace event pairs linkage strength value between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files.

6. The event correlation system according to claim 5, wherein the trace event pairs link score determination module is to further determine a maximum trace event pairs link score value that corresponds to a minimal time difference between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files, and the maximum trace event pairs linkage strength value between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files.

7. The event correlation system according to claim 6, wherein the trace event pairs link score determination module is to further determine at least one link that represents the event correlation between the at least one event from the first log file of the plurality of log files and the at least one event of the plurality of events from the second log file of the plurality of log files based on the maximum trace event pairs link score value.

8. The event correlation system according to claim 7, further comprising:

an event correlation graphing module, executed by the at least one processor, to display the at least one link between the at least one event from the first log file of the plurality of log files and the at least one event of the plurality of events from the second log file of the plurality of log files.

9. The event correlation system according to claim 1, wherein the feature set includes at least one feature related to an identifier, a timestamp, an event category, originator information, destination information, and location information.

10. The event correlation system according to claim 1, wherein the trace event pairs linkage strength values are based on a ratio of an intersection of the feature set for the at least one event from the first log file of the plurality of log files and the feature set for each of the plurality of events from the second log file of the plurality of log files, and a union of the feature set for the at least one event from the first log file of the plurality of log files and the feature set for each of the plurality of events from the second log file of the plurality of log files.

11. A method for event correlation, the method comprising:

identifying, by a processor, a feature set for each log file of a plurality of log files;

extracting the feature set for each event of a plurality of events in each log file of the plurality of log files;

determining a timestamp for each event of the plurality of events in each log file of the plurality of log files;

determining a plurality of trace event pairs linkage strength values for at least one event from a first log file of the plurality of log files and a plurality of events from a second log file of the plurality of log files, wherein

the plurality of trace event pairs linkage strength values represent an overlap of the feature set for the at least one event from the first log file and the feature set for each of the plurality of events from the second log file, wherein each linkage strength value increases as the overlap of the feature set increases,

the trace event pairs linkage strength values are based on an intersection of the feature set for the at least one event from the first log file of the plurality of log files and the feature set for each of the plurality of events from the second log file of the plurality of log files, and a union of the feature set for the at least one event from the first log file of the plurality of log files and the feature set for each of the plurality of events from the second log file of the plurality of log files,

an event correlation between the at least one event from the first log file of the plurality of log files and at least one event of the plurality of events from the second log file of the plurality of log files is identified based on the plurality of trace event pairs linkage strength values,

the at least one event of the plurality of events from the second log file of the plurality of log files represents an anomaly associated with the second log file of the plurality of log files, and

the anomaly associated with the second log file of the plurality of log files is related to the at least one event from the first log file of the plurality of log files; and

determining trace event pairs link time strength values between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files, wherein

the trace event pairs link time strength values are based on a time difference between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files, and a highest absolute difference of all timestamp pairs between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files.

12. The method for event correlation according to claim 11, further comprising:

using the timestamps associated with the at least one event from the first log file of the plurality of log files and the plurality of events from the second log file of the plurality of log files to determine the time difference between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files.

13. The method for event correlation according to claim 11, wherein the trace event pairs link time strength values represent a strength of time difference between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files.

14. The method for event correlation according to claim 11, further comprising:

determining trace event pairs link score values based on the trace event pairs linkage strength values and the trace event pairs link time strength values, wherein the trace event pairs link score values are based on the time difference between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files, and a maximum trace event pairs linkage strength value between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files.

15. The method for event correlation according to claim 14, further comprising:

determining a maximum trace event pairs link score value that corresponds to a minimal time difference between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files, and the maximum trace event pairs linkage strength value between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files.

16. The method for event correlation according to claim 15, further comprising:

determining at least one link that represents the event correlation between the at least one event from the first log file of the plurality of log files and the at least one event of the plurality of events from the second log file of the plurality of log files based on the maximum trace event pairs link score value.

17. The method for event correlation according to claim 16, further comprising:

displaying the at least one link between the at least one event from the first log file of the plurality of log files and the at least one event of the plurality of events from the second log file of the plurality of log files.

18. A non-transitory computer readable medium having stored thereon machine readable instructions for event correlation, the machine readable instructions when executed cause a processor to:

identify a feature set for each log file of a plurality of log files;

extract the feature set for each event of a plurality of events in each log file of the plurality of log files;

determine a timestamp for each event of the plurality of events in each log file of the plurality of log files;

determine a plurality of trace event pairs linkage strength values for at least one event from a first log file of the plurality of log files and a plurality of events from a second log file of the plurality of log files, wherein

the plurality of trace event pairs linkage strength values represent an overlap of the feature set for the at least one event from the first log file and the feature set for each of the plurality of events from the second log file, wherein each linkage strength value increases as the overlap of the feature set increases; and

determine trace event pairs link time strength values between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files, wherein

the trace event pairs link time strength values represent a strength of time difference between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files,

and the trace event pairs link time strength values are based on a time difference between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files, and a highest absolute difference of all timestamp pairs between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files,

an event correlation between the at least one event from the first log file of the plurality of log files and at least one event of the plurality of events from the second log file of the plurality of log files is identified based on the plurality of trace event pairs linkage strength values, and the trace event pairs link time strength values,

the at least one event of the plurality of events from the second log file of the plurality of log files represents an anomaly associated with the second log file of the plurality of log files, and

the anomaly associated with the second log file of the plurality of log files is related to the at least one event from the first log file of the plurality of log files.

19. The non-transitory computer readable medium according to claim 18, wherein the trace event pairs link time strength values are based on the time difference between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files, and a maximum trace event pairs linkage strength value between the at least one event from the first log file of the plurality of log files and each of the plurality of events from the second log file of the plurality of log files.

20. The non-transitory computer readable medium according to claim 18, wherein the trace event pairs link time strength values are modified by application of a kernel function to scale the trace event pairs link time strength values.