Automatic troubleshooting from computer system monitoring data based on analyzing sequences of changes

• US 10,303,539 B2
• Filed: 02/22/2016
• Issued: 05/28/2019
• Est. Priority Date: 02/23/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Meng, Fan Jing, Rajan, Vadakkedathu T., Wegman, Mark N., Xu, Jing Min, Yang, Lin Y.
• Primary Examiner(s): Schell, Joseph O
• Application Number:
  US15/049,855
• Publication Number:
  US 20160246662A1
• Time in Patent Office: 1,191 Days
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/0757 by exceeding a time limit, i.e. time-out, e.g. watchdogs
• CPC Class Codes: G06F 11/0787 Storage of error reports, e.g. persistent data storage, storage using memory protection
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/3058 Monitoring arrangements for monitoring environmental properties or parameters of the computing system or of the computing system component, e.g. monitoring of power, currents, temperature, humidity, position, vibrations thermal management in cooling arrangements of a computing system G06F1/206
• CPC Class Codes: G06F 11/3065 Monitoring arrangements determined by the means or processing involved in reporting the monitored data error or fault reporting or logging G06F11/0766
• CPC Class Codes: G06F 11/3072 where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
• CPC Class Codes: G06F 11/3075 the data filtering being achieved in order to maintain consistency among the monitored data, e.g. ensuring that the monitored data belong to the same timeframe, to the same system or component
• CPC Class Codes: G06F 11/3079 the data filtering being achieved by reporting only the changes of the monitored data
• CPC Class Codes: G06F 11/34 Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting

Abstract

A method for automatically detecting and diagnosing problems in computer system functioning includes determining changed objects from computer system monitoring data, calculating temporal correlations from errors and changes sequences for each changed object, identifying and ranking suspicious computer system behavior patterns from the temporal correlations, and outputting said ranked suspicious computer system behavior patterns.

Claims

1. A method for automatically detecting and diagnosing problems in computer system functioning, comprising the steps of:

receiving a diagnosis time window for computer system monitoring data to be diagnosed;

determining changed objects by analyzing the computer system monitoring data, including obtaining all changes in the diagnosis time window, identifying a list of changed objects, and extracting features from the list of changed objects, wherein the features include errors sequences and change sequences, wherein the changes include file/directory changes, package changes, operating system configuration changes, and network service changes;

calculating temporal correlations between problematic behaviors and changes for each changed object;

identifying and ranking suspicious computer system behavior patterns from the temporal correlations;

removing irrelevant pattern changes, wherein irrelevant pattern changes include change behavior patterns and change sequence patterns; and

outputting said ranked suspicious computer system behavior patterns and remediation actions associated with the identified suspicious computer system behavior patterns.

2. The method of claim 1, wherein features further include change frequencies and change densities.

3. The method of claim 1, wherein calculating temporal correlations between problematic behaviors and changes for each changed object comprises:

obtaining a time of an error or performance anomaly from the computer system monitoring data;

filtering changes by narrowing the diagnosis time window using the time of the error; and

for each changed object, fusing the error sequences with the change sequences to generate a change-error sequence, and extracting temporal correlations between errors and changes from the change-error sequence.

4. The method of claim 1, wherein identifying and ranking suspicious computer system behavior patterns from the temporal correlations comprises:

discovering normal changes and abnormal behavior patterns from the temporal correlations;

calculating confidence levels of the normal changes and abnormal behavior patterns from the extracted features to identify suspicious behavior patterns; and

ranking identified suspicious behavior patterns.

5. A non-transitory media fear automatically detecting and diagnosing problems in computer system functioning, comprising;

a feature extraction module that extracts features, including change frequencies and change sequences, of changed objects from change records of computer system monitoring data of a computer system;

a data cleaning module that removes irrelevant changes based on domain independent rules or patterns, wherein the irrelevant changes include adding a new file, accessing a file, and frequently changed objects;

a drift annotation module that identifies suspicious computer system behavior patterns using normal and abnormal patterns; and

a ranking module that calculates confidence levels of the identified suspicious behavior patterns, ranks said suspicious behavior patterns, and outputs the ranked suspicious behavior patterns and remediation actions associated with the identified suspicious computer system behavior patterns.

6. A non-transitory program storage device readable by a computer, tangibly embodying a program of instructions executed by the computer to perform the method steps for automatically detecting and diagnosing problems in computer system functioning, the method comprising the steps of:

receiving a diagnosis time window for computer system monitoring data to be diagnosed;

determining changed objects by analyzing the computer system monitoring data, including obtaining all changes in the diagnosis time window, identifying a list of changed objects, and extracting features from the list of changed objects, wherein the features include errors sequences and change sequences, wherein the changes include file/directory changes, package changes, operating system configuration changes, and network service changes;

calculating temporal correlations between problematic behaviors and changes for each changed object;

identifying and ranking suspicious computer system behavior patterns from the temporal correlations;

removing irrelevant pattern changes, wherein irrelevant pattern changes include change behavior patterns and change sequence patterns; and

outputting said ranked suspicious computer system behavior patterns and remediation actions associated with the identified suspicious computer system behavior patterns.

7. The computer readable program storage device of claim 6, wherein features further include change frequencies and change densities.

8. The computer readable program storage device of claim 6, wherein calculating temporal correlations between problematic behaviors and changes for each changed object comprises:

obtaining a time of an error or performance anomaly from the computer system monitoring data;

filtering changes by narrowing the diagnosis time window using the time of the error; and

for each changed object, fusing the error sequences with the change sequences to generate a change-error sequence, and extracting temporal correlations between errors and changes from the change-error sequence.

9. The computer readable program storage device of claim 6, wherein identifying and ranking suspicious computer system behavior patterns from the temporal correlations comprises:

discovering normal changes and abnormal behavior patterns from the temporal correlations;

calculating confidence levels of the normal changes and abnormal behavior patterns from the extracted features to identify suspicious behavior patterns; and

ranking identified suspicious behavior patterns.