Systems and methods for fast detection and diagnosis of system outages

• US 8,972,783 B2
• Filed: 06/28/2011
• Issued: 03/03/2015
• Est. Priority Date: 06/28/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gupta, Manish, Kothari, Ravi, Mann, Vijay, Vishnoi, Anil Kumar
• Primary Examiner(s): DUNCAN, MARC M
• Application Number:
  US13/170,941
• Publication Number:
  US 20130007538A1
• Time in Patent Office: 1,344 Days
• Field of Search: 714/26, 714/47.1, 714/46, 714/57
• US Class Current: 714/26
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0748 in a remote unit communicating with a single-box computer node experiencing an error/fault remote testing G06F11/2294
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3082 the data filtering being achieved by aggregating or compressing the monitored data
• CPC Class Codes: G06F 11/3093 Configuration details thereof, e.g. installation, enabling, spatial arrangement of the probes
• CPC Class Codes: G06F 11/327 Alarm or error message display

Abstract

Methods and arrangements for detecting and diagnosing system outages. A system outage is ascertained and aberrant user activities are categorized, as possible contributors to the system outage, based on system impact. User activities and system impact are learned, and user activities and system impact are compared against predetermined rules. A system outage alert is generated, and a user activity responsible for the system outage is displayed.

Claims

1. An apparatus comprising:

at least one processor; and

a computer readable storage medium having computer readable program code embodied therewith and executable by the at least one processor, the computer readable program code comprising;

computer readable program code configured to ascertain a system outage;

computer readable program code configured to categorize aberrant user activities, as possible contributors to the system outage, based on system impact in terms of at least one member taken from the group consisting of;

process forking and closure activities, file system activities, network activities, memory activities, user activity parameter values;

computer readable program code configured to learn user activities and system impact;

computer readable program code configured to compare user activities and system impact against predetermined rules;

computer readable program code configured to generate a system outage alert; and

computer readable program code configured to display a user activity responsible for the system outage; and

said computer readable program code being configured to;

measure system impact of user activities via employing data collector agents; and

aggregate data collected by the data collector agents based on process hierarchy.

2. A computer program product comprising:

a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising;

computer readable program code configured to ascertain a system outage;

computer readable program code configured to categorize aberrant user activities, as possible contributors to the system outage, based on system impact in terms of at least one member taken from the group consisting of;

process forking and closure activities, file system activities, network activities, memory activities, user activity parameter values;

computer readable program code configured to learn user activities and system impact;

computer readable program code configured to compare user activities and system impact against predetermined rules;

computer readable program code configured to generate a system outage alert; and

computer readable program code configured to display a user activity responsible for the system outage; and

said computer readable program code being configured to;

measure system impact of user activities via employing data collector agents; and

aggregate data collected by the data collector agents based on process hierarchy.

3. The computer program product according to claim 2, wherein said computer readable program code is configured to:

detect the start and end of a change window; and

compare process and network snapshots at the start and end of a change window to assimilate information on a transitional event taking place during the change window.

4. The computer program product according to claim 3, wherein:

said computer readable program code is configured to detect the start and end of a change window via at least one taken from the group consisting of;

reading a file, detecting a fork of a remote shell process, detecting a death of a remote shell process;

said computer program product is configured to undertake the comparing via comparing the transitional event information with a predetermined list of essential processes and ports;

the transitional event comprises at least one taken from the group consisting of;

death of a process, closure of a network port, closure of a network connection, deletion of a file, modification of a file; and

the predetermined list comprises at least one taken from the group consisting of;

a list created from policy documents, a list created from a firewall policy.

5. The computer program product according to claim 2, wherein the predetermined rules comprise at least one taken from the group consisting of:

learnt rules, static rules.

6. The computer program product according to claim 5, wherein:

the learnt rules comprises rules learnt through association of a user activity and its side effects over a period of time; and

the static rules comprise rules retrieved from at least one of;

a policy document, a firewall policy.

7. The computer program product according to claim 2, wherein said computer readable program code is configured to display and highlight on a dashboard a user activity responsible for the system outage.

8. The computer program product according to claim 2, wherein said computer readable program code is configured to measure system impact of a user activity based on at least one taken from the group consisting of:

files modified, files deleted;

network ports opened, network ports closed, network connections created, network connections closed, new processes forked, processes killed;

in-memory data structures modified.

9. The computer program product according to claim 2, wherein said computer readable program code is configured to measure system impact of a user activity based on supplied parameter values.

10. The computer program product according to claim 2, wherein said computer readable program code is further configured to send the aggregated data to at least one central server for processing.