Hybrid method for event prediction and system control

• US 7,451,210 B2
• Filed: 11/24/2003
• Issued: 11/11/2008
• Est. Priority Date: 11/24/2003
• Status: Expired due to Fees

• Current Assignee: GlobalFoundries, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Gupta, Manish, Oliner, Adam J., Sahoo, Ramendra K., Moreira, Jose E.
• Primary Examiner(s): Salad; Abdullahi
• Assistant Examiner(s): Mejia; Anthony
• Application Number:
  US10/720,300
• Publication Number:
  US 20050114739A1
• Time in Patent Office: 1,814 Days
• Field of Search: 709/223, 709/224, 709/226
• US Class Current: 709/224
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3058 Monitoring arrangements for monitoring environmental properties or parameters of the computing system or of the computing system component, e.g. monitoring of power, currents, temperature, humidity, position, vibrations thermal management in cooling arrangements of a computing system G06F1/206
• CPC Class Codes: G06F 11/3447 Performance evaluation by modeling
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: Y10S 706/908 Electronic or computer, internal or network, circuit
• CPC Class Codes: Y10S 706/916 Electronic or computer, internal or network, circuit

Abstract

A hybrid method of predicting the occurrence of future critical events in a computer cluster having a series of nodes records system performance parameters and the occurrence of past critical events. A data filter filters the logged to data to eliminate redundancies and decrease the data storage requirements of the system. Time-series models and rule based classification schemes are used to associate various system parameters with the past occurrence of critical events and predict the occurrence of future critical events. Ongoing processing jobs are migrated to nodes for which no critical events are predicted and future jobs are routed to more robust nodes.

Claims

1. A method of predicting an occurrence of a critical event in a computer cluster having a plurality of nodes, said method comprising steps of:

A) maintaining an event log comprising information concerning critical events that occur in the computer cluster, wherein said critical events adversely affect performance of the cluster or one of its nodes, the maintaining step comprising;

i) aligning the information concerning the critical events;

ii) categorizing the information concerning the critical events according to time-dependency;

B) maintaining a system parameter log comprising information concerning system performance parameters for each node in the cluster, the maintaining step comprising;

i) recording a temperature of the nodes in the cluster and a corresponding time value;

ii) recording a utilization parameter of a central processing unit of a node in the cluster and a corresponding time value;

C) filtering the event log and the system parameter log such that some critical event information and some system parameter information is eliminated in order to reduce storage requirements of the cluster;

D) implementing a hybrid prediction system comprising rule based prediction algorithms, time-dependent variable prediction algorithms, and a warning window;

wherein the rule-based prediction algorithms use associative rules based upon the critical event information and the system parameter information for predicting a probable occurrence of the critical events within a specified time window and the variables that are likely to indicate a potential occurrence of such an event;

wherein the time-dependent variable prediction algorithms generate time-series mathematical models that predict future values of the system performance parameters; and

wherein the warning window is formed for only those nodes in the cluster in which at least one error has occurred in order to reduce system requirements, wherein said warning window comprises a predicted performance parameter or critical event occurrence for the node for a predetermined future period of time;

E) for only those nodes in which an error has occurred, loading the information from the event log and the system performance information pertaining to said error-prone nodes from the system parameter log into a Bayesian network model representing a correspondence between the system performance parameters and occurrence of the critical events;

F) using the Bayesian network model to predict a future critical event within a specified time-limit based upon the hybrid prediction system;

G) making future scheduling and current data migration selections based upon the hybrid prediction system; and

H) adapting the Bayesian Network Model by feeding the scheduling and data migration selections into said Bayesian Network Model.