Hardware/software based indirect time stamping methodology for proactive hardware/software event detection and control

• US 20050144532A1
• Filed: 12/12/2003
• Published: 06/30/2005
• Est. Priority Date: 12/12/2003
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Sahoo, Ramendra K., Sugavanam, Krishnan, Dombrowa, Marc B., Hoenicke, Dirk I.
• Granted Patent: US 7,529,979 B2
• Time in Patent Office: Days
• US Class Current: 714/39
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/348 Circuit details, i.e. tracer hardware
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/835 Timestamp
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 2201/88 Monitoring involving counting

Abstract

An improved method and apparatus for time stamping events occurring on a large scale distributed network uses a local counter associated with each processor of the distributed network. Each counter resets at the same time globally so that all events are recorded with respect to a particular time. The counter is stopped when a critical event is detected. The events are masked or filtered in an online or offline fashion to eliminate non-critical events from triggering a collection by the system monitor or service/host processor. The masking can be done dynamically through the use of an event history logger. The central system may poll the remote processor periodically to receive the accurate counter value from the local counter and device control register. Remedial action can be taken when conditional probability calculations performed on the historical information indicate that a critical event is about to occur.

Claims

1. A distributed network having a plurality of processors, the network hardware and software comprising:

a local counter associated with each of the processors in the distributed network;

an event register associated with each of the local counters; and

an event logger for receiving a counter value from the local counter in response to an event being registered in the event register.

2. The distributed network of claim 1 comprising a global clock wherein a time stamp is calculated based on the counter value received from a counter associated with a processor in the distributed network.

3. The distributed network of claim 1 wherein the event logger records data concerning a type of event registered by the event register and a time an event occurred.

4. The distributed network of claim 1 wherein the event register remains frozen until the event register is read by the system monitor.

5. The distributed network of claim 1 comprising dynamic masking mechanisms for filtering the event register outputs to differentiate between critical and non-critical events.

6. The network of claim 5 wherein the masking is dynamically updated during online processing.

7. The network of claim 1 comprising software for performing conditional probability calculations based on event information stored in a history table wherein the calculations are performed to determine if a probability of an event occurring has exceeded a minimum threshold level and, if the threshold is exceeded, to migrate a process or schedule maintenance to avoid consequences of the predicted event.

8. The network of claim 7 wherein the conditional probability calculations are based upon events occurring within a selected time window.

9. The network of claim 1 wherein the event register comprises an error time stamp register that receives a value from the local counter when an event occurs.

10. The network of claim 1 wherein the event register stores an error occurred value that indicates to the network monitor that a critical event has occurred.

11. A method of producing a time stamp for an event occurring on a distributed network including a plurality of processors comprising:

producing a local counter value for each of a plurality of processors in the distributed network with an associated counter;

synchronizing the local counter at each of the processors with a global clock; and

freezing the local counter for a processor when a critical event associated with the processor occurs.

12. The method of claim 11 comprising establishing a history table containing information concerning events associated with the critical event and the conditional probabilities of the associated events during offline processing.

13. The method of claim 12 comprising determining during an offline phase if an event is critical and whether or not online processing is possible.

14. The method of claim 12 comprising dynamically filtering the events based on a recorded history of information associated with the occurrence of events such that only certain critical events produce global interrupts.

15. The method of claim 12 comprising updating the conditional probability information and history table during offline processing.

16. The method of claim 11 comprising determining during online processing a type of event that occurred and determining whether to produce a global alert, synch stop or machine check alert signal based upon the type of event that occurred.

17. The method of claim 11 comprising dynamically masking events that occur based on conditional probabilistic calculations using machine learning algorithms to predict an occurrence of a critical event during a specified time period.

28. The medium of claim 11 comprising an instruction for determining a type of event that occurred and determining whether to produce a global alert, synch stop or machine check alert signal based upon the type of event that occurred.

29. The medium of claim 11 comprising an instruction for dynamically masking events that occur based on conditional probabilistic calculations using machine learning algorithms.

18. A distributed computer system having hardware and software for implementing a time stamping process for producing a time stamp associated with an occurrence of an error event, the computer system comprising:

a plurality of local counters wherein each counter is associated with a particular processor or system in the distributed computer system;

an event register for recording event information concerning an occurrence of an event associated with the processor and event register; and

an event logger for receiving and logging information concerning the occurrence of the events.

19. The distributed computer system of claim 18 comprising a global clock for synchronizing the local counters.

20. The distributed computer system of claim 19 wherein the event logger records a time stamp based upon the global clock and a local counter value received from a local counter.

21. The distributed computer system of claim 18 comprising dynamic masks created based upon historical event information for filtering events such that only information concerning critical events result is stored.

22. The distributed computer system of claim 21 comprising software for evaluating events based on conditional probabilistic calculations and scheduling remedial or preventative action during online processing.

23. A computer-executable medium comprising instructions for producing a time stamp for an event occurring on a distributed network including a plurality of processors, the medium comprising instructions for:

producing a local counter value for each of a plurality of processors in the distributed network with an associated counter;

synchronizing the local counter at each of the processors with a global clock; and

freezing the local counter for a processor when an event associated with the processor occurs.

24. The medium of claim 23 comprising an instruction for monitoring the local counter with a system monitor through the use of online and offline processing.

25. The medium of claim 23 comprising an instruction for periodically polling the local counters and storing information received in a history table.

26. The medium of claim 23 comprising an instruction for dynamically filtering the events based on a recorded history of information associated with the occurrence of events such that only critical events are reported to a system monitor.

27. The medium of claim 23 comprising an instruction for performing conditional probability calculations to determine if a probability that a critical event will occur exceeds a threshold level and performing or scheduling preventative action if such threshold is exceeded.