Method and apparatus for automating the root cause analysis of system failures

• US 7,516,362 B2
• Filed: 03/19/2004
• Issued: 04/07/2009
• Est. Priority Date: 03/19/2004
• Status: Expired due to Fees

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Connelly, Jon Christopher, Loy, Eric William
• Primary Examiner(s): Beausoliel; Robert
• Assistant Examiner(s): Manoskey; Joseph D
• Application Number:
  US10/804,275
• Publication Number:
  US 20050210331A1
• Time in Patent Office: 1,845 Days
• Field of Search: 714/25, 714/26, 714/31, 714/37, 714/39, 709/248
• US Class Current: 714/26
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0748 in a remote unit communicating with a single-box computer node experiencing an error/fault remote testing G06F11/2294
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: H04L 41/0631 using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
• CPC Class Codes: H04L 41/069 using logs of notifications; Post-processing of notifications
• CPC Class Codes: H04L 43/0805 by checking availability
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

A method for analyzing the root cause of system failures on one or more computers. An event is generated when a computer system detects a system failure. The cause of the failure is determined. The event, including the cause is transmitted from the computer system to a central repository. And the system failure is analyzed in the central repository.

Claims

1. A method for analyzing the root cause of system failures on one or more computers, comprising:

generating an event when a computer system detects a system failure;

determining the cause of the system failure;

transmitting the event, including the determined cause, from the computer system to a central repository;

analyzing the system failure event in the central repository;

storing the event in a local repository located on the computer system; and

synchronizing the local repository and the central repository,

wherein the synchronizing step comprises;

transmitting missing events in the central repository from the computer systems,

wherein the missing events correspond to system failure events for which causes were still being determined by the computer system at a time when the central repository made a request for event information to be sent thereto, and for which the causes have subsequently been determined by the computer system.

2. The method of claim 1, further comprising:

re-transmitting the event if a receipt confirmation message is not received from the central repository.

3. The method of claim 1, further comprising:

holding the event in a queue if a receipt confirmation message is not received from the central repository; and

re-transmitting the event in the queue after a period of time.

4. The method of claim 1, further comprising:

determining if the system failure was due to a hardware problem by analyzing a file log.

5. The method of claim 1, further comprising:

determining if the system failure was due to a software problem by analyzing system core files.

6. The method of claim 1, further comprising:

assigning a sequence number to each event generated;

receiving a status request from the central repository; and

synchronizing the local repository and the central repository if the sequence number does not match the expected sequence number.

7. The method of claim 1, wherein the synchronizing step further comprises:

transmitting missing events in the local repository from the central repository.

8. The method of claim 1, wherein the synchronizing step further comprises:

discarding events that have already been received.

9. The method of claim 1, further comprising:

retransmitting the information stored in the central repository to another computer system for further analysis.

10. The method of claim 1, wherein the central repository is in a SLEEP mode at all times other than when events are transmitted to the central repository from the computer system and analyzed by the central repository.

11. An apparatus for analyzing the root cause of system failures on one or more computers, comprising:

a network;

a local support computer coupled to said network;

a computer system coupled to the network, the computer system programmed to monitor itself and another computer system for system failures, to determine the cause of the system failure, and to transmit system failure events to the local support computer;

storing the event in a local repository located on the computer system; and

synchronizing the local repository and a repository of the local support computer,

wherein the synchronizing step comprises;

transmitting missing events in the repository of the local support computer from the computer systems,

wherein the missing events correspond to system failure events for which causes were still being determined by the computer system at a time when the repository of the local support computer made a request for event information to be sent thereto, and for which the causes have subsequently been determined by the computer system.

12. The apparatus of claim 11, further comprising:

the local support computer programmed to collect and analyze the system failure information.

13. The apparatus of claim 11, further comprising:

the computer system programmed to re-transmit the event if a receipt confirmation message is not received from the local support computer.

14. The apparatus of claim 13, further comprising:

the computer system programmed to assign a sequence number to each event generated;

the local support computer programmed to send a status request to the computer system, and to synchronize the local repository with the local support computer if the sequence number does not match the expected sequence number.

15. The apparatus of claim 11, further comprising:

the computer system programmed to hold the event in a queue if a receipt confirmation message is not received from the central repository, and to re-transmit the event in the queue after a period of time.

16. The apparatus of claim 11, further comprising:

the computer system programmed to determine if the system failure was due to a hardware problem by analyzing a file log.

17. The apparatus of claim 11, further comprising:

the computer system programmed to determine if the system failure was due to a software problem by analyzing system core files.

18. The apparatus of claim 11, further comprising:

a remote support computer connectable to the local support computer for receiving system failure data from the local support computer.

19. The apparatus of claim 11, wherein the central repository is in a SLEEP mode at all times other than when events are transmitted to the central repository from the computer system and analyzed by the central repository.

20. A means for analyzing the root cause of system failures on one or more computers, comprising:

a means for transmitting data from one computer to another,

a local support computer coupled to the means for transmitting data,

a computer system coupled to the means for transmitting data,

a means for the computer system to monitor itself or another computer system for system failures and determining the causes of the failures,

a means for transmitting the causes of the failures to the local support computer;

a local repository located on the computer system for storing the event; and

a means for synchronizing the local repository and a repository of the local support computer,

wherein the synchronizing means comprises;

a means for transmitting missing events in the repository of the local support computer from the computer system,

wherein the missing events correspond to system failure events for which causes were still being determined by the computer system at a time when the repository of the local support computer made a request for event information to be sent thereto, and for which the causes have subsequently been determined by the computer system.

21. The means of claim 20, wherein the central repository is in a SLEEP mode at all times other than when events are transmitted to the central repository from the computer system and analyzed by the central repository.