System and method for automated problem diagnosis

• US 7,379,846 B1
• Filed: 06/29/2004
• Issued: 05/27/2008
• Est. Priority Date: 06/29/2004
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Williams, Emrys, Rudoff, Andrew
• Primary Examiner(s): Barlow, Jr.; John E.
• Assistant Examiner(s): LE, JOHN H
• Application Number:
  US10/880,253
• Time in Patent Office: 1,428 Days
• Field of Search: 702/35, 702/185, 702/183, 714/25, 714/26, 714/33, 714/35, 714/48, 370/450, 709/224, 709/223, 706/52, 700/79
• US Class Current: 702/185
• CPC Class Codes: G06F 11/0706 the processing taking place on a specific hardware platform or in a specific software environment
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36

Abstract

A method and system are provided for automated diagnosis for a system. In one embodiment, the method includes providing a fault tree representation of the system, the fault tree specifying propagations of errors generated in the system by problems to produce error reports. At least some of the propagations have timing information associated therewith. One or more error reports having timing information associated therewith are received and analysed using the fault tree representation to determine a suspect list of problems. The suspect list contains those problems that could have generated errors to produce the received error reports compatible with the propagations in the fault tree, and consistent with the timing information associated with the propagations and the received error reports.

Claims

1. A method of automated diagnosis for a system, comprising:

providing a fault tree representation of the system, wherein said fault tree representation indicates hierarchical cause and effect relationships between system events including propagations of errors generated in the system by problems to produce error reports, at least some of said propagations having timing information associated therewith;

receiving one or more error reports having timing information associated therewith;

analyzing the received one or more error reports using the fault tree representation to determine one or more suspected problems, said suspected problems comprising those problems that could have generated errors to produce the received error reports compatible with the propagations in the fault tree representation, and consistent with the timing information associated with the propagations and the received error reports; and

producing one or more suspect lists dependent on said analyzing.

2. The method of claim 1, wherein said fault tree representation further specifies constraints associated with at least some of said propagations, and said suspected problems comprise those problems for which said constraints are satisfied.

3. The method of claim 2, wherein at least some of said constraints are dependent upon system configuration at the time of the propagation.

4. The method of claim 3, further comprising interacting with a configuration manager in order to evaluate any constraints that are dependent upon system configuration at the time of the propagation.

5. The method of claim 2, wherein at least some of said constraints are dependent upon a parameter.

6. The method of claim 5, wherein information regarding the parameter is included in a received error report.

7. The method of claim 5, wherein information regarding the parameter is not included in a received error report.

8. The method of claim 1, wherein the fault tree representation is independent of configuration.

9. The method of claim 8, further comprising combining the fault tree representation with system configuration information at run-time.

10. The method of claim 9, wherein the diagnosis algorithm uses said system configuration information to evaluate the fault tree representation at run time.

11. The method of claim 1, further comprising representing the times of events as earliest and latest pairs.

12. The method of claim 1, wherein said producing one or more suspect lists comprises producing a sequence of suspect lists of decreasing size, wherein each of said suspect lists contains an actual problem in the system.

13. The method of claim 1, wherein said producing one or more suspect lists comprises producing a suspect list of minimum size consistent with the information available at a particular time.

14. The method of claim 1, further comprising producing a wait time associated with said one or more suspected problems, wherein said wait time represents the earliest that it is useful to re-examine the fault tree representation to create a new suspect list.

15. The method of claim 1, further comprising producing a list of desirable pollers containing those pollers that might improve diagnosis.

16. The method of claim 1, wherein certain propagations are associated with pollers, further comprising recording the start and end time of a poller run for a propagation.

17. The method of claim 1, further comprising recording the times of configuration changes in the system, including the initial and final configuration states for each configuration change.

18. The method of claim 17, wherein the times of configuration changes specify the earliest time that the configuration was not in its initial state, and the latest time that the configuration was not in its final state.

19. The method of claim 17, wherein a request for a configuration variable returns the entire recent history of the variable in terms of the initial and final configuration states for each configuration change.

20. The method of claim 1, wherein each of said one or more suspected problems is annotated with a likelihood of having occurred.

21. The method of claim 1, further comprising forming a suspect list comprising said one or more problems.

22. The method of claim 21, further comprising publishing said suspect list if it contains only a single problem.

23. The method of claim 21, further comprising publishing said suspect list if there are no more pollers or no more waiting that can reduce the size of the suspect list.

24. The method of claim 21, further comprising publishing said suspect list if all the problems on the suspect list come from one field replaceable unit.

25. The method of claim 21, further comprising publishing said suspect list if all the problems on the suspect list come from one automatic system reconfiguration unit.

26. The method of claim 21, further comprising updating the suspect list to eliminate one or more suspected problems as a result of the passage of time.

27. The method of claim 21, further comprising updating the suspect list to eliminate one or more suspected problems as a result of receiving one or more error reports.

28. The method of claim 1, wherein said analyzing is performed by ascending the fault tree representation from a received error report to find a problem that could have caused the error report.

29. The method of claim 28, wherein said analyzing further comprises descending the fault tree representation from said problem to the received error report to confirm whether or not the problem could have caused the error.

30. The method of claim 29, wherein said descending includes performing tests on each event on the route.

31. The method of claim 30, wherein said tests including checking that each event on the route could have been caused.

32. The method of claim 30, wherein said tests including checking that each event on the route was not certainly inhibited.

33. The method of claim 30, wherein said tests including checking that the consequences of each event on the route could have occurred.

34. A computer-readable storage medium, comprising program instructions computer-executable for performing automated diagnosis for a system by:

providing a fault tree representation of the system, said fault tree representation indicating hierarchical cause and effect relationships between system events including propagations of errors generated in the system by problems to produce error reports, at least some of said propagations having timing information associated therewith;

receiving one or more error reports having timing information associated therewith;

analyzing the received one or more error reports using the fault tree representation to determine one or more suspected problems, said suspected problems comprising those problems that could have generated errors to produce the received error reports compatible with the propagations in the fault tree representation, and consistent with the timing information associated with the propagations and the received error reports; and

producing one or more suspect lists dependent on said analyzing.

35. Apparatus for automated diagnosis for a system, comprising:

a fault tree representation of the system, said fault tree representation indicating hierarchical cause and effect relationships between system events including propagations of errors generated in the system by problems to produce error reports, at least some of said propagations having timing information associated therewith; and

a diagnosis engine for analyzing one or more received error reports having timing information associated therewith using the fault tree representation to determine one or more suspected problems, said suspected problems comprising those problems that could have generated errors to produce the received error reports compatible with the propagations in the fault tree representation, and consistent with the timing information associated with the propagations and the received error reports.