Methods and apparatus for root cause identification and problem determination in distributed systems

• US 7,096,459 B2
• Filed: 09/11/2002
• Issued: 08/22/2006
• Est. Priority Date: 09/11/2002
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Keller, Alexander, Kar, Gautam
• Primary Examiner(s): Khatri, Anil
• Application Number:
  US10/241,213
• Publication Number:
  US 20040049565A1
• Time in Patent Office: 1,441 Days
• Field of Search: 717124-129, 717/100, 717/101, 717120-123, 707/100
• US Class Current: 717/124
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36

Abstract

A technique for determining a root cause of a condition (e.g., service outage) of at least one subject component in a computing environment comprises the following steps/operations. First, one or more components in the computing environment upon which the at least one subject component depends (e.g., antecedents) are identified. Identification comprises traversing at least a portion of a model representative of an existence of one or more relationships associated with at least a portion of components of the computing environment and which is capable of accounting for a full lifecycle (e.g., including deployment, installation and runtime) associated with at least one component of the computing environment. Then, one or more procedures are performed in accordance with the one or more identified components to determine a condition status associated with each of the one or more identified components. By way of example, the inventive techniques may be applied to a distributed computing environment. The computing environment may also be an autonomic computing environment.

Claims

1. A computer-based method for determining a root cause of a condition of at least one subject component in a computing environment, wherein the computing environment comprises a plurality of managed components, the method comprising the steps of:

identifying one or more managed components in the computing environment upon which the at least one subject component depends, wherein identification comprises traversing at least a portion of a model representative of an existence of one or more relationships associated with at least a portion of the managed components of the computing environment and which is capable of accounting for a full lifecycle associated with at least one managed component of the computing environment; and

performing one or more procedures in accordance with the one or more identified managed components to determine a condition status associated with each of the one or more identified managed components.

2. The method of claim 1, wherein the computing environment comprises a distributed computing environment.

3. The method of claim 1, wherein the computing environment comprises an autonomic computing environment.

4. The method of claim 1, wherein the identifying step further comprises identifying the one or more managed components upon which the at least one subject component directly depends.

5. The method of claim 1, wherein the identifying step further comprises identifying a complete set of managed components upon which the at least one subject component depends.

6. The method of claim 1, wherein the performing step further comprises performing the one or more procedures in a stepwise manner.

7. The method of claim 1, wherein the performing step further comprises performing the one or more procedures in a combined manner.

8. The method of claim 1, wherein the model is in a form comprising a functional categorization, a structural categorization and an operational categorization.

9. The method of claim 8, wherein the identifying step further comprises computing the operational categorization of the model based on at least one input criterion and traversing the operational categorization to identify the one or more managed components in the computing environment upon which the at least one subject component depends.

10. The method of claim 9, wherein the at least one input criterion comprises at least one of a name of the at least one subject component, a host associated with the at least one subject component, a direction of the traversal, a depth of the traversal, a path of the traversal and at least one filtering criterion.

11. The method of claim 1, wherein the one or more procedures of the performing step comprise at least one of a component inspection procedure, a component exercising procedure, a component heartbeat detection procedure and a component status indication procedure.

12. The method of claim 1, wherein the condition of the at least one subject component is one of failed, not failed and degraded.

13. The method of claim 1, wherein a component is one of a service, an application, middleware, hardware, a device driver, an operating system and a system associated with the computing environment.

14. The method of claim 1, wherein at least a portion of results of the root cause determination method is one of persistently stored and not persistently stored.

15. The method of claim 1, further comprising the step of maintaining a history of results associated with the root cause determination method.

16. A computer-based method for determining a root cause of a condition of at least one subject component in a computing environment, the method comprising the steps of:

identifying one or more components in the computing environment upon which the at least one subject component depends, wherein identification comprises traversing at least a portion of a model representative of an existence of one or more relationships associated with at least a portion of components of the computing environment and which is capable of accounting for a full lifecycle associated with at least one component of the computing environment; and

performing one or more procedures in accordance with the one or more identified components to determine a condition status associated with each of the one or more identified components;

wherein the root cause determination method is performed for two or more subject components substantially concurrently.

17. The method of claim 16, wherein at least a portion of the history is used to identify a most likely root cause.

18. Apparatus for determining a root cause of a condition of at least one subject component in a computing environment, wherein the computing environment comprises a plurality of managed components, the apparatus comprising:

at least one processor operative to;

(i) identify, one or more managed components in the computing environment upon which the at least one subject component depends, wherein identification comprises traversing at least a portion of a model representative of an existence of one or more relationships associated with at least a portion of the managed components of the computing environment and which is capable of accounting for a full lifecycle associated with at least one managed component of the computing environment; and

(ii) cause performance of one or more procedures in accordance with the one or more identified managed components to determine a condition status associated with each of the one or more identified managed components; and

memory, coupled to the at least one processor, operative to store at least a portion of results associated with the identifying and performing operations.

19. The apparatus of claim 18, wherein the computing environment comprises a distributed computing environment.

20. The apparatus of claim 18, wherein the computing environment comprises an autonomic computing environment.

21. The apparatus of claim 18, wherein the identifying operation further comprises identifying the one or more managed components upon which the at least one subject component directly depends.

22. The apparatus of claim 18, wherein the identifying operation further comprises identifying a complete set of managed components upon which the at least one subject component depends.

23. The apparatus of claim 18, wherein the performing operation further comprises causing the performance of the one or more procedures in a stepwise manner.

24. The apparatus of claim 18, wherein the performing operation further comprises causing the performance of the one or more procedures in a combined manner.

25. The apparatus of claim 18, wherein the model is in a form comprising a functional categorization, a structural categorization and an operational categorization.

26. The apparatus of claim 25, wherein the identifying operation further comprises computing the operational categorization of the model based on at least one input criterion and traversing the operational categorization to identify the one or more managed components in the computing environment upon which the at least one subject component depends.

27. The apparatus of claim 26, wherein the at least one input criterion comprises at least one of a name of the at least one subject component, a host associated with the at least one subject component, a direction of the traversal, a depth of the traversal, a path of the traversal and at least one filtering criterion.

28. The apparatus of claim 18, wherein the one or more procedures of the performing operation comprise at least one of a component inspection procedure, a component exercising procedure, a component heartbeat detection procedure and a component status indication procedure.

29. The apparatus of claim 18, wherein the condition of the at least one subject component is one of failed, not failed and degraded.

30. The apparatus of claim 18, wherein a component is one of a service, an application, middleware, hardware, a device driver, an operating system and a system associated with the computing environment.

31. The apparatus of claim 18, wherein at least a portion of results of the root cause determination performed by the at least one processor is one of persistently stored and not persistently stored.

32. The apparatus of claim 18, wherein the at least one processor is further operative to maintain a history of results associated with the root cause determination method.

33. The apparatus of claim 32, wherein at least a portion of the history is used to identify a most likely root cause.

34. Apparatus for determining a root cause of a condition of at least one subject component in a computing environment, the apparatus comprising:

at least one processor operative to;

(i) identify one or more components in the computing environment upon which the at least one subject component depends, wherein identification comprises traversing at least a portion of a model representative of an existence of one or more relationships associated with at least a portion of components of the computing environment and which is capable of accounting for a full lifecycle associated with at least one component of the computing environment; and

(ii) cause performance of one or more procedures in accordance with the one or more identified components to determine a condition status associated with each of the one or more identified components; and

memory, coupled to the at least one processor, operative to store at least a portion of results associated with the identifying and performing operations;

wherein root cause determination is performed by the at least one processor for two or more subject components substantially concurrently.

35. An article of manufacture for determining a root cause of a condition of at least one subject component in a computing environment, wherein the computing environment comprises a plurality of managed components, comprising a machine readable medium containing one or more programs which when executed implement the steps of:

identifying one or more managed components in the computing environment upon which the at least one subject component depends, wherein identification comprises traversing at least a portion of a model representative of an existence of one or more relationships associated with at least a portion of the managed components of the computing environment and which is capable of accounting for a full lifecycle associated with at least one managed component of the computing environment; and

performing one or more procedures in accordance with the one or more identified managed components to determine a condition status associated with each of the one or more identified managed components.