Apparatus and method for event correlation and problem reporting

• US 20050137832A1
• Filed: 01/12/2005
• Published: 06/23/2005
• Est. Priority Date: 05/25/1994
• Status: Active Grant

• Current Assignee: VMware, Inc. (Broadcom, Inc.)
• Original Assignee: System Management Arts, Inc. (Dell Technologies Inc.)
• Inventors: Kliger, Shmuel, Yemini, Yechiam, Yemini, Shaula
• Granted Patent: US 7,003,433 B2
• Time in Patent Office: Days
• US Class Current: 702/183
• CPC Class Codes: G06F 11/2257 using expert systems
• CPC Class Codes: G06F 11/2273 Test methods
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 2201/86 Event-based monitoring

Abstract

A computer implemented method on a computer readable media is provided for determining the source of a problem in a complex system of managed components based upon symptoms. The problem source identification process is split into different activities. Explicit configuration non-specific representations of types of managed components, their problems, symptoms and the relations along which the problems or symptoms propagate are created that can be manipulated by executable computer code. A data structure is produced for determining the source of a problem by combining one or more of the representations based on information of specific instances of managed components in the system. Computer code is then executed which uses the data structure to determine the source of the problem from one or more symptoms.

Claims

2. A computer readable media containing code thereon, said code providing instructions to a computer system to determine the effects of one or more events in a system of managed components by executing the steps of:

creating one or more configuration non-specific representations of types of managed components;

creating one or more configuration non-specific representations of events of said types of managed components;

creating configuration non-specific representations of relations along which the events and/or effects of said events propagate amongst the types of managed components,

producing a data structure for determining the effects of an event by combining one or more of said configuration non-specific representations based on information of specific instances of managed components in the system; and

utilizing said data structure to determine the corresponding effects on one or more managed components caused by the one or more events.

3. The computer readable media as recited in claim 2, wherein said corresponding effects are associated with symptoms and said events are associated with problems.

4. The computer readable media as recited in claim 2, wherein system is a distributed system.

5. The computer readable media as recited in claim 4, wherein said distributed system is partitioned into a plurality of system domains each of the system domains containing a subset of instances of managed components of the distributed system.

6. The computer readable media as recited in claim 5, the code further providing instruction to the computer system for executing the step of:

producing a data structure for each of the system domains for determining the effects of one or more events by combining a plurality of said configuration non-specific representations based on information of specific instances of managed components.

7. The computer readable media as recited in claim 6, the code further providing instruction to the computer system for executing the step of:

combining the determined effects in each of the selected smaller domains into one or more effects in said system.

8-1. The computer readable media as recited in claim 9, wherein the mismatch measure possesses a different weight to absence of a symptom data value than to presence of a symptom data value.

9. A computer readable media containing code thereon, said code providing instructions to a computer system for detecting in a distributed system problems among a plurality of problems which generate a plurality of symptoms, said problems and symptoms being mapped in a matrix of values by executing the steps of:

monitoring a plurality of symptom data values representing said plurality of symptoms;

determining a mismatch measure between each of a plurality of groups of said values in said matrix and said plurality of symptom data values; and

selecting one of said plurality of problems corresponding to one of said plurality of groups having the smallest mismatch measure.

1. A computer readable media containing code thereon, said code providing instructions to a computer system for executing the steps:

creating one or more configuration non-specific representations of types of managed components,

creating one or more configuration non-specific representations of problems of said types of managed components,

creating configuration non-specific representations of symptoms of said problems, and

creating configuration non-specific representations of relations along which the problems or symptoms propagate amongst the types of managed components,

said configuration non-specific representations of types of managed components, said configuration non-specific representations of problems of said types of managed components, said configuration non-specific representations of symptoms of said problems, and said configuration non-specific representations of relations along which the problems or symptoms propagate amongst the types of managed components,

producing a data structure for diagnosing a problem from the one or more symptoms by combining a plurality of said configuration non-specific representations based on information of specific instances of managed components in the system, and

diagnosing a problem of a managed component from the one or more symptoms using the data structure.

8. A computer readable media containing code thereon, said code providing instructions to a computer system for determining the effects of one or more events in a system of managed components by executing the steps of:

creating one or more configuration non-specific representations of types of managed components;

creating one or more configuration non-specific representations of events of said types of managed components; and

creating configuration non-specific representations of relations along which the events and/or effects of said events propagate amongst the types of managed components;

partitioning a system domain into a plurality of smaller domains, each said smaller domain containing a subset of instances of managed components of the system;

for each smaller domain, producing a data structure for determining the effects of one or more events by combining a plurality of said configuration non-specific representations based on information of specific instances of managed components in the smaller domains;

determining corresponding effects on one or more managed components caused by the one or more events for selected ones of the smaller domains utilizing corresponding data structures; and

combining the determined effects in each of the selected smaller domains into one or more effects in said system.

10. The computer readable media as recited in claim 9, wherein said code further providing instruction to the computer system for executing the step of:

generating a report comprising at least one of the plurality of problems.

11. The computer readable media as recited in claim 9, wherein the mismatch measure is a Hamming distance between each of said plurality of groups and said plurality of symptom data values.

12. The computer readable media as recited in claim 9, wherein the mismatch measure is the sum of individual mismatch measures taken over a plurality of pairs, each pair comprising one of the plurality of symptom data values and one of the plurality of values.

13. The computer readable media as recited in claim 9, wherein each of said values in said matrix of values reflects a probability that a likelihood that a symptom selected from the plurality of symptoms was caused by a problem selected from the plurality of problems.

14. The computer readable media as recited in claim 9, wherein said matrix values are specified as a pair comprising a first datum designating the probability and a second datum designating a temporal indicator corresponding to a time frame within which the probability is valid.

16. The computer readable media as recited in claim 9, wherein said code further providing instruction to the computer system for executing the step of:

generating the mapping of the problems and symptoms from a larger mapping of problems and symptoms by eliminating redundant information from the larger mapping of problems and symptoms.

17. The computer readable media as recited in claim 16, wherein the larger mapping of problems and symptoms is selected from the group consisting of:

a causality matrix and a causality graph.

18. The computer readable media as recited in claim 17, wherein the redundant information is stored in rows of the causality matrix.

19. The computer readable media as recited in claim 17, wherein the redundant information is represented as nodes in the causality graph.

20. The computer readable media as recited in claim 19, wherein the nodes are selected from the group consisting of:

symptom nodes and indistinguishable problem nodes.

21. The computer readable media as recited in claim 16, wherein the redundant information is eliminated until a desired degree of distinction between groups of said plurality of symptoms is achieved.

27. The computer readable media as recited in claim 9, wherein said code further providing instruction to the computer system for executing the step of:

specifying each of said values in said matrix of values as a probability reflecting a likelihood that an event was caused by at least one other event.

28. The method of claim 27, The computer readable media as recited in claim 9, wherein said code further providing instruction to the computer system for executing the step of:

specifying each of said values in said matrix of values as a pair of data, said pair of data comprising a first datum designating said probability and a second datum designating a temporal indicator corresponding to a time frame within which said probability holds true.

29. The method of claim 27, wherein each of said probabilities is a discrete value.

15. The computer readable media wherein the mismatch measure is a predetermined value.

22. A computer readable media containing code thereon, said code providing instructions to a computer system for analyzing events in a system by executing the steps of:

(1) partitioning a system domain representative of the system into a plurality of subdomains, each of said subdomains generating domain events, each domain event comprising one of the events in the system;

(2) providing a matrix of values for each subdomain, wherein each value corresponds to a mapping between one of said domain events and one of a plurality of likely other events in said system;

(3) monitoring for each subdomain event data values representing said domain events generated by said subdomain;

(4) determining for selected ones of said subdomains a mismatch measure between each of said matrix of values and said event data values for said subdomain and selecting the event having the smallest mismatch measure as the most likely cause event; and

(5) combining said selected mostly likely cause event in each of said selected subdomains to determine one or more likely cause events in said system.

23. The computer readable media as recited in claim 22, wherein said code further providing instruction to the computer system for executing the step of:

generating a subdomain report identifying the most likely event determined for each subdomain.

24. The computer readable media as recited in claim 22, the step of determining a mismatch measure comprises the step of:

determining a Hamming distance between said matrix of values corresponding to said subdomain and said event data values.

25. The computer readable media as recited in claim 22, wherein the step of determining a mismatch measure comprises the step of:

accumulating individual mismatch measures across a plurality of pairs, wherein each pair comprising one of the plurality of event data values and one of the plurality of values.

26. The computer readable media as recited in claim 24, wherein said code further providing instruction to the computer system for executing the step of:

providing a mismatch measure that gives a different weight to absence of an event than to presence of an event.