Methods for decomposing events from managed infrastructures

• US 9,535,973 B2
• Filed: 04/28/2014
• Issued: 01/03/2017
• Est. Priority Date: 04/29/2013
• Status: Active Grant

• Current Assignee: Dell Products LP (Dell Technologies Inc.)
• Original Assignee: Moogsoft Inc. (Dell Technologies Inc.)
• Inventors: Tee, Philip, Harper, Robert Duncan, Silvey, Charles Mike
• Primary Examiner(s): Reyes, Mariela
• Assistant Examiner(s): Vu, Thong
• Application Number:
  US14/262,890
• Publication Number:
  US 20140324794A1
• Time in Patent Office: 981 Days
• Field of Search: 707/692, 707/739, 707/639, 714/1, 714/37, 714/47.1, 714/25, 714/15
• US Class Current: 1/1
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/0769 Readable error formats, e.g. cross-platform generic formats, human understandable formats
• CPC Class Codes: G06F 11/0772 Means for error signaling, e.g. using interrupts, exception flags, dedicated error registers
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/358 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: H04L 41/0631 using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
• CPC Class Codes: H04L 41/065 involving logical or physical relationship, e.g. grouping and hierarchies
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 51/216 Handling conversation history, e.g. grouping of messages in sessions or threads
• CPC Class Codes: H04L 51/224 providing notification on incoming messages, e.g. pushed notifications of received messages

Abstract

Methods are provided for clustering events. Data is received at an extraction engine from managed infrastructure. Events are converted into alerts and the alerts mapped to a matrix M. One or more common steps are determined from the events and clusters of events are produced relating to the alerts and or events.

Claims

1. A method for clustering events, comprising:

receiving messages at an extraction engine from managed infrastructure that includes managed infrastructure physical hardware that supports the flow and processing of information;

producing events that relate to the managed infrastructure and converting the events into words and subsets used to group the events that relate to failures or errors in the managed infrastructure, including the managed infrastructure physical hardware;

providing a sigalizer engine that includes one or more of an NMF engine, a k-means clustering engine and a topology proximity engine, the sigalizer engine determining one or more common characteristics of events and producing clusters of events relating to the failure or errors in the managed infrastructure, the topology proximity engine using a source address for each event and a graph topology of the managed infrastructure which represents node to node connectivity of the topology proximity engine and assigns a graph coordinate to the event with an optional subset of attributes being extracted for each event and turned into a vector, the topology engine inputting a list of devices and a list a connections between components or nodes in the managed infrastructure;

where membership in a cluster indicates a common factor of the events that is a failure or an actionable problem in the physical hardware managed infrastructure directed to supporting the flow and processing of information; and

in response to production of the clusters making one or more physical changes in the managed infrastructure hardware.

2. The method of claim 1, further comprising:

using a source address for each event to assign a graph coordinate to the event with an optional subset of attributes being extracted for each event and turning that into a vector.

3. The method of claim 2, further comprising:

using the graph coordinates to cluster the events.

4. The method of claim 1, further comprising:

determining hop and a proximity of the source of the event.

5. The method of claim 4, further comprising:

calculating one or more distances between tile vectors.

6. The method of claim 5, further comprising:

bringing together events whose vectors are optimally close.

7. The method of claim 1, further comprising:

factoring the matrix M into A and B, where A is inspected and significant clusters are extracted, and B is used to assign a start and end time to each cluster.

8. The method of claim 1, further comprising:

collecting alerts into time buckets.

9. The method of claim 1, further comprising:

breaking messages into subsets of messages that relate to failures or errors in the managed infrastructure.

10. The method of claim 4, further comprising:

grouping subsets of messages into clusters.

11. The method of claim 10, further comprising:

extracting text components from the event messages and converting them into word and subtext.

12. The method of claim 11, further comprising, normalizing the works and subtexts.

13. The method of claim 12, further comprising:

mapping the works and subtexts that are normalized to a common, 0.0 and a non-common, 1.0.

14. The method of claim 1, further comprising:

normalizing entropy for events.

15. The method of claim 1, further comprising:

assigning entropy to the alerts.

16. The method of claim 1, further comprising:

running alerts in parallel with an activities of the extraction engine.

17. The method of claim 1, wherein membership in a cluster indicates a common underlying failure in the infrastructure.

18. The method of claim 1, further comprising:

eliminating duplicate copies of repeating data.

19. The method of claim 1, further comprising:

reducing a number of bytes in network data transfers that need to be sent.

20. The method of claim 1, further comprising:

determining sigalizer common steps to ascertain how many clusters to extract from events.

21. The method of claim 1, further comprising:

using entropy and a severity of an event for clustering.