System for decomposing events from managed infrastructures with semantic clustering

• US 10,693,707 B2
• Filed: 07/24/2018
• Issued: 06/23/2020
• Est. Priority Date: 01/27/2015
• Status: Active Grant

• Current Assignee: Dell Products LP (Dell Technologies Inc.)
• Original Assignee: Moogsoft Inc. (Dell Technologies Inc.)
• Inventors: Tee, Philip, Harper, Robert Duncan
• Primary Examiner(s): Lazaro, David R
• Assistant Examiner(s): Khurshid, Zia
• Application Number:
  US16/043,168
• Publication Number:
  US 20190036760A1
• Time in Patent Office: 700 Days
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 41/064 involving time analysis
• CPC Class Codes: H04L 41/065 involving logical or physical relationship, e.g. grouping and hierarchies
• CPC Class Codes: H04L 41/0886 Fully automatic configuration
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 41/145 involving simulating, designing, planning or modelling of a network
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 43/067 using time frame reporting
• CPC Class Codes: H04L 43/0823 Errors, e.g. transmission errors
• CPC Class Codes: H04L 43/10 Active monitoring, e.g. heartbeat, ping or trace-route
• CPC Class Codes: H04L 63/0272 Virtual private networks
• CPC Class Codes: H04L 63/029 Firewall traversal, e.g. tunnelling or, creating pinholes
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416

Abstract

A system is provided for decomposing events from managed infrastructures. A first engine is configured to receive message data from a managed infrastructure that includes managed infrastructure physical hardware that supports the flow and processing of information, the at least one engine is configured to determine common characteristics of events and produce clusters of events relating to the failure of errors in the managed infrastructure. Membership in a cluster indicates a common factor of the events that is a failure or an actionable problem in a physical hardware of the managed infrastructure directed to supporting the flow and processing of information. The first engine is configured to create one or more situations that is a collection of one or more events or alerts representative of the actionable problem in the managed infrastructure. A second engine is configured to determine one or more common steps from events and produces clusters relating to events. The second engine determines one or more common characteristics of events and produces clusters of events relating to the failure or errors in the managed infrastructure. The system is configured to use data-driven fault localization, more particularly using semantic clustering.

Claims

1. A system for decomposing events from a managed infrastructure,

comprising:

one or more processors;

a memory including instructions that when executed by the one or more processors executes the following;

a first engine receiving data from a managed infrastructure that includes managed infrastructure physical hardware which supports the flow and processing of information;

a second engine coupled to the first engine determining common characteristics of events and produces clusters of events relating to the failure of errors in the managed infrastructure, where membership in a cluster indicates a common factor of the events that is a failure or an actionable problem in the physical hardware managed infrastructure directed to supporting the flow and processing of information, and producing events that relate to the managed infrastructure while converting the events into words and subsets used to group the events that relate to failures or errors in the managed infrastructure, including the managed infrastructure physical hardware; and

a semantic clustering engine coupled to the first and second engines taking text from an event source, manipulates the text to produce a feature vector for each of an event, and the feature vector is feed into a clustering engine, with semantic clustering looking at text description of events, analyses the words in the text description, and groups them with other events when there is significant overlap; and

wherein a change to a managed infrastructure physical hardware component is made on the semantic clustering.

2. The system of claim 1, further comprising:

using a vector quantization for cluster analysis.

3. The system of claim 2, wherein the vector quantization is a K-means algorithm.

4. The system of claim 1, wherein the clustering engine executes on a sliding window of data on a period basis.

5. The system of claim 4, wherein a variety of the period basis is utilized.

6. The system of claim 1, wherein the system periodically takes a snapshot of the prior X minutes of data to group data alerts.

7. The system of claim 4, wherein attributes are used for grouping data.

8. The system of claim 7, wherein a choice of attributes is taken at a deployment level that is determined by a client.

9. The system of claim 8, wherein the attributes are turned into a feature vector that is executed using the sliding data worth of data.

10. The system of claim 8, wherein a similarity of the attributes is grouped together.

11. The system of claim 1, wherein the clustering engine operates in a streaming manner for data.

12. The system of claim 1, wherein the clustering engine operates in a streaming manner of the data that streams continuously for a selected period of time.

13. The system of claim 12, wherein every time an event arrives it is at least one of:

allocated to a cluster; and

a new cluster is created.

14. The system of claim 13, wherein attributes are turned into a feature vector that is executed using the streaming manner of data.

15. The system of claim 1, wherein the first engine is an extraction engine.

16. The system of claim 1, wherein the second engine is a signaliser engine.

17. The system of claim 1, further comprising:

a compare and merge engine coupled to the first and second engines and receiving outputs from the second engine, the compare and merge engine communicating with one or more user interfaces in a situation room.

18. The system of claim 1, wherein the first engine in operation receives messages from the managed infrastructure.

19. The system of claim 1, wherein the first engine in operation produces events that relate to the managed infrastructure.

20. The system of claim 19, wherein the events are converted into words and subsets used to group the events into clusters that relate to failures or errors in the managed infrastructure.

21. The system of claim 1, wherein the second engine includes one or more of an Non-negative Matrix Factorization NMF engine, a k-means clustering engine and a topology proximity engine.