Systems and method for event parsing

• US 10,719,375 B2
• Filed: 03/13/2018
• Issued: 07/21/2020
• Est. Priority Date: 03/13/2018
• Status: Active Grant

• Current Assignee: ServiceNow Incorporated
• Original Assignee: ServiceNow Incorporated
• Inventors: Tucker, Stephen, Li, Qingbin
• Primary Examiner(s): Onat, Umut
• Application Number:
  US15/920,071
• Publication Number:
  US 20190286500A1
• Time in Patent Office: 861 Days
• Field of Search: None
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06F 9/547 Remote procedure calls [RPC]; Web services
• CPC Class Codes: H04L 41/0213 Standardised network management protocols, e.g. simple network management protocol [SNMP]
• CPC Class Codes: H04L 43/028 by filtering
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level

Abstract

A system includes a non-transitory memory and a hardware processors configured to perform operations including receiving a plurality of events from one or more network monitoring systems, wherein each event includes a message output by a network monitoring system communicating a status of a network resource connected to a network, clustering similar events into one or more event clusters, extracting an event template for each event cluster, extracting a regular expression (regex) for each event cluster, grouping the events into one or more groups of events having the same or similar extracted regexes, and outputting the one or more groups of events.

Claims

1. A system, comprising:

a non-transitory memory; and

one or more hardware processors configured to read instructions from the non-transitory memory to perform operations comprising;

receiving a plurality of events from one or more network monitoring systems, wherein each event of the plurality of events comprises a message output by a network monitoring system of the one or more network monitoring systems that communicates a status of a network resource connected to a network;

normalizing the plurality of events comprising removing punctuation, unnecessary words or characters, or both, from the plurality of events to generate a plurality of normalized events;

clustering similar events of the plurality of normalized events into one or more normalized event clusters;

after clustering the similar events into the one or more normalized event clusters;

extracting an event template for each of the one or more normalized event clusters, wherein the event template comprises boiler plate language included in the message of one or more of the similar events and a placeholder for a character string; and

extracting a regular expression (regex) for each of the one or more normalized event clusters that, when searched, returns the character string;

automatically grouping the plurality of events into one or more groups of events, wherein each group of events comprises one or more events of the plurality of events having identical extracted regexes or similar extracted regexes; and

outputting the one or more groups of events.

2. The system of claim 1, wherein the operations comprise retrieving the plurality of events from an events database.

3. The system of claim 1, wherein clustering similar events of the plurality of normalized events into the one or more normalized event clusters comprises graph-based clustering of the plurality of normalized events.

4. The system of claim 3, wherein graph-based clustering of the plurality of normalized events comprises:

generating a node for each event of the plurality of normalized events;

graphing each node for each event of the plurality of normalized events, wherein nodes for events having identical event templates or similar event templates form a cluster; and

connecting two or more of the nodes with branches.

5. The system of claim 1, wherein clustering similar events of the plurality of normalized events into the one or more normalized event clusters comprises performing a similarity calculation.

6. The system of claim 5, wherein performing the similarity calculation comprises calculating a Levenshtein distance.

7. The system of claim 1, wherein the operations comprise populating or updating an alerts database.

8. A system, comprising:

a plurality of external network monitors, each configured to monitor and to output event data corresponding to performance of one or more resources connected to a network;

an application server having a memory and a processor configured to execute instructions stored within the memory; and

an application instance that executes on the application server, wherein the application instance is configured to receive the event data via a web service API, store the event data in an events database, and process the event data stored in the events database, wherein the application instance comprises an events processor configured to;

retrieve a plurality of events from the event data stored in the events database;

normalize the plurality of events comprising removing punctuation, unnecessary words or characters, or both, from the plurality of events to generate a plurality of normalized events;

cluster similar events of the plurality of normalized events into one or more normalized event clusters;

after clustering the similar events into the one or more normalized event clusters;

extract an event template for each of the one or more normalized event clusters, wherein the event template comprises boiler plate language included in the event data of one or more of the similar events and a placeholder for a character string; and

extract a regular expression (regex) for each of the one or more normalized event clusters that, when searched, returns the character string;

automatically group the plurality of events into one or more groups of events, wherein each group of events comprises one or more events of the plurality of events having identical extracted regexes or similar extracted regexes; and

output the one or more groups of events to an alerts database.

9. The system of claim 8, wherein clustering similar events of the plurality of normalized events into the one or more normalized event clusters comprises graph-based clustering of the plurality of normalized events.

10. The system of claim 8, wherein clustering similar events of the plurality of normalized events into the one or more normalized event clusters comprises performing a similarity calculation.

11. The system of claim 8, wherein the application instance is configured to determine an impact of one or more alerts stored in the alerts database.

12. The system of claim 8, wherein the application instance is configured to take remedial action to address one or more alerts stored in the alerts database.

13. The system of claim 12, wherein the remedial action comprises creating a task for a member of an information technology service team to address the one or more alerts.

14. The system of claim 8, comprising a user interface client running on a client computing device, wherein the user interface client is configured to provide access to the application instance.

15. The system of claim 14, wherein the user interface client comprises:

a dashboard configured to communicate the performance of the one or more resources connected to the network;

an alert console configured to display one or more alerts stored in the alerts database; and

a service map comprising a graphical representation of the one or more resources connected to the network and one or more relationships between the one or more resources.

16. A method, comprising:

receiving a plurality of events from one or more network monitoring systems, wherein each event of the plurality of events comprises a message output by a network monitoring system of the one or more network monitoring systems that communicates a status of a network resource connected to a network;

normalizing the plurality of events comprising removing punctuation, unnecessary words or characters, or both, from the plurality of events to generate a plurality of normalized events;

clustering similar events of the plurality of normalized events into one or more normalized event clusters;

after clustering the similar events into the one or more normalized event clusters;

extracting an event template for each of the one or more normalized event clusters, wherein the event template comprises boiler plate language included in the message of one or more of the similar events and a placeholder for a character string; and

extracting a regular expression (regex) for each of the one or more normalized event clusters that, when searched, returns the character string;

automatically grouping the plurality of events into one or more groups of events, wherein each group of events comprises one or more events of the plurality of events having identical extracted regexes or similar extracted regexes; and

outputting the one or more groups of events to an alerts database.

17. The method of claim 16, comprising outputting the event templates to an event template database.

18. The method of claim 16, comprising outputting the identical extracted regexes and the similar extracted regexes to a regex database.

19. The method of claim 16, wherein clustering similar events of the plurality of normalized events into the one or more normalized event clusters comprises graph-based clustering of the plurality of normalized events.

20. The method of claim 16, wherein clustering similar events of the plurality of normalized events into the one or more normalized event clusters comprises performing a similarity calculation.