Automatic creation of related event groups for IT service monitoring

• US 10,505,825 B1
• Filed: 04/11/2017
• Issued: 12/10/2019
• Est. Priority Date: 10/09/2014
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Bettaiah, Vineetha, Fletcher, Tristan Antonio, Lazerowitz, Ross Andrew, Choudhary, Hemendra Singh
• Primary Examiner(s): Choi, David E
• Application Number:
  US15/485,222
• Time in Patent Office: 973 Days
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/24573 using data annotations, e.g. user-defined metadata
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06F 40/177 of tables; using ruled lines
• CPC Class Codes: H04L 43/028 by filtering
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 43/20 the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

The operation of an automatic service monitoring system (SMS) is directed by stored control information. Methods and mechanisms are provided to create control information that directs operations of the SMS regarding the grouping together of related notable events for unified display and processing. The control information directs grouping operations that automatically correlate the events without requiring, for example, a set of declarative grouping rules.

Claims

1. A method comprising:

(a) receiving a plurality of notable events of a service monitoring system (SMS) that performs service monitoring of an information technology (IT) environment;

(b) populating a candidate pool with level-1 group definitions, each level-1 group definition comprising a distinct fieldname-value pair identified among the data of the notable events;

(c) identifying zero or more subsets of level-1 group definitions in the candidate pool as satisfying merger criteria, and for each identified subset creating a level-N group definition and removing the subset from the candidate pool, wherein N is determined by the number of level-1 definitions in the subset, and wherein the level-N group definition comprises the N fieldname-value pairs of the level-1 definitions of the subset;

(d) establishing a level number, N, wherein N is initially 2, and wherein N is increased for subsequent iterations;

(e) identifying permutations between level-(N−

1) definitions of the candidate pool and level-1 definitions of the candidate pool that satisfy a permutation criteria, and for each identified permutation creating a level-N definition in the candidate pool, wherein the level-N definition comprises the N fieldname-value pairs from the permuted definitions; and

(f) performing iteratively (d) and (e) until a first termination criteria is satisfied;

wherein the method is performed by one or more processors.

2. The method of claim 1 wherein (b) further comprises omitting level-1 group definitions having low membership as determined using a culling threshold.

3. The method of claim 1 wherein the merger criteria includes consideration of a prominence threshold.

4. The method of claim 1 wherein the merger criteria includes consideration of a prominence threshold for identifying group definitions of the candidate pool having high membership.

5. The method of claim 1 wherein the merger criteria includes consideration of a prominence threshold for identifying group definitions of the candidate pool having high membership, and (c) further comprising:

identifying any group definition of the candidate pool having high membership based at least in part on the prominence threshold; and

promoting each group definition of the candidate pool identified as having high membership to a results pool.

6. The method of claim 1 wherein the merger criteria includes consideration of an event overlap threshold.

7. The method of claim 1 wherein the identifying the subsets of (c) includes a consideration of a total number of subsets identified and/or a determined portion of definitions of the candidate pool identified as having high membership based at least in part on a prominence threshold.

8. The method of claim 1 wherein created level-N group definition of (c) is created in the candidate pool.

9. The method of claim 1 wherein created level-N group definition of (c) is created in a results pool.

10. The method of claim 1 wherein the permutation criteria of (e) includes consideration of the membership size of a permuted definition.

11. The method of claim 1 wherein the permutation criteria of (e) includes consideration of the membership size of a permuted definition in comparison to a threshold determined at least in part on the average membership size of level-(N−

1) definitions of the candidate pool.

12. The method of claim 1 wherein the first termination criteria includes consideration of the number of definitions created in the candidate pool for an iteration of (f).

13. The method of claim 1 further comprising:

(g) promoting each level-N definition from the candidate pool to the results pool;

(h) identifying each level-(N−

1) definition in the candidate pool satisfying an overlap criteria, and removing each identified definition from the candidate pool;

(i) decreasing the level number, N, and performing iteratively (g) and (h) until a second termination criteria is satisfied; and

(j) storing control information for the SMS based at least in part on the results pool, wherein the control information determines realtime notable event grouping operations of the SMS.

14. The method of claim 1 further comprising:

(g) promoting each level-N definition from the candidate pool to the results pool;

(h) identifying each level-(N−

1) definition in the candidate pool satisfying a factor overlap criteria, and removing each identified definition from the candidate pool;

(i) decreasing the level number, N, and performing iteratively (g) and (h) until a second termination criteria is satisfied;

(j) storing control information for the SMS based at least in part on the results pool, wherein the control information determines realtime notable event grouping operations of the SMS; and

wherein the factor overlap criteria of (h) is based at least in part on N.

15. The method of claim 1 further comprising:

(g) promoting each level-N definition from the candidate pool to the results pool;

(h) identifying each level-(N−

1) definition in the candidate pool satisfying a factor overlap criteria, and removing each identified definition from the candidate pool;

(i) decreasing the level number, N, and performing iteratively (g) and (h) until a second termination criteria is satisfied;

(j) storing control information for the SMS based at least in part on the results pool, wherein the control information determines realtime notable event grouping operations of the SMS; and

wherein the second termination criteria of (i) is based on exhaustion of the candidate pool.

16. A system comprising:

a memory; and

a processing device coupled with the memory to;

(a) receive a plurality of notable events of a service monitoring system (SMS) that performs service monitoring of an information technology (IT) environment;

(b) populate a candidate pool with level-1 group definitions, each level-1 group definition comprising a distinct fieldname-value pair identified among the data of the notable events;

(c) identify zero or more subsets of level-1 group definitions in the candidate pool as satisfying merger criteria, and for each identified subset creating a level-N group definition and removing the subset from the candidate pool, wherein N is determined by the number of level-1 definitions in the subset, and wherein the level-N group definition comprises the N fieldname-value pairs of the level-1 definitions of the subset;

(d) establish a level number, N, wherein N is initially 2, and wherein N is increased for subsequent iterations;

(e) identify permutations between level-(N−

1) definitions of the candidate pool and level-1 definitions of the candidate pool that satisfy a permutation criteria, and for each identified permutation creating a level-N definition in the candidate pool, wherein the level-N definition comprises the N fieldname-value pairs from the permuted definitions; and

(f) perform iteratively (d) and (e) until a first termination criteria is satisfied.

17. The system of claim 16 wherein (b) further comprises omitting level-1 group definitions having low membership as determined using a culling threshold.

18. The system of claim 16 wherein the merger criteria includes consideration of a prominence threshold.

19. The system of claim 16 wherein the merger criteria includes consideration of a prominence threshold for identifying group definitions of the candidate pool having high membership.

20. The system of claim 16 wherein the merger criteria includes consideration of a prominence threshold for identifying group definitions of the candidate pool having high membership, and (c) further comprising:

identifying any group definition of the candidate pool having high membership based at least in part on the prominence threshold; and

promoting each group definition of the candidate pool identified as having high membership to a results pool.

21. The system of claim 16 wherein the merger criteria includes consideration of an event overlap threshold.

22. The system of claim 16 wherein the identifying the subsets of (c) includes a consideration of a total number of subsets identified and/or a determined portion of definitions of the candidate pool identified as having high membership based at least in part on a prominence threshold.

23. The system of claim 16 wherein created level-N group definition of (c) is created in the candidate pool.

24. The system of claim 16 wherein created level-N group definition of (c) is created in a results pool.

25. The system of claim 16 wherein the permutation criteria of (e) includes consideration of the membership size of a permuted definition.

26. The system of claim 16 wherein the permutation criteria of (e) includes consideration of the membership size of a permuted definition in comparison to a threshold determined at least in part on the average membership size of level-(N−

1) definitions of the candidate pool.

27. The system of claim 16 further comprising:

(g) promoting each level-N definition from the candidate pool to the results pool;

(h) identifying each level-(N−

1) definition in the candidate pool satisfying an overlap criteria, and removing each identified definition from the candidate pool;

(i) decreasing the level number, N, and performing iteratively (g) and (h) until a second termination criteria is satisfied; and

(j) storing control information for the SMS based at least in part on the results pool, wherein the control information determines realtime notable event grouping operations of the SMS.

28. The system of claim 16 further comprising:

(g) promoting each level-N definition from the candidate pool to the results pool;

(h) identifying each level-(N−

1) definition in the candidate pool satisfying a factor overlap criteria, and removing each identified definition from the candidate pool;

(i) decreasing the level number, N, and performing iteratively (g) and (h) until a second termination criteria is satisfied;

(j) storing control information for the SMS based at least in part on the results pool, wherein the control information determines realtime notable event grouping operations of the SMS; and

wherein the factor overlap criteria of (h) is based at least in part on N.

29. The system of claim 16 further comprising:

(g) promoting each level-N definition from the candidate pool to the results pool;

(h) identifying each level-(N−

1) definition in the candidate pool satisfying a factor overlap criteria, and removing each identified definition from the candidate pool;

(i) decreasing the level number, N, and performing iteratively (g) and (h) until a second termination criteria is satisfied;

(j) storing control information for the SMS based at least in part on the results pool, wherein the control information determines realtime notable event grouping operations of the SMS; and

wherein the second termination criteria of (i) is based on exhaustion of the candidate pool.

30. A non-transitory computer readable storage medium encoding instructions thereon that, in response to execution by one or more processing devices, cause the one or more processing devices to perform operations comprising:

(a) receiving a plurality of notable events of a service monitoring system (SMS) that performs service monitoring of an information technology (IT) environment;

(b) populating a candidate pool with level-1 group definitions, each level-1 group definition comprising a distinct fieldname-value pair identified among the data of the notable events;

(c) identifying zero or more subsets of level-1 group definitions in the candidate pool as satisfying merger criteria, and for each identified subset creating a level-N group definition and removing the subset from the candidate pool, wherein N is determined by the number of level-1 definitions in the subset, and wherein the level-N group definition comprises the N fieldname-value pairs of the level-1 definitions of the subset;

(d) establishing a level number, N, wherein N is initially 2, and wherein N is increased for subsequent iterations;

(e) identifying permutations between level-(N−

1) definitions of the candidate pool and level-1 definitions of the candidate pool that satisfy a permutation criteria, and for each identified permutation creating a level-N definition in the candidate pool, wherein the level-N definition comprises the N fieldname-value pairs from the permuted definitions; and

(f) performing iteratively (d) and (e) until a first termination criteria is satisfied.