Heterogeneous log analysis
First Claim
Patent Images
1. A method for heterogeneous log analysis, comprising:
- performing hierarchical log clustering on heterogeneous logs to generate a log cluster hierarchy for the heterogeneous logs by pruning search space for efficient querying, the pruning reducing processing time and memory requirements for the querying, the log cluster hierarchy including log clusters conforming to heterogenous and unknown log formats, the unknown log formats being domain-knowledge-free, and including logs from arbitrary and unknown applications;
performing, by a log pattern recognizer device having a processor, log pattern recognition on the log cluster hierarchy to generate log pattern representations, the log pattern recognition determining optimal representative layouts and log fields by executing context-free motif discovery within the log clusters;
generating log field statistics by automatically performing domain-knowledge-free log field analysis on the log pattern representations;
performing log indexing on the log pattern representations to generate searchable log indexes; and
searching the log cluster hierarchy based on the log indexes and the log field statistics responsive to a log query to generate log query results.
2 Assignments
0 Petitions
Accused Products
Abstract
A method and system are provided for heterogeneous log analysis. The method includes performing hierarchical log clustering on heterogeneous logs to generate a log cluster hierarchy for the heterogeneous logs. The method further includes performing, by a log pattern recognizer device having a processor, log pattern recognition on the log cluster hierarchy to generate log pattern representations. The method also includes performing log field analysis on the log pattern representations to generate log field statistics. The method additionally includes performing log indexing on the log pattern representations to generate log indexes.
10 Citations
18 Claims
-
1. A method for heterogeneous log analysis, comprising:
-
performing hierarchical log clustering on heterogeneous logs to generate a log cluster hierarchy for the heterogeneous logs by pruning search space for efficient querying, the pruning reducing processing time and memory requirements for the querying, the log cluster hierarchy including log clusters conforming to heterogenous and unknown log formats, the unknown log formats being domain-knowledge-free, and including logs from arbitrary and unknown applications; performing, by a log pattern recognizer device having a processor, log pattern recognition on the log cluster hierarchy to generate log pattern representations, the log pattern recognition determining optimal representative layouts and log fields by executing context-free motif discovery within the log clusters; generating log field statistics by automatically performing domain-knowledge-free log field analysis on the log pattern representations; performing log indexing on the log pattern representations to generate searchable log indexes; and searching the log cluster hierarchy based on the log indexes and the log field statistics responsive to a log query to generate log query results. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A system for heterogeneous log analysis, comprising:
-
a hierarchical log clusterer for performing hierarchical log clustering on heterogeneous logs to generate a log cluster hierarchy for the heterogeneous logs by pruning search space for efficient querying, the pruning reducing processing time and memory requirements for the querying, the log cluster hierarchy including log clusters conforming to heterogenous and unknown log formats, the unknown log formats being domain-knowledge-free, and including logs from arbitrary and unknown applications; a log pattern recognizer device having a processor for performing log pattern recognition on the log cluster hierarchy to generate log pattern representations, the log pattern recognition determining optimal representative layouts and log fields by executing context-free motif discovery within the log clusters; a log field analyzer for generating log field statistics by automatically performing domain-knowledge-free log field analysis on the log pattern representations; a log indexer for performing log indexing on the log pattern representations to generate searchable log indexes; and a heterogeneous log analyzer for searching the log cluster hierarchy based on the log indexes and the log field statistics responsive to a log query to generate log query results. - View Dependent Claims (15, 16, 17, 18)
-
Specification