AUTOMATIC ANALYSIS OF LOG ENTRIES THROUGH USE OF CLUSTERING

• US 20110131453A1
• Filed: 12/02/2009
• Published: 06/02/2011
• Est. Priority Date: 12/02/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Fernandess, Yaacov, Rodeh, Ohad, Shpigelman, Lavi
• Granted Patent: US 8,230,259 B2
• Time in Patent Office: Days
• US Class Current: 714/37
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36

Abstract

A set of log entries is automatically inspected to determine a bug. A training set is utilized to determine clustering of log identifications. Log entries are examined in real-time or retroactively and matched to clusters. Timeframe may also be matched to a cluster based on log entries associated with the timeframe. Error indications may be outputted to a user of the system in respect to a log entry or a timeframe.

Claims

1. A computerized system comprising:

an input module configured to obtain log entries;

a learning module configured to determine a matching function based on a first portion of the log entries, the matching function is configured to match a log entry with an at least one cluster;

a cluster associator configured to associate a second portion of the log entries with the at least one cluster, said cluster associator is configured to utilize the matching function determined by said learning module;

an output module configured to provide an indication referring to the at least one cluster associated with the first portion of the log entries and the second portion of the log entries; and

a processor.

2. The computerized system of claim 1, wherein said learning module comprises a learning dataset obtainer.

3. The computerized system of claim 1, wherein said learning module comprises a number-of-clusters selector.

4. The computerized system of claim 1, wherein said learning module is configured to determine a matching function between the log entry and a hierarchical arrangement of the at least one cluster.

5. The computerized system of claim 1 further comprising a cluster labeler, said cluster labeler is configured to label the at least one cluster.

6. The computerized system of claim 1 further comprising a timeframe associator, said timeframe associator is configured to associate an at least one timeframe with the at least one cluster.

7. The computerized system of claim 6, wherein said timeframe associator further comprising a cluster identification threshold module.

8. The computerized system of claim 6 further comprising a timeframe labeler, said timeframe labeler is configured to label a portion of the at least one timeframe.

9. The computerized system of claim 8 further comprising a validation unit, said validation unit is configured to validate the association of the at least one label with the portion of the at least one timeframe.

10. The computerized system of claim 6 further comprising an error indicator, the error indicator is configured to determine a possibility of a failure event associated with the at least one timeframe.

11. The computerized system of claim 10 further comprising a score determinator configured to determine a score of the at least one timeframe.

12. The computerized system of claim 11 further comprising a log entry filter configured to remove association of a log entry with the at least one timeframe, and wherein said score determinator is configured to determine the score in respect to log entries associated with the at least one timeframe.

13. A method in a computerized environment, said method comprising:

obtaining log entries;

determining a matching function between a log entry and an at least one cluster based on a first portion of the log entries;

associating a second portion of the log entries with the at least one cluster, based on the matching function; and

providing an indication referring to the at least one cluster associated with the first portion of the log entries and the second portion of the log entries;

whereby the first portion of log entries and the second portion of log entries are transformed to an at least one indication of the at least one cluster.

14. The method of claim 13 further comprising obtaining a learning dataset.

15. The method of claim 13 further comprises determining a number-of-clusters.

16. The method of claim 13, wherein determining a matching function between the log entry and a hierarchical arrangement of the at least one cluster.

17. The method of claim 13 further comprises labeling the at least one cluster.

18. The method of claim 13 further comprises associating an at least one timeframe with the at least one cluster.

19. The method of claim 18 further comprises determining a cluster identification threshold.

20. The method of claim 18 further comprises labeling a portion of the at least one timeframe.

21. The method of claim 20 further comprises validating the association of an at least one label with the portion of the at least one timeframe.

22. The method of claim 18 further comprises:

determining a possibility of a failure event associated with the at least one timeframe; and

providing an indication associated with the failure event.

23. The method of claim 22 further comprises:

filtering a portion of the log entries associated with the at least one timeframe; and

determining a score of the at least one timeframe; and

wherein the score is indicative of the possibility of the failure event.

24. A computer program product comprising:

a computer readable media;

first program instruction for obtaining log entries;

second program instruction for determining a matching function between a log entry and an at least one cluster based on a first portion of the log entries;

third program instruction for associating a second portion of the log entries with the at least one cluster, based on the matching function;

fourth program instruction for providing an indication referring to the at least one cluster associated with the first portion of the log entries and the second portion of the log entries;

wherein said first, second, third and fourth program instructions are stored on said computer readable media.