Automatic analysis of log entries through use of clustering

• US 8,386,854 B2
• Filed: 03/12/2012
• Issued: 02/26/2013
• Est. Priority Date: 12/02/2009
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Fernandess, Yaacov, Rodeh, Ohad, Shpigelman, Lavi
• Primary Examiner(s): Lottich, Joshua P
• Application Number:
  US13/417,429
• Publication Number:
  US 20120173466A1
• Time in Patent Office: 351 Days
• Field of Search: 714/4.1, 714/4.4, 714/20, 714/43, 714/47.1, 714/47.2, 714/48
• US Class Current: 714/47.1
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36

Abstract

A set of log entries is automatically inspected to determine a bug. A training set is utilized to determine clustering of log identifications. Log entries are examined in real-time or retroactively and matched to clusters. Timeframe may also be matched to a cluster based on log entries associated with the timeframe. Error indications may be outputted to a user of the system in respect to a log entry or a timeframe.

Claims

1. A method in a computerized environment, said method comprising:

obtaining a training set of log entries, wherein the log entries describe an operation of a system, wherein the log entries comprise at least one log entry which is indicative of a non-erroneous operation of the system;

automatically determining an approximated matching function between a log entry and at least one cluster based on the training set of the log entries, wherein said automatically determining the approximated matching function is performed using a machine learning process;

obtaining another set of log entries describing operation of the system;

associating the other set of log entries with the at least one cluster, based on the approximated matching function; and

providing an indication referring to the at least one cluster associated with the training set of the log entries and the other set of the log entries.

2. The method of claim 1 further comprising determining a number-of-clusters.

3. The method of claim 1, wherein said automatically determining the approximated matching function further comprises determining an approximated matching function between a log entry and a hierarchical arrangement of the at least one cluster.

4. The method of claim 1 further comprising labeling the at least one cluster.

5. The method of claim 1 further comprising associating at least one timeframe with the at least one cluster.

6. The method of claim 5 further comprising determining a cluster identification threshold.

7. A computerized apparatus, the apparatus comprising a hardware processor which is arranged to:

obtain a training set of log entries, wherein the log entries describe an operation of a system, wherein the log entries comprise at least one log entry which is indicative of a non-erroneous operation of the system;

automatically determine an approximated matching function between a log entry and at least one cluster based on the training set of the log entries, wherein said automatically determining the approximated matching function is performed using a machine learning process;

obtain another set of log entries describing operation of the system;

associate the other set of log entries with the at least one cluster, based on the approximated matching function; and

provide an indication referring to the at least one cluster associated with the training set of the log entries and the other set of the log entries.

8. The apparatus of claim 7, wherein said processor is further arranged to determine a number-of-clusters.

9. The apparatus of claim 7, wherein the automatic determination of the approximated matching function comprises determining an approximated matching function between a log entry and a hierarchical arrangement of the at least one cluster.

10. The apparatus of claim 7, wherein said processor is further arranged to label the at least one cluster.

11. The apparatus of claim 7, wherein said processor is further arranged to associate at least one timeframe with the at least one cluster.

12. The apparatus of claim 11, wherein said processor is further arranged to determine a cluster identification threshold.

13. A computer program product, said computer program product comprising a non-transitory computer readable medium, in which computer instructions are stored, which instructions, when read by a computer, cause the computer to:

obtain a training set of log entries, wherein the log entries describe an operation of a system, wherein the log entries comprise at least one log entry which is indicative of a non-erroneous operation of the system;

automatically determine an approximated matching function between a log entry and at least one cluster based on the training set of the log entries, wherein said automatically determining the approximated matching function is performed using a machine learning process;

obtain another set of log entries describing operation of the system;

associate the other set of log entries with the at least one cluster, based on the approximated matching function; and

provide an indication referring to the at least one cluster associated with the training set of the log entries and the other set of the log entries.

14. The computer program product of claim 13, wherein said instructions, when read by the computer, further cause the computer to determine a number-of-clusters.

15. The computer program product of claim 13, wherein the automatic determination of the approximated matching function comprises determining an approximated matching function between a log entry and a hierarchical arrangement of the at least one cluster.

16. The computer program product of claim 13, wherein said instructions, when read by the computer, further cause the computer to label the at least one cluster.

17. The computer program product of claim 13, wherein said instructions, when read by the computer, further cause the computer to associate at least one timeframe with the at least one cluster.

18. The computer program product of claim 17, wherein said instructions, when read by the computer, further cause the computer to determine a cluster identification threshold.