Log summarization and diff

• US 10,635,682 B2
• Filed: 04/15/2016
• Issued: 04/28/2020
• Est. Priority Date: 12/15/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Parthasarathy, Srivatsan, Bhardwaj, Rohit, Gupta, Chirag, Malhotra, Vipul, Hissey, Evan Herschel Brodie
• Primary Examiner(s): Peng, Huawen A
• Application Number:
  US15/130,651
• Publication Number:
  US 20170169080A1
• Time in Patent Office: 1,474 Days
• CPC Class Codes: G06F 11/323 Visualisation of programs or trace data
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/2462 Approximate or statistical queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/26 Visual data mining; Browsing structured data
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 17/40 Data acquisition and logging for input to computer G06F3/00
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

Analyzing log data. The method includes obtaining a first bucket of a log data. The first bucket of log data includes a plurality of log lines. The method further includes analyzing the first bucket of log data to identify different sets of similar log lines. The method further includes providing to a user in a user interface one or more summaries of the different sets of similar lines. The summary comprises at least one user selectable indicator representing differences in log lines in a set of similar log lines that when selected by a user in the user interface reveals specific differences in the log lines in the set of similar log lines.

Claims

1. A computing system comprising:

one or more processors; and

one or more computer-readable media having stored thereon instructions that are executable by the one or more processors to configure the computer system to analyze log data, including instructions that are executable to configure the computer system to perform at least the following;

obtain a first bucket of a log data, the first bucket of log data comprising a plurality of log lines;

analyze the first bucket of log data to identify a first plurality of different sets of similar log lines, wherein identifying the first plurality of different sets comprises grouping log lines into different sets based on a similarity of log lines in a given set;

obtain a second bucket of log data;

analyze the log data in the second bucket to identify a second plurality of different sets of similar log lines;

cause a user interface to provide a comparison, to the user, of the first plurality of different sets from the first bucket to the second plurality of different sets from the second bucket including at least one graphical indication that indicates;

which sets occur in both the buckets, which sets occur in only one of the buckets, or changes in the size of sets between the buckets, wherein a size for a given set defines the number of log lines in the given set.

2. The system of claim 1, wherein one or more computer-readable media further have stored thereon instructions that are executable by the one or more processors to configure the computer system to show in the user interface one or more abbreviated summaries of different sets of similar lines, wherein the one or more abbreviated summaries comprise one or more templates showing how the members of each set are similar.

3. The system of claim 1, wherein one or more computer-readable media further have stored thereon instructions that are executable by the one or more processors to configure the computer system to display a distribution of differences between members of a given set of similar log lines.

4. The system of claim 1, wherein the first bucket and second bucket are selected based on a time dimension.

5. The system of claim 1, wherein the first bucket and second bucket are selected based on a machine dimension.

6. The system of claim 1, wherein the first bucket and second bucket are selected based on a transaction dimension.

7. The system of claim 1, wherein the first bucket and second bucket are selected based on a user dimension.

8. The system of claim 4, wherein log lines in the first bucket are collected from a first time in a sliding window and log lines in the second bucket are collected from a second time in a sliding window.

9. The computing system of claim 1, wherein each set of log lines in the first and second plurality of different sets of similar log lines is represented as a shaded element, wherein shading for each set of log lines in the first and second plurality of different sets of similar log lines is selected from a shading scheme that includes shades indicating that a set occurs in both buckets, that a set shrunk from the first bucket to the second bucket, that a set only occurs in the first bucket, that a set grew from the first bucket to the second bucket, and that a set only occurs in the second bucket.

10. The computing system of claim 1, wherein each set of log lines in the first and second plurality of different sets of similar log lines is represented over time to show a distribution of instances in a set over time.

11. In a computing environment, a method of analyzing log data, the method comprising:

obtaining a first bucket of a log data, the first bucket of log data comprising a plurality of log lines;

analyzing the first bucket of log data to identify a first plurality of different sets of similar log lines, wherein identifying the first plurality of different sets comprises grouping log lines into different sets based on a similarity of log lines in a given set;

obtain a second bucket of log data;

analyze the log data in the second bucket to identify a second plurality of different sets of similar log lines;

cause a user interface to provide a comparison, to the user, of the first plurality of different sets from the first bucket to the second plurality of different sets from the second bucket including at least one graphical indication that indicates;

which sets occur in both the buckets, which sets occur in only one of the buckets, or changes in the size of sets between buckets, wherein a size for a given set defines the number of log lines in the given set.

12. The method of claim 11, further comprising showing in the user interface one or more abbreviated summaries of different sets of similar lines, wherein the one or more abbreviated summaries comprise one or more templates showing how the members of each set are similar.

13. The method of claim 11, further comprising displaying a distribution of differences between members of a given set of similar log lines.

14. The method of claim 11, wherein the first bucket and second bucket are selected based on a time dimension.

15. The method of claim 11, wherein the first bucket and second bucket are selected based on a machine dimension.

16. The method of claim 11, wherein the first bucket and second bucket are selected based on a transaction dimension.

17. The method of claim 11, wherein the first bucket and second bucket are selected based on a user dimension.

18. The method of claim 14, wherein log lines are collected for the first bucket at a first time and log lines are collected for the second bucket at a predetermined offset from the first time.

19. A computing system comprising:

one or more processors;

a user interface, wherein the user interface is configured to;

obtain a first bucket of a log data, the first bucket of log data comprising a plurality of log lines;

analyze the first bucket of log data to identify a first plurality of different sets of similar log lines, wherein identifying the first plurality of different sets comprises grouping log lines into different sets based on a similarity of log lines in a given set; and

obtain a second bucket of log data;

analyze the log data in the second bucket to identify a second plurality of different sets of similar log lines;

cause a user interface to provide a comparison, to the user, of the first plurality of different sets from the first bucket to the second plurality of different sets from the second bucket including at least one graphical indication that indicates;

which sets occur in both the buckets, which sets occur in only one of the buckets, or changes in the size of sets between buckets, wherein a size for a given set defines the number of log lines in the given set.

20. The computing system of claim 19, wherein the first bucket and second bucket are selected based on a time dimension.

21. The computing system of claim 19, wherein the first bucket and second bucket are selected based on a machine dimension.

22. The computing system of claim 20, wherein log lines in the first bucket are collected from a first time in a sliding window and log lines in the second bucket are collected from a second time in a sliding window.