Pattern matching framework for log analysis

• US 9,400,733 B2
• Filed: 12/01/2010
• Issued: 07/26/2016
• Est. Priority Date: 12/01/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Abraham, Robin
• Primary Examiner(s): SOMERS, MARC S
• Application Number:
  US12/958,099
• Publication Number:
  US 20120143893A1
• Time in Patent Office: 2,064 Days
• Field of Search: 707/705, 707/706, 707/758, 707/769
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 16/2428 Query predicate definition using graphical user interfaces, including menus and forms G06F16/2423 takes precedence
• CPC Class Codes: G06F 2201/86 Event-based monitoring

Abstract

A pattern matching framework for log analysis is described. In one or more implementations, one or more inputs are received via a user interface of a computing device that describe a filter pattern that specifies data that is to be matched and extracted from a log and a projection pattern that specifies how at least a portion of the data extracted using the filter pattern is to be output. A query is formed from the filter pattern and the projection pattern by the computing device that is configured to analyze the log.

Claims

1. A method comprising:

outputting, by a computing device, a user interface configured to display data elements that are user-selectable to arrange and represent data patterns that include a filter pattern defining a search for data and a projection pattern defining a data output for the data that is matched by the filter pattern;

receiving one or more inputs via the user interface of the computing device to define and arrange the data elements describing the filter pattern that specifies the data to be matched and extracted from a log, and the data elements describing the projection pattern that specifies a format of the data output for at least a portion of the data extracted from the log using the filter pattern, the specification by the projection pattern including global variables, bound variables from the filter pattern, aggregation functions, and type cast operations; and

forming, by the computing device, a query from the filter pattern and the projection pattern to analyze the log.

2. A method as described in claim 1, wherein:

the data elements are configured to specify particular data for respective said data elements that is to be matched from the log; and

the arrangement of the data elements, one to another, is configured to specify an arrangement of the data in the log that is to be considered as a match of the filter pattern.

3. A method as described in claim 1, wherein:

the data elements are configured to specify particular data for respective said data elements that is to be output from the data extracted from the log by the filter pattern; and

the arrangement of the data elements, one to another, is configured to specify an arrangement of the data extracted from the log that is to be considered as a match of the projection pattern.

4. A method as described in claim 1, wherein the user interface is configured to specify that values are to be bound to corresponding variables that are represented by respective said data elements.

5. A method as described in claim 1, wherein the specification by the projection pattern includes injecting one or more labels to the portion of the data extracted.

6. A method as described in claim 1, wherein the user interface is configured to specify a number of occurrences of a match in the log of at least a portion of a respective said filter pattern or projection pattern for finding a match for the respective said filter pattern or projection pattern.

7. A method as described in claim 1, wherein the user interface is configured to specify separation of a number of lines in the log of respective portions of a respective said filter pattern or projection pattern for finding a match for the respective said filter pattern or projection pattern.

8. A method as described in claim 1, wherein the user interface is configured to define portions of a respective said filter pattern or projection pattern as separated by an undefined number of lines in the log for finding a match for the respective said filter pattern or projection pattern.

9. A method as described in claim 1, wherein the global variables include a data file filename, a first timestamp indicating a start of query processing, a second timestamp indicating an end of the query processing, and a current date.

10. A computing device, comprising:

a display device configured to display a user interface that displays data elements, which are user-selectable to arrange in the user interface and represent data patterns that include a filter pattern defining a search for data and a projection pattern defining a data output for the data that is matched by the filter pattern;

a memory and processor system to implement a query builder module that is configured to;

receive one or more inputs via the user interface to define and arrange the data elements describing the filter pattern that specifies the data to be matched and extracted from a log, and the data elements describing the projection pattern that specifies a format of the data output for at least a portion of the data extracted from the log using the filter pattern, the specification by the projection pattern including global variables, bound variables from the filter pattern, aggregation functions, and type cast operations; and

form a query from the filter pattern and the projection pattern, as defined by the arrangement of the data elements, one to another, to analyze the log.

11. A computing device as described in claim 10, wherein the filter pattern is represented as a two-dimensional pattern by the arrangement of the data elements, and the filter pattern specifies the data that is to be matched and extracted from the log.

12. A computing device as described in claim 11, wherein the projection pattern that specifies whether one or more labels are injected into at least a portion of the data extracted using the filter pattern is to be output.

13. A computing device as described in claim 10, wherein the user interface is configured to specify that values are to be bound to corresponding variables that are represented by respective said arrangement of the data elements to represent the data patterns.

14. A computing device as described in claim 10, wherein the user interface is configured to specify a number of occurrences of a match in the log of at least a portion of the filter pattern for finding a match for the filter pattern.

15. A computing device as described in claim 10, wherein the user interface is configured to specify separation of a number of lines in the log of portions of the filter pattern for finding a match for the filter pattern.

16. A computing device as described in claim 10, wherein the user interface is configured to define portions of the filter pattern as separated by an undefined number of lines in the log for finding a match for the filter pattern.

17. A method comprising:

receiving a query at a computing device, the query including a filter pattern and a projection pattern, the filter pattern defining a search for data and the projection pattern defining a data output for the data that is matched by the filter pattern, each of the filter pattern and the projection pattern defined by data patterns represented by data elements that are user-selectable and arranged in a user interface;

analyzing a log by the computing device using the query, the analyzing including;

extracting the data from the log that matches the filter pattern; and

outputting the extracted data that matches the projection pattern, which specifies a format of the data output for at least a portion of the data that is to be output to a data file based in part on the arrangement of the data elements, the specification by the projection pattern including global variables, bound variables from the filter pattern, aggregation functions, and type cast operations.

18. A method as described in claim 17, wherein the filter pattern or the projection pattern is configured to specify a number of occurrences of a match in the log of at least a portion of a respective said filter pattern or projection pattern for finding a match for the respective said filter pattern or projection pattern.

19. A method as described in claim 17, wherein the filter pattern or the projection pattern is configured to specify separation of a number of lines in the log of respective portions of a respective said filter pattern or projection pattern for finding a match for the respective said filter pattern or projection pattern.

20. A method as described in claim 17, wherein the filter pattern or the projection pattern is configured to define portions of a respective said filter pattern or projection pattern as separated by an undefined number of lines in the log for finding a match for the respective said filter pattern or projection pattern.