Log processing and analysis

• US 10,642,712 B2
• Filed: 10/23/2017
• Issued: 05/05/2020
• Est. Priority Date: 10/24/2016
• Status: Active Grant

• Current Assignee: Yscope Inc.
• Original Assignee: Yscope Inc.
• Inventors: Luo, Yu, Rodrigues, Kirk, Stumm, Michael, Yuan, Ding, Zhao, Xu
• Primary Examiner(s): Wu, Daxin
• Application Number:
  US15/790,918
• Publication Number:
  US 20180113783A1
• Time in Patent Office: 925 Days
• Field of Search: None
• CPC Class Codes: G06F 11/323 Visualisation of programs or trace data
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 11/3604 Software analysis for verifying properties of programs byte-code verification G06F9/44589
• CPC Class Codes: G06F 11/3612 by runtime analysis performance monitoring G06F11/3466
• CPC Class Codes: G06F 11/3636 by tracing the execution of the program
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 2201/865 Monitoring of software

Abstract

A log of execution of an executable program is obtained. Log messages contained in the log are parsed to generate object identifiers representative of instances of programmatic elements in the executable program. Relationships among the object identifiers are identified. A representation of identified relationships is constructed and outputted as, for example, a visual representation.

Claims

1. A process comprising:

obtaining a log of execution of an executable program;

parsing log messages contained in the log without prior knowledge of a structure of the log messages, wherein parsing a particular log message involves parsing the particular log message into an object identifier and one or more of a string constant and a non-identifier, the object identifier being one or more segments of the particular log message not parsed as a string constant or a non-identifier, the object identifier representative of one or more instances of programmatic elements in the executable program;

identifying relationships among object identifiers of parsed log messages by inferring object types of objects represented by the object identifiers and identifying relationships among the object types of objects to obtain identified relationships, wherein an object type of the object identifier in the particular log message is inferred from a schema of the object identifier and the one or more of a string constant and a non-identifier in the particular log message;

constructing a representation of the identified relationships; and

outputting the representation, wherein the representation comprises a set of event timelines, each event timeline corresponding to an object, each event timeline depicting one or more nodes which each represent an event identified in a log message which contains the corresponding object, and wherein the event timelines are organized in nested hierarchical relationships corresponding to identified relationships of the corresponding object.

2. The process of claim 1, wherein the object types are empirical types that are not constrained to correspond to object types defined by a programming language of the executable program.

3. The process of claim 1, further comprising determining events from log messages and including the events in the representation.

4. The process of claim 1, wherein the log messages are generated by log statements provided to the executable program by a human programmer.

5. The process of claim 1, further comprising identifying relationship types of the identified relationships, and wherein the representation comprises indications of the relationship types of the identified relationships.

6. The process of claim 1, wherein the representation includes a visual connection between a node of a first event timeline and a node of a second event timeline, the connection indicating an event that contains object identifiers corresponding to the objects of the first and second event timelines.

7. A non-transitory machine-readable storage medium comprising instructions that, when executed by a processor:

discover a log of execution of an executable program;

extract a string constant from the executable program;

use the string constant to parse an unstructured log message contained in the log to generate an object identifier representative of an instance of a programmatic element in the executable program, wherein generating the object identifier involves identifying a segment of the unstructured log message that does not match the string constant or another non-identifier;

identify a relationship between the object identifier and another object identifier to obtain an identified relationship for output in a representation, wherein identifying the relationship involves inferring an object type of the object identifier from a schema of the object identifier and the string constant or another non-identifier in the unstructured log message,

construct a representation of the identified relationships; and

output the representation, wherein the representation comprises a set of event timelines, each event timeline corresponding to an object, each event timeline depicting one or more nodes which each represent an event identified in a log message which contains the corresponding object, and wherein the event timelines are organized in nested hierarchical relationships corresponding to identified relationships of the corresponding object.

8. The storage medium of claim 7, wherein the instructions are further to locate the executable program in a file system of a host.

9. The storage medium of claim 7, wherein the object type is an empirical type that is not constrained to correspond to object types defined by a programming language of the executable program.

10. The storage medium of claim 7, wherein the instructions are further to transmit an indication of the identified relationship to a server via a network.

11. The storage medium of claim 7, wherein the unstructured log message is generated by a log statement provided to the executable program by a human programmer.

12. The storage medium of claim 7, wherein the instructions are further to identify relationship types of the identified relationships, and wherein the representation comprises indications of the relationship types of the identified relationships.

13. The storage medium of claim 7, wherein the representation includes a visual connection between a node of a first event timeline and a node of a second event timeline, the connection indicating an event that contains object identifiers corresponding to the objects of the first and second event timelines.

14. A non-transitory machine-readable storage medium comprising instructions that, when executed by a processor:

extract string constants from an executable program;

obtain log data that includes identified relationships among object identifiers representative of instances of programmatic elements in the executable program;

construct a representation of the identified relationships; and

output the representation, wherein the log data is based on unstructured log messages generated by log statements provided to the executable program and parsed using the string constants to generate the object identifiers, wherein generating the object identifiers involves identifying segments of the unstructured log messages that do not match the string constants or other non-identifiers in the unstructured log messages, and wherein the identified relationships are identified by inferring the object types of the object identifiers from schemas of the object identifiers, string constants, and other non-identifiers in the unstructured log messages;

wherein the representation comprises a set of event timelines, each event timeline corresponding to an object, each event timeline depicting one or more nodes which each represent an event identified in a log message which contains the corresponding object, and wherein the event timelines are organized in nested hierarchical relationships corresponding to identified relationships of the corresponding object.

15. The storage medium of claim 14, wherein the instructions are further to identify relationship types of the identified relationships, and wherein the representation comprises indications of the relationship types of the identified relationships.

16. The storage medium of claim 14, wherein the representation includes a visual connection between a node of a first event timeline and a node of a second event timeline, the connection indicating an event that contains object identifiers corresponding to the objects of the first and second event timelines.