Merging multi-line log entries

• US 20070260931A1
• Filed: 04/05/2006
• Published: 11/08/2007
• Est. Priority Date: 04/05/2006
• Status: Active Grant

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Arcsight Incorporated (HP Inc.)
• Inventors: Aguilar-Macias, Hector, Mantry, Girish
• Granted Patent: US 7,437,359 B2
• Time in Patent Office: Days
• US Class Current: 714/39
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99937 Sorting

Abstract

A system and method for building merged events from log entries received from multiple devices. Multiple log events generally contribute to a single merged event. In the described embodiment, the mapping module receives log entries associated with specific merged events and maps them to fields in the merged event data structure in accordance with mapping properties. The described embodiments of the invention use regular expressions in the merge properties to describe values that are searched for in the received log entries. A described embodiment of the present invention gives the mapping module access to the event under construction. A new conditional operator, oneOf, is introduced that selects the first token that is bound to a value out of a list of tokens.

Claims

1. A method for merging multiple log entries received by a data processing system, comprising:

receiving a plurality of log entries;

for each received log entry;

determining if the log entry contains an ID common to any potential merged event in accordance with merge properties,

beginning a new merged event, if the log entry is a beginning log entry of a merged event in accordance with the merge properties;

ending an existing merged event, if the log entry is an ending log entry of an existing merged event in accordance with the merge properties; and

mapping each log entry containing an ID common to an existing merged event to that merged event in accordance with mapping properties for the merged event.

2. The method of claim 1, further comprising:

ending an existing merged event if a timeout as defined in the merge properties for that merged event occurs.

3. The method of claim 2, further comprising identifying a log entry indicating a beginning of the merged event.

4. The method of claim 2, further comprising identifying a log entry indicating an end of the merged event.

5. The method of claim 2, further comprising identifying a log entry that neither implies a start nor an end of the merged event.

6. The method of claim 2, further comprising the ability to consider the merged event as it exists so far when merging in a new entry'"'"'s token.

7. The method of claim 2, further wherein the ability is used in a mapping operation.

8. The method of claim 1, further comprising:

determining whether each received log entry is to be considered for merging in accordance with the merge properties.

9. The method of claim 1, wherein mapping a log entry to a merged event further comprises determining a time of the merged event, the time being the time of a beginning log event for the merged event.

10. The method of claim 1, wherein mapping the log entry to a merged event further comprises determining a time of the merged event, the time being the time of an ending log event for the merged event.

11. The method of claim 1, wherein mapping a log entry to a merged event further comprises mapping an event ID in accordance with the mapping properties.

12. The method of claim 1, wherein mapping a log entry to a merged event further comprises mapping an event name in accordance with the mapping properties.

13. The method of claim 1, wherein mapping a log entry to a merged event further comprises mapping a name parsed from the log entry, the mapping performed in accordance with a oneOf function in the mapping properties.

14. The method of claim 1, wherein mapping a log entry to a merged event further comprises mapping a device action in accordance with the mapping properties.

15. The method of claim 1, wherein the received log entries contain log entries corresponding to more than one merged event mixed together.

16. The method of claim 1, wherein one received log entry is used to build more than one merged event.

17. The method of claim 1, wherein an ID comprises multiple fields in the log entry, the multiple fields acting to identify log entries that contribute to a merged event.

19. A system for merging multiple log entries received by a data processing system, comprising:

a module for receiving a plurality of log entries;

a parser for parsing the log entries into tokens;

a grouper that, for each received log entry;

determines if the log entry contains an ID common to any potential merged event in accordance with merge properties,

begins a new merged event, if the log entry is a beginning log entry of a merged event in accordance with the merge properties;

ends an existing merged event, if the log entry is an ending log entry of an existing merged event in accordance with the merge properties; and

a mapper that mapping each log entry containing an ID common to an existing merged event to that merged event in accordance with mapping properties for the merged event.

20. A computer program product comprising instructions stored on a computer readable medium for causing a computer to perform a method, comprising:

receiving a plurality of log entries;

for each received log entry;

determining if the log entry contains an ID common to any potential merged event in accordance with merge properties,

beginning a new merged event, if the log entry is a beginning log entry of a merged event in accordance with the merge properties;

ending an existing merged event, if the log entry is an ending log entry of an existing merged event in accordance with the merge properties; and

mapping each log entry containing an ID common to an existing merged event to that merged event in accordance with mapping properties for the merged event.