METHOD AND SYSTEM FOR IMPLEMENTING TARGET MODEL CONFIGURATION METADATA FOR A LOG ANALYTICS SYSTEM

• US 20160294649A1
• Filed: 04/01/2016
• Published: 10/06/2016
• Est. Priority Date: 04/03/2015
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: RUSSELL, Jerry Paul, ZHU, Shushuai
• Granted Patent: US 10,592,521 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/0766 Error or fault reporting or storing
• CPC Class Codes: G06F 11/0775 Content or structure details of the error report, e.g. specific table structure, specific error fields
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3072 where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
• CPC Class Codes: G06F 11/3086 where the reporting involves the use of self describing data formats, i.e. metadata, markup languages, human readable formats
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/84 Mapping; Conversion
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 40/16 Automatic learning of transformation rules, e.g. from examples
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 9/44505 Configuring for program initiating, e.g. using registry, configuration files
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H04L 41/145 involving simulating, designing, planning or modelling of a network
• CPC Class Codes: H04L 41/5074 Handling of user complaints or trouble tickets
• CPC Class Codes: H04L 43/04 : Processing captured monitoring data, e.g. for logfile generation

Abstract

Disclosed is a system, method, and computer program product for implementing a log analytics method and system that can configure, collect, and analyze log records in an efficient manner. The log analytics system, method, and computer program product provide target-based configuration of log monitoring metadata. The log analytics system, method, and computer program product can be implemented in a cloud-based/SaaS-based architecture.

Claims

1. A method, for configuring log monitoring to be performed in a system that implements a log analytics system, wherein the log monitoring comprises storing log data according to an association between log monitoring configuration metadata and a target, the target representing a component in a computing environment for which log data is generated, the method comprising:

generating the log monitoring configuration metadata by;

identifying a log source corresponding to a target, the log source identifying a location to monitor for the log data generated for the target;

identifying a parser for the log source;

configuring log monitoring configuration metadata comprising target-side log monitoring configuration metadata and server-side log monitoring configuration metadata, the target-side log monitoring configuration metadata including information regarding the log source and the server-side log monitoring configuration metadata including information regarding the parser for the log source;

distributing the target-side log monitoring configuration metadata to a host for the target, wherein the target is configured according to the target-side log monitoring configuration metadata;

configuring a log analytics server according to the server-side log monitoring configuration metadata;

gathering log data for the target from the host according to the target-side log monitoring configuration metadata; and

performing log processing on the log data at the log analytics server according to the server-side log monitoring configuration metadata.

2. The method of claim 1, wherein the log monitoring configuration metadata comprises at least one of a base parser, a log rule that is associated with the target, an identification of the target, or a type identification for the target.

3. The method of claim 1, wherein the parser comprises a regular expression to define how to parse the log data for the target.

4. The method of claim 1, wherein for a single host having multiple targets, the log monitoring configuration metadata corresponds to a single file having metadata portions that pertain to the multiple targets.

5. The method of claim 1, wherein the log monitoring configuration metadata corresponds to an XML file.

6. The method of claim 1, wherein the log monitoring configuration metadata is generated, modified, or deleted upon a change to the association between the target and the log source, a change to the target, or a change to a status of a log collection agent on the host.

7. The method of claim 1, wherein the log monitoring configuration metadata is periodically synchronized to one or more master metadata databases.

8. A computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method for configuring log monitoring to be performed in a system that implements a log analytics system, wherein the log monitoring comprises storing log data according to an association between log monitoring configuration metadata and a target, the target representing a component in a computing environment for which log data is generated, the method comprising:

generating the log monitoring configuration metadata by;

identifying a log source corresponding to a target, the log source identifying a location to monitor for the log data generated for the target;

identifying a parser for the log source;

configuring log monitoring configuration metadata comprising target-side log monitoring configuration metadata and server-side log monitoring configuration metadata, the target-side log monitoring configuration metadata including information regarding the log source and the server-side log monitoring configuration metadata including information regarding the parser for the log source;

distributing the target-side log monitoring configuration metadata to a host for the target, wherein the target is configured according to the target-side log monitoring configuration metadata;

configuring a log analytics server according to the server-side log monitoring configuration metadata;

gathering log data for the target from the host according to the target-side log monitoring configuration metadata; and

performing log processing on the log data at the log analytics server according to the server-side log monitoring configuration metadata.

9. The computer readable medium of claim 8, wherein the log monitoring configuration metadata comprises at least one of a base parser, a log rule that is associated with the target, an identification of the target, or a type identification for the target.

10. The computer readable medium of claim 8, wherein the parser comprises a regular expression to define how to parse the log data for the target.

11. The computer readable medium of claim 8, wherein for a single host having multiple targets, the log monitoring configuration metadata corresponds to a single file having metadata portions that pertain to the multiple targets.

12. The computer readable medium of claim 8, wherein the log monitoring configuration metadata corresponds to an XML, file.

13. The computer readable medium of claim 8, wherein the log monitoring configuration metadata is generated, modified, or deleted upon a change to the association between the target and the log source, a change to the target, or a change to a status of a log collection agent on the host.

14. The computer readable medium of claim 8, wherein the log monitoring configuration metadata is periodically synchronized to one or more master metadata databases.

15. A log analytics system implemented in a cloud or software-as-a-service architecture, the log analytics system comprising:

a configuration mechanism to configure log monitoring comprising a memory having stored thereon a sequence of instructions which, when executed by a processor, causes the processor to execute operations comprising by;

identifying a log source corresponding to a target, the log source identifying a location to monitor for log data generated for the target;

identifying a parser for the log source;

configuring log monitoring configuration metadata comprising target-side log monitoring configuration metadata and server-side log monitoring configuration metadata, the target-side log monitoring configuration metadata including information regarding the log source and the server-side log monitoring configuration metadata including information regarding the parser for the log source;

distributing the target-side log monitoring configuration metadata to a host for the target, wherein the target is configured according to the target-side log monitoring configuration metadata;

configuring a log analytics pipeline at a log analytics server according to the server-side log monitoring configuration metadata;

the log analytics pipeline receiving log data gathered from the host for the target according to the target-side log monitoring configuration metadata, wherein the log analytics pipeline performs log processing at the log analytics server according to the server-side log monitoring configuration metadata.

16. The log analytics system of claim 15, wherein the log monitoring configuration metadata comprises at least one of a base parser, a log rule that is associated with the target, an identification of the target, or a type identification for the target.

17. The log analytics system of claim 15, wherein the server-side log monitoring configuration metadata corresponds to a parser having a regular expression that defines how to parse the log data for the target.

18. The log analytics system of claim 15, wherein the log monitoring configuration metadata corresponds to an XML file.

19. The log analytics system of claim 15, wherein the log monitoring configuration metadata is generated, modified, or deleted upon a change to an association between the target and the log source, a change to the target, or a change to a status of a log collection agent on the host.

20. The log analytics system of claim 15, wherein the log monitoring configuration metadata is periodically synchronized to one or more master metadata databases.