Method and system for implementing an operating system hook in a log analytics system

• US 9,767,171 B2
• Filed: 04/01/2016
• Issued: 09/19/2017
• Est. Priority Date: 04/03/2015
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Russell, Jerry Paul, He, Haobo, Ma, Greg, Xu, Xin
• Primary Examiner(s): Ho, Andy
• Application Number:
  US15/089,049
• Publication Number:
  US 20160378577A1
• Time in Patent Office: 536 Days
• Field of Search: 719318
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/0766 Error or fault reporting or storing
• CPC Class Codes: G06F 11/0775 Content or structure details of the error report, e.g. specific table structure, specific error fields
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3072 where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
• CPC Class Codes: G06F 11/3086 where the reporting involves the use of self describing data formats, i.e. metadata, markup languages, human readable formats
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/84 Mapping; Conversion
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 40/16 Automatic learning of transformation rules, e.g. from examples
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 9/44505 Configuring for program initiating, e.g. using registry, configuration files
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H04L 41/145 involving simulating, designing, planning or modelling of a network
• CPC Class Codes: H04L 41/5074 Handling of user complaints or trouble tickets
• CPC Class Codes: H04L 43/04 : Processing captured monitoring data, e.g. for logfile generation

Abstract

Disclosed is a system, method, and computer program product for implementing a log analytics method and system that can configure, collect, and analyze log records in an efficient manner. An improved approach is provided for identifying log files that have undergone a change in status that would require retrieve of its log data, by including a module directly into the operating system that allows the log collection component to be reactively notified of any changes to pertinent log files.

Claims

1. A method implemented with a processor, comprising:

to monitor for changes to one or more log files, configuring an operating system (OS) module to generate an event within an event log when any of a target set of operating-system-level system calls is made;

loading the OS module into an operating system of a host computing system;

operating the OS module within the operating system of the host computing system to detect an invocation of the any of the target set of operating-system-level system calls and to execute the invocation of the any of the target set of operating-system-level system calls, wherein the OS module does not detect invocation of untargeted operating-system-level system calls, and wherein the one or more log files are changed by one or more processes associated with one or more particular calls in the target set of operating-system-level system calls; and

identifying one or more events corresponding to the one or more log files changed by invocation of the any of the target set of operating-system-level system calls, wherein the one or more events correspond to a filtered subset of all log files that are changed by invocation of the any of the target set of operating-system-level system calls, and the one or more events are reviewable by a log collector to collect the one or more log files for a log analytics system.

2. The method of claim 1, wherein a filter is applied to generate the filtered subset that is recorded into the event log, the filtered subset identifiable based upon application of a filtering criteria to the invocation of the any of the target set of operating-system-level system calls.

3. The method of claim 2, wherein (a) the filter is applied before the one or more events are recorded into the event log, (b) the filter is applied asynchronously to remove filtered items from the event log, or (c) the filter is applied both before the one or more events are recorded into the event log and asynchronously to remove the filtered items from the event log.

4. The method of claim 2, wherein both a filtered event log and an unfiltered event log are maintained, the filtered event log storing events that meet the filtering criteria, and the unfiltered event log storing events regardless of whether the events meet the filtering criteria.

5. The method of claim 2, wherein the filtering criteria comprises at least one of a filename pattern, a pathname pattern, or an operation pattern.

6. The method of claim 2, wherein the event log corresponds to a named pipe.

7. The method of claim 1, wherein the OS module corresponds to a loadable kernel module, and the OS module is inserted within an operating system kernel.

8. The method of claim 1, further comprising determining whether the invocation of the any of the target set of operating-system-level system calls has successfully completed, and recording an associated event after determining successful completion.

9. The method of claim 1, wherein the one or more events are reviewed by the log collector on a periodic basis.

10. The method of claim 1, wherein the OS module is operated within the operating system of the host computing system to detect a change to the one or more log files by:

saving an original address of an operating system function;

intercepting a call by an application to the operating system function; and

calling the operating system function from the OS module, wherein a function call parameter from the application is passed to the operating system function.

11. The method of claim 10, wherein the operating system function comprises at least one of a write function, a rename function, a delete function, or a move function.

12. The method of claim 1, wherein the target set of operating-system-level system calls comprises one or more calls that indicate a possible change to the one or more log files being monitored and exclude one or more calls that do not indicate a change to any file.

13. A computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method, the method comprising:

to monitor for changes to one or more log files, configuring an operating system (OS) module to generate an event within an event log when any of a target set of operating-system-level system calls is made;

loading the OS module into an operating system of a host computing system;

operating the OS module within the operating system of the host computing system to detect an invocation of the any of the target set of operating-system-level system calls and to execute the invocation of the any of the target set of operating-system-level system calls, wherein the OS module does not detect invocation of untargeted operating-system-level system calls, and wherein the one or more log files are changed by one or more processes associated with one or more particular calls in the target set of operating-system-level system calls; and

identifying one or more events corresponding to the one or more log files changed by invocation of the any of the target set of operating-system-level system calls, wherein the one or more events correspond to a filtered subset of all log files that are changed by invocation of the any of the target set of operating-system-level system calls, and the one or more events are reviewable by a log collector to collect the one or more log files for a log analytics system.

14. The computer readable medium of claim 13, wherein a filter is applied to generate the filtered subset that is recorded into the event log, the filtered subset identifiable based upon application of a filtering criteria to the invocation of the any of the target set of operating-system-level system calls.

15. The computer readable medium of claim 14, wherein the filtering criteria comprises at least one of a filename pattern, a pathname pattern, or an operation pattern.

16. The computer readable medium of claim 13, wherein the OS module corresponds to a loadable kernel module, and the OS module is inserted within an operating system kernel.

17. The computer readable medium of claim 13, further comprising determining whether the invocation of the any of the target set of operating-system-level system calls has successfully completed, and recording an associated event after determining successful completion.

18. The computer readable medium of claim 13, wherein the OS module is operated within the operating system of the host computing system to detect a change to the one or more log files by:

saving an original address of an operating system function;

intercepting a call by an application to the operating system function; and

calling the operating system function from the OS module, wherein a function call parameter from the application is passed to the operating system function.

19. A system, comprising:

a processor;

a memory having stored thereon a sequence of instructions which, when executed by the processor causes the processor to execute operations comprising;

to monitor for changes to one or more log files, configuring an operating system (OS) module to generate an event within an event log when any of a target set of operating-system-level system calls is made;

loading the OS module into an operating system of a host computing system;

operating the OS module within the operating system of the host computing system to detect an invocation of the any of the target set of operating-system-level system calls and to execute the invocation of the any of the target set of operating-system-level system calls, wherein the OS module does not detect invocation of untargeted operating-system-level system calls, and wherein the one or more log files are changed by one or more processes associated with one or more particular calls in the target set of operating-system-level system calls; and

identifying one or more events corresponding to the one or more log files changed by invocation of the any of the target set of operating-system-level system calls, wherein the one or more events correspond to a filtered subset of all log files that are changed by invocation of the any of the target set of operating-system-level system calls, and the one or more events are reviewable by a log collector to collect the one or more log files for a log analytics system.

20. The system of claim 19, wherein a filter is applied to generate the filtered subset that is recorded into the event log, the filtered subset identifiable based upon application of a filtering criteria to the invocation of the any of the target set of operating-system-level system calls.

21. The system of claim 19, wherein the sequence of instructions which, when executed by the processor, causes the processor to operate the OS module within the operating system of the host computing system to detect a change to the one or more log files by:

saving an original address of an operating system function;

intercepting a call by an application to the operating system function; and

calling the operating system function from the OS module, wherein a function call parameter from the application is passed to the operating system function.