Automatic log sensor tuning

• US 9,449,072 B2
• Filed: 06/18/2014
• Issued: 09/20/2016
• Est. Priority Date: 09/27/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Beisiegel, Michael, Joseph, Dinakaran, Nadgir, Devaprasad K.
• Primary Examiner(s): Truong, Dennis
• Application Number:
  US14/307,752
• Publication Number:
  US 20150094990A1
• Time in Patent Office: 825 Days
• Field of Search: 707/758, 709/224, 370/253, 702/127
• US Class Current: 1/1
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: H04L 41/069 using logs of notifications; Post-processing of notifications
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04Q 2209/823 where the data is sent when the measured values exceed a threshold, e.g. sending an alarm
• CPC Class Codes: H04Q 9/00 Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom

Abstract

A process for automatic tuning a set of collectors and/or sensors includes: collecting first machine data by a first sensor in a collection framework, processing the first machine data by a first collector in the collection framework to yield first collected machine data, performing analytics on the first collected machine data to generate analytics output, and tuning, based, at least in part, on the analytics output, at least one of the following: the first sensor and the first collector.

Claims

1. A method comprising:

identifying first machine data from a first enterprise component by a first sensor in a collection framework, the first sensor tuned to a first setting for identifying the first machine data;

processing a portion of the first machine data by a first collector in the collection framework, having a first configuration, to generate first collected machine data, the first collector programmed to distribute the first collected machine data to a search cluster;

responsive to an alert condition, generating a piped HTTP request for performing analytics on a set of collected machine data in the search cluster including the first collected machine data;

receiving a single threaded piped HTTP response to the piped HTTP request as analytics output;

determining a second configuration for the first collector responsive to the analytics output;

executing a sync instruction to the first collector to replace the first configuration of the first collector with the second configuration;

receiving a second machine data from the first sensor; and

processing a portion of the second machine data according to the second configuration of the first collector to generate second collected machine data;

wherein;

the second collected machine data includes event-specific data determined to be relevant by the performing analytics on the set of collected machine data.

2. The method of claim 1 wherein:

the performing analytics occurs in real time; and

the receiving a single threaded piped HTTP response occurs in real time.

3. The method of claim 1 further comprising:

determining an alert condition is met;

wherein;

the performing analytics is performed automatically in response to the alert condition being met; and

the alert condition is a threshold stored memory size of the search cluster.

4. The method of claim 1 further comprising:

responsive to a completion of a pre-determined time period during which the first collector operates according to the second configuration, returning the first collector to the first configuration to generate the first collected machine data.

5. The method of claim 1 further comprising:

causing the first sensor to identify the second machine data from at least a second enterprise component having an effect of adjusting a scope of data capture corresponding to identifying the second machine data by the first sensor.