System and method for smart alerts

• US 10,628,801 B2
• Filed: 08/05/2016
• Issued: 04/21/2020
• Est. Priority Date: 08/07/2015
• Status: Active Grant

• Current Assignee: TATA Consultancy Services Limited (Tata Sons Pvt Ltd.)
• Original Assignee: TATA Consultancy Services Limited (Tata Sons Pvt Ltd.)
• Inventors: Natu, Maitreya, Venkateswaran, Praveen, Sadaphal, Vaishali Paithankar
• Primary Examiner(s): Gart, Matthew S
• Assistant Examiner(s): Swartz, Stephen S
• Application Number:
  US15/230,120
• Publication Number:
  US 20170039530A1
• Time in Patent Office: 1,355 Days
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0715 in a system implementing multitasking multitasking per se G06F9/46
• CPC Class Codes: G06F 11/0757 by exceeding a time limit, i.e. time-out, e.g. watchdogs
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06Q 10/06316 Sequencing of tasks or work
• CPC Class Codes: G06Q 10/1097 Task assignment

Abstract

A system for smart alerts in a batch system for an IT enterprise. The method includes alert configuration by identifying recent steady state of a batch job and deriving schedules for the steady state. The normal behaviour is then computed within the schedules. The method further includes aggregating the one or more alerts by identifying correlated group of alerts by pruning of one or more jobs and alerts, detecting correlations between the two or more alerts and deriving causality of the grouped alerts. The method finally includes predicting of future alerts of a batch job.

Claims

1. A processor-implemented method for generating alerts in a batch system, the method comprising:

configuring of one or more alerts, wherein the configuring comprises;

identifying a recent steady state of a batch job, wherein the steady state of the batch job is identified by analyzing change in a metric value associated with the steady state;

deriving at least one schedule within the identified recent steady state of the batch job using Classification and Regression Trees (CARTS);

computing an overlap between the identified groups of metric values, wherein the overlap indicates a similarity between the identified groups; and

identifying each group of metric values with overlap as a schedule;

computing a normal behavior within the at least one schedule, wherein the normal behavior is defined by a range of normal values within an upper threshold and a lower threshold, and wherein the upper threshold and the lower threshold is calculated by one or more of median and median absolute deviation methods;

aggregating the one or more alerts by identifying correlated group of alerts based at least one of a historical and a real-time analysis, wherein the identifying of the correlated group of alerts comprises;

pruning of one or more jobs and alerts based on one or more metrics conditions, wherein the one or more metric conditions comprise dependencies of one or more batch jobs, execution conditions of the one or more batch jobs, volumes of alerts generated by the one or more batch jobs and type of alert generated by the one or more batch jobs;

detecting correlations between two or more alerts by using one or more correlation rules for grouping the alerts; and

deriving causality of the grouped alerts using one or more causality rules to identify potential causes and effects;

predicting of future alerts of a batch job based on at least one or more of univariate metric forecasting, multivariate metric forecasting, and system behavior.

2. The method as claimed in claim 1, wherein the configuring of the alerts are updated incrementally for next batch jobs on observing changes in the job behavior.

3. The method as claimed in claim 1, wherein the lower and upper threshold is computed based on the skewness of distribution of the metric values, wherein if the distribution exhibits skewness the lower threshold is computed by medianleft-2*MAD left and upper threshold is computed by median right+2*MAD right, wherein median left and median right are median values of two groups of the metric values, and MAD left and MAD right are median absolute deviation of two groups of the metric values.

4. The method as claimed in claim 1, wherein the identifying of correlated group of alerts further includes applying a plurality of correlation rules for rule chaining and grouping of alerts, wherein the grouped alerts are assigned to one or more resolvers.

5. The method as claimed in claim 1, wherein the one or more metrics for pruning of the one or more alerts comprises, dependencies of the one or more batch jobs, execution conditions, volume of alerts generated and type of alert generated by the one or more batch jobs.

6. A computer-implemented system for providing alerts in a batch system, the system comprising:

at least one processor; and

at least one memory, the at least one memory coupled to the at least one processor, wherein the at least one processor configured by said instructions for;

configuring of one or more alerts upon triggering of an abnormal behavior in a batch job, wherein the configuring comprises;

identifying a steady state of the batch job, wherein the steady state of the batch job identified by analyzing change in a metric value associated with the steady state;

deriving at least one schedule within the identified steady state of the batch job by;

identifying one or more groups of metric values of the batch job using Classification and Regression Trees (CARTs);

computing an overlap between the identified groups of metric values, wherein the overlap indicates a similarity between the identified groups; and

identifying each group of metric values with overlap as a schedule;

computing a normal behavior within the at least one schedule, wherein the normal behavior is defined by a range of normal values within an upper threshold and a lower threshold, and wherein the upper threshold and the lower threshold is calculated by one or more of median and median absolute deviation methods; and

aggregating the one or more alerts by identifying a correlated group of alerts based on at least one of a historical and a real-time analysis, wherein the identifying of the correlated group of alerts comprises;

pruning of one or more jobs and alerts based on one or more metrics conditions;

detecting correlations between two or more alerts by using one or more correlation rules for grouping the alerts; and

deriving causality of the grouped alerts using one or more causality rules to identify potential causes and effects;

predict future alerts of a batch job based on at least one or more of univariate metric forecasting, multivariate metric forecasting, and system behavior.

7. The system as claimed in claim 6, wherein the lower and upper threshold is computed based on the skewness of distribution of the metric values, wherein if the distribution exhibits skewness the lower threshold is computed by medianleft-2*MADleft and upper threshold is computed by medianright+2*MADright, wherein medianleft and medianright are median values of two groups of the metric values, and MADleft and MADright are median absolute deviation of two groups of the metric values.

8. The system as claimed in claim 6, wherein the identifying of correlated group of alerts further includes applying a plurality of correlation rules for rule chaining and grouping of alerts, wherein the grouped alerts are assigned to one or more resolvers.

9. The system as claimed in claim 6, wherein the one or more metrics for pruning of the one or more alerts comprises, dependencies of the one or more batch jobs, execution conditions, volume of alerts generated and type of alert generated by the one or more batch jobs.

10. A non-transitory computer-readable medium having embodied thereon a computer program for executing a method for providing alerts, the method comprising:

configuring of one or more alerts upon triggering of an abnormal behavior in a batch job, wherein the configuring comprises;

identifying a steady state of the batch job;

deriving at least one schedule within the identified steady state of the batch job by;

identifying one or more groups of metric values of the batch job using Classification and Regression Trees (CARTs);

computing an overlap between the identified groups of metric values, wherein the overlap indicates a similarity between the identified groups; and

identifying each group of metric values with overlap as a schedule;

computing a normal behavior within the at least one schedule, wherein the normal behavior is defined by a range of normal values for an upper threshold and a lower threshold, and wherein the upper threshold and the lower threshold is calculated by one or more of median and median absolute deviation methods;

aggregating the one or more alerts by identifying correlated group of alerts based at least one of a historical and a real-time analysis, wherein the identifying of the correlated group of alerts comprises;

pruning of one or more jobs and alerts based on one or more metrics conditions;

detecting correlations between two or more alerts by using one or more correlation rules for grouping the alerts; and

deriving causality of the grouped alerts using one or more causality rules to identify potential causes and effects;

predicting of future alerts of a batch job based on at least one or more of univariate metric forecasting, multivariate metric forecasting, and system behavior.