Systems and methods for robust anomaly detection
First Claim
Patent Images
1. A system, comprising:
- a distributed cache configured to store state information for a plurality of configuration items (CIs);
a plurality of management, instrumentation, and discovery (MID) servers forming a cluster, each of the plurality of MID servers comprising;
one or more processors, configured to execute machine-readable instructions;
a tangible, non-transitory, machine-readable medium, comprising the machine-readable instructions that, when executed by the one or more processors, cause a corresponding MID server to;
receive, from the distributed cache, a subset of the state information associated with assigned CIs; and
perform a statistical analysis on the subset of the state information; and
an anomaly detector, configured to;
identify statistical outliers of the statistical analysis; and
identify an anomaly of the statistical outliers by tracking a history of the statistical outliers;
wherein the anomaly is determined based upon a magnitude of deviation between the subset of the state information and a statistical model;
a change detector, configured to;
identify the statistical outliers using a first filter applied to the subset of the state information, the first filter comprising a Kalman filter that produces estimates of unknown variables using Bayesian inference and joint probability distribution estimation over the unknown variables for a timeframe;
determine when a data transition associated with the statistical outliers indicates noise;
determine when the data transition associated with the statistical outliers indicates a level change, byupon identifying the statistical outliers, feed data to a second filter that represents data indicative of no statistical outlier occurring;
upon stabilization of a change in the first filter, use an output of the second filter as a reference to determine is the change is statically significant;
when the change is statistically significant, classify the data transition associated with the statistical outliers as a level shift, andotherwise, when the change is not statistically significant, classify the data transition associated with the statistical outliers as noise; and
when the data transition associated with the statistical outliers indicates the level change, present a notification of the level change.
1 Assignment
0 Petitions
Accused Products
Abstract
A system, includes: a distributed cache that stores state information for a plurality of configuration items (CIs). Management, instrumentation, and discovery (MID) servers form a cluster, each of the MID servers including one or more processors that receive, from the distributed cache, a subset of the state information associated with assigned CIs and perform a statistical analysis on the subset of the state information.
31 Citations
12 Claims
-
1. A system, comprising:
-
a distributed cache configured to store state information for a plurality of configuration items (CIs); a plurality of management, instrumentation, and discovery (MID) servers forming a cluster, each of the plurality of MID servers comprising; one or more processors, configured to execute machine-readable instructions; a tangible, non-transitory, machine-readable medium, comprising the machine-readable instructions that, when executed by the one or more processors, cause a corresponding MID server to; receive, from the distributed cache, a subset of the state information associated with assigned CIs; and perform a statistical analysis on the subset of the state information; and an anomaly detector, configured to; identify statistical outliers of the statistical analysis; and identify an anomaly of the statistical outliers by tracking a history of the statistical outliers;
wherein the anomaly is determined based upon a magnitude of deviation between the subset of the state information and a statistical model;a change detector, configured to; identify the statistical outliers using a first filter applied to the subset of the state information, the first filter comprising a Kalman filter that produces estimates of unknown variables using Bayesian inference and joint probability distribution estimation over the unknown variables for a timeframe; determine when a data transition associated with the statistical outliers indicates noise; determine when the data transition associated with the statistical outliers indicates a level change, by upon identifying the statistical outliers, feed data to a second filter that represents data indicative of no statistical outlier occurring; upon stabilization of a change in the first filter, use an output of the second filter as a reference to determine is the change is statically significant; when the change is statistically significant, classify the data transition associated with the statistical outliers as a level shift, and otherwise, when the change is not statistically significant, classify the data transition associated with the statistical outliers as noise; and when the data transition associated with the statistical outliers indicates the level change, present a notification of the level change. - View Dependent Claims (2)
-
-
3. A system comprising,
a distributed cache configured to store state information for a plurality of configuration items (CIs); -
a plurality of management, instrumentation, and discovery (MID) servers forming a cluster, each of the plurality of MID servers comprising; one or more processors, configured to execute machine-readable instructions; a tangible, non-transitory, machine-readable medium, comprising the machine-readable instructions that, when executed by the one or more processors, cause a corresponding MID server to; receive, from the distributed cache, a subset of the state information associated with assigned CIs; and perform a statistical analysis on the subset of the state information; and a time-series analyzer, configured to; classify the subset of the state information into at least one of a plurality of classifications based at least in part upon historical time-series data; and construct a statistical model representative of a stream of current time-series data based upon the at least one of the plurality of classifications; an anomaly detector, configured to; monitor the stream of current time-series data; perform a statistical analysis on the stream of current time-series data based at least in part upon the statistical model constructed by the time-series analyzer; identify statistical outliers of the stream of current time-series data based at least on the statistical analysis; determine an anomalous score for the statistical outliers based at least on a history of the statistical outliers;
wherein the anomalous score indicates a magnitude of deviation between the current time-series data and the statistical model; andpresent the anomalous score to a client communicatively coupled to the system, for subsequent reporting, client action, or both; wherein the plurality of classifications comprise; a quasi-normal classification, indicating that the subset of the state information appears to adhere to a stationary process; a seasonal classification, indicating that the subset of the state information appears to adhere to a process having periodic non-normal distribution, a general trend over time, or both; a near-constant classification, indicating that the subset of the state information appears to adhere to a constant value; a trendy classification, indicating that the subset of the state information comprises noisy data with a trend; an accumulator classification, indicating that the subset of the state information comprises an accumulation of low-noise data; a categorical classification, indicating that the subset of the state information appears to adhere to a process having a set of discrete values; a switching classification, indicating that the subset of the state information appears to adhere to a process of jumping from at least one first state to at least one second state via one or more state transitions; and an arrival classification, indicating that the subset of the state information appears to adhere to a process dependent upon a timing of a particular event. - View Dependent Claims (4, 5, 6)
-
-
7. A tangible, non-transitory, machine-readable medium, comprising machine-readable instructions that, when executed by one or more processors, cause the one or more processors to:
-
receive a stream of current time-series data; at periodic intervals, classify the stream of current time-series data into at least one of a plurality of classifications based at least in part upon historical time-series data; identify a statistical model representative of the stream of current time-series data based upon the classification of the stream; when the classification comprises a seasonal classification, identify the statistical model representative of the stream of current time-series data from a selection between a weekly statistical model and a daily statistical model; and perform a statistical analysis on the stream of the current time-series data based at least in part upon the statistical model, wherein the statistical model is constructed by a time-series analyzer; identify a statistical outlier of the stream of the current time-series data based at least on the statistical analysis; and classify the statistical outlier as either noise or a level shift, by; feeding unobserved data that that suggests that no statistical outlier has occurred into a filter; determining, via filtering of the unobserved data by the filter, whether a data transition associated with the statistical outlier is statistically significant; classifying the statistical outlier as a level shift when the data transition is statistically significant; and classifying the statistical outlier as noise when the data transition is not statistically significant; and provide a graphical indication of results of the statistical analysis at a communicatively coupled instance. - View Dependent Claims (8, 9)
-
-
10. A management, instrumentation, and discovery (MID) server, comprising:
-
one or more processors, configured to execute machine-readable instructions; a tangible, non-transitory, machine-readable medium, comprising the machine-readable instructions that, when executed by the one or more, cause the MID server to; receive a subset of a stream of current time-series data that is associated with assigned configuration items (CIs) of the MID server; classify the subset of the stream of current time-series data as seasonal data; select a statistical model representative of the subset of the stream of current time-series data from a selection between a weekly statistical model and a daily statistical model; perform a statistical analysis on the stream of the current time-series data based at least in part upon the statistical model constructed by a time-series analyzer; identify statistical outliers of the stream of the current time-series data based at least on the statistical analysis; classify the statistical outliers as either transient or a level shift using a first Kalman filter; and when the statistical outliers are classified as transient, identify an anomaly from the statistical outliers by tracking a history of the statistical outliers using a second Kalman filter;
wherein the anomaly is identified based upon a magnitude of deviation between the current time-series data and the statistical model. - View Dependent Claims (11, 12)
-
Specification