Systems and/or methods for dynamic anomaly detection in machine sensor data
First Claim
1. A system for detecting anomalies in data dynamically received from a plurality of sensors that are associated with at least one of a plurality of machines, the system comprising:
- a knowledgebase;
a model store; and
processing resources including at least one processor and a memory, the processing resources being configured, for each instance of data that is received from the plurality of sensors, to at least;
select a model from the model store based on which machine of the plurality of machines is associated with at least one corresponding sensor that provided the data;
classify, using the selected model, the respective instance of data as being one of a normal instance type and an anomalous instance type, wherein classification of the instance of data as an anomalous instance type indicates a potential operating issue with the machine that corresponds to the selected model;
in response to a classification of the respective instance being a normal instance type, use the data in the respective instance to train the selected model without a supervised designation of the respective instance;
in response to a classification of the respective instance being an anomalous instance type that is not new, determine from the knowledgebase an action to be taken and take the determined action; and
in response to a classification of the respective instance being an anomalous instance type that is new, seek confirmation from an authorized user as to whether the respective instance should be designated as a confirmed new anomalous instance type, and;
responsive to confirmation from the authorized user that the respective instance is a new anomalous instance type, update the knowledgebase with information about the respective instance and/or an action to be taken should the new anomalous instance type be detected again; and
train the selected model by using both the data in the respective instance and the corresponding designation from the authorized user;
wherein each model in the model store is implemented using a k-means cluster algorithm modified to be continually trainable as a result of the dynamic reception of data over a time period, wherein clusters within each model are built incrementally and in connection with an updatable distance threshold that indicates when a new cluster is to be created; and
wherein each said model has a respective total number of clusters that is dynamic and learned over time, whereina given data stream X from a given one of the machines includes instances x1 . . . xn with a number of variables d;
the modified k-means cluster algorithm uses a cluster initialization window p, a distance threshold t, an instance-weighting window w, a number of clusters k, clusters c1 . . . cn, sample covariance matrices S1 . . . Sk for respective clusters, and μ
1 . . . μ
k as centroids of respective clusters; and
the modified k-means algorithm is programmed to;
initialize centroid μ
1 of cluster c1 as the mean of instances x1 . . . xp, and matrix S1 as the covariance of instances x1 . . . xp, cluster c1 and instances x1 . . . xp being predicted as normal instance types; and
for each instance i from xp+1 . . . x∞
in the given data stream X;
temporarily assign instance xi to the cluster with the nearest centroid μ
1 . . . μ
k;
if the distance of xi to that centroid is greater than the distance threshold t, obtain a cluster assignment for xi from an or the authorized user; and
if the cluster assignment is for a confirmed new anomalous instance type, (a) create a new cluster cj+1, and set centroid μ
j+1=xi and covariance matrix Sj+1 as the mean of existing covariance matrices S1 . . . Sj, and (b) predict the class of cj+1 for xi; and
otherwise;
update the centroid μ
j as the w window-weighted mean of the instances xi that have been assigned to the cluster;
if the number of instances xi that have been assigned to the cluster is greater than the cluster initialization window p, update the matrix Sj as the w window-weighted covariance of the instances xi that have been assigned to the cluster; and
predict the class of cj for xi.
2 Assignments
0 Petitions
Accused Products
Abstract
Certain example embodiments relate to techniques for detecting anomalies in streaming data. More particularly, certain example embodiments use an approach that combines both unsupervised and supervised machine learning techniques to create a shared anomaly detection model in connection with a modified k-means clustering algorithm and advantageously also enables concept drift to be taken into account. The number of clusters k need not be known in advance, and it may vary over time. Models are continually trainable as a result of the dynamic reception of data over an unknown and potentially indefinite time period, and clusters can be built incrementally and in connection with an updatable distance threshold that indicates when a new cluster is to be created. Distance thresholds also are dynamic and adjustable over time.
-
Citations
36 Claims
-
1. A system for detecting anomalies in data dynamically received from a plurality of sensors that are associated with at least one of a plurality of machines, the system comprising:
-
a knowledgebase; a model store; and processing resources including at least one processor and a memory, the processing resources being configured, for each instance of data that is received from the plurality of sensors, to at least; select a model from the model store based on which machine of the plurality of machines is associated with at least one corresponding sensor that provided the data; classify, using the selected model, the respective instance of data as being one of a normal instance type and an anomalous instance type, wherein classification of the instance of data as an anomalous instance type indicates a potential operating issue with the machine that corresponds to the selected model; in response to a classification of the respective instance being a normal instance type, use the data in the respective instance to train the selected model without a supervised designation of the respective instance; in response to a classification of the respective instance being an anomalous instance type that is not new, determine from the knowledgebase an action to be taken and take the determined action; and in response to a classification of the respective instance being an anomalous instance type that is new, seek confirmation from an authorized user as to whether the respective instance should be designated as a confirmed new anomalous instance type, and; responsive to confirmation from the authorized user that the respective instance is a new anomalous instance type, update the knowledgebase with information about the respective instance and/or an action to be taken should the new anomalous instance type be detected again; and train the selected model by using both the data in the respective instance and the corresponding designation from the authorized user; wherein each model in the model store is implemented using a k-means cluster algorithm modified to be continually trainable as a result of the dynamic reception of data over a time period, wherein clusters within each model are built incrementally and in connection with an updatable distance threshold that indicates when a new cluster is to be created; and wherein each said model has a respective total number of clusters that is dynamic and learned over time, wherein a given data stream X from a given one of the machines includes instances x1 . . . xn with a number of variables d; the modified k-means cluster algorithm uses a cluster initialization window p, a distance threshold t, an instance-weighting window w, a number of clusters k, clusters c1 . . . cn, sample covariance matrices S1 . . . Sk for respective clusters, and μ
1 . . . μ
k as centroids of respective clusters; andthe modified k-means algorithm is programmed to; initialize centroid μ
1 of cluster c1 as the mean of instances x1 . . . xp, and matrix S1 as the covariance of instances x1 . . . xp, cluster c1 and instances x1 . . . xp being predicted as normal instance types; andfor each instance i from xp+1 . . . x∞
in the given data stream X;temporarily assign instance xi to the cluster with the nearest centroid μ
1 . . . μ
k;if the distance of xi to that centroid is greater than the distance threshold t, obtain a cluster assignment for xi from an or the authorized user; and if the cluster assignment is for a confirmed new anomalous instance type, (a) create a new cluster cj+1, and set centroid μ
j+1=xi and covariance matrix Sj+1 as the mean of existing covariance matrices S1 . . . Sj, and (b) predict the class of cj+1 for xi; andotherwise; update the centroid μ
j as the w window-weighted mean of the instances xi that have been assigned to the cluster;if the number of instances xi that have been assigned to the cluster is greater than the cluster initialization window p, update the matrix Sj as the w window-weighted covariance of the instances xi that have been assigned to the cluster; and predict the class of cj for xi. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
-
16. A system for detecting anomalies in data dynamically received from a plurality of sensors, each said sensor being associated with one or more machines, the system comprising:
-
a model store, each said machine having an associated model stored therein; and processing resources including at least one processor and a memory, the processing resources being configured to train each said model using a modified k-means cluster algorithm in which there are defined a cluster initialization window p, a distance threshold t, an instance-weighting window w, a number of clusters k, clusters c1 . . . cn, sample covariance matrices S1 . . . Sk for respective clusters, and μ
1 . . . μ
k as centroids of respective clusters;wherein each said cluster has an associated class, the class being one of an anomalous type class and a non-anomalous type class; wherein, for each given data stream X from a given one of the machines that includes data instances x1 . . . xn with a number of variables d, the modified k-means algorithm is programmed to; initialize centroid μ
1 of cluster c1 as the mean of instances x1 . . . xp, and matrix S1 as the covariance of instances x1 . . . xp, cluster c1 and instances x1 . . . xp being predicted as normal instance types; andfor each instance i from xp+1 . . . x∞
in the given data stream X;temporarily assign instance xi to the cluster with the nearest centroid μ
1 . . . μ
k;if the distance of xi to that centroid is greater than the distance threshold t, obtain a cluster assignment for xi from an authorized user; and if the cluster assignment is for a confirmed new anomalous instance type, (a) create a new cluster cj+1, and set centroid μ
j+1=xi and covariance matrix Sj+1 as the mean of existing covariance matrices S1 . . . Sj, and (b) predict the class of cj+1 for xi; andotherwise; update the centroid μ
j as the w window-weighted mean of the instances xi that have been assigned to the cluster;if the number of instances xi that have been assigned to the cluster is greater than the cluster initialization window p, update the matrix Sj as the w window-weighted covariance of the instances xi that have been assigned to the cluster; and predict the class of cj for xi. - View Dependent Claims (17, 18, 19, 20, 21, 22, 23, 24, 25, 26)
-
-
27. A method of detecting anomalies in data dynamically received from a plurality of sensors that are associated with at least one of a plurality of machines, the method comprising:
-
receiving data from the plurality of sensors; and for each instance of data received, using processing resources including at least one processor and a memory, to at least; selecting a model from the model store based on which machine of the plurality of machines is associated with at least one corresponding sensor that provided the data; classifying, using the selected model, the respective instance of data as being one of a normal instance type and an anomalous instance type, wherein classification of the instance of data as an anomalous instance type indicates a potential operating issue with the machine that corresponds to the selected model; in response to a classification of the respective instance being a normal instance type, training the selected model by using unsupervised learning with the data; in response to a classification of the respective instance being an anomalous instance type that is not new, determining an action to be taken and take the determined action; and in response to a classification of the respective instance being an anomalous instance type that is new, seeking confirmation from an authorized user as to whether the respective instance should be designated as a confirmed new anomalous instance type, and; responsive to confirmation from the authorized user that the respective instance is a new anomalous instance type, updating a knowledgebase with information about the respective instance and/or an action to be taken should the new anomalous instance type be detected again; and training the selected model by using supervised learning with both the data in the respective instance and the corresponding designation from the authorized user; wherein each model in the model store is implemented using a k-means cluster algorithm modified to be continually trainable as a result of the dynamic reception of data over a time period, wherein clusters within each model are built incrementally and in connection with an updatable distance threshold that indicates when a new cluster is to be created; and wherein each said model has a respective total number of clusters that is dynamic and learned over time, wherein a given data stream X from a given one of the machines includes instances x1 . . . xn with a number of variables d; the modified k-means cluster algorithm uses a cluster initialization window p, a distance threshold t, an instance-weighting window w, a number of clusters k, clusters c1 . . . cn, sample covariance matrices S1 . . . Sk for respective clusters, and μ
1 . . . μ
k as centroids of respective clusters; andthe modified k-means algorithm is programmed to; initialize centroid μ
1 of cluster c1 as the mean of instances x1 . . . xp, and matrix S1 as the covariance of instances x1 . . . xp, cluster c1 and instances x1 . . . xp being predicted as normal instance types; andfor each instance i from xp+1 . . . x∞
in the given data stream X;temporarily assign instance xi to the cluster with the nearest centroid μ
1 . . . μ
k;if the distance of xi to that centroid is greater than the distance threshold t, obtain a cluster assignment for xi from an authorized user; and if the cluster assignment is for a confirmed new anomalous instance type, (a) create a new cluster cj+1, and set centroid μ
j+1=xi and covariance matrix Sj+1 as the mean of existing covariance matrices S1 . . . Sj, and (b) predict the class of cj+1 for xi; andotherwise; update the centroid μ
j as the w window-weighted mean of the instances xi that have been assigned to the cluster;if the number of instances xi that have been assigned to the cluster is greater than the cluster initialization window p, update the matrix Sj as the w window-weighted covariance of the instances xi that have been assigned to the cluster; and predict the class of cj for xi . . . . - View Dependent Claims (28)
-
-
29. A method of detecting anomalies in data dynamically received from a plurality of sensors, each said sensor being associated with one or more machines, the method comprising:
-
maintaining a model store, each said machine having an associated model stored therein; receiving data from the plurality of sensors; and using processing resources including at least one processor and a memory to train each said model using a modified k-means cluster algorithm in which there are defined a cluster initialization window p, a distance threshold t, an instance-weighting window w, a number of clusters k, clusters c1 . . . cn, sample covariance matrices S1 . . . Sk for respective clusters, and μ
1 . . . μ
k as centroids of respective clusters;wherein each said cluster has an associated class, the class being one of an anomalous type class and a non-anomalous type class; wherein, for each given data stream X from a given one of the machines that includes data instances x1 . . . xn with a number of variables d, the modified k-means algorithm is programmed to; initialize centroid μ
1 of cluster c1 as the mean of instances x1 . . . xp, and matrix S1 as the covariance of instances x1 . . . xp, cluster c1 and instances x1 . . . xp being predicted as normal instance types; andfor each instance i from xp+1 . . . x∞
in the given data stream X;temporarily assign instance xi to the cluster with the nearest centroid μ
1 . . . μ
k;if the distance of xi to that centroid is greater than the distance threshold t, obtain a cluster assignment for xi from an authorized user; and if the cluster assignment is for a confirmed new anomalous instance type, (a) create a new cluster cj+1, and set centroid μ
j+1=xi and covariance matrix Sj+1 as the mean of existing covariance matrices S1 . . . Sj, and (b) predict the class of cj+1 for xi; andotherwise; update the centroid μ
j as the w window-weighted mean of the instances xi that have been assigned to the cluster;if the number of instances xi that have been assigned to the cluster is greater than the cluster initialization window p, update the matrix Sj as the w window-weighted covariance of the instances xi that have been assigned to the cluster; and predict the class of cj for xi. - View Dependent Claims (30, 31, 32, 33, 34, 35)
-
-
36. A non-transitory computer readable storage medium comprising instructions that, when executed in connection with processing resources including at least one processor and a memory, aid in detecting anomalies in data dynamically received from a plurality of sensors, each said sensor being associated with one or more machines, by at least:
-
maintaining a model store, each said machine having an associated model stored therein; receiving data from the plurality of sensors; and training each said model using a modified k-means cluster algorithm in which there are defined a cluster initialization window p, a distance threshold t, an instance-weighting window w, a number of clusters k, clusters c1 . . . cn, sample covariance matrices S1 . . . Sk for respective clusters, and μ
1 . . . μ
k as centroids of respective clusters;wherein each said cluster has an associated class, the class being one of an anomalous type class and a non-anomalous type class; wherein, for each given data stream X from a given one of the machines that includes data instances x1 . . . xn with a number of variables d, the modified k-means algorithm is programmed to; initialize centroid μ
1 of cluster c1 as the mean of instances x1 . . . xp, and matrix S1 as the covariance of instances x1 . . . xp, cluster c1 and instances x1 . . . xp being predicted as normal instance types; andfor each instance i from xp+1 . . . x∞
in the given data stream X;temporarily assign instance xi to the cluster with the nearest centroid μ
1 . . . μ
k;if the distance of xi to that centroid is greater than the distance threshold t, obtain a cluster assignment for xi from an authorized user; and if the cluster assignment is for a confirmed new anomalous instance type, (a) create a new cluster cj+1, and set centroid μ
j+1=xi and covariance matrix Sj+1 as the mean of existing covariance matrices S1 . . . Sj, and (b) predict the class of cj+1 for xi; andotherwise; update the centroid μ
j as the w window-weighted mean of the instances xi that have been assigned to the cluster;if the number of instances xi that have been assigned to the cluster is greater than the cluster initialization window p, update the matrix Sj as the w window-weighted covariance of the instances xi that have been assigned to the cluster; and predict the class of cj for xi.
-
Specification