Computer system and method for creating and deploying an anomaly detection model based on streaming data

• US 10,579,932 B1
• Filed: 07/10/2018
• Issued: 03/03/2020
• Est. Priority Date: 07/10/2018
• Status: Active Grant

• Current Assignee: Uptake Technologies, Inc.
• Original Assignee: Uptake Technologies, Inc.
• Inventors: Cantrell, Michael
• Primary Examiner(s): Tabor, Amare F
• Application Number:
  US16/031,584
• Time in Patent Office: 602 Days
• Field of Search: 726 12
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/00 Computing arrangements based on specific mathematical models

Abstract

A computing system may operate in a first mode during which it calculates a set of training metrics on a running basis as a stream of multivariate data points originating from a data source is being received. While operating in the first mode, the computing system may determine that the set of training metrics has reached a threshold level of stability. In response, the computing system may transition to a second mode during which its extracts a model object and calculates a set of model parameters for an anomaly detection model. While operating in the second mode, the computing system may determine that the set of model parameters has reached a threshold level of stability. In response, the computing system may transition to a third mode during which it uses the anomaly detection model to monitor for anomalies in the stream of multivariate data points originating from the data source.

Claims

1. A computing system comprising:

a communication interface configured to receive data from at least one data source;

at least one processor;

a non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to;

receive, via the communication interface, a stream of multivariate data points originating from a given data source;

operate in a first mode during which the computing system calculates a set of training metrics on a running basis as the computing system receives the stream of multivariate data points originating from the given data source;

while operating in the first mode, make a determination that the set of training metrics being calculated on a running basis has reached a first threshold level of stability, wherein the set of training metrics at the time of the determination is defined as an initial set of training metrics to use for creating an anomaly detection model that comprises an initial model object and an initial set of model parameters;

in response to the determination that the set of training metrics being calculated on a running basis has reached the first threshold level of stability, transition to a second mode during which the computing system (a) uses the initial set of training metrics to extract the initial model object for the anomaly detection model and (b) uses the initial model object for the anomaly detection model to calculate a set of model parameters for the anomaly detection model on a running basis as the computing system continues to receive the stream of multivariate data points originating from the given data source;

while operating in the second mode, make a determination that the set of model parameters being calculated on a running basis has reached a second threshold level of stability, wherein the set of model parameters at the time of the determination is defined as the initial set of model parameters for the anomaly detection model;

in response to the determination that the set of model parameters being calculated on a running basis has reached the second threshold level of stability, transition to a third mode during which the computing system uses the anomaly detection model to monitor for anomalies as the computing system continues to receive the stream of multivariate data points originating from the given data source by;

(i) using the initial model object for the anomaly detection model to score each of at least a subset of the multivariate data points in the stream of multivariate data points received during the third mode,

(ii) evaluating whether a threshold extent of multivariate data points within a given window of time violate the initial set of model parameters for the anomaly detection model, and

(iii) determining that an anomaly has occurred if it is determined that a threshold extent of multivariate data points within the given window of time violate the initial set of model parameters for the anomaly detection model.

2. The computing system of claim 1, wherein the stream of multivariate data points originating from the given data source comprises a stream of operating data originating from a given asset.

3. The computing system of claim 1, wherein:

the anomaly detection model comprises a regression model based on principal component analysis (PCA);

the set of training metrics comprise a mean, a variance, a max, a min, a streaming object, and a correlation matrix;

the initial model object comprises a projection matrix; and

the initial set of model parameters comprises a set of anomaly thresholds.

4. The computing system of claim 1, wherein a first temporal range of the stream of multivariate data points is received during the first mode, a second temporal range of the stream of multivariate data points is received during the second mode, and a third temporal range of the stream of multivariate data points is received during the third mode.

5. The computing system of claim 4, further comprising program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

after the initial set of training metrics is defined, continue calculating the set of training metrics on a running basis as the computing system continues to receive one or both of the second temporal range and the third temporal range of the stream of multivariate data points;

make a determination that the set of training metrics being calculated on a running basis has diverged from the initial set of training metrics by a threshold extent, wherein the set of training metrics at the time of the determination is defined as an updated set of training metrics to use for creating an updated anomaly detection model that comprises an updated model object and an updated set of model parameters;

in response to the determination that the set of training metrics being calculated on a running basis has diverged from the initial set of training metrics by the threshold extent, use the updated set of training metrics as a basis for extracting the updated model object and defining the updated set of model parameters; and

begin to use the updated anomaly detection model to monitor for anomalies as the computing system continues to receive the stream of multivariate data points originating from the given data source.

6. The computing system of claim 5, wherein the program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to continue calculating the set of training metrics on a running basis as the computing system continues to receive one or both of the second temporal range and the third temporal range of the stream of multivariate data points comprise program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to continue calculating the set of training metrics on a running basis using only the multivariate data points in the third temporal range of the stream of multivariate data points that do not violate the initial set of model parameters for the anomaly detection model.

7. The computing system of claim 4, further comprising program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

after the initial set of model parameters is defined, continue calculating the set of model parameters on a running basis as the computing system continues to receive the third temporal range of the stream of multivariate data points;

make a determination that the set of model parameters being calculated on a running basis has diverged from the initial set of model parameters by a threshold extent, wherein the set of model parameters at the time of the determination is defined as an updated set of model parameters to use for creating an updated anomaly detection model that comprises the initial model object and the updated set of model parameters; and

begin to use the updated anomaly detection model to monitor for anomalies as the computing system continues to receive the stream of multivariate data points originating from the given data source.

8. The computing system of claim 7, wherein the program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to continue calculating the set of model parameters on a running basis as the computing system continues to receive the third temporal range of the stream of multivariate data points comprise program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to continue calculating the set of model parameters on a running basis using only the multivariate data points in the third temporal range of the stream of multivariate data points that do not violate the initial set of model parameters for the anomaly detection model.

9. The computing system of claim 1, wherein the given data source is capable of outputting multivariate data points belonging to a plurality of different modes, wherein the anomaly detection model is associated with one given mode of the plurality of different modes, and wherein the computing system further comprises program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

while receiving the stream of multivariate data points originating from the given data source, detect a mode to which each data point in the stream of multivariate data points belongs; and

proceed to use only the multivariate data points within the stream of multivariate data points that belong to the one given mode.

10. A method performed by a computing system, the method comprising:

receiving, at the computing system, a stream of multivariate data points originating from a given data source;

operating in a first mode during which the computing system calculates a set of training metrics on a running basis as the computing system receives the stream of multivariate data points originating from the given data source;

while operating in the first mode, making a determination that the set of training metrics being calculated on a running basis has reached a first threshold level of stability, wherein the set of training metrics at the time of the determination is defined as an initial set of training metrics to use for creating an anomaly detection model that comprises an initial model object and an initial set of model parameters;

in response to the determination that the set of training metrics being calculated on a running basis has reached the first threshold level of stability, transitioning to a second mode during which the computing system (a) uses the initial set of training metrics to extract the initial model object for the anomaly detection model and (b) uses the initial model object for the anomaly detection model to calculate a set of model parameters for the anomaly detection model on a running basis as the computing system continues to receive the stream of multivariate data points originating from the given data source;

while operating in the second mode, making a determination that the set of model parameters being calculated on a running basis has reached a second threshold level of stability, wherein the set of model parameters at the time of the determination is defined as the initial set of model parameters for the anomaly detection model;

in response to the determination that the set of model parameters being calculated on a running basis has reached the second threshold level of stability, transitioning to a third mode during which the computing system uses the anomaly detection model to monitor for anomalies as the computing system continues to receive the stream of multivariate data points originating from the given data source by;

(i) using the initial model object for the anomaly detection model to score each of at least a subset of the multivariate data points in the stream of multivariate data points received during the third mode,

(ii) evaluating whether a threshold extent of multivariate data points within a given window of time violate the initial set of model parameters for the anomaly detection model, and

(iii) determining that an anomaly has occurred if it is determined that a threshold extent of multivariate data points within the given window of time violate the initial set of model parameters for the anomaly detection model.

11. The method of claim 10, wherein:

the anomaly detection model comprises a regression model based on principal component analysis (PCA);

the set of training metrics comprise a mean, a variance, a max, a min, a streaming object, and a correlation matrix;

the initial model object comprises a projection matrix; and

the initial set of model parameters comprises a set of anomaly thresholds.

12. The method of claim 10, wherein a first temporal range of the stream of multivariate data points is received during the first mode, a second temporal range of the stream of multivariate data points is received during the second mode, and a third temporal range of the stream of multivariate data points is received during the third mode.

13. The method of claim 12, further comprising:

after the initial set of training metrics is defined, continuing to calculate the set of training metrics on a running basis as the computing system continues to receive one or both of the second temporal range and the third temporal range of the stream of multivariate data points;

making a determination that the set of training metrics being calculated on a running basis has diverged from the initial set of training metrics by a threshold extent, wherein the set of training metrics at the time of the determination is defined as an updated set of training metrics to use for creating an updated anomaly detection model that comprises an updated model object and an updated set of model parameters;

in response to the determination that the set of training metrics being calculated on a running basis has diverged from the initial set of training metrics by the threshold extent, using the updated set of training metrics as a basis for extracting the updated model object and defining the updated set of model parameters; and

beginning to use the updated anomaly detection model to monitor for anomalies as the computing system continues to receive the stream of multivariate data points originating from the given data source.

14. The method of claim 13, wherein continuing to calculate the set of training metrics on a running basis as the computing system continues to receive one or both of the second temporal range and the third temporal range of the stream of multivariate data points comprises:

continuing to calculate the set of training metrics on a running basis using only the multivariate data points in the third temporal range of the stream of multivariate data points that do not violate the initial set of model parameters for the anomaly detection model.

15. The method of claim 12, further comprising:

after the initial set of model parameters is defined, continuing to calculate the set of model parameters on a running basis as the computing system continues to receive the third temporal range of the stream of multivariate data points;

making a determination that the set of model parameters being calculated on a running basis has diverged from the initial set of model parameters by a threshold extent, wherein the set of model parameters at the time of the determination is defined as an updated set of model parameters to use for creating an updated anomaly detection model that comprises the initial model object and the updated set of model parameters; and

beginning to use the updated anomaly detection model to monitor for anomalies as the computing system continues to receive the stream of multivariate data points originating from the given data source.

16. The method of claim 15, wherein continuing to calculate the set of model parameters on a running basis as the computing system continues to receive the third temporal range of the stream of multivariate data points comprises continuing to calculate the set of model parameters on a running basis using only the multivariate data points in the third temporal range of the stream of multivariate data points that do not violate the initial set of model parameters for the anomaly detection model.

17. The method of claim 10, wherein the given data source is capable of outputting multivariate data points belonging to a plurality of different modes, and wherein the anomaly detection model is associated with one given mode of the plurality of different modes, the method further comprising:

while receiving the stream of multivariate data points originating from the given data source, detecting a mode to which each data point in the stream of multivariate data points belongs; and

proceeding to use only the multivariate data points within the stream of multivariate data points that belong to the one given mode.

18. A non-transitory computer-readable storage medium having program instructions stored thereon that are executable to cause a computing system to:

receive a stream of multivariate data points originating from a given data source;

operate in a first mode during which the computing system calculates a set of training metrics on a running basis as the computing system receives the stream of multivariate data points originating from the given data source;

while operating in the first mode, make a determination that the set of training metrics being calculated on a running basis has reached a first threshold level of stability, wherein the set of training metrics at the time of the determination is defined as an initial set of training metrics to use for creating an anomaly detection model that comprises an initial model object and an initial set of model parameters;

in response to the determination that the set of training metrics being calculated on a running basis has reached the first threshold level of stability, transition to a second mode during which the computing system (a) uses the initial set of training metrics to extract the initial model object for the anomaly detection model and (b) uses the initial model object for the anomaly detection model to calculate a set of model parameters for the anomaly detection model on a running basis as the computing system continues to receive the stream of multivariate data points originating from the given data source;

while operating in the second mode, make a determination that the set of model parameters being calculated on a running basis has reached a second threshold level of stability, wherein the set of model parameters at the time of the determination is defined as the initial set of model parameters for the anomaly detection model;

in response to the determination that the set of model parameters being calculated on a running basis has reached the second threshold level of stability, transition to a third mode during which the computing system uses the anomaly detection model to monitor for anomalies as the computing system continues to receive the stream of multivariate data points originating from the given data source by;

(i) using the initial model object for the anomaly detection model to score each of at least a subset of the multivariate data points in the stream of multivariate data points received during the third mode,

(ii) evaluating whether a threshold extent of multivariate data points within a given window of time violate the initial set of model parameters for the anomaly detection model, and

(iii) determining that an anomaly has occurred if it is determined that a threshold extent of multivariate data points within the given window of time violate the initial set of model parameters for the anomaly detection model.