Systems and Methods for Dynamic Anomaly Detection

• US 20090234899A1
• Filed: 03/11/2008
• Published: 09/17/2009
• Est. Priority Date: 03/11/2008
• Status: Active Grant

• Current Assignee: Paragon Science, Inc.
• Original Assignee: Paragon Science, Inc.
• Inventors: Kramer, Stephen Patrick
• Granted Patent: US 8,738,652 B2
• Time in Patent Office: Days
• US Class Current: 708/200
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases

Abstract

Methods and systems for detecting anomalies in sets of data are disclosed, including: computing components of one or more types of feature vectors at a plurality of values of one or more independent variables, each type of the feature vectors characterizing a set of input data being dependent on the one or more independent variables; computing one or more types of output values corresponding to each type of feature vectors as a function of the one or more independent variables using a nonlinear sequence analysis method; and detecting anomalies in how the one or more types of output values change as functions of the one or more independent variables.

Claims

1. A method for detecting anomalies, the method comprising:

computing components of one or more types of feature vectors at a plurality of values of one or more independent variables, each type of the feature vectors characterizing a set of input data being dependent on the one or more independent variables;

computing one or more types of output values corresponding to each type of feature vectors as a function of the one or more independent variables using a nonlinear sequence analysis method; and

detecting anomalies in how the one or more types of output values change as functions of the one or more independent variables.

2. The method of claim 1, further comprising comparing anomalies detected across each type of the one or more output values.

3. The method of claim 2, further comprising determining one or more subsets of the input data corresponding to the detected anomalies.

4. The method of claim 1, where the nonlinear sequence analysis method is at least one of:

computing Finite-Time Lyapunov Exponents (FTLEs) for the one or more feature vectors, computing off-diagonal complexity for the one or more feature vectors, and computing temporal correlations for the one or more feature vectors.

5. The method of claim 4, where computing the FTLEs comprises estimating a Jacobian of a flow map using the one or more feature vectors.

6. The method of claim 5, where the Jacobian estimation is performed using at least one of:

finite differencing, interpolation, and tessellation.

7. The method of claim 1, further comprising clustering the input data into a plurality of clusters of input data and detecting anomalies in how the one or more output values vary across the clusters of input data.

8. The method of claim 1, where detecting the anomalies comprises monitoring for uncharacteristic changes in results generated using the nonlinear sequence analysis methods.

9. The method of claim 7, where detecting the anomalies comprises monitoring for uncharacteristic changes in results generated using the nonlinear sequence analysis methods across the plurality of clusters.

10. The method of claim 1, where computing components of one or more types of feature vectors at a plurality of values of one or more independent variables further comprises first preprocessing the input data to calculate a modified data set, wherein the modified data set is used in place of the original input data in all subsequent steps.

11. The method of claim 10, where the input data set comprises multiple sets of input data of the same type or of different types, which are preprocessed into a single data set for subsequent analysis.

12. An information handling system for identifying mediated communications in a network of nodes, the system comprising:

one or more memory units;

one or more processor units; and

one or more input/output devices,

wherein the system is operable to;

compute components of one or more types of feature vectors at a plurality of values of one or more independent variables, each type of the feature vectors characterizing a set of input data being dependent on the one or more independent variables;

compute one or more types of output values corresponding to each type of feature vectors as a function of the one or more independent variables using nonlinear sequence analysis methods; and

detect anomalies in how the one or more types of output values change as functions of the one or more independent variables.

13. The system of claim 12, the system being further configured to compare anomalies detected across each type of the one or more output values.

14. The system of claim 13, the system being further configured to determine one or more subsets of the input data corresponding to the detected anomalies.

15. The system of claim 12, where the nonlinear sequence analysis method is at least one of:

computing Finite-Time Lyapunov Exponents (FTLEs) for the one or more feature vectors, computing off-diagonal complexity for the one or more feature vectors, and computing temporal correlations for the one or more feature vectors.

16. The system of claim 12, the system being further configured to cluster the input data into a plurality of clusters of input data and to detect anomalies in how the one or more output values vary across the clusters of input data.

17. A computer program product stored on a computer operable medium, the computer program product comprising software code being effective to:

compute components of one or more types of feature vectors at a plurality of values of one or more independent variables, each type of the feature vectors characterizing a set of input data being dependent on the one or more independent variables;

compute one or more types of output values corresponding to each type of feature vectors as a function of the one or more independent variables using nonlinear sequence analysis methods; and

detect anomalies in how the one or more types of output values change as functions of the one or more independent variables.

18. The product of claim 17, the code being further effective to compare anomalies detected across each type of the one or more output values.

19. The product of claim 18, the code being further effective to determine one or more subsets of the input data corresponding to the detected anomalies.

20. The product of claim 17, where the nonlinear sequence analysis method is at least one of:

computing Finite-Time Lyapunov Exponents (FTLEs) for the one or more feature vectors, computing off-diagonal complexity for the one or more feature vectors, and computing temporal correlations for the one or more feature vectors.

21. The product of claim 17, the code being further effective to cluster the input data into a plurality of clusters of input data and to detect anomalies in how the one or more output values vary across the clusters of input data.