Detecting anomalies in an internet of things network

• US 10,013,303 B2
• Filed: 04/25/2017
• Issued: 07/03/2018
• Est. Priority Date: 10/07/2015
• Status: Active Grant

• Current Assignee: Business Objects Incorporated (SAP SE)
• Original Assignee: Business Objects Incorporated (SAP SE)
• Inventors: Pallath, Paul, Razavi, Rouzbeh
• Primary Examiner(s): McCarthy, Christopher S
• Application Number:
  US15/496,348
• Publication Number:
  US 20170228278A1
• Time in Patent Office: 434 Days
• CPC Class Codes: G05B 23/02 Electric testing or monitoring
• CPC Class Codes: G05B 23/0254 based on a quantitative model, e.g. mathematical relationships between inputs and outputs; functions: observer, Kalman filter, residual calculation, Neural Networks
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/0787 Storage of error reports, e.g. persistent data storage, storage using memory protection
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Abstract

The present disclosure describes methods, systems, and computer program products for detecting anomalies in an Internet-of-Things (IoT) network. One computer-implemented method includes receiving, by operation of a computer system, a dataset of a plurality of data records, each of the plurality of data records comprising a plurality of features and a target variable, the plurality of features and target variable including information of a manufacturing environment; identifying a set of normal data records from the dataset based on the target variable; identifying inter-feature correlations by performing correlation analysis on the set of normal data records; and detecting anomaly based on the inter-feature correlations for predictive maintenance.

Claims

1. A computer-implemented method comprising:

receiving, by operation of a computer system, a dataset of a plurality of data records, each of the plurality of data records comprising a plurality of features and a target variable associated with the plurality of features, each feature of the plurality of features representing a reading of a separate sensor coupled to a machine and measuring a condition of the machine, and the target variable representing a status of the machine;

identifying a set of normal data records from the dataset based on the target variable;

identifying inter-feature correlations by performing correlation analysis on the set of normal data records; and

performing predictive maintenance on the machine based on a detection of an anomaly based on the inter-feature correlations for predictive maintenance, wherein the detection of the anomaly comprises;

identifying a cluster of correlated features based on the inter-feature correlations, wherein a set of correlated features forms the cluster of correlated features if a correlation between each pair of correlated features of the set of correlated features exceeds a minimum threshold value; and

building a model that estimates a first feature in the cluster of correlated features based on one or more other features in the cluster of correlated features.

2. The method of claim 1, wherein detecting an anomaly further comprises detecting a data record to be an abnormal data record in response to determining that a number of previous time stamped data records are determined as abnormal data records.

3. The method of claim 1, wherein detecting an anomaly using the inter-feature correlations for predictive maintenance further comprises:

receiving a data record that includes an observed value of the first feature and observed values of the one or more other features in the cluster of correlated features;

determining an estimated value of the first feature according to the model based on the values of the one or more other features in the cluster of correlated features;

comparing the estimated value of the first feature with the observed value of the first feature; and

determining whether the data record is a normal data record or an abnormal data record based on the comparison.

4. The method of claim 3, wherein determining whether the data record is a normal data record or an abnormal data record based on the comparison comprises determining a probability of anomaly of the data record based on the comparison.

5. The method of claim 3, wherein comparing the estimated value of the first feature with the observed value of the first feature comprises computing an error measure between the estimated value of the first feature with the observed value of the first feature.

6. The method of claim 5, further comprising:

building an impact model that represents a relationship between the target variable and the respective error measure of each feature in the cluster of correlated features; and

identifying impact of a deviation from an estimated value of a feature in the cluster of correlated features on the target variable.

7. The method of claim 5, wherein determining whether the data record is a normal data record or an abnormal data record based on the comparison comprises increasing a probability of an anomaly of the data record in response to determining the error measure between the estimated value of the first feature with the observed value of the first feature exceeds a threshold.

8. The method of claim 3, further comprising:

for each feature besides the first feature in the cluster of correlated features;

building a respective model that estimates the feature based on one or more features other than the feature in the cluster of correlated features;

determining an estimated value of the feature according to the respective model based on observed values of the one or more other features other than the feature in the cluster of correlated features;

comparing the estimated value of the feature with the observed value of the feature; and

determining whether the data record is a normal data record or an abnormal data record based on the comparison.

9. The method of claim 3, further comprising:

identifying a second cluster of correlated features based on the inter-feature correlations of the set of normal data records;

building a second model that determines a second feature in the second cluster of correlated features based on one or more other features in the second cluster of correlated features;

receiving a second data record that includes an observed value of the second feature and observed values of the one or more other features in the second cluster of correlated features;

determining an estimated value of the second feature based on the values of the one or more other features in the second cluster of correlated features;

comparing the estimated value of the second feature with the observed value of the second feature; and

determining whether the data record is a normal data record or an abnormal data record based on the comparison.

10. The method of claim 9, wherein determining whether the data record is a normal data record or an abnormal data record comprises determining that the data record is an abnormal data record in response to determining that at least one of:

the observed value of the first feature is outside a first tolerance range of the estimated value of the first feature;

or

the observed value of the second feature is outside a second tolerance range of the estimated value of the second feature.

11. A non-transitory, computer-readable medium storing computer-readable instructions executable by a computer and configured to perform operations comprising:

receiving a dataset of a plurality of data records, each of the plurality of data records comprising a plurality of features and a target variable associated with the plurality of features, each feature of the plurality of features representing a reading of a separate sensor coupled to a machine and measuring a condition of the machine, and the target variable representing a status of the machine;

identifying a set of normal data records from the dataset based on the target variable;

identifying inter-feature correlations by performing correlation analysis on the set of normal data records; and

performing predictive maintenance on the machine based on a detection of an anomaly based on the inter-feature correlations for predictive maintenance, wherein the detection of the anomaly comprises;

identifying a cluster of correlated features based on the inter-feature correlations, wherein a set of correlated features forms the cluster of correlated features if a correlation between each pair of correlated features of the set of correlated features exceeds a minimum threshold value; and

building a model that estimates a first feature in the cluster of correlated features based on one or more other features in the cluster of correlated features.

12. The medium of claim 11, wherein detecting an anomaly using the inter-feature correlations for predictive maintenance comprises:

receiving a data record that includes an observed value of the first feature and observed values of the one or more other features in the cluster of correlated features;

determining an estimated value of the first feature according to the model based on the values of the one or more other features in the cluster of correlated features;

comparing the estimated value of the first feature with the observed value of the first feature; and

determining whether the data record is a normal data record or an abnormal data record based on the comparison.

13. The medium of claim 12, wherein comparing the estimated value of the first feature with the observed value of the first feature comprises computing an error measure between the estimated value of the first feature with the observed value of the first feature, and the operations further comprising:

building an impact model that represents a relationship between the target variable and the respective error measure of each feature in the cluster of correlated features; and

identifying impact of a deviation from an estimated value of a feature in the cluster of correlated features on the target variable.

14. The medium of claim 12, the operations further comprising:

for each feature besides the first feature in the cluster of correlated features;

building a respective model that estimates the feature based on one or more features other than the feature in the cluster of correlated features;

determining an estimated value of the feature according to the respective model based on observed values of the one or more other features other than the feature in the cluster of correlated features;

comparing the estimated value of the feature with the observed value of the feature; and

determining whether the data record is a normal data record or an abnormal data record based on the comparison.

15. The medium of claim 12, the operations further comprising:

identifying a second cluster of correlated features based on the inter-feature correlations of the set of normal data records;

building a second model that determines a second feature in the second cluster of correlated features based on one or more other features in the second cluster of correlated features;

receiving a second data record that includes an observed value of the second feature and observed values of the one or more other features in the second cluster of correlated features;

determining an estimated value of the second feature based on the values of the one or more other features in the second cluster of correlated features;

comparing the estimated value of the second feature with the observed value of the second feature; and

determining whether the data record is a normal data record or an abnormal data record based on the comparison.

16. A system, comprising:

a memory;

at least one hardware processor interoperably coupled with the memory and configured to perform operations comprising;

receiving a dataset of a plurality of data records, each of the plurality of data records comprising a plurality of features and a target variable associated with the plurality of features, each feature of the plurality of features representing a reading of a separate sensor coupled to a machine and measuring a condition of the machine, and the target variable representing a status of the machine;

identifying a set of normal data records from the dataset based on the target variable;

identifying inter-feature correlations by performing correlation analysis on the set of normal data records; and

performing predictive maintenance on the machine based on a detection of an anomaly based on the inter-feature correlations for predictive maintenance, wherein the detection of the anomaly comprises;

identifying a cluster of correlated features based on the inter-feature correlations, wherein a set of correlated features forms the cluster of correlated features if a correlation between each pair of correlated features of the set of correlated features exceeds a minimum threshold value; and

building a model that estimates a first feature in the cluster of correlated features based on one or more other features in the cluster of correlated features.

17. The system of claim 16, wherein detecting an anomaly using the inter-feature correlations for predictive maintenance comprises:

receiving a data record that includes an observed value of the first feature and observed values of the one or more other features in the cluster of correlated features;

determining an estimated value of the first feature according to the model based on the values of the one or more other features in the cluster of correlated features;

comparing the estimated value of the first feature with the observed value of the first feature; and

determining whether the data record is a normal data record or an abnormal data record based on the comparison.

18. The system of claim 17, wherein comparing the estimated value of the first feature with the observed value of the first feature comprises computing an error measure between the estimated value of the first feature with the observed value of the first feature;

and wherein determining whether the data record is a normal data record or an abnormal data record based on the comparison comprises increasing a probability of an anomaly of the data record in response to determining the error measure between the estimated value of the first feature with the observed value of the first feature exceeds a threshold.

19. The system of claim 17, the operations further comprising:

for each feature besides the first feature in the cluster of correlated features;

building a respective model that estimates the feature based on one or more features other than the feature in the cluster of correlated features;

determining an estimated value of the feature according to the respective model based on observed values of the one or more other features other than the feature in the cluster of correlated features;

comparing the estimated value of the feature with the observed value of the feature; and

determining whether the data record is a normal data record or an abnormal data record based on the comparison.

20. The system of claim 17, the operations further comprising:

identifying a second cluster of correlated features based on the inter-feature correlations of the set of normal data records;

building a second model that determines a second feature in the second cluster of correlated features based on one or more other features in the second cluster of correlated features;

receiving a second data record that includes an observed value of the second feature and observed values of the one or more other features in the second cluster of correlated features;

determining an estimated value of the second feature based on the values of the one or more other features in the second cluster of correlated features;

comparing the estimated value of the second feature with the observed value of the second feature; and

determining whether the data record is a normal data record or an abnormal data record based on the comparison, wherein determining whether the data record is a normal data record or an abnormal data record comprises determining that the data record is an abnormal data record in response to determining that at least one of;

the observed value of the first feature is outside a first tolerance range of the estimated value of the first feature;

or

the observed value of the second feature is outside a second tolerance range of the estimated value of the second feature.