ANOMALY DETECTION FOR CONTEXT-DEPENDENT DATA

• US 20160328654A1
• Filed: 05/04/2015
• Published: 11/10/2016
• Est. Priority Date: 05/04/2015
• Status: Abandoned Application

• Current Assignee: AGT International GMBH
• Original Assignee: AGT International GMBH
• Inventors: BAUER, Alexander, Heidtke, Nico, Niessen, Maria, Merentitis, Andreas
• Application Number:
  US14/703,502
• Publication Number:
  US 20160328654A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/048 Fuzzy inferencing
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/40 for vehicles, e.g. vehicle-to-pedestrians [V2P]

Abstract

The present invention is a new method directed for detecting anomalies in monitored data having plurality of data-segments partitioned to context related initial-subspaces, the method comprising:

• • training an association-map between the initial-subspaces and feature-clusters of the plurality of data-segments, the training is responsive to a fit-criterion;
  • concatenating the initial-subspaces into cluster-subspaces, responsive to being associated to similar feature-clusters according to the association-map, to obtain a generalized-association-map;
  • pinpointing at least one anomaly of at least one new data-segment of the data, responsive to deviation-criterion for deviation of the new data-segment from its association to one of the feature-clusters, according to the generalized-association-map; and
  • triggering an automatic-act responsive to a trigger-criterion for the at least one anomaly.

Claims

1. A method directed for detecting anomalies in monitored data having plurality of data-segments partitioned to context related initial-subspaces, said method comprising:

training an association-map between said initial-subspaces and feature-clusters of said plurality of data-segments, said training is responsive to a fit-criterion;

concatenating said initial-subspaces into cluster-subspaces, responsive to being associated to similar said feature-clusters according to said association-map, to obtain a generalized-association-map;

pinpointing at least one anomaly of at least one new data-segment of said data, responsive to deviation-criterion for deviation of said new data-segment from its associated one of said feature-clusters, according to said generalized-association-map; and

triggering an automatic-act responsive to a trigger-criterion for said at least one anomaly.

2. The method according to claim 1, wherein said data is continuous measurement-data collected from at least one sensor;

and wherein said plurality of data-segments are feature-vectors extracted from plurality of sections of said data.

3. The method according to claim 2, further comprising extracting said plurality of said feature-vectors from said plurality of sections.

4. The method according to claim 3, wherein said extracting is performed by a method selected from the group consisting of:

principal component analysis (PCA), independent component analysis, minimum noise fraction, random forest embedding, non-negative matrix factorization, and any combination thereof.

5. The method according to claim 1, wherein each of said plurality of data-segments is labeled with at least one context-label;

and wherein said method further comprising partitioning said plurality of data-segments to said context related initial-subspaces, responsive to a predetermined similarity in their said at least one context-label.

6. The method according to claim 5, further comprising selecting said at least one context-label from the group consisting of:

days of the week, midweek- or weekend-days, time of the day, light- or dark-hours, holidays, public events, weather conditions, visibility, temperature, locations, measuring scenarios, population, and any combination thereof.

7. The method according to claims 2 and 5, wherein said data is vehicle traffic measured data.

8. The method according to claim 1 or 2, further comprising clustering said feature-clusters, using an unsupervised clustering-method.

9. The method according to claim 8, wherein at least one of the following holds true:

said unsupervised clustering-method is selected from the group consisting of;

K-means nearest neighbor, Density-based spatial clustering of applications with noise (DBSCAN), hierarchical clustering, Gaussian mixture and any combination thereof;

said deviation-criterion and said pinpointing are determined by said unsupervised clustering-method.

10. The method according to claim 8, wherein at least one of the following holds true:

said clustering is incremental;

said training and said concatenating are incremental.

11. The method according to claim 1 or 10, wherein said training further comprising defining at least one additional feature-cluster associated to said data-segments of at least one of said initial-subspaces, responsive to a failure of said one of said initial-subspaces to comply with said fit-criterion.

12. The method according to claim 11, further comprising repeating said training and said concatenating, responsive to said defining of said at least one additional feature-cluster.

13. The method according to claims 5 and 8, further comprising repeating said partitioning with a different said predetermined similarity and/or repeating said clustering with a different number of clusters, responsive to a failure of at least one of said initial-subspaces to comply with said fit-criterion.

14. The method according to claim 1, further comprising selecting said fit-criterion from the group consisting of:

frequency threshold, average deviation threshold, statistical properties deviation threshold, dedicated matrices, Silhouette coefficients, and any combination thereof.

15. The method according to claim 1, wherein said pinpointing and said triggering are in real-time.

16. The method according to claim 1, wherein at least one of the following holds true:

said deviation is distance of said new data-segment from center from its said associated one of said feature-clusters;

said deviation is distance of said new data-segment from nearest data-segment in its said associated one of said feature-clusters.

17. The method according to claim 1, further comprising selecting said trigger-criterion from the group consisting of:

a predetermined number of consecutive said at least one anomaly;

a predetermined number of said at least one anomaly within a selected group of said data-segments;

a magnitude-threshold for said deviation; and

any combination thereof.

18. A computer system for detection of anomalies in monitored data having plurality of data-segments partitioned to context related initial-subspaces, said detection according to method steps comprising:

training an association-map between said initial-subspaces and feature-clusters of said plurality of data-segments, said training is responsive to a fit-criterion;

concatenating said initial-subspaces into cluster-subspaces, responsive to being associated to similar said feature-clusters according to said association-map, to obtain a generalized-association-map;

pinpointing at least one anomaly of at least one new data-segment of said data, responsive to deviation-criterion for deviation of said new data-segment from its associated one of said feature-clusters, according to said generalized-association-map; and

triggering an automatic-act responsive to a trigger-criterion for said at least one anomaly;

wherein said computer system comprising;

an interface component, configured to receive said data-segments;

a feature-extractor component, configured to extract said feature-clusters;

a context-identifier component, configured for partitioning of said plurality of data-segments to said context related initial-subspaces;

a mapping-machine component, configured to produce and update said generalized-association-map according to said steps of training and concatenating; and

an anomaly-detector, configured for said pinpointing of said at least one anomaly and for said triggering of said automatic act.

19. A non-transitory computer readable medium (CRM) that, when loaded into a memory of a computing device and executed by at least one processor of said computing device, configured to execute the steps of a computer implemented method for detecting anomalies in monitored data having plurality of data-segments partitioned to context related initial-subspaces, said steps comprising:

training an association-map between said initial-subspaces and feature-clusters of said plurality of data-segments, said training is responsive to a fit-criterion;

concatenating said initial-subspaces into cluster-subspaces, responsive to being associated to similar said feature-clusters according to said association-map, to obtain a generalized-association-map;

pinpointing at least one anomaly of at least one new data-segment of said data, responsive to deviation-criterion for deviation of said new data-segment from its associated one of said feature-clusters, according to said generalized-association-map; and

triggering an automatic-act responsive to a trigger-criterion for said at least one anomaly.

20. The CRM according to claim 19, wherein at least one of the following holds true:

said CRM further configured to execute step of partitioning said plurality of data-segments to said context related initial-subspaces, responsive to a predetermined similarity in their said context;

said CRM further configured to execute step of clustering said feature-clusters, using an unsupervised clustering-method;

said data is continuous measurement-data collected from at least one sensor, and wherein said plurality of data-segments are feature-vectors extracted from plurality of sections of said data, and said CRM further configured for extracting said plurality of said feature-vectors from said plurality of sections;

said CRM further configured to execute step of defining at least one additional feature-cluster associated to said data-segments of at least one of said initial-subspaces, responsive to a failure of said one of said initial-subspaces to comply with said fit-criterion;

said steps of pinpointing and triggering are in real-time.