METHOD AND APPARATUS FOR COMPUTING CELL DENSITY BASED RARENESS FOR USE IN ANOMALY DETECTION

• US 20160359685A1
• Filed: 04/05/2016
• Published: 12/08/2016
• Est. Priority Date: 06/04/2015
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Yadav, Navindra, Scheib, Ellen, Agasthy, Rachita
• Granted Patent: US 10,505,819 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 41/16 using machine learning or artificial intelligence
• CPC Class Codes: H04L 43/062 related to network traffic

Abstract

In one embodiment, a method includes receiving network data at an analytics device, grouping features of the network data into multivariate bins, generating a density for each of the multivariate bins, computing a rareness metric for each of the multivariate bins based on a probability of obtaining a feature in a bin and the probability for all other of the multivariate bins with equal or smaller density, and identifying anomalies based on computed rareness metrics. An apparatus and logic are also disclosed herein.

Claims

1. A method comprising:

receiving network data at an analytics device;

grouping features of the network data into multivariate bins at the analytics device;

generating a density for each of said multivariate bins at the analytics device;

computing at the analytics device, a rareness metric for each of said multivariate bins based on a probability of obtaining a feature in a bin and said probability for all other of said multivariate bins with equal or smaller density; and

identifying anomalies based on computed rareness metrics.

2. The method of claim 1 wherein said multivariate bins comprise bins of varying width.

3. The method of claim 2 wherein bin boundaries are based on univariate transition points.

4. The method of claim 1 wherein said density comprises a time weighted binned feature density.

5. The method of claim 4 wherein generating said time weighted binned feature density comprises applying an exponential decay to the features based on time of observation.

6. The method of claim 1 wherein generating said density comprises a nonparametric process.

7. The method of claim 1 wherein computing said rareness metric comprises computing rareness for different time categories corresponding to different days and time of day.

8. The method of claim 1 further comprising comparing the features to historical features corresponding to a same type of feature.

9. The method of claim 1 wherein said rareness metric is computed based on a context.

10. The method of claim 9 wherein said context is based on a tenant, a provider IP (Internet Protocol) address, a protocol, and a provider port and type.

11. The method of claim 1 wherein said rareness metric is computed for different units of analysis.

12. The method of claim 11 wherein said units of analysis are selected from a group consisting of IP (Internet Protocol) address, applications, users, and roles.

13. The method of claim 11 further comprising comparing said rareness metrics for one unit of analysis at different time periods.

14. The method of claim 1 wherein said probability comprises a cumulative probability taking into account historical data.

15. The method of claim 1 wherein the network data is collected from a plurality of sensors distributed throughout a network to monitor network flows within the network from multiple perspectives in the network.

16. An apparatus comprising:

an interface for receiving network data; and

a processor for grouping features of the network data into multivariate bins, generating a density for each of said multivariate bins, computing a rareness metric for each of said multivariate bins based on a probability of obtaining a feature in a bin and said probability for all other of said multivariate bins with equal or smaller density, and identifying anomalies based on computed rareness metrics.

17. The apparatus of claim 16 wherein said multivariate bins comprise bins of varying width.

18. The apparatus of claim 16 wherein said density comprises a time weighted binned feature density and said probability comprises a cumulative probability, and wherein generating said density comprises a nonparametric process.

19. Logic encoded on one or more non-transitory computer readable media for execution and when executed operable to:

process network data at an analytics device;

group features of the network data into multivariate bins;

generate a density for each of said multivariate bins;

compute a rareness metric for each of said multivariate bins based on a probability of obtaining a feature in a bin and said probability for all other of said multivariate bins with equal or smaller density; and

identify anomalies based on computed rareness metrics.

20. The logic of claim 19 wherein said multivariate bins comprise bins of varying width.