Method and apparatus for grouping features into bins with selected bin boundaries for use in anomaly detection

• US 10,154,053 B2
• Filed: 04/05/2016
• Issued: 12/11/2018
• Est. Priority Date: 06/04/2015
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Yadav, Navindra, Scheib, Ellen, Agasthy, Rachita
• Primary Examiner(s): Kanaan, Simon P
• Application Number:
  US15/090,992
• Publication Number:
  US 20160359886A1
• Time in Patent Office: 980 Days
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 41/16 using machine learning or artificial intelligence
• CPC Class Codes: H04L 43/062 related to network traffic
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

In one embodiment, a method includes receiving network data at an analytics device, identifying features for the network data at the analytics device, grouping each of the features into bins of varying width at the analytics device, the bins comprising bin boundaries selected based on a probability that data within each of the bins follows a discrete uniform distribution, and utilizing the binned features for anomaly detection. An apparatus and logic are also disclosed herein.

Claims

1. A method comprising:

receiving network data at a processor of an analytics device, the network data collected from a plurality of sensors distributed throughout a network to monitor network flows within the network from multiple perspectives in the network;

processing the network data at the processor of the analytics device, wherein processing comprises;

identifying features for the network data;

determining transition points for each of said features in a histogram;

grouping each of said features into bins of varying width in the histogram, wherein said width defines a range of said features in each of said bins;

wherein said transition points define bin boundaries in the histogram, said transition points selected based on a probability that data within each of the bins follows a discrete uniform distribution; and

inputting said binned features into an algorithm for anomaly detection.

2. The method of claim 1 wherein said bin boundaries are assigned at transition points within the features.

3. The method of claim 1 further comprising updating said bin boundaries upon receiving new network data.

4. The method of claim 1 wherein each of said features comprises univariate data.

5. The method of claim 1 wherein said features comprise numeric features.

6. The method of claim 1 further comprising defining said bin boundaries utilizing a Pearson chi-square test to determine said probability.

7. The method of claim 1 further comprising identifying observation counts for minimum and maximum values of one of said bins for use in testing said probability.

8. The method of claim 1 wherein said probability is compared to a predetermined value selected based on a desired granularity of data and available storage space.

9. The method of claim 1 wherein said selected bin boundaries retain distribution characteristics comprising spikes and areas of sparseness in said binned features.

10. The method of claim 1 wherein the network data is collected from a plurality of sensors distributed throughout a network to monitor network flows within the network from multiple perspectives in the network.

11. An apparatus comprising:

an interface for receiving network data; and

a processor for identifying features for the network data, determining transition points for each of said features in a histogram, grouping each of said features into bins of varying width in the histogram, and inputting said binned features into an algorithm for anomaly detection;

wherein the transition points define bin boundaries in the histogram, said transition points selected based on a probability that data within each of the bins follows a discrete uniform distribution and wherein said width defines a range of said features in each of said bins.

12. The apparatus of claim 11 wherein said bin boundaries are assigned at transition points within the features.

13. The apparatus of claim 11 wherein the processor is further configured to define said bin boundaries utilizing a Pearson chi-square test to determine said probability.

14. The apparatus of claim 11 wherein said probability is compared to a predetermined value selected based on a desired granularity of data and available storage space.

15. The apparatus of claim 11 wherein said selected bin boundaries retain distribution characteristics comprising spikes and areas of sparseness in said binned features.

16. The apparatus of claim 11 wherein the network data is collected from a plurality of sensors distributed throughout a network to monitor network flows within the network from multiple perspectives in the network.

17. Logic encoded on one or more non-transitory computer readable media for execution and when executed operable to:

identify features for network data;

determine transition points for each of said features in a histogram;

group each of said features into bins of varying width in the histogram, wherein said width defines a range of said features in each of said bins; and

input said binned features into an algorithm for anomaly detection;

wherein said transition points define bin boundaries in the histogram, said transition points selected based on a probability that data within each of the bins follows a discrete uniform distribution.

18. The logic of claim 17 further operable to identify observation counts for minimum and maximum values of one of said bins for use in testing said probability.

19. The logic of claim 17 wherein said probability is compared to a predetermined value selected based on a desired granularity of data and available storage space.

20. The logic of claim 17 wherein the network data is collected from a plurality of sensors distributed throughout a network to monitor network flows within the network from multiple perspectives in the network.