Dynamic application degrouping to optimize machine learning model accuracy

• US 10,318,887 B2
• Filed: 06/21/2016
• Issued: 06/11/2019
• Est. Priority Date: 03/24/2016
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Vasseur, Jean-Philippe, Savalle, Pierre-Andre, Honore, Alexandre
• Primary Examiner(s): Barot, Bharat
• Application Number:
  US15/188,140
• Publication Number:
  US 20170279696A1
• Time in Patent Office: 1,085 Days
• Field of Search: 709223-226
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 43/062 related to network traffic
• CPC Class Codes: H04L 43/14 using software, i.e. software packages network security related monitoring H04L63/1408
• CPC Class Codes: H04L 43/50 Testing arrangements
• CPC Class Codes: H04L 63/1433 Vulnerability analysis

Abstract

In one embodiment, a device in a network identifies a plurality of applications from observed traffic in the network. The device forms two or more application clusters from the plurality of applications. Each of the application clusters includes one or more of the applications, and wherein a particular application in the plurality of applications is included in each of the application clusters. The device generates anomaly detection models for each of the application clusters. The device tests the anomaly detection models, to determine a measure of efficacy for each of the models with respect to traffic associated with the particular application. The device selects a particular anomaly detection model to analyze the traffic associated with the particular application based on the measures of efficacy for each of the models.

Claims

1. A method, comprising:

identifying, by a device in a network, a plurality of applications from observed traffic in the network;

forming, by the device, two or more application clusters from the plurality of applications, wherein each of the application clusters includes one or more of the applications, and wherein a particular application in the plurality of applications is included in each of the application clusters;

generating, by the device, anomaly detection models for each of the application clusters;

testing, by the device, the anomaly detection models, to determine a measure of efficacy for each of the models with respect to traffic associated with the particular application; and

selecting, by the device, a particular anomaly detection model to analyze the traffic associated with the particular application based on the measures of efficacy for each of the models.

2. The method as in claim 1, wherein testing the anomaly detection models comprises:

determining, for each of the anomaly detection models, the measure of efficacy of the model based in part on an estimated set of inputs that the model would deem anomalous.

3. The method as in claim 2, wherein the selected anomaly detection model is selected based in part on an amount of overlap between the estimated sets of inputs that each of the anomaly detection models would deem anomalous.

4. The method as in claim 3, wherein the selected anomaly detection model models the traffic associated with the particular application and traffic associated with at least one other application.

5. The method as in claim 1, wherein testing the anomaly detection models comprises:

using the anomaly detection models to detect anomalous traffic; and

reporting the detected anomalous traffic to a supervisory device.

6. The method as in claim 5, wherein the measures of efficacy comprise feedback from the supervisory device regarding the reported anomalous traffic.

7. The method as in claim 1, further comprising:

receiving, at the device, a request from a supervisory device to use a separate anomaly detection model for the traffic associated with the particular application;

determining, by the device, whether the device has sufficient resource to execute a separate anomaly detection model for the traffic associated with the particular application; and

notifying, by the device, the supervisory device when there are not sufficient resources on the device to execute a separate anomaly detection model for the traffic associated with the particular application.

8. The method as in claim 7, wherein the request to use a separate anomaly detection model for the traffic associated with the particular application is sent by the supervisory device based in part on a threat intelligence index of compromise.

9. The method as in claim 1, wherein the anomaly detection models are statistical behavioral models.

10. An apparatus, comprising:

one or more network interfaces to communicate with a network;

a processor coupled to the network interfaces and configured to execute one or more processes; and

a memory configured to store a process executable by the processor, the process when executed configured to;

identify a plurality of applications from observed traffic in the network;

form two or more application clusters from the plurality of applications, wherein each of the application clusters includes one or more of the applications, and wherein a particular application in the plurality of applications is included in each of the application clusters;

generate anomaly detection models for each of the application clusters;

test the anomaly detection models, to determine a measure of efficacy for each of the models with respect to traffic associated with the particular application; and

select a particular anomaly detection model to analyze the traffic associated with the particular application based on the measures of efficacy for each of the models.

11. The apparatus as in claim 10, wherein the apparatus tests the anomaly detection models by:

determining, for each of the anomaly detection models, the measure of efficacy of the model based in part on an estimated set of inputs that the model would deem anomalous.

12. The apparatus as in claim 11, wherein the apparatus selects the selected anomaly detection model based in part on an amount of overlap between the estimated sets of inputs that each of the anomaly detection models would deem anomalous.

13. The apparatus as in claim 12, wherein the selected anomaly detection model models the traffic associated with the particular application and traffic associated with at least one other application.

14. The apparatus as in claim 10, wherein the apparatus tests the anomaly detection models by:

using the anomaly detection models to detect anomalous traffic; and

reporting the detected anomalous traffic to a supervisory device.

15. The apparatus as in claim 14, wherein the measures of efficacy comprise feedback from the supervisory device regarding the reported anomalous traffic.

16. The apparatus as in claim 10, wherein the process when executed is further operable to:

receive a request from a supervisory device to use a separate anomaly detection model for the traffic associated with the particular application;

determine whether the apparatus has sufficient resource to execute a separate anomaly detection model for the traffic associated with the particular application; and

notify the supervisory device when there are not sufficient resources on the apparatus to execute a separate anomaly detection model for the traffic associated with the particular application.

17. The apparatus as in claim 16, wherein the request to use a separate anomaly detection model for the traffic associated with the particular application is sent by the supervisory device based in part on a threat intelligence index of compromise.

18. The apparatus as in claim 10, wherein the anomaly detection models are statistical behavioral models.

19. A tangible, non-transitory, computer-readable medium storing program instructions that cause a device in a network to execute a process comprising:

identifying, by the, a plurality of applications from observed traffic in the network;

forming, by the device, two or more application clusters from the plurality of applications, wherein each of the application clusters includes one or more of the applications, and wherein a particular application in the plurality of applications is included in each of the application clusters;

generating, by the device, anomaly detection models for each of the application clusters;

testing, by the device, the anomaly detection models, to determine a measure of efficacy for each of the models with respect to traffic associated with the particular application; and

selecting, by the device, a particular anomaly detection model to analyze the traffic associated with the particular application based on the measures of efficacy for each of the models.

20. The tangible, non-transitory, computer-readable medium as in claim 19, wherein the anomaly detection models are statistical behavioral models.