Assessing detectability of malware related traffic

• US 10,630,709 B2
• Filed: 02/13/2018
• Issued: 04/21/2020
• Est. Priority Date: 02/13/2018
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Brabec, Jan, Machlica, Lukas
• Primary Examiner(s): Avery, Jeremiah L
• Application Number:
  US15/895,072
• Publication Number:
  US 20190253442A1
• Time in Patent Office: 798 Days
• Field of Search: None
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

In one embodiment, a computing device trains a multi-class classifier (having a plurality of classes) on a training dataset, and evaluates the multi-class classifier on a testing dataset to determine a performance of each of the plurality of classes. The plurality of classes may then be partitioned into either learnable or unlearnable based on whether the performance each particular class surpasses a particular threshold, and then a predicting classifier can be trained on the training dataset, where data of the training dataset is labelled as either learnable or unlearnable based on the particular class to which the data corresponds. Accordingly, the computing device may then use the predicting classifier on a new class to predict whether samples associated with the new class are learnable or unlearnable, and may retrain the multi-class classifier with the samples associated with the new class in response to predicting that the samples are learnable.

Claims

1. A method, comprising:

training, by a computing device, a multi-class classifier on a training dataset, the multi-class classifier having a plurality of classes;

evaluating, by the computing device, the multi-class classifier on a testing dataset to determine a performance of each class of the plurality of classes of the multi-class classifier;

partitioning, by the computing device, the plurality of classes into either learnable or unlearnable based on whether the performance each particular class surpasses a particular threshold;

training, by the computing device, a predicting classifier on the training dataset, wherein data of the training dataset is labelled as either learnable or unlearnable based on the particular class to which the data corresponds;

using, by the computing device, the predicting classifier on a new class to predict whether samples associated with the new class are learnable or unlearnable; and

retraining, by the computing device, the multi-class classifier with the samples associated with the new class in response to predicting that the samples are learnable.

2. The method as in claim 1, wherein the predicting classifier is trained on only non-negative data from the training dataset.

3. The method as in claim 1, wherein the training dataset comprises benign negative data and malicious non-negative data.

4. The method as in claim 1, wherein the training dataset and testing dataset comprise samples related to one or more of network flow data, proxy log data, and encrypted traffic analysis (ETA) data.

5. The method as in claim 1, wherein the predicting classifier is a regression model predicting a level of performance of a given class, and wherein the training dataset is labelled according to corresponding levels of how learnable is each particular class.

6. The method as in claim 1, wherein the particular threshold is set separately for each individual class of the plurality of classes.

7. The method as in claim 1, further comprising:

using multiple instance learning to classify all samples of each given class at once.

8. The method as in claim 1, wherein the new class is based on a set of samples representing the new class, wherein the set of samples has a set of numerical features used to describe the samples.

9. The method as in claim 1, wherein the predicting classifier is based on a random forest classifier.

10. The method as in claim 1, wherein the multi-class classifier is from one malware family.

11. The method as in claim 1, wherein the plurality of classes consist of numerical features selected from a group consisting of:

a list of cipher suite type codes;

transferred bytes;

uniform resource locator (URL) length; and

port number.

12. The method as in claim 1, wherein the plurality of classes are malware classes.

13. A tangible, non-transitory, computer-readable medium storing program instructions that cause a computer to execute a process comprising:

training a multi-class classifier on a training dataset, the multi-class classifier having a plurality of classes;

evaluating the multi-class classifier on a testing dataset to determine a performance of each class of the plurality of classes of the multi-class classifier;

partitioning the plurality of classes into either learnable or unlearnable based on whether the performance each particular class surpasses a particular threshold;

training a predicting classifier on the training dataset, wherein data of the training dataset is labelled as either learnable or unlearnable based on the particular class to which the data corresponds;

using the predicting classifier on a new class to predict whether samples associated with the new class are learnable or unlearnable; and

retraining the multi-class classifier with the samples associated with the new class in response to predicting that the samples are learnable.

14. The computer-readable medium as in claim 13, wherein the predicting classifier is trained on only non-negative data from the training dataset.

15. The computer-readable medium as in claim 13, wherein the predicting classifier is a regression model predicting a level of performance of a given class, and wherein the training dataset is labelled according to corresponding levels of how learnable is each particular class.

16. The computer-readable medium as in claim 13, wherein the particular threshold is set separately for each individual class of the plurality of classes.

17. An apparatus, comprising:

one or more network interfaces to communicate with a computer network;

a processor coupled to the network interfaces and configured to execute one or more process; and

a memory configured to store a process executable by the processor, the process when executed configured to;

train a multi-class classifier on a training dataset, the multi-class classifier having a plurality of classes;

evaluate the multi-class classifier on a testing dataset to determine a performance of each class of the plurality of classes of the multi-class classifier;

partition the plurality of classes into either learnable or unlearnable based on whether the performance each particular class surpasses a particular threshold;

train a predicting classifier on the training dataset, wherein data of the training dataset is labelled as either learnable or unlearnable based on the particular class to which the data corresponds;

use the predicting classifier on a new class to predict whether samples associated with the new class are learnable or unlearnable; and

retrain the multi-class classifier with the samples associated with the new class in response to predicting that the samples are learnable.

18. The apparatus as in claim 17, wherein the predicting classifier is trained on only non-negative data from the training dataset.

19. The apparatus as in claim 17, wherein the predicting classifier is a regression model predicting a level of performance of a given class, and wherein the training dataset is labelled according to corresponding levels of how learnable is each particular class.

20. The apparatus as in claim 17, wherein the particular threshold is set separately for each individual class of the plurality of classes.