Automatically grouping malware based on artifacts

• US 10,581,892 B2
• Filed: 01/18/2019
• Issued: 03/03/2020
• Est. Priority Date: 02/29/2016
• Status: Active Grant

• Current Assignee: Palo Alto Networks Incorporated
• Original Assignee: Palo Alto Networks Incorporated
• Inventors: Rostami-Hesarsorkh, Shadi, Vasudevan, Sudarshan, Hewlett, II, William Redington, Rostamabadi, Farshad
• Primary Examiner(s): Murphy, J. Brant
• Application Number:
  US16/252,421
• Publication Number:
  US 20190158525A1
• Time in Patent Office: 410 Days
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 21/56 Computer malware detection or handling, e.g. anti-virus arrangements
• CPC Class Codes: G06F 21/562 Static detection
• CPC Class Codes: G06F 21/566 Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities
• CPC Class Codes: G06F 2221/034 Test or assess a computer or a system
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: H04L 63/0254 Stateful filtering
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

Techniques for automatically grouping malware based on artifacts are disclosed. In some embodiments, a system, process, and/or computer program product for automatically grouping malware based on artifacts includes receiving a plurality of samples for performing automated malware analysis to generate log files based on the automated malware analysis; processing the log files to extract features associated with malware; clustering the plurality of samples based on the extracted features; and performing an action based on the clustering output.

Claims

1. A computer-implemented method, comprising:

clustering a plurality of samples based on a plurality of features associated with malware, wherein each of the features corresponds to a line or a sub-line in one or more log files determined to be an artifact associated with malware based on an automated malware analysis, wherein clustering the plurality of samples based on the plurality of features further comprises;

selecting one or more of the plurality of features and assigning values to each indicator, wherein selecting one or more of the plurality of features includes performing a pre-filtering operation to select the plurality of features for clustering based on a threshold association between the line or the sub-line in the one or more of the log files and known malware;

collecting the assigned values in an array for each of the plurality of samples;

comparing the assigned values of the array between two of the plurality of samples; and

calculating a distance between the two samples, wherein the samples within a defined threshold of distance are clustered; and

performing an action based on an output of clustering the plurality of samples based on the plurality of features, wherein the action based on the output of clustering the plurality of samples based on the plurality of features further comprises validate the output of clustering the plurality of samples based on the plurality of features based on tags to identify previously identified malware groups.

2. The method of claim 1, wherein the plurality of features correspond to high-risk artifacts, and wherein each high-risk artifact is determined to be associated with a malware sample based on the automated malware analysis.

3. The method of claim 1, wherein performing the automated malware analysis includes performing a dynamic analysis and a static analysis.

4. The method of claim 1, further comprising:

receiving the plurality of samples for performing the automated malware analysis to generate the log files based on the automated malware analysis; and

processing the log files to extract the plurality of features associated with malware.

5. The method of claim 1, further comprising:

clustering the plurality of samples based on the plurality of features using a decision tree clustering process.

6. The method of claim 1, further comprising:

clustering the plurality of samples based on the plurality of features using a k-means++ clustering process.

7. The method of claim 1, wherein a log file for a sample comprises one or more lines based on results of the automated malware analysis for the sample.

8. The method of claim 1, further comprising:

clustering the plurality of samples based on the plurality of features using a decision tree clustering process; and

clustering the plurality of samples based on the plurality of features using a k-means++ clustering process.

9. A system, comprising:

a processor configured to;

cluster a plurality of samples based on a plurality of features associated with malware, wherein each of the features corresponds to a line or a sub-line in one or more log files determined to be an artifact associated with malware based on an automated malware analysis, wherein clustering the plurality of samples based on the plurality of features further comprises;

select one or more of the plurality of features and assigning values to each indicator, wherein selecting one or more of the plurality of features includes performing a pre-filtering operation to select the plurality of features for clustering based on a threshold association between the line or the sub-line in the one or more of the log files and known malware;

collect the assigned values in an array for each of the plurality of samples;

compare the assigned values of the array between two of the plurality of samples; and

calculate a distance between the two samples, wherein the samples within a defined threshold of distance are clustered; and

perform an action based on an output of clustering the plurality of samples based on the plurality of features, wherein the action based on the output of clustering the plurality of samples based on the plurality of features further comprises validate the output of clustering the plurality of samples based on the plurality of features based on tags to identify previously identified malware groups; and

a memory coupled to the processor and configured to provide the processor with instructions.

10. The system recited in claim 9, wherein the plurality of features correspond to high-risk artifacts, and wherein each high-risk artifact is determined to be associated with a malware sample based on the automated malware analysis.

11. The system recited in claim 9, wherein performing the automated malware analysis includes performing a dynamic analysis and a static analysis.

12. The system recited in claim 9, wherein a log file for a sample comprises one or more lines based on results of the automated malware analysis for the sample.

13. The system recited in claim 9, wherein the processor is further configured to:

receive the plurality of samples for performing the automated malware analysis to generate the log files based on the automated malware analysis; and

process the log files to extract the plurality of features associated with malware.

14. The system recited in claim 9, wherein the processor is further configured to:

cluster the plurality of samples based on the plurality of features using a decision tree clustering process.

15. The system recited in claim 9, wherein the processor is further configured to:

cluster the plurality of samples based on the plurality of features using a k-means++ clustering process.

16. The system recited in claim 9, wherein the processor is further configured to:

cluster the plurality of samples based on the plurality of features using a decision tree clustering process; and

cluster the plurality of samples based on the plurality of features using a k-means++ clustering process.

17. A computer program product, the computer program product being embodied in a non-transitory tangible computer readable storage medium and comprising computer instructions for:

clustering a plurality of samples based on a plurality of features associated with malware, wherein each of the features corresponds to a line or a sub-line in one or more log files determined to be an artifact associated with malware based on an automated malware analysis, wherein clustering the plurality of samples based on the plurality of features further comprises;

selecting one or more of the plurality of features and assigning values to each indicator, wherein selecting one or more of the plurality of features includes performing a pre-filtering operation to select the plurality of features for clustering based on a threshold association between the line or the sub-line in the one or more of the log files and known malware;

collecting the assigned values in an array for each of the plurality of samples;

comparing the assigned values of the array between two of the plurality of samples; and

calculating a distance between the two samples, wherein the samples within a defined threshold of distance are clustered; and

performing an action based on an output of clustering the plurality of samples based on the plurality of features, wherein the action based on the output of clustering the plurality of samples based on the plurality of features further comprises validate the output of clustering the plurality of samples based on the plurality of features based on tags to identify previously identified malware groups.

18. The computer program product recited in claim 17, wherein the plurality of features correspond to high-risk artifacts, and wherein each high-risk artifact is determined to be associated with a malware sample based on the automated malware analysis.

19. The computer program product recited in claim 17, wherein performing the automated malware analysis includes performing a dynamic analysis or static analysis.

20. The computer program product recited in claim 17, further comprising computer instructions for:

receiving the plurality of samples for performing the automated malware analysis to generate the log files based on the automated malware analysis; and

processing the log files to extract the plurality of features associated with malware.

21. The computer program product recited in claim 17, further comprising computer instructions for:

clustering the plurality of samples based on the plurality of features using a decision tree clustering process.

22. The computer program product recited in claim 17, further comprising computer instructions for:

clustering the plurality of samples based on the plurality of features using a k-means++ clustering process.

23. The computer program product recited in claim 17, wherein a log file for a sample comprises one or more lines based on results of the automated malware analysis for the sample.

24. The computer program product recited in claim 17, further comprising computer instructions for:

clustering the plurality of samples based on the plurality of features using a decision tree clustering process; and

clustering the plurality of samples based on the plurality of features using a k-means++ clustering process.