Automated malware family signature generation

• US 10,432,648 B1
• Filed: 08/28/2017
• Issued: 10/01/2019
• Est. Priority Date: 08/28/2017
• Status: Active Grant

• Current Assignee: Palo Alto Networks Incorporated
• Original Assignee: Palo Alto Networks Incorporated
• Inventors: Xu, Zhi, Wang, Jiajie, Zhang, Xiao, Hu, Wenjun
• Primary Examiner(s): Smithers, Matthew
• Application Number:
  US15/688,649
• Time in Patent Office: 764 Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
• CPC Class Codes: H04W 12/128 Anti-malware arrangements, e.g. protection against SMS fraud or mobile malware

Abstract

The automatic generation of malware family signatures is disclosed. A set of metadata associated with a plurality of samples is received. The samples are clustered. For members of a first cluster, a set of similarities shared among at least a portion of the members of the first cluster is determined. The similarities are evaluated for suitability as a malware family signature. In the event the similarities are determined to be suitable as a malware family signature, a signature is generated.

Claims

1. A system, comprising:

a processor configured to;

receive a set of metadata associated with a plurality of samples;

cluster the samples;

determine, for members of a first cluster, a set of similarities shared among at least a portion of the members of the first cluster; and

evaluate the similarities for suitability as a malware family signature, including by generating a query encompassing the similarities and performing the query against a malware repository; and

a memory coupled to the processor and configured to provide the processor with instructions.

2. The system of claim 1 wherein the processor is further configured to determine, for a first sample included in the plurality of samples, a set of features comprising name-value pairs.

3. The system of claim 2 wherein determining the set of features includes performing a tokenization.

4. The system of claim 1 wherein the processor is further configured to assign weights to a set of tokens.

5. The system of claim 4 wherein the weights are assigned using term frequency-inverse document frequency analysis.

6. The system of claim 1 wherein the processor is further configured to generate a vector list that indicates, for a given sample, a set of tokens hit by the sample.

7. The system of claim 1 wherein clustering the samples includes performing multiple rounds of k-means clustering and selecting as output those clusters with consistent membership across the multiple rounds.

8. The system of claim 1 wherein determining the similarities includes determining a portion of metadata that is present in all members of the first cluster.

9. The system of claim 8 wherein evaluating the similarities includes comparing a size of the first cluster to a number of samples in a corpus that also includes the portion of metadata.

10. The system of claim 1 wherein evaluating the similarities further includes determining a quality score based at least in part on a number of members in the first cluster and a number of results provided in response to the query.

11. The system of claim 1 wherein the processor is further configured to iteratively perform the clustering, determining, and evaluating until a low quality threshold is reached.

12. The system of claim 11 wherein the processor is further configured to exclude metadata associated with samples for which malware signatures were assigned in a previous iteration, prior to performing a current iteration.

13. The system of claim 1 wherein the processor is further configured to generate the malware family signature.

14. The system of claim 1 wherein the processor is further configured to provide as output a list of malware samples matching the malware family signature.

15. A method, comprising:

receiving a set of metadata associated with a plurality of samples;

clustering the samples;

determining, for members of a first cluster, a set of similarities shared among at least a portion of the members of the first cluster; and

evaluating the similarities for suitability as a malware family signature, including by generating a query encompassing the similarities and performing the query against a malware repository.

16. The method of claim 15 further comprising generating a vector list that indicates, for a given sample, a set of tokens hit by the sample.

17. The method of claim 15 wherein clustering the samples includes performing multiple rounds of k-means clustering and selecting as output those clusters with consistent membership across the multiple rounds.

18. The method of claim 15 further comprising providing as output a list of malware samples matching the malware family signature.

19. The method of claim 15 further comprising determining, for a first sample included in the plurality of samples, a set of features comprising name-value pairs.

20. The method of claim 19 wherein determining the set of features includes performing a tokenization.

21. The method of claim 15 further comprising assigning weights to a set of tokens.

22. The method of claim 21 wherein the weights are assigned using term frequency-inverse document frequency analysis.

23. The method of claim 15 wherein determining the similarities includes determining a portion of metadata that is present in all members of the first cluster.

24. The method of claim 23 wherein evaluating the similarities includes comparing a size of the first cluster to a number of samples in a corpus that also includes the portion of metadata.

25. The method of claim 15 wherein evaluating the similarities further includes determining a quality score based at least in part on a number of members in the first cluster and a number of results provided in response to the query.

26. The method of claim 15 further comprising iteratively performing the clustering, determining, and evaluating until a low quality threshold is reached.

27. The method of claim 26 further comprising excluding metadata associated with samples for which malware signatures were assigned in a previous iteration, prior to performing a current iteration.

28. The method of claim 15 further comprising generating the malware family signature.

29. A computer program product embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving a set of metadata associated with a plurality of samples;

clustering the samples;

determining, for members of a first cluster, a set of similarities shared among at least a portion of the members of the first cluster; and

evaluating the similarities for suitability as a malware family signature, including by generating a query encompassing the similarities and performing the query against a malware repository.