Generation and use of trained file classifiers for malware detection

• US 10,304,010 B2
• Filed: 05/31/2017
• Issued: 05/28/2019
• Est. Priority Date: 05/01/2017
• Status: Active Grant

• Current Assignee: SparkCognition Inc.
• Original Assignee: SparkCognition Inc.
• Inventors: Sai, Na
• Primary Examiner(s): Zarrineh, Shahriar
• Application Number:
  US15/610,191
• Publication Number:
  US 20180314983A1
• Time in Patent Office: 727 Days
• Field of Search: 726 23
• CPC Class Codes: G06F 21/562 Static detection
• CPC Class Codes: G06F 2221/033 Test or assess software
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms

Abstract

A method includes receiving one or more n-gram vectors for a file as input to a file classifier, where the one or more n-gram vectors indicate occurrences of groups of entropy indicators in a sequence of entropy indicators representing the file. The method also includes generating, based on the one or more n-gram vectors, output including classification data associated with the file, the classification data indicating whether the file includes malware.

Claims

1. A computing device comprising:

a memory configured to store instructions to execute a file classifier; and

a processor configured to execute the instructions from the memory to perform operations comprising;

receiving, via a network from a remote computing device, a feature vector representing a file stored in a memory of the remote computing device, the feature vector including an entropy indicator n-gram vector, the entropy indicator n-gram vector including data indicating occurrences of a plurality of n-grams in a sequence of entropy indicators representing the file, a first entropy indicator of the sequence of entropy indicators corresponding to a first bin name associated with a first range of entropy values, and a second entropy indicator of the sequence of entropy indicators corresponding to a second bin name associated with a second range of entropy values;

generating, based on the feature vector, output including classification data associated with the file, the classification data indicating whether the file includes malware; and

transmitting the classification data to the remote computing device via the network, wherein access to the file or execution of the file at the remote computing device is restricted responsive to the classification data indicating that the file includes malware.

2. The computing device of claim 1, wherein the entropy indicator n-gram vector includes a Boolean vector indicating the occurrences of the plurality of n-grams in the sequence of entropy indicators.

3. The computing device of claim 1, wherein the entropy indicator n-gram vector includes a vector of counts of the occurrences of the plurality of n-grams in the sequence of entropy indicators.

4. The computing device of claim 1, wherein the entropy indicator n-gram vector includes a zero-skip n-gram vector indicating occurrences of groups of adjacent entropy indicators, and includes at least one skip n-gram vector indicating occurrences of groups of non-adjacent entropy indicators.

5. The computing device of claim 1, wherein each entropy indicator represents an entropy value calculated for a respective chunk of data of the file.

6. The computing device of claim 1, wherein the file classifier corresponds to a decision tree, a neural network, or a support vector machine.

7. The computing device of claim 1, wherein each of the plurality of n-grams is a bigram or a trigram.

8. The computing device of claim 1, wherein the operations further comprise generating printable characters corresponding to the file, and wherein the output is generated further based on the printable characters.

9. A method comprising:

receiving, via a network from a remote computing device, a feature vector representing a file stored in a memory of the remote computing device, the feature vector including an entropy indicator n-gram vector, the entropy indicator n-gram vector including data indicating occurrences of a plurality of n-grams in a sequence of entropy indicators representing the file, a first entropy indicator of the sequence of entropy indicators corresponding to a first bin name associated with a first range of entropy values, and a second entropy indicator of the sequence of entropy indicators corresponding to a second bin name associated with a second range of entropy values;

generating, based on the feature vector, output including classification data associated with the file, the classification data indicating whether the file includes malware; and

transmitting the classification data to the remote computing device via the network, wherein access to the file or execution of the file at the remote computing device is restricted responsive to the classification data indicating that the file includes malware.

10. The method of claim 9, wherein the entropy indicator n-gram vector includes a Boolean vector indicating the occurrences of the plurality of n-grams in the sequence of entropy indicators.

11. The method of claim 9, wherein the entropy indicator n-gram vector includes a vector of counts of the occurrences of the plurality of n-grams in the sequence of entropy indicators.

12. The method of claim 9, wherein each n-gram of the plurality of n-grams represents an occurrence of a group of entropy indicators in a vector representing binned entropy values for chunks of data of the file.

13. The method of claim 9, wherein the first bin name indicates that a first chunk of data of the file has a first calculated entropy value within the first range of entropy values and the second bin name indicates that a second chunk of data of the file has a second calculated entropy value within the second range of entropy values.

14. The method of claim 13, wherein the first entropy indicator is adjacent to the second entropy indicator in the sequence of entropy indicators based on the first chunk of data being adjacent to the second chunk of data in the file.

15. A computer-readable storage device storing instructions that, when executed, cause a computer to perform operations comprising:

receiving, via a network from a remote computing device, a feature vector representing a file stored in a memory of the remote computing device, the feature vector including an entropy indicator n-gram vector, the entropy indicator n-gram vector including data indicating occurrences of a plurality of n-grams in a sequence of entropy indicators representing the file, a first entropy indicator of the sequence of entropy indicators corresponding to a first bin name associated with a first range of entropy values, and a second entropy indicator of the sequence of entropy indicators corresponding to a second bin name associated with a second range of entropy values;

generating, based on the feature vector, output including classification data associated with the file, the classification data indicating whether the file includes malware; and

transmitting the classification data to the remote computing device via the network, wherein access to the file or execution of the file at the remote computing device is restricted responsive to the classification data indicating that the file includes malware.

16. The computer-readable storage device of claim 15, wherein the feature vector includes a zero-skip n-gram vector indicating occurrences of groups of adjacent entropy indicators in the sequence of entropy indicators, and includes at least one skip n-gram vector indicating occurrences of groups of non-adjacent entropy indicators in the sequence of entropy indicators.

17. The computer-readable storage device of claim 15, wherein each entropy indicator represents an entropy value calculated for a respective chunk of data of the file.

18. The computer-readable storage device of claim 15, wherein each n-gram of the plurality of n-grams represents an occurrence of a group of entropy indicators in a vector representing binned entropy values for chunks of data of the file.

19. The computer-readable storage device of claim 15, wherein the feature vector further comprises one or more n-gram vectors indicating occurrences of character pairs in printable characters representing the file.

20. The computer-readable storage device of claim 19, wherein the printable characters representing the file include American Standard Code for Information Interchange (ASCII) characters.