STATIC FEATURE EXTRACTION FROM STRUCTURED FILES

• US 20160246800A1
• Filed: 02/12/2016
• Published: 08/25/2016
• Est. Priority Date: 01/31/2014
• Status: Active Grant

• Current Assignee: Cylance Inc. (Blackberry Limited)
• Original Assignee: Cylance Inc. (Blackberry Limited)
• Inventors: Soeder, Derek A., Permeh, Ryan, Golomb, Gary, Wolff, Matthew
• Granted Patent: US 9,959,276 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/36 Preventing errors by testing or debugging software
• CPC Class Codes: G06F 11/3604 Software analysis for verifying properties of programs byte-code verification G06F9/44589
• CPC Class Codes: G06F 16/116 Details of conversion of file system types or formats
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 21/562 Static detection
• CPC Class Codes: G06F 21/563 by source code analysis
• CPC Class Codes: G06F 21/564 by virus signature recognition
• CPC Class Codes: G06F 21/565 by checking file integrity
• CPC Class Codes: G06F 40/205 Parsing

Abstract

Data is received or accessed that includes a structured file encapsulating data required by an execution environment to manage executable code wrapped within the structured file. Thereafter, code and data regions are iteratively identified in the structured file. Such identification is analyzed so that at least one feature can be extracted from the structured file. Related apparatus, systems, techniques and articles are also described.

Claims

1. A method comprising:

receiving or accessing data comprising a structured file encapsulating data required by an execution environment to manage executable code wrapped within the structured file;

iteratively identifying code and data regions in the structured file; and

extracting at least one feature from the structured file by analyzing at least a portion of the identified code and data regions.

2. The method of claim 1, wherein the structured file is selected from a group consisting of:

Portable Executable (PE) format files, disk operating system (DOS) executable files, New Executable (NE) files, Linear Executable (LE) files, Executable and Linkable Format (ELF) files, JAVA Archive (JAR) files, and SHOCKWAVE/FLASH (SWF) files.

3. The method of claim 1, wherein the execution environment is an operating system or a virtual machine.

4. The method of claim 1, further comprising:

determining that the structured file is valid by examining at least one header within the structured file to determine whether it encapsulates a valid signature.

5. The method of claim 1, wherein the extracted at least one feature is a first order feature.

6. The method of claim 5, further comprising:

deriving the extracted at least one first order feature into a higher-order feature.

7. The method of claim 1, further comprising:

analyzing negative space within the structured file to extract at least one additional feature, the negative space being different from the identified code and data regions.

8. The method of claim 1, further comprising:

transforming the extracted at least one feature.

9. The method of claim 8, wherein the transforming comprises one or more of:

sanitizing the extracted at least one feature,

truncating the extracted at least one feature, and

encoding at least a portion of the at least one feature.

10. The method of claim 1, wherein the identifying code and data regions in the structured file comprises parsing and disassembling the structured file.

11. The method of claim 1, wherein data within the structured file is arranged hierarchically and the structured file comprises a top level header encapsulating a first data structure that encapsulates a second data structure.

12. The method of claim 1, wherein the iteratively identifying code and data regions in the structured file comprises identifying types of code and/or data regions in the structured file.

13. The method of claim 1, further comprising:

providing the extracted at least one feature to a machine learning model.

14. The method of claim 13, wherein the machine learning model consumes the extracted at least one feature without intermediate processing or parsing.

15. A non-transitory computer program product storing instructions which, when executed by at least one data processor forming part of at least one computing system, result in operations comprising:

receiving or accessing data comprising a structured file encapsulating data required by an execution environment to manage executable code wrapped within the structured file;

iteratively identifying code and data regions in the structured file; and

extracting at least one feature from the structured file by analyzing at least a portion of the identified code and data regions.

16. A system comprising:

at least one data processor; and

memory storing instructions which, when executed by the at least one data processor, result in operations comprising;

receiving or accessing data comprising a structured file encapsulating data required by an execution environment to manage executable code wrapped within the structured file;

iteratively identifying code and data regions in the structured file; and

extracting at least one feature from the structured file by analyzing at least a portion of the identified code and data regions.