Automated unpacking of executables packed by multiple layers of arbitrary packers

• US 7,996,904 B1
• Filed: 12/19/2007
• Issued: 08/09/2011
• Est. Priority Date: 12/19/2007
• Status: Active Grant

• Current Assignee: NortonLifeLock Inc.
• Original Assignee: Symantec Corporation (NortonLifeLock Inc.)
• Inventors: Chiueh, Tzi-cker, Guo, Fanglu
• Primary Examiner(s): Orgad; Edan
• Assistant Examiner(s): Tolentino; Roderick
• Application Number:
  US11/960,426
• Time in Patent Office: 1,329 Days
• Field of Search: 713/150, 713/188, 713/189, 726/23, 726/22, 726/24
• US Class Current: 726/24
• CPC Class Codes: G06F 21/566 Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities

Abstract

The packing manager provides an automated method that allows existing AV scanning technology to be applied to detect known malware samples packed by one or more packers that are potentially proprietary. The packing manager tracks the memory areas to which an executable binary writes and executes, and so can unpack programs packed by multiple arbitrary packers without requiring reverse-engineering of the packers or any human intervention. By tracking page modification and execution of an executable binary at run time, the packing control module can detect the instant at which the program'"'"'s control is first transferred to a page whose content is dynamically generated, so AV scanning can then be invoked. Thus, code cannot be executed under the packing control manager without being scanned by an AV scanner first.

Claims

1. A computer program product for automated detection of dynamically unpacked malicious code, the computer program product comprising a non-transitory computer-readable medium containing computer program code for performing the method comprising:

detecting an attempt by a program to dynamically generate content to a memory page, wherein the program is suspected to be unpacking malicious executable code to the memory page;

marking the memory page to which the program attempted to dynamically generate content;

allowing the program to dynamically generate the content to the memory page until the program attempts to execute code stored on the memory page;

detecting an attempt by the program to execute the code stored in the memory page, the detection occurring before the execution occurs; and

providing to a malicious code detection module the memory page for analysis for the presence of malicious code.

2. The computer program product of claim 1, further comprising initially marking the memory page as executable but not writable prior to detecting the attempt to dynamically generate content to the memory page, wherein the detection of the attempt to dynamically generate content further comprises intercepting a first exception generated by a memory manager when the program attempts to dynamically generate the content to the memory page set to be executable but not writable.

3. The computer program product of claim 1, wherein the marking comprises marking the memory page as writable but not executable, and wherein the detection of the attempt execute the program further comprises intercepting a second exception generated by the memory manager when the program attempts to execute the code written to the memory page set to be writable but not executable.

4. The computer program product of claim 3, wherein the memory manager is implemented in hardware.

5. The computer program product of claim 1, wherein the program is packed by multiple layers of packers, and wherein the method further comprises repeating the steps for each layer of unpacking that occurs.

6. The computer program product of claim 5, wherein one or more of the multiple layers of packers are arbitrary packers for which antivirus signatures are not available.

7. The computer program product of claim 1, wherein the program is packed by a packer and is being unpacked by an unpacking process, wherein detection of the attempt to execute further comprises detecting a point in the unpacking process after unpacking has concluded but before the program executes the memory page with dynamically generated content, and wherein the providing step further comprises providing the unpacked program to an antivirus scanner to scan for malicious code.

8. A computer-implemented method of automated detection of dynamically unpacked malicious code, the method comprising:

detecting an attempt by a program to dynamically generate content to a memory page, wherein the program is suspected to be unpacking malicious executable code to the memory page;

marking the memory page to which the program attempted to dynamically generate content;

allowing the program to dynamically generate the content to the memory page until the program attempts to execute code stored on the memory page;

detecting an attempt by the program to execute the code stored in the memory page, the detection occurring before the execution occurs; and

providing to a malicious code detection module the memory page for analysis for the presence of malicious code.

9. The method of claim 8, further comprising initially marking the memory page as executable but not writable prior to detecting the attempt to dynamically generate content to the memory page, wherein the detection of the attempt to dynamically generate content further comprises intercepting a first exception generated by a memory manager when the program attempts to dynamically generate the content to the memory page set to be executable but not writable.

10. The method of claim 9, wherein the marking comprises marking the memory page as writable but not executable, and wherein the detection of the attempt execute the program further comprises intercepting a second exception generated by the memory manager when the program attempts to execute the code written to the memory page set to be writable but not executable.

11. The method of claim 10, wherein the memory manager is implemented in hardware.

12. The method of claim 8, wherein the program is packed by multiple layers of packers, and wherein the method further comprises repeating the steps for each layer of unpacking that occurs.

13. The method of claim 8, wherein the program is packed by a packer and is being unpacked by an unpacking process, wherein detection of the attempt to execute further comprises detecting a point in the unpacking process after unpacking has concluded but before the program executes the memory page with dynamically generated content, and wherein the providing step further comprises providing the unpacked program to an antivirus scanner to scan for malicious code.

14. A computer system automated detection of dynamically unpacked malicious code, the system comprising:

a dynamic content detection module for detecting an attempt by a program to dynamically generate content to a memory page, wherein the program is suspected to be unpacking malicious executable code to the memory page;

a marking module for marking the memory page to which the program attempted to dynamically generate content;

a permission module for allowing the program to dynamically generate the content to the memory page until the program attempts to execute code stored on the memory page;

an execution detection module detecting an attempt by the program to execute the code stored in the memory page, the detection occurring before the execution occurs; and

an analysis module for providing to a malicious code detection module the memory page for analysis for the presence of malicious code.

15. The system of claim 14, further comprising the marking module initially marking the memory page as executable but not writable prior to detecting the attempt to dynamically generate content to the memory page, wherein the execution detection module further comprises an exception handler for intercepting a first exception generated by a memory manager when the program attempts to dynamically generate the content to the memory page set to be executable but not writable.

16. The system of claim 15, wherein the marking module is further configured for marking the memory page as writable but not executable, and wherein execution detection module further comprises an exception handler for intercepting a second exception generated by the memory manager when the program attempts to execute the code written to the memory page set to be writable but not executable.

17. The system of claim 16, wherein the memory manager is implemented in hardware.

18. The system of claim 16, wherein the memory manager is implemented in software.

19. The system of claim 14, wherein the program is packed by multiple layers of packers, and wherein the modules are further configured to repeat the steps for each layer of unpacking that occurs.

20. The system of claim 14, wherein the program is packed by a packer and is being unpacked by an unpacking process, wherein the execution detection module is further configured for detecting a point in the unpacking process after unpacking has concluded but before the program executes the memory page with dynamically generated content, and wherein the analysis module is further configured for providing the unpacked program to an antivirus scanner to scan for malicious code.