Method, computer program and computer for analyzing an executable computer file

• US 8,479,174 B2
• Filed: 03/30/2007
• Issued: 07/02/2013
• Est. Priority Date: 04/05/2006
• Status: Active Grant

• Current Assignee: Carbonite LLC (Open Text Corporation)
• Original Assignee: Prevx Ltd (Open Text Corporation)
• Inventors: Chiriac, Mihai
• Primary Examiner(s): Kang, Insun
• Application Number:
  US11/694,261
• Publication Number:
  US 20080040710A1
• Time in Patent Office: 2,286 Days
• Field of Search: None
• US Class Current: 717/136
• CPC Class Codes: G06F 21/566 Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities

Abstract

In one aspect, an executable computer file is partitioned into basic blocks of code. At least some basic blocks are translated into translated basic blocks. At least some translated basic blocks are linked in memory of a computer. At least some translated basic blocks on the computer are executed so as to enable the file to be unpacked or decrypted. In this way, the file can be analyzed to determine whether the file is or should be classed as malware. In another aspect, at least a read page of cache memory is created for at least some basic blocks and at least a write page of cache memory is created for at least some basic blocks. During the execution of a basic block, at least one of the read page and the write page is checked for a cached real address corresponding to the virtual address that is being accessed for said basic block.

Claims

1. A method of unpacking or decrypting an executable computer file in a malware scanner using a host computer having a hardware processor, the method comprising:

during emulation in a virtual memory on the host computer for malware detection;

partitioning the executable computer file into plural basic blocks of code;

creating at least a read page of a cache memory for at least some of the basic blocks, the read page of the cache memory storing a read cached real address corresponding to a read cached virtual memory address for a respective basic block, and creating at least a write page of the cache memory for at least some of the basic blocks, the write page of the cache memory storing a write cached real address corresponding to a write cached virtual memory address for a respective basic block;

emulating the executable file by executing some basic blocks of code in the virtual memory on the host computer;

checking, during the execution of said some basic blocks of code, at least one of the read page and the write page of the cache memory for a cached real address corresponding to the virtual address that is being accessed for a respective basic block of said some basic blocks of code;

translating other basic blocks of code during the emulation into translated basic blocks of code that are functionally equivalent to said other basic blocks and which can be executed directly by the hardware processor of the host computer rather than by the virtual computer;

linking at least some of the translated basic blocks of code in a real memory of the host computer; and

executing at least some of the translated basic blocks of code on the host computer so as to enable the executable computer file to be unpacked or decrypted in the virtual memory, whereupon the unpacked or decrypted executable computer file can be analyzed to determine whether the executable computer file is or should be classed as malware.

2. The method according to claim 1, comprising, when a basic block of code is newly translated during the translating step, checking whether an already translated basic block of code has a jump instruction to the newly translated basic block of code and, if so, linking the newly translated basic block of code and the already translated basic block of code in the real memory.

3. The method according to claim 1, wherein a first translated basic block is linked to a second translated basic block by updating a branch instruction of the first translated basic block to point to the second translated basic block, or by inserting an unconditional branch instruction into the first translated basic block to jump to the second translated basic block, or by removing any existing branch instructions from the first translated basic block and physically moving the second translated basic block to the memory address that follows the first translated basic block.

4. The method according to claim 1, wherein the linking of the translated basic blocks of code is carried out dynamically as the blocks are translated.

5. The method according to claim 1, comprising passing flow of the method to a virtual machine in the case of one or more of (i) a cache miss and (ii) a CPU exception arising during execution of said at least some of the translated basic blocks.

6. The method according to claim 1, wherein the basic blocks of code that are translated are only those that execute more than a predetermined number of times during the emulation.

7. The method according to claim 6, wherein the value for said predetermined number of times is dynamically variable during the emulation.

8. The method according to claim 1, comprising creating at least a stack page of the cache memory for at least some of the basic blocks, the stack page of the cache memory storing a stack cached real address corresponding to a stack cached virtual memory address for a respective basic block, and comprising, during the execution of a basic block, checking the stack page of the cache memory for a cached real address corresponding to the virtual address that is being accessed for said basic block.

9. The method according to claim 1, wherein the partitioning is carried out dynamically.

10. The method according to claim 1, wherein the executable file is a Portable Executable File.

11. The host computer comprising a hardware processor programmed to carry out a method according to claim 1.

12. A computer program for unpacking or decrypting an executable computer file in a malware scanner using a host computer having a hardware processor, the computer program recorded on a non-transitory computer readable medium and comprising program instructions for causing a said host computer to carry out a method of:

during emulation in a virtual memory on the host computer for malware detection by the malware scanner;

partitioning the executable computer file into plural basic blocks of code;

creating at least a read page of a cache memory for at least some of the basic blocks, the read page of the cache memory storing a read cached real address corresponding to a read cached virtual memory address for a respective basic block, and creating at least a write page of the cache memory for at least some of the basic blocks, the write page of the cache memory storing a write cached real address corresponding to a write cached virtual memory address for a respective basic block;

emulating the executable file by executing some basic blocks of code in the virtual memory on the host computer;

checking, during the execution of said some basic blocks of code, at least one of the read page and the write page of cache memory for a cached real address corresponding to the virtual address that is being accessed for a respective basic block of said some basic blocks of code;

translating other basic blocks of code during the emulation into translated basic blocks of code that are functionally equivalent to said other basic blocks and which can be directly by the hardware processor of the host computer rather than by the virtual computer;

linking at least some of the translated basic blocks of code in a real memory of the host computer; and

executing at least some of the translated basic blocks of code on the host computer so as to enable the executable computer file to be unpacked or decrypted in the virtual memory, whereupon the unpacked or decrypted executable computer file can be analyzed in the virtual memory to determine whether the executable computer file is or should be classed as malware.

13. The computer program according to claim 12, wherein the computer program is arranged so that the method comprises, when a basic block of code is newly translated during the translating step, checking whether an already translated basic block of code has a jump instruction to the newly translated basic block of code and, if so, linking the newly translated basic block of code and the already translated basic block of code in the real memory.

14. The computer program according to claim 12, wherein the computer program is arranged so that a first translated basic block is linked to a second translated basic block by updating a branch instruction of the first translated basic block to point to the second translated basic block, or by inserting an unconditional branch instruction into the first translated basic block to jump to the second translated basic block, or by removing any existing branch instructions from the first translated basic block and physically moving the second translated basic block to the memory address that follows the first translated basic block.

15. The computer program according to claim 12, wherein the computer program is arranged so that the linking of the translated basic blocks of code is carried out dynamically as the blocks are translated.

16. The computer program according to claim 12, wherein the computer program is arranged so that flow of the method passes to a virtual machine in the case of one or more of (i) a cache miss and (ii) a CPU exception arising during execution of said at least some of the translated basic blocks.

17. A computer program according to claim 12, wherein the basic blocks of code that are translated are only those that execute more than a predetermined number of times during the emulation.

18. The computer program according to claim 17, wherein the computer program is arranged so that the value for said predetermined number of times is dynamically variable during the emulation.

19. A computer program according to claim 12, wherein the computer program is arranged so that the method comprises creating at least a stack page of the cache memory for at least some of the basic blocks, the stack page of the cache memory storing a stack cached real address corresponding to a stack cached virtual memory address for a respective basic block, and comprises, during the execution of a basic block, checking the stack page of the cache memory for a cached real address corresponding to the virtual address that is being accessed for said basic block.

20. computer program according to claim 12, wherein the computer program is arranged so that the partitioning is carried out dynamically.

21. The computer program according to claim 12, wherein the computer program is arranged so that the executable computer file is a Portable Executable File.

22. A computer program for unpacking or decrypting an executable computer file, the computer program recorded on a non-transitory computer readable medium and comprising program instructions for causing a host computer to carry out a method of:

during emulation in a virtual memory on a host computer for malware detection;

partitioning the executable computer file into plural basic blocks of code;

creating at least a read page of a cache memory for at least some of the basic blocks, the read page of the cache memory storing a read cached real address corresponding to a read cached virtual memory address for a respective basic block, and creating at least a write page of the cache memory for at least some of the basic blocks, the write page of the cache memory storing a write cached real address corresponding to a write cached virtual memory address for a respective basic block;

emulating the executable file by executing some of the basic blocks of code;

during the execution of said some of the basic blocks of code, checking at least one of the read page and the write page of the cache memory for a cached real address corresponding to the virtual address that is being accessed for a respective basic block of said some of the basic blocks of code;

translating basic blocks of code that execute more than a predetermined number of times during the emulation into translated basic blocks of code that can be executed by the host computer;

linking at least some of the translated basic blocks of code in a real memory of the host computer; and

executing at least some of the translated basic blocks of code on the host computer so as to enable the executable computer file to be unpacked or decrypted, whereupon the unpacked or decrypted executable computer file can be analyzed to determine whether the executable computer file is or should be classed as malware.

23. The computer program according to claim 22, wherein the computer program is arranged so that the method comprises creating at least a stack page of a cache memory for at least some of the basic blocks, the stack page of the cache memory storing a stack cached real address corresponding to a stack cached virtual memory address for the respective basic block, and comprises, during the execution of a basic block, checking the stack page of the cache memory for a cached real address corresponding to the virtual address that is being accessed for said basic block.

24. The computer program according to claim 22, wherein the computer program is arranged so that the partitioning is carried out dynamically.

25. The computer program according to claim 22, wherein the computer program is arranged so that the executable computer file is a Portable Executable file.

26. A method of analyzing an executable computer file using a host computer, the method comprising:

during emulation in a virtual memory on a host computer for malware detection;

partitioning the executable computer file into plural basic blocks of code;

creating at least a read page of a cache memory of the host computer for at least some of the basic blocks, the read page of the cache memory storing a read cached real address corresponding to a read cached virtual memory address for a respective basic block, and creating at least a write page of the cache memory for at least some of the basic blocks, the write page of the cache memory storing a write cached real address corresponding to a write cached virtual memory address for a respective basic block;

emulating the executable file by executing some basic blocks of code, wherein, during the execution of a basic block of said some basic blocks of code, at least one of the read page and the write page of the cache memory is checked for a cached real address corresponding to the virtual address that is being accessed for said basic block;

translating basic blocks of code that execute more than a predetermined number of times during the emulation into translated basic blocks of code that can be executed by the host computer;

linking at least some of the translated basic blocks of code in a real memory of the host computer; and

executing at least some of the translated basic blocks of code on the host computer so as to enable the executable computer file to be unpacked or decrypted, whereupon the unpacked or decrypted executable computer file can be analyzed to determine whether the executable computer file is or should be classed as malware.

27. The method according to claim 26, comprising, when a basic block of code is newly translated during the translating step, checking whether an already translated basic block of code has a jump instruction to the newly translated basic block of code and, if so, linking the newly translated basic block of code and the already translated basic block of code in the real memory.

28. The method according to claim 26, wherein a first translated basic block is linked to a second translated basic block by updating a branch instruction of the first translated basic block to point to the second translated basic block, or by inserting an unconditional branch instruction into the first translated basic block to jump to the second translated basic block, or by removing any existing branch instructions from the first translated basic block and physically moving the second translated basic block to the memory address that follows the first translated basic block.

29. The method according to claim 26, wherein the linking of the translated basic blocks of code is carried out dynamically as the blocks are translated.

30. The method according to claim 26, comprising passing flow of the method to a virtual machine in the case of one or more of (i) a cache miss and (ii) a CPU exception arising during execution of said at least some of the translated basic blocks.

31. The method according to claim 26, comprising creating at least a stack page of the cache memory for at least some of the basic blocks, the stack page of the cache memory storing a stack cached real address corresponding to a stack cached virtual memory address for the respective basic block, and comprising, during the execution of a basic block, checking the stack page of the cache memory for a cached real address corresponding to the virtual address that is being accessed for said basic block.

32. The method according to claim 26, wherein the value for said predetermined number of times is dynamically variable during the emulation.

33. The method according to claim 26, wherein the partitioning is carried out dynamically.

34. The method according to claim 26, wherein the executable computer file is a Portable Executable file.

35. The host computer comprising a hardware processor programmed to carry out a method according to claim 26.