System and method to mitigate malicious calls

• US 9,767,283 B2
• Filed: 06/27/2014
• Issued: 09/19/2017
• Est. Priority Date: 06/27/2014
• Status: Active Grant

• Current Assignee: McAfee, LLC
• Original Assignee: McAfee, Inc. (McAfee, LLC)
• Inventors: Szor, Peter, Mathur, Rachit
• Primary Examiner(s): SHAIFER HARRIMAN, DANT B
• Application Number:
  US14/318,242
• Publication Number:
  US 20150379267A1
• Time in Patent Office: 1,180 Days
• Field of Search: 726 23
• CPC Class Codes: G06F 21/566 Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities

Abstract

Systems and methods are provided in example embodiments for mitigating malicious calls. The system can be configured to receive a function call, determine the location of a memory page that initiated the function call, determine if the memory page is associated with a trusted module, and block the function call if the memory page is not associated with the trusted module. In addition, the system can determine the return address for the function call and block the function call if the return address does not belong to the trusted module. Further, the system can determine a parameter for the function call, determine if the parameter is a known parameter used by the process that called the function, and block the function call if the parameter is not the known parameter used by the process that called the function.

Claims

1. At least one non-transitory computer-readable medium comprising one or more instructions that, when executed by a processor, perform a method comprising:

receiving a function call from a security program, wherein the function call is an asynchronous procedure call;

determining a return address for the function call on an operating system stack above an address of the function call; and

pushing parameters on the operating system stack for a call for a dynamic link library to lead an operating system kernel back to restore a routine, if the return address does not belong to a trusted module.

2. The at least one non-transitory computer-readable medium of claim 1, the method further comprising:

allowing the function call if the return address does belong to the trusted module.

3. The at least one non-transitory computer-readable medium of claim 2, the method further comprising:

determining a location of a memory page that initiated the function call;

determining if the memory page is associated with the trusted module; and

blocking the function call if the memory page is not associated with the trusted module.

4. The at least one non-transitory computer-readable medium of claim 3, the method further comprising:

allowing the function call if the memory page is associated with the trusted module.

5. The at least one non-transitory computer-readable medium of claim 1, wherein the trusted module is part of the security program.

6. The at least one non-transitory computer-readable medium of claim 1, the method further comprising:

determining whether the return address on the operating system stack points back to the security program.

7. The at least one non-transitory computer-readable medium of claim 1, further comprising:

determining whether the return address is within a predetermined distance on the operating system stack from the function call.

8. The at least one non-transitory computer-readable medium of claim 1, the method further comprising:

determining whether the operating system stack includes an address to an executable page that does not belong to an active process dynamic link library.

9. The at least one non-transitory computer-readable medium of claim 1, wherein the parameters are pushed on the operating system stack with a security program interface module to prevent a termination of a process.

10. An apparatus, comprising:

a processor configured to

receive a function call from a security program, wherein the function call is an asynchronous procedure call;

determine if a memory page is associated with a trusted module by determining a return address for the function call on an operating system stack above an address of the function call; and

push parameters on the operating system stack for a call for a dynamic link library to lead an operating system kernel back to restore a routine, if the memory page is not associated with the trusted module.

11. The apparatus of claim 10, wherein the function call is allowed if the memory page is associated with the trusted module.

12. The apparatus of claim 10, wherein the processor is further configured to determine whether the return address on the operating system stack points back to the security program.

13. The apparatus of claim 10, wherein the processor is further configured to determine whether the return address is within a predetermined distance on the operating system stack from the function call.

14. The apparatus of claim 10, wherein the processor is further configured to determine whether the operating system stack includes an address to an executable page that does not belong to an active process dynamic link library.

15. The apparatus of claim 10, wherein the parameters are pushed on the operating system stack with a security program interface module to prevent a termination of a process.

16. The apparatus of claim 10, wherein the trusted module is part of the security program.

17. A method, comprising:

receiving an exit process function call from a process, wherein the exit process function call is an asynchronous procedure call;

determining a parameter for the exit process function call;

determining if the parameter is an exit code for the exit process function call used by the process, wherein the process identifies the exit code; and

blocking the exit process function call if the parameter is not the exit code.

18. The method of claim 17, further comprising:

allowing the exit process function call if the parameter is the exit code.

19. The method of claim 17, wherein the process is a security program.

20. A system for mitigating malicious calls, the system comprising:

a processor configured for

receiving a function call from a security program, wherein the function call is an asynchronous procedure call;

determining a return address for the function call on an operating system stack above an address of the function call; and

pushing parameters on the operating system stack for a call for a dynamic link library to lead an operating system kernel back to restore a routine, if the return address does not belong to a trusted module.

21. The system of claim 20, wherein the processor is further configured for determining whether the return address on the operating system stack points back to the security program.

22. The system of claim 20, wherein the processor is further configured for determining whether the return address is within a predetermined distance on the operating system stack from the function call.

23. The system of claim 20, wherein the processor is further configured for determining whether the operating system stack includes an address to an executable page that does not belong to an active process dynamic link library.

24. The system of claim 20, wherein the parameters are pushed on the operating system stack with a security program interface module to prevent a termination of a process.

25. The system of claim 20, wherein the processor is further configured for allowing the function call if the return address does belong to the trusted module.