Return-oriented programming detection

• US 9,594,912 B1
• Filed: 06/20/2014
• Issued: 03/14/2017
• Est. Priority Date: 06/06/2014
• Status: Active Grant

• Current Assignee: Fireeye Security Holdings US LLC
• Original Assignee: FireEye, Inc.
• Inventors: Thioux, Emmanuel, Lin, Yichong
• Primary Examiner(s): Pyzocha, Michael
• Assistant Examiner(s): Massie, David
• Application Number:
  US14/311,014
• Time in Patent Office: 998 Days
• Field of Search: 726/25
• US Class Current: 1/1
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 21/566 Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities
• CPC Class Codes: G06F 21/577 Assessing vulnerabilities and evaluating computer system security
• CPC Class Codes: G06F 2212/1052 Security improvement
• CPC Class Codes: G06F 2221/033 Test or assess software

Abstract

According to one embodiment, a threat detection system is integrated with at least a dynamic analysis engine. The dynamic analysis engine is configured to automatically detect a function call by an application, responsive to detecting the function call, analyze contents located at one or more addresses located within a portion of memory allocated for the application, and, based on the analysis, determine whether one or more objects included in received network traffic is associated with a return-oriented programming (ROP) exploit.

Claims

1. A computerized method, comprising:

detecting a function call by an application;

responsive to detecting the function call, capturing and preserving contents in a range of a stack of memory addresses surrounding a current stack pointer;

analyzing contents located at a first valid address within the preserved contents to detect a first gadget and contents located at a second valid address within the preserved contents to detect a second gadget, the first valid address and the second valid address being located within a portion of a region of memory allocated for the application, wherein the first gadget comprises a first sequence of a first number of instructions less than a predetermined number of instructions followed by a return instruction, and the second gadget comprises a second sequence of a second number of instructions less than the predetermined number of instructions followed by a return instruction;

assigning a first weight to the first gadget based on the first number of instructions and a second weight to the second gadget based on the second number of instructions, wherein the first weight is different than the second weight; and

determining that a return-oriented programming (ROP) exploit is present within the portion of the region of allocated memory within the preserved contents based on at least whether a combination of at least the first weight and the second weight exceeds a predetermined weight threshold.

2. The computerized method of claim 1, wherein the first gadget includes (1) a sequence of one or more computer instructions other than a return instruction and (2) the return instruction following the sequence of one or more computer instructions, where a number of instructions forming the sequence of one or more computer instructions is less than a predefined threshold.

3. The computerized method of claim 2, wherein the function call is a system call.

4. The computerized method of claim 2, wherein the function call is an application programming interface (API) call.

5. The computerized method of claim 2, wherein the first gadget includes a first sequence of computer instructions and the second gadget includes a second sequence of computer instructions.

6. The computerized method of claim 2, further comprising:

determining whether any valid addresses are present within the portion of the region of allocated memory.

7. The computerized method of claim 2, wherein a valid address is an address in memory of a software component loaded by the application.

8. The computerized method of claim 7, wherein the software component loaded by the application is a dynamically-loaded library (DLL).

9. The computerized method of claim 1, wherein the content of the first gadget is a length of the sequence of one or more computer instructions preceding the return instruction.

10. The computerized method of claim 1, wherein the first gadget includes at least one instruction but less than a threshold number of instructions preceding the return instruction.

11. The computerized method of claim 1, wherein the second gadget includes only Rap the return instruction.

12. The computerized method of claim 2, further comprising:

prior to detecting the function call by the application, detecting a stack discrepancy.

13. The computerized method of claim 12, wherein the detection of the stack discrepancy is accomplished by analyzing a Thread Information Block of the application.

14. The computerized method of claim 2, further comprising:

dynamically configuring a virtual machine with a software image representing a current operating state of a targeted client device, the software image representing content and structure of a storage volume for the targeted client device at a time of configuring the virtual machine; and

detecting the function call, responsive to detecting the function call, analyzing the contents located at one or more of the valid addresses, and determining that the ROP exploit is present within the portion of the region of allocated memory within the virtual machine.

15. The computerized method of claim 14, wherein the virtual machine includes a module, the application and an operating system of the targeted client device.

16. The computerized method of claim 15, wherein the module queries one or more of the application or the operating system to determine what memory has been allocated to the application.

24. The computerized method of claim 1, wherein the preserved contents includes a copy of the range of the stack of memory addresses surrounding the current stack pointer when the function call is detected.

26. The computerized method of claim 1, wherein the first gadget includes (1) a sequence of one or more computer instructions other than a return instruction and (2) a return instruction following the sequence of one or more computer instructions, where a number of instructions forming the sequence of one or more computer instructions is less than a predefined threshold, and the second gadget includes only a return instruction.

28. The method of claim 1, wherein determining the ROP exploit is present is based on at least a combination of the first weight, the second weight and weights of one or more additional gadgets, each detected at valid addresses located within the portion of the region of memory allocated for the application within the preserved contents.

17. A system comprising:

one or more processors;

a storage module communicatively coupled to the one or more processors, the storage module includes logic to;

detect a function call by an application;

responsive to detecting the function call, capture and preserve contents in a range of a stack of memory addresses surrounding a current stack pointer;

analyze contents located at a first valid address within the preserved contents to detect a first gadget and contents located at a second valid address within the preserved contents to detect a second gadget, the first valid address and the second valid address being located within a portion of a region of memory allocated for the application, wherein the first gadget comprises a first sequence of a first number of instructions less than a predetermined number of instructions followed by a return instruction, and the second gadget comprises a second sequence of a second number of instructions less than the predetermined number of instructions followed by a return instruction;

assign a first weight to the first gadget based on the first number of instructions and a second weight to the second gadget based on the second number of instructions, wherein the first weight is different than the second weight; and

determine that a return-oriented programming (ROP) exploit is present within the portion of the region of allocated memory within the preserved contents based on at least whether a combination of at least the first weight and the second weight exceeds a predetermined weight threshold.

18. The system of claim 17, wherein the first gadget includes (1) a sequence of one or more computer instructions other than a return instruction and (2) the return instruction following the sequence of one or more computer instructions, where a number of instructions forming the sequence of one or more computer instructions is less than a predefined threshold.

19. The system of claim 18, wherein the function call is a system call.

20. The system of claim 18, wherein the function call is an application programming interface (API) call.

21. The system of claim 18, wherein a valid address is an address in memory of a software component loaded by the application.

22. The system of claim 21, wherein the software component loaded by the application is a dynamically-loaded library (DLL).

23. The system of claim 18, wherein presence of the ROP exploit is based on a combined weight of all detected gadgets present within the portion of the region of allocated memory.

25. The system of claim 17, wherein the preserved contents includes a copy of the range of the stack of memory addresses surrounding the current stack pointer when the function call is detected.

27. The system of claim 17, wherein the first gadget includes (1) a sequence of one or more computer instructions other than a return instruction and (2) a return instruction following the sequence of one or more computer instructions, where a number of instructions forming the sequence of one or more computer instructions is less than a predefined threshold, and the second gadget includes only a return instruction.

29. The system of claim 18, wherein determining the ROP exploit is present is based on at least a combination of the first weight, the second weight and weights of one or more additional gadgets, each detected at valid addresses located within the portion of the region of memory allocated for the application within the preserved contents.