System and Method for Aggressive Self-Modification in Dynamic Function Call Systems

• US 20120284792A1
• Filed: 04/04/2012
• Published: 11/08/2012
• Est. Priority Date: 10/08/2009
• Status: Active Grant

• Current Assignee: Irdeto B.V. (MultiChoice Group Ltd.)
• Original Assignee: Irdeto Canada Corporation (MultiChoice Group Ltd.)
• Inventors: Liem, Clifford
• Granted Patent: US 9,195,476 B2
• Time in Patent Office: Days
• US Class Current: 726/22
• CPC Class Codes: G06F 21/52 during program execution, e.g. stack integrity ; Preventing unwanted data erasure; Buffer overflow
• CPC Class Codes: G06F 9/4486 Formation of subprogram jump address

Abstract

Provided are a system and method for software obfuscation for transforming a program from a first form to more secure form that is resistant to static and dynamic attacks. The method utilizes a sophisticated pre-analysis step to comprehend the function-call structure, the function-call layout, and the entire function call graph of the program, in order to determine strategic points in the program for changing the program. This provides resistance to static attacks by transforming the original function-call layout to a new layout. Changing the layout may include changing the function boundaries. The method also provides resistance to static attacks by transforming the original function-call structure to a new structure to be able to self modify as the transformed program executes in memory. Changing the function-call structure may include modifying when and how functions are called, and/or choosing random paths of execution that lead to the same result.

Claims

1. A method of transforming a software program from an original form to a more secure form by changing the control flow structure of the program to protect the program against static and dynamic attacks, comprising:

a) analyzing the original function-call structure and function-call layout of the program;

b) transforming the original function-call layout to a new function-call layout;

c) transforming the original function-call structure to a new function-call structure that is able to perform dynamic self modifications;

d) producing a transformed program having a transformed control flow structure, but which is semantically equivalent to the original program;

said transformed program configured to transform the original function-call graph to a new function-call graph upon execution.

2. The method of claim 1, wherein the dynamic self modifications change the time and the manner in which functions are called.

3. The method of claim 1, wherein steps a) to c) are pre-modifications performed at build time and transforming the original function-call graph to a new function-call graph is performed at run time to execute dynamic changes planned in steps a) to c).

4. The method of claim 1, wherein step a) further comprises determining a level of importance for each function in the program for emphasizing important functions when transforming the program.

5. The method of claim 1, wherein steps b) and c) comprise changing function boundaries to disguise the function-call layout.

6. The method of claim 5, wherein changing function boundaries includes breaking function boundaries and joining at least two functions in one.

7. The method of claim 5, wherein changing function boundaries includes inserting new function boundaries to divide a function into at least two functions

8. The method of claim 1, wherein steps b) and c) further comprise inserting function duplicates for making multiple code paths possible during execution of the transformed program.

9. The method of claim 8, further comprising using non-determinism for randomly choosing paths of execution through the function call graph.

10. The method of claim 1, further comprising changing the order of execution of functions in the call graph.

11. The method of claim 1, further comprising creating new functions and inserting new function calls.

12. The method of claim 1, further comprising inserting decoys at strategic points.

13. The method of claim 1, further comprising modifying instructions to change their behavior.

14. The method of claim 1, further comprising removing instructions and replacing them with no operation (NOP) instructions.

15. The method of claim 1, further comprising performing damages to the program, which damages include at least one corruptive damage followed by at least one fix-up damage for further disguising the function-call structure and maintaining functional equivalence to the original program.

16. The method of claim 15, further comprising removing evidence of functions called.

17. The method of claim 15, wherein each corruptive damage has more than one fix-up damage.

18. The method of claim 1, further comprising setting the transformed program to have different function-call graphs whenever it is executed.

19. The method of claim 1, wherein step a) comprises determining strategic points in the program for modifying the program.

20. The method of claim 19, wherein step a) comprises performing a sophisticated pre-analysis step to comprehend the function-call structure, the function-call layout, and the entire function call graph of the program,

21. The method of claim 19, wherein steps b) and c) further comprise placing modification points in the program at a physically and temporally distant point from the corresponding execution points.

22. The method of claim 21, further comprising providing multiple modification points for an execution point.

23. The method of claim 1, wherein the transformed control-flow structure includes the new function call layout and the new function call structure.

24. The method of claim 23, wherein the transformed control flow structure further includes one of more of the following:

jumps, branches, returns, and exceptions.

25. A method of transforming a software program from an original form to a more secure form by changing the control flow structure of the program to protect the program against static and dynamic attacks, said method including a build time phase and a run time phase, said build time phase comprising the steps of:

a) analyzing original function-call structure and function-call layout of the program;

b) transforming the original function-call layout of the program to a new function-call layout;

c) transforming the function-call structure to a new function-call structure that is able to perform dynamic modifications; and

d) producing a transformed program having a transformed control flow structure, but which is semantically equivalent to the original program;

said run time phase comprising;

e) transforming the original function-call graph of the program to a new function-call graph upon execution of the program;

wherein the dynamic modifications performed at run time are complementary to the changes performed at build time to produce a transformed program that is semantically equivalent to the original program.

26. A computer readable memory having recorded thereon statements and instructions for transforming a software program from an original form to a more secure form by changing the control flow structure of the program to protect the program against static and dynamic attacks, said statements and instructions when executed by a processor, cause the processor to perform the steps of:

a) analyzing original function-call structure, and function-call layout of the program;

b) transforming the original function-call layout to a new layout;

c) transforming the original function-call structure to a new structure that is able to perform dynamic self modifications;

d) producing a transformed program having a transformed control flow structure, but which is semantically equivalent to the original program;

said transformed program configured to transform the original function-call graph to a new function-call graph upon execution.