Method of relocating the stack in a computer system for preventing overrate by an exploit program

• US 5,949,973 A
• Filed: 07/25/1997
• Issued: 09/07/1999
• Est. Priority Date: 07/25/1997
• Status: Expired due to Term

• Current Assignee: Computer Associates Think Inc. (Broadcom, Inc.)
• Original Assignee: MEMCO Software Ltd (Broadcom, Inc.)
• Inventors: Yarom, Yuval
• Primary Examiner(s): Ellis, Richard L.
• Application Number:
  US08/900,518
• Time in Patent Office: 774 Days
• Field of Search: 395/670, 395/186, 395/187.01, 395/188.01, 380/4.25
• US Class Current: 726/23
• CPC Class Codes: G06F 21/52 during program execution, e.g. stack integrity ; Preventing unwanted data erasure; Buffer overflow
• CPC Class Codes: G06F 9/4486 Formation of subprogram jump address

Abstract

A stack override prevention method provides protection against a computer attack that utilizes the technique of stack override to gain control of a computer system. The method of the protection is to permit the stack to be executable but to add functionality that blocks the possibility of passing control via stack override to code inserted into the stack by means of the exploit program. This method includes relocating the entire stack to a random memory location in memory and subsequently erasing the old stack area. By moving the entire stack associated with a process to a random location, the attacker cannot predict the address in which potentially all permitting code resides and thus cannot put the correct value in the location of the return address within the stack frame. The invention is applicable to operating systems which use the stack as means for passing control to and returning from functions and in which the stack is executable.

Claims

1. In a computer system including an operating system executable thereon, a program executable under said operating system, a stack associated with said program, said stack composed of memory that is executable, said stack having a stack pointer associated therewith, said stack memory located at a first location in memory and having a memory return address placed thereon, a method of preventing stack override in the computer system, said method comprising the steps of:

moving the contents of the stack to a second location in memory, said second location being a random memory address, the contents of said stack being moved intact;

modifying said stack pointer in accordance with said random memory address;

modifying the memory contents of the former stack located at said first location such that no executable code remains therein; and

wherein upon each launch of said program a new random memory address is generated and the contents of said stack moved in accordance thereto.

2. The method according to claim 1, wherein said step of modifying the memory contents of the former stack comprises the step of zeroing the memory contents of the former stack.

3. In a computer system including an operating system executable thereon, a program executable under said operating system, a stack associated with said program, said stack composed of memory that is executable, said stack having a stack pointer associated therewith, said stack memory located at a first location in memory and having a memory return address placed thereon, a method of preventing stack override in the computer system, said method comprising the steps of:

determining the length of the stack;

generating a random memory address, wherein upon each launch of said program a new random memory address is generated;

allocating sufficient memory at said random memory address to hold the stack having said length;

copying the contents of the stack intact to said allocated memory;

modifying said stack pointer in accordance with said random memory address; and

modifying the memory contents of the former stack at said first location such that no executable code remains therein.

4. The method according to claim 3, wherein said step of modifying the memory contents of the former stack comprises the step of zeroing the memory contents of the former stack.

5. In a computer system including an operating system having a kernel executable thereon, an application program executable under said operating system, a stack associated with said application program, said stack composed of memory that is executable, said stack having a stack pointer associated therewith, said stack memory located at a first location in memory and having a memory return address placed thereon, a method of preventing stack override in the computer system, said method comprising the steps of:

inserting a code module into the kernel portion of the operating system, said code module containing program code that functions to relocate the stack to a random location in memory, wherein upon each launch of said application program a new random location is generated by said program code;

generating and adding a user exit to the computer system from within the operating system;

executing said program code located within said code module which functions to relocate the stack; and

wherein the stack is relocated intact.

6. The method according to claim 5, wherein said computer system includes environment variables and argument vector areas, wherein said step of executing said program code comprises the steps of:

generating a random offset value;

subtracting said offset from the top of stack value to yield a new top of stack;

copying the data lying between the original top of stack and the bottom of stack to said new top of stack;

subtracting said offset value from each pointer value within the environment variables and the argument vector areas; and

modifying the contents of the stack at the original location such that said memory return address no longer points to valid executable code.

7. The method according to claim 6, wherein said step of modifying the contents of the stack comprises the step of modifying an area of memory starting from the original top of stack or the new bottom of stack, whichever is higher, and ending at the original bottom of stack.

8. The method according to claim 6, wherein said step of modifying the contents of the stack comprises the step of zeroing the memory contents of the former stack.

9. In a computer system including an operating system executable therein, a program executable under operating system, a stack associated with said program said stack composed of memory that is executable, a method of preventing stack override in the computer system, said method comprising the steps of:

generating a random memory address upon the invocation of said program, wherein upon each invocation of said program a new random memory address is generated;

creating the stack at said random memory address location; and

utilizing the stack created at said random memory address location during the course of execution of said program.