Pointguard: method and system for protecting programs against pointer corruption attacks

• US 20030182572A1
• Filed: 12/06/2002
• Published: 09/25/2003
• Est. Priority Date: 12/06/2001
• Status: Active Grant

• Current Assignee: RPX Corporation
• Original Assignee: Novell Incorporated (Open Text Corporation)
• Inventors: Wagle, Perry Michael, Arnold, Seth Richard, Cowan, Stanley Crispin, Beattie, Steven Michael
• Granted Patent: US 7,752,459 B2
• Time in Patent Office: Days
• US Class Current: 713/200
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 21/125 by manipulating the program code, e.g. source code, compiled code, interpreted code, machine code
• CPC Class Codes: G06F 21/52 during program execution, e.g. stack integrity ; Preventing unwanted data erasure; Buffer overflow
• CPC Class Codes: G06F 9/4486 Formation of subprogram jump address

Abstract

To protect computer programs against security attacks that attempt to corrupt pointers within the address space of the program, the value of a pointer is encrypted each time the pointer is initialized or modified, and then the value is decrypted before use, i.e., each time the pointer is read. Preferably, the encrypting and decrypting steps are effected by instructions generated by a compiler during compilation of the program. One convenient method of implementing the encrypting and decrypting steps is by XOR'"'"'ing the pointer with a predetermined encryption key value, which could be specially selected or selected at random.

Claims

1. A method of protecting a computer program against pointer corruption attacks, the method comprising the steps of:

encrypting a value of a pointer each time the value of the pointer is modified; and

decrypting the encrypted value of the pointer before use each time the pointer is read.

2. A method according to claim 1 wherein said encrypting and decrypting steps are effected by instructions generated by a compiler during compilation of the said program.

3. A method according to claim 1 wherein said encrypting and decrypting steps are effected by XOR'"'"'ing the pointer with a predetermined encryption key value.

4. A method according to claim 3 including selecting the key value at random.

5. A method according to claim 1 including storing the pointer value in the clear only in CPU registers and storing the encrypted value of the pointer in addressable memory.

6. A method according to claim 1 wherein instructions to implement said encryption and decryption steps are inserted into the program in a preprocessor.

7. A method according to claim 1 wherein instructions to implement said encryption and decryption steps are inserted by a compiler during manipulation of an intermediate representation of the program.

9. A method according to claim 1 to protect the encryption key from tampering.

8. A method according to claim I wherein instructions to implement said encryption and decryption steps are inserted by a compiler in the context of an architecture-dependent representation by transforming instructions to load pointer values from memory into registers to add the decryption instructions, and transform the saving of pointer values from registers to memory to add the encryption instructions.

10. A method of protecting a computer program against pointer corruption attacks while employing a shared library, the method comprising the steps of:

in the library, responsive to a function call, exporting at least two versions of the called function, including a first version and a second version,

wherein one of the first and second versions effects pointer protection by encrypting a value of a pointer each time the value of the pointer is modified; and

decrypting the encrypted value of the pointer each time the pointer is read.