Method and apparatus for verifying a suspect return pointer in a stack
First Claim
1. A computer implemented method for verifying a return address, the method comprising:
- a) storing the return address into a stack based on a function call;
b) generating a first hash based on a first stack frame and a second stack frame;
c) storing the first hash in a first canary location, wherein the first canary location is in the first stack frame;
d) executing at least one instruction of a routine referenced by the function call;
e) reading the first canary location to form a first suspect hash;
f) calculating a first verification hash based on the first stack frame and the second stack frame;
g) determining that the first verification hash matches the first suspect hash to form a first positive determination;
h) responsive to the first positive determination, reading a second canary location to form a second suspect hash;
i) calculating a second verification hash based on the second stack frame;
j) determining that the second verification hash matches the second suspect hash to form a second positive determination;
k) responsive to the first positive determination and the second positive determination, popping the return address off the stack; and
l) executing at least one instruction at a memory location pointed to by the return address.
1 Assignment
0 Petitions
Accused Products
Abstract
The present invention provides a computer implemented method, data processing system, and computer program product for verifying a return address. A computer stores the return address into a stack based on a function call. The computer generates a first hash based on a first stack frame and a second stack frame. The computer stores the first hash in a first canary location, wherein the first canary location is in the first stack frame. The computer executes at least one instruction of a routine referenced by the function call. The computer reads the first canary location to form a first suspect hash. The computer calculates a first verification hash based on the first stack frame and the second stack frame. The computer determines that the first verification hash matches the first suspect hash to form a first positive determination. The computer responsive to the first positive determination, the computer reads a second canary location to form a second suspect hash. The computer calculates a second verification hash based on the second stack frame. The computer determines that the second verification hash matches the second suspect hash to form a second positive determination. The computer responsive to the first positive determination and the second positive determination, the computer pops the return address off the stack. The computer executes at least one instruction at a memory location pointed to by the return address.
-
Citations
20 Claims
-
1. A computer implemented method for verifying a return address, the method comprising:
-
a) storing the return address into a stack based on a function call; b) generating a first hash based on a first stack frame and a second stack frame; c) storing the first hash in a first canary location, wherein the first canary location is in the first stack frame; d) executing at least one instruction of a routine referenced by the function call; e) reading the first canary location to form a first suspect hash; f) calculating a first verification hash based on the first stack frame and the second stack frame; g) determining that the first verification hash matches the first suspect hash to form a first positive determination; h) responsive to the first positive determination, reading a second canary location to form a second suspect hash; i) calculating a second verification hash based on the second stack frame; j) determining that the second verification hash matches the second suspect hash to form a second positive determination; k) responsive to the first positive determination and the second positive determination, popping the return address off the stack; and l) executing at least one instruction at a memory location pointed to by the return address. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A data processing system comprising:
-
a bus; a storage device connected to the bus, wherein computer usable code is located in the storage device; a communication unit connected to the bus; a processing unit connected to the bus, wherein the processing unit executes the computer usable code for verifying a return address, the processing unit further executes the computer usable code to store the return address into a stack based on a function call;
generate a first hash based on a first stack frame and a second stack frame;
store the first hash in a first canary location, wherein the first canary location is in the first stack frame;
execute at least one instruction of a routine referenced by the function call;
read the first canary location to form a first suspect hash;
calculate a first verification hash based on the first stack frame and the second stack frame;
determine that the first verification hash matches the first suspect hash to form a first positive determination;
responsive to the first positive determination, read a second canary location to form a second suspect hash;
calculate a second verification hash based on the second stack frame;
determine that the second verification hash matches the second suspect hash to form a second positive determination;
responsive to the first positive determination and the second positive determination, pop the return address off the stack; and
execute at least one instruction at a memory location pointed to by the return address. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. A computer program product the computer program product comprising:
a computer-readable tangible storage device having computer usable program code for verifying a return address, said computer program product including; computer usable program code for storing the return address into a stack based on a function call; computer usable program code for generating a first hash based on a first stack frame and a second stack frame; computer usable program code for storing the first hash in a first canary location, wherein the first canary location is in the first stack frame; computer usable program code for executing at least one instruction of a routine referenced by the function call; computer usable program code for reading the first canary location to form a first suspect hash; computer usable program code for calculating a first verification hash based on the first stack frame and the second stack frame; computer usable program code for determining that the first verification hash matches the first suspect hash to form a first positive determination; computer usable program code for responsive to the first positive determination, reading a second canary location to form a second suspect hash; computer usable program code for calculating a second verification hash based on the second stack frame; computer usable program code for determining that the second verification hash matches the second suspect hash to form a second positive determination; computer usable program code for responsive to the first positive determination and the second positive determination, popping the return address off the stack; and computer usable program code for executing at least one instruction at a memory location pointed to by the return address. - View Dependent Claims (16, 17, 18, 19, 20)
Specification