Heap buffer overflow exploitation prevention system and method

• US 7,328,323 B1
• Filed: 03/08/2004
• Issued: 02/05/2008
• Est. Priority Date: 03/08/2004
• Status: Active Grant

• Current Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Original Assignee: Symantec Corporation (NortonLifeLock Inc.)
• Inventors: Conover, Matthew
• Primary Examiner(s): Shah; Sanjiv
• Assistant Examiner(s): Ayash; Marwan
• Application Number:
  US10/796,358
• Time in Patent Office: 1,429 Days
• Field of Search: None
• US Class Current: 711/170
• CPC Class Codes: G06F 12/1441 for a range
• CPC Class Codes: G06F 21/556 involving covert channels, i.e. data leakage between processes inhibiting the analysis of circuitry or operation with measures against power attack G06F21/755

Abstract

A method includes stalling a call to a heap allocation function originating from a request by an application for a block of heap buffer, predicting a block of the heap buffer to fulfill the request, and determining if a forward link (F-link) and a backward link (B-link) of the predicted block are addresses within a heap segment associated with the predicted block. If a determination is made that the F-link or the B-link point outside the associated heap segment, e.g., have been overwritten by a heap buffer overflow attack, corrective action is taken to correct the stray F-link or B-link. After the corrective action is taken, the heap allocation function call is released and the block of heap buffer is allocated. In this manner, a heap buffer overflow attack is defeated.

Claims

1. A method comprising:

stalling a heap allocation function call to a heap allocation function originating from a request by an application for a block of heap buffer;

predicting a predicted block of said heap buffer to fulfill said request, said predicted block comprising a header portion and a data portion reserved for data; and

determining if a forward link (F-link) in a F-link field and a backward link (B-link) in a B-link field of said header portion of said predicted block are addresses within a heap segment associated with said predicted block, wherein upon a determination that said F-link and said B-link of said predicted block are not addresses within said heap segment, said method further comprising taking corrective action comprising setting said F-link and said B-link to be an address of a list head of a freelist comprising said predicted block.

2. The method of claim 1 further comprising hooking said heap allocation function.

3. The method of claim 1 further comprising determining a size of said block.

4. The method of claim 3 wherein said predicted block has said size.

5. The method of claim 3 wherein said freelist comprises a plurality of free blocks having said size, said predicted block being on said freelist.

6. The method of claim 1 further comprising determining whether a F-link of said list head of said freelist points into said heap segment.

7. The method of claim 1 further comprising determining whether a B-link of a predicted next block of said freelist points into said heap segment.

8. A method comprising:

stalling a heap deallocation function call to a heap deallocation function originating from a release by an application of a block of heap buffer, wherein said block is a deallocation block that is being deallocated to a deallocation freelist; and

determining if a forward link (F-link) in a F-link field of a header portion of a list head of said deallocation freelist and a backward link (B-link) in a B-link field of a header portion of a first block of said deallocation freelist are addresses within a heap segment associated with said deallocation freelist, said first block further comprising a data portion reserved for data, wherein upon a determination that said F-link is not an address within said heap segment, said method further comprising taking corrective action comprising setting said F-link and a B-link in a B-link field of a header portion of said list head to be an address of said list head.

9. The method of claim 8 further comprising reading said F-link and said B-link in said B-link field of said header portion of said first block.

10. The method of claim 8 further comprising hooking said heap deallocation function.

11. The method of claim 8 further comprising determining said block being released by said application.

12. The method of claim 8 wherein upon a determination that said F-link and said B-link in said B-link field of said header portion of said first block are addresses within said heap segment, said method further comprising releasing said heap deallocation function call.

13. The method of claim 8 wherein said F-link or said B-link in said B-link field of said header portion of said first block is a stray F-link or stray B-link, said method further comprising determining if said stray F-link or stray B-link is a known false positive.

14. The method of claim 8 further comprising determining if said block is to be coalesced with other free blocks.

15. The method of claim 8 wherein said block is to be coalesced with a coalesced block, said method further comprising:

determining if a F-link and a B-link of said coalesced block are addresses within a heap segment associated with said coalesced block.

16. The method of claim 15 further comprising determining if there are other blocks to be coalesced with said block.

17. A computer-program product comprising a tangible computer-readable storage medium containing computer program code comprising:

a heap buffer overflow exploitation prevention application for stalling a heap allocation function call to a heap allocation function originating from a request by an application for a block of heap buffer;

said heap buffer overflow exploitation prevention application further for predicting a predicted block of said heap buffer to fulfill said request, said predicted block comprising a header portion and a data portion reserved for data; and

said heap buffer overflow exploitation prevention application further for determining if a forward link (F-link) in a F-link field and a backward link (B-link) in a B-link field of said header portion of said predicted block are addresses within a heap segment associated with said predicted block, wherein upon a determination that said F-link and said B-link of said predicted block are not addresses within said heap segment, said heap buffer overflow exploitation prevention application further for taking corrective action comprising setting said F-link and said B-link to be an address of a list head of a freelist comprising said predicted block.