Single instruction method of seizing control of program execution flow in a multiprocessor computer system

• US 6,014,740 A
• Filed: 04/11/1997
• Issued: 01/11/2000
• Est. Priority Date: 04/11/1997
• Status: Expired due to Term

• Current Assignee: BMC Software Incorporated (KKR & Co., Inc.)
• Original Assignee: BMC Software Incorporated (KKR & Co., Inc.)
• Inventors: Shearer, Bennie L. Jr.
• Primary Examiner(s): Lim, Krisna
• Application Number:
  US08/834,025
• Time in Patent Office: 1,005 Days
• Field of Search: 712/233, 712/210
• US Class Current: 712/233
• CPC Class Codes: G06F 8/66 of program code stored in read-only memory [ROM]
• CPC Class Codes: G06F 9/32 Address formation of the next instruction, e.g. by incrementing the instruction counter G06F9/38 takes precedence
• CPC Class Codes: G06F 9/328 for runtime instruction patching
• CPC Class Codes: G06F 9/4486 Formation of subprogram jump address

Abstract

A single-instruction method of diverting or "hooking" the operation of software entails setting up a work area of code at an address within the range of a relative branching instruction type defined in the operating-system architecture. Hooking is accomplished by patching the software in question to insert a single relative-branch instruction. That instruction causes program execution to be diverted to the work area. In one embodiment, the work area is an area of memory that normally is used only for system initialization and thus would not otherwise be executed by the processor system after system initialization is complete.

Claims

1. A method of hooking an operating system in a computer system, the operating system including program code, said method comprising:

(a) preparing an area within an address range of a relative branch instruction supported by the computer system by storing in said area an instruction or instructions to be performed or data to be used after hooking; and

(b) inserting into the operating system program code a relative branch instruction that specifies said prepared area, wherein, when executed, said relative branch instruction will direct said operating system to perform the instruction or instructions stored in said prepared area.

2. The method of claim 1, wherein the computer system is a multi-processor system.

3. The method of claim 1, wherein the work area is within an address range that is used by the operating system during initialization but is not used by the operating system thereafter.

7. A program storage device readable by a computer and encoding instructions executable by the computer for performing the operations recited in a specified one of claims 1-6.

4. A method of dynamically altering a program of instructions, referred to as code, being executed by a computer that supports a relative branch instruction, the method comprising:

(a) providing new code within a permissible address range for relative branch instructions by storing in said range an instruction or instructions to be performed or data to be used after hooking; and

(b) inserting into the code a relative branch instruction directing the computer system to begin executing the new code, wherein, when executed, said relative branch instruction will direct said operating system to perform the instruction or instructions stored in said prepared area.

5. The method of claim 4, wherein the computer system is a multi-processor system.

6. The method of claim 4, wherein the new code is provided within an address range that is used by the computer system during initialization but not thereafter.