Method and apparatus for remote debugging of kernel and application software
First Claim
1. A method of debugging a remote computer, comprising:
- running a debugger on a host computer;
running an operating system on the target computer;
when debugging is required, loading a debug agent from persistent store into memory and executing the debug agent;
executing initialization code of said debug agent, wherein replacing selected OS kernel code and data that are referenced, accessed, and otherwise used in the processing of debugging traps by the OS kernel, and whereas said replaced code and data reside in or reference to said debug agent code and data images in memory;
while the target is being debugged, the debug agent intercepting and processing one or more processor debugging traps generated;
when debugging is no longer required, unloading the debug agent, wherein restoring replaced OS kernel and data to original values.
0 Assignments
0 Petitions
Accused Products
Abstract
A method and apparatus for debugging of OS kernel and applications software that does not require use of a hardware probe; can debug both user-mode programs and a significant body of the OS kernel code; allows the OS to continue servicing exceptions while debugging; leverages OS built-in device drivers for communicating devices to communicate with the host debugger; and can debug a production version of the OS kernel. When debugging is required, the running OS kernel dynamically loads a software-based debug agent on demand whereby such debug agent dynamically modifies the running production OS kernel code and data to intercept debugging traps and provide run-control. To provide debugging of loadable module, the debug agent implement techniques to intercept the OS module loading system call; set breakpoints in the loaded module initialization function; calculate the start address of the debugged module in memory; and asynchronously put the system under debug. By structuring command loop to execute in non-exception mode, and devising a process to transfer execution from the debug agent exception handler to the debug agent command loop and back, the debug agent can communicate with the host debugger using interrupt-driven input/output devices as well as allowing the system to service interrupts while under debug.
-
Citations
45 Claims
-
1. A method of debugging a remote computer, comprising:
-
running a debugger on a host computer;
running an operating system on the target computer;
when debugging is required, loading a debug agent from persistent store into memory and executing the debug agent;
executing initialization code of said debug agent, wherein replacing selected OS kernel code and data that are referenced, accessed, and otherwise used in the processing of debugging traps by the OS kernel, and whereas said replaced code and data reside in or reference to said debug agent code and data images in memory;
while the target is being debugged, the debug agent intercepting and processing one or more processor debugging traps generated;
when debugging is no longer required, unloading the debug agent, wherein restoring replaced OS kernel and data to original values. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A method of intercepting the OS loadable module loading system call, comprising:
-
a. running a debug agent on the target system;
b. saving an entry in the OS syscall table pointing to a sys_init_module function, which services the OS module loading system call;
c. replacing said entry with a pointer to a proxy sys_init_module function residing in the debug agent memory image;
responsive to system call to load the loadable module by the loading utility program, executing the steps comprising;
d. invoking said proxy sys_init_module function via said replacement;
e. responsive to determining that said module has been selected for debugging, initiating debugging of the loadable module by the said proxy sys_init_module function. - View Dependent Claims (8, 9, 10, 11)
-
-
12. A method for transferring execution flow from the debug agent exception handler to and from the debug agent command loop after the occurrence of a debugging trap, comprising:
-
a. prior to the debugging trap occurrence, the debug agent capturing the execution context at the destination within the debug agent command loop;
b. saving of debugged entity context at trap occurrence in the context saved area and invoking the debug agent trap handler;
c. the debug agent trap handler saving and replacing the contents of the context saved area with said captured context at the destination;
d. the debug agent trap handler executing the exception return code to resume system execution to the specified destination within the debug agent command loop, whereas said specified destination context is stored in the context saved area;
e. the debug agent responding to one or more access requests from the host debugger;
the debug agent, responsive to a run-control request, performing steps comprising;
f. setting a global variable to a value denoting transference of command loop to trap handler, whereas such variable is accessible to debug agent command loop and debug agent trap handler;
g. executing an instruction causing the system to enter exception mode, wherein said instruction is a selected one of;
a BREAK instruction or an illegal instruction, further wherein the BREAK code or the illegal instruction opcode denotes transference;
h. the system invoking the debug agent trap handler on entrance to exception mode;
responsive to determining that both the value of the global variable and the break code or the illegal instruction opcode denotes transference, the debug agent trap handler resuming execution to the debugged entity, comprising the steps;
i. restoring original execution context of the debugged entity to the context saved area;
j. executing the exception return code, resuming system execution to the destination at or near the point of the debugging trap occurrence in the debugged entity. - View Dependent Claims (13, 14, 15)
-
-
16. An apparatus comprising:
-
a target computer, comprising one or more processors;
a memory coupled to the processor;
a hardware bus coupling the processor and one or more peripheral devices;
one or more communicating peripheral devices coupled to the hardware bus;
an operating system running on the processor;
one or more programs, each residing in memory and executing on the processor as one or more processes or threads;
one or more device driver drivers, each loaded by the OS on demand;
a host computer, connecting to the target computer via communicating peripheral devices;
a host debugger executing on the host computer;
a debug agent, loaded by the OS on demand, residing in memory and executing on the target computer, wherein;
a. said debug agent is loaded from persistent store into memory and executed when debugging is required;
b. said debug agent initialization code replacing selected OS kernel code and data that are referenced, accessed, and otherwise used in the processing of debugging traps by the OS kernel, and whereas said replaced code and data reside in or reference to said debug agent code and data images in memory;
c. while the target is under debug, said debug agent intercepting and processing one or more processor debugging traps generated;
d. unloading the debug agent, wherein restoring replaced OS kernel and data to original values when debugging is no longer required. - View Dependent Claims (17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30)
-
-
31. A program product comprising:
-
a debugger running on a host computer;
a debug agent, when loaded from persistent store into memory and executed;
comprising the steps;
a. executing the debug agent initialization, wherein replacing selected OS kernel code and data that are referenced, accessed, and otherwise used in the processing of debugging traps by the OS kernel, and whereas said replaced code and data reside in or reference to said debug agent code and data images in memory;
b. while the target is being debugged, intercepting and processing one or more processor debugging traps generated;
c. when debugging is no longer required, unloading and restoring replaced OS kernel and data to original values. - View Dependent Claims (32, 33, 34, 35, 36)
-
-
37. A program product for intercepting the OS loadable module loading system call, comprising:
-
a. running a debug agent on the target system;
b. saving an entry in the OS syscall table pointing to a sys_init_module function, which services the OS module loading system call;
c. replacing said entry with a pointer to a proxy sys_init_module function residing in the debug agent memory image;
responsive to system call to load the loadable module by the loading utility program, executing the steps comprising;
d. invoking said proxy sys_init_module function via said replacement;
e. responsive to determining that said module has been selected for debugging, initiating debugging of the loadable module by the said proxy sys_init_module function. - View Dependent Claims (38, 39, 40, 41)
-
-
42. A program product for transferring execution flow from the debug agent exception handler to and from the debug agent command loop after the occurrence of a debugging trap, comprising:
-
a. prior to the debugging trap occurrence, the debug agent capturing the execution context at the destination within the debug agent command loop;
b. saving of system context at trap occurrence in the context saved area and invoking the debug agent trap handler;
c. the debug agent trap handler saving and replacing the contents of the context saved area with said captured context at the destination;
d. the debug agent trap handler executing the exception return code to resume system execution to the specified destination within the debug agent command loop, whereas said specified destination context is stored in the context saved area;
e. the debug agent responding to one or more access requests from the host debugger;
the debug agent, responsive to a run-control request, performing steps comprising;
f. setting a global variable to a value denoting transference of command loop to trap handler, whereas such variable is accessible to debug agent command loop and debug agent trap handler;
g. executing an instruction causing the system to enter exception mode, wherein said instruction is a selected one of;
a BREAK instruction or an illegal instruction, further wherein the BREAK code or the illegal instruction opcode denotes transference;
h. the system invoking the debug agent trap handler on entrance to exception mode;
responsive to determining that both the value of the global variable and the break code or the illegal instruction opcode denotes transference, the debug agent trap handler resuming execution to the debugged entity, comprising the steps;
i. restoring original execution context of the debugged entity to the context saved area;
j. executing the exception return code, resuming system execution to the destination at or near the point of the debugging trap occurrence in the debugged entity. - View Dependent Claims (43, 44, 45)
-
Specification