Debugging techniques in a multithreaded environment

• US 6,480,818 B1
• Filed: 11/13/1998
• Issued: 11/12/2002
• Est. Priority Date: 11/13/1998
• Status: Expired due to Term

• Current Assignee: CRAY Incorporated (Hewlett-Packard Enterprise Company)
• Original Assignee: CRAY Incorporated (Hewlett-Packard Enterprise Company)
• Inventors: Alverson, Gail A., Kaplan, Laurence S., Niehaus, Mark L., Smith, Burton J.
• Primary Examiner(s): Jones, Hugh M.
• Assistant Examiner(s): PHAN, THAI Q
• Application Number:
  US09/191,985
• Time in Patent Office: 1,460 Days
• Field of Search: 703/26, 703/27, 714/30, 712/244, 712/245, 712/224
• US Class Current: 703/26
• CPC Class Codes: G06F 11/362 Software debugging

Abstract

A system for debugging targets using various techniques, some of which are particularly useful in a multithread environment. These techniques include implementing breakpoints using out-of-line instruction emulation so that an instruction replaced with a breakpoint instruction does not need to be returned to its original location for single-step execution, executing a debugger nub for each target as part of the target task but using a nub task thread for the nub execution that is separate from the target task threads, providing immunity from breakpoints for specified threads such as the nub thread via specialized breakpoint handlers used by those threads, and virtualizing the debugger nub such that a shared root nub provides a uniform interface between the debugger and the target while specialized nubs provide differing functionality based on the type of target being debugged.

Claims

1. A method for debugging a task executing on a computer system with a processor having multiple streams for executing threads of the task, the method comprising:

executing a debugger nub of a specialized type using one thread of the task, the specialized type of the debugger nub chosen based on a type of the task, the debugger nub thread having a breakpoint handler distinct from breakpoint handlers of the other task threads;

when the debugger nub thread receives a request from a debugger to set a breakpoint at a specified location in the task, performing the request in a specialized manner determined by the specialized type by,

identifying an executable instruction at the specified location;

generating a group of instructions for emulating the identified instruction out-of-line at a location other than the specified location;

loading the generated group of instructions into the other location; and

replacing the identified instruction at the specified location with an inserted instruction that when executed will create a break;

when a thread other than the debugger nub thread encounters the inserted instruction, executing the identified instruction by,

transferring control of execution for the thread to the breakpoint handler for the thread;

notifying the debugger nub of the encounter with the inserted instruction so that the debugger nub can notify the debugger of the encounter; and

after receiving an indication from the debugger via the debugger nub to resume execution, executing the group of instructions loaded at the other location;

when the debugger nub thread encounters the inserted instruction, executing the identified instruction by,

transferring control of execution for the nub thread to the breakpoint handler for the nub thread; and

without notifying the debugger of the encounter and without receiving an indication from the debugger to resume execution, executing the group of instructions loaded at the other location; and

when the debugger nub thread receives a request from another thread to perform an action for the another thread, masking exceptions that occur during performing of the action so that execution of the debugger nub is not halted and so that the debugger nub can notify the debugger of the exceptions.

2. A computer-readable medium containing instructions for causing a computer system to debug a task executing on a computing device that has a processor with multiple streams for executing threads of the task, by:

executing a debugger nub using a thread of the task that has a breakpoint handler distinct from breakpoint handlers of the other task threads;

when the debugger nub receives a request from a debugger to set a breakpoint at a specified location in the task, replacing an executable instruction identified at the specified location with an inserted instruction that when executed will create a break;

when a thread other than the debugger nub thread encounters the inserted instruction, executing the identified instruction by,

transferring control of execution for the thread to the breakpoint handler for the thread;

notifying the debugger of the encounter with the inserted instruction; and

after receiving an indication from the debugger to resume execution, emulating the execution of the identified instruction at the specified location;

when the debugger nub thread encounters the inserted instruction, executing the identified instruction by,

transferring control of execution for the nub thread to the breakpoint handler for the nub thread; and

without notifying the debugger of the encounter and without receiving an indication from the debugger to resume execution, emulating the execution of the identified instruction at the specified location; and

when the debugger nub thread receives a request from another thread to perform an action for the another thread, masking exceptions that occur during performing of the action so that execution of the debugger nub is not halted.

3. A computer system including components to debug a task executing on multiple processor streams each executing a thread of the task, comprising:

an executing debugger nub that uses a task thread with a breakpoint handler distinct from breakpoint handlers of the other task threads, and that is capable of,

when a request is received from a debugger to set a breakpoint in the task, identifying an executable instruction corresponding to the breakpoint and setting the breakpoint such that attempted execution of the identified instruction will generate a break;

when a request is received from another thread to perform an action, masking exceptions that occur during performance of the action so that execution of the debugger nub is not halted; and

when attempting execution of the identified instruction, generating the break such that the breakpoint handler of the debugger nub thread executes the identified instruction without halting execution of the debugger nub; and

multiple executing tasks threads other than the debugger nub thread that are each capable of, when attempting execution of the identified instruction, generating the break such that the breakpoint handler for the task thread halts execution of the task thread and, after receiving an indication from the debugger, continues the execution of the task thread in order to execute the identified instruction.

4. The computer system of claim 3 including:

an executing debugger that is capable of sending requests to the debugger nub to set breakpoints in the task and of, after a break is generated by an executing task thread other than the debugger nub thread, indicating to the executing task thread to continue execution.

5. A computer-implemented method for debugging an executing target program using a debugger nub, the target program having multiple software threads each able to execute without halting execution of the other threads, the method comprising:

executing a debugger nub using a thread of the target program;

executing the target program using the other target program threads; and

under control of the executing debugger nub, repeatedly responding to an executing debugger by,

receiving a request from the executing debugger to provide indicated information about a current state of the executing target program;

in response and without support from an operating system, obtaining the requested information from the executing target program; and

sending the obtained information to the debugger.

6. The method of claim 5 wherein the obtaining of the requested information includes reading information from the program memory of the executing target program.

7. The method of claim 5 wherein the obtaining of the requested information includes reading information from the data memory of the executing target program.

8. The method of claim 5 wherein the obtaining of the requested information includes interacting with at least one of the other target program threads.

9. The method of claim 5 wherein the obtaining of the requested information is performed without support from any other executing program.

10. The method of claim 5 wherein the received request is to evaluate an indicated expression using the current state of the executing target program, and wherein the obtaining of the requested information includes evaluating the expression.

11. The method of claim 10 wherein the indicated expression is a condition for a conditional breakpoint encountered by one of the other target program threads.

12. The method of claim 5 wherein the repeated responding to the executing debugger includes:

receiving a request from the executing debugger to provide an indicated functionality related to the executing target program; and

in response and without support from the operating system, providing the requested functionality.

13. The method of claim 12 wherein the requested functionality is to halt execution of at least one of the other target program threads.

14. The method of claim 12 wherein the requested functionality is to set a watchpoint in the executing target program.

15. The method of claim 12 wherein the requested functionality is to set a breakpoint in the executing target program.

16. The method of claim 15 wherein the providing of the requested functionality includes:

identifying an executable instruction of the target program to correspond to the breakpoint;

generating a group of instructions for emulating execution of the identified instruction out-of-line;

loading the generated group of instructions into memory; and

replacing the identified instruction with an inserted instruction that when executed will create a break, so that when execution continues after the created break, execution of the identified instruction will be emulated by executing the loaded generated group of instructions.

17. The method of claim 15 wherein the breakpoint is a conditional breakpoint.

18. The method of claim 5 including:

under control of the executing debugger nub, responding to one of the other target program threads by,

receiving a request from the other target program thread to provide indicated functionality; and

in response and without support from an operating system, providing the indicated functionality.

19. The method of claim 18 wherein the requested functionality is to provide a notification to the debugger.

20. The method of claim 18 including masking exceptions during the providing of the indicated functionality.

21. The method of claim 5 wherein the executing of the debugger nub is caused by code that is part of an executable version of the target program, and wherein the code is added to the executable version of the target program during creation of the executable version as part of compilation of the target program.

22. The method of claim 5 wherein the executing of the debugger nub is performed by code that is provided to the executing target program during execution.

23. The method of claim 5 wherein multiple distinct target programs are each executing simultaneously and each have multiple executing software threads, and wherein each of the distinct target programs has a distinct debugger nub executing on one of the software threads of that target program to respond to the executing debugger.

24. The method of claim 23 wherein the multiple distinct target programs are of multiple distinct types, and wherein the target programs of each distinct type have debugger nubs of a type that is distinct from debugger nubs types of the target programs of other distinct types.

25. The method of claim 24 wherein the distinct types of debugger nubs share a common root functionality so that the debugger can communicate with debugger nubs of differing types in a common manner.

26. The method of claim 5 wherein the target program is executing on multiple distinct processors, and including executing a copy of the debugger nub on each of those distinct processors.

27. The method of claim 26 wherein at least a portion of the target program is loaded into program memory on each of the distinct processors, and including, under control of each of the debugger nub copies executing on one of the distinct processors, when a breakpoint is to be set at a specified location in the target program and when the portion of the target program loaded on that one distinct processor includes the specified location, setting the breakpoint at the specified location in the loaded program memory on that one distinct processor.

28. The method of claim 26 wherein one of the executing debugger nub copies is a master debugger nub that coordinates activities of all of the debugger nub copies.

29. The method of claim 5 wherein the debugger is executing on a remote computer.

30. The method of claim 5 wherein the debugger nub executes at a same privilege level as the other target program threads.

31. The method of claim 5 wherein the target program executes on a processor that has multiple hardware streams each able to execute at least one of the target program threads.

32. The method of claim 31 wherein the processor has multiple protection domains that are each able to execute a program, and wherein the target program executes in at least one of the protection domains.

33. A computer-readable medium whose contents cause a computing device to debug a target program having multiple software threads each able to execute without halting execution of the other threads, by:

executing a debugger nub using a thread of the target program; and

under control of the executing debugger nub,

receiving a request from an executing debugger to provide indicated information about a current state of the executing target program or to provide an indicated functionality related to the executing target program;

when the request is to provide the indicated information, responding to the request by obtaining the indicated information from the executing target program and sending the obtained information to the debugger; and

when the request is to provide the indicated functionality, responding to the request by providing the indicated functionality.

34. The computer-readable medium of claim 33 wherein the responding to the request is performed without support from another program.

35. The computer-readable medium of claim 33 wherein the computer-readable medium is a memory of a computer.

36. The computer-readable medium of claim 35 wherein the computer-readable medium is a transmission medium containing generated data that includes the contents.

37. A computing device for debugging an executing target program having multiple software threads each able to execute without halting execution of the other threads, comprising:

a target program execution component that is capable of executing the target program using multiple of the target program threads; and

a debugger nub execution component that is capable of executing the debugger nub using a thread of the target program, the executing debugger nub responding to a request from an executing debugger to provide indicated information about a current state of the executing target program by obtaining the requested information from the executing target program and by sending the obtained information to the debugger.

38. The computing device of claim 37 further comprising a debugger execution component that is capable of executing the debugger.

39. The computing device of claim 37 further comprising at least one processor that has multiple hardware streams each able to execute at least one of the target program threads.

40. The computing device of claim 37 further comprising a processor that has multiple protection domains each able to execute a program, and wherein the target program executes in at least one of the protection domains.

41. The computing device of claim 40 wherein the debugger executes in a protection domain that is distinct from the protection domains in which the target program executes.

42. A computing device for debugging an executing target program having multiple software threads each able to execute without halting execution of the other threads, comprising:

means for executing the target program using multiple of the target program threads; and

means for executing a debugger nub using a thread of the target program, the executing debugger nub capable of responding to a request from an executing debugger to provide indicated information about a current state of the executing target program by obtaining the requested information from the executing target program and by sending the obtained information to the debugger.

43. The computing device of claim 42 further comprising means for executing the debugger.

44. A computer-implemented method for debugging an executing target program using a debugger nub, the target program having multiple software threads each able to execute without halting execution of the other threads and having at least one breakpoint set in such a manner that encountering the breakpoint causes a break to occur, the method comprising:

executing a debugger nub using a thread of the target program;

executing the target program using the other target program threads;

when one of the executing other target program threads encounters one of the set breakpoints during execution of the target program and causes a break to occur, halting execution of the thread until an indication is received from the debugger nub to resume execution; and

when the executing debugger nub thread encounters one of the set breakpoints and causes a break to occur, continuing execution of the thread without receiving an external indication to continue the execution.

45. The method of claim 44 including, when one of the executing other target program threads encounters one of the set breakpoints during execution of the target program and causes a break to occur, notifying a debugger of the encounter.

46. The method of claim 45 wherein the indication that is received by the other target program thread from the debugger nub to resume execution is forwarded from the debugger to that other target program thread by the debugger nub.

47. The method of claim 45 wherein the notifying of the debugger includes notifying the debugger nub so that the debugger nub can notify the debugger.

48. The method of claim 44 wherein, when the executing debugger nub thread encounters one of the set breakpoints and causes a break to occur, the continuing of the execution of the thread is performed without notifying a debugger of the encounter.

49. The method of claim 44 wherein the target program includes multiple executable instructions that are stored in memory, wherein the setting of a breakpoint includes replacing an executable instruction stored in memory with a breakpoint instruction, and wherein the encounter of the set breakpoint includes attempting to execute one of the target program instructions that was replaced by a breakpoint instruction.

50. The method of claim 49 wherein the resuming of the execution includes executing that one replaced target program instruction without returning that one replaced target program instruction to its original memory location.

51. The method of claim 49 wherein the continuing of the execution includes executing that one replaced target program instruction without returning that one replaced target program instruction to its original memory location.

52. The method of claim 44 including:

when one of the executing other target program threads performs an operation that triggers an exception, immediately transferring control of execution for that thread to an exception handler for that thread so that the exception handler can process the exception; and

when the executing debugger nub thread performs an operation that triggers an exception, deferring transferring control of execution for the debugger nub thread to an exception handler for the thread until the debugger nub has completed a specified assignment.

53. The method of claim 52 wherein the deferring of the transferring of the control includes masking exceptions that occur during performance of the specified assignment.

54. The method of claim 52 wherein the deferring of the transferring of the control is permanent such that the specified assignment continues until execution of the debugger nub is completed.

55. The method of claim 52 including, when the executing debugger nub thread performs an operation that triggers an exception, notifying an executing debugger of the exception.

56. The method of claim 44 including:

when one of the executing other target program threads performs an operation that triggers a fatal trap, immediately transferring control of execution for that thread to a trap handler for that thread to halt execution of the thread; and

when the executing debugger nub thread performs an operation that triggers a fatal trap, blocking transferring control of execution for the debugger nub thread to a trap handler that will halt execution of the thread.

57. The method of claim 56 including, before the performance of the operation by the executing debugger nub thread that triggers the fatal trap, rebinding the trap handler for the debugger nub thread to a different trap handler that will not halt execution of the thread, and wherein the blocking of the transferring of the control includes transferring the control of execution for the debugger nub thread to the different trap handler.

58. The method of claim 56 wherein the blocking of the transferring of the control includes notifying an executing debugger of the fatal trap.

59. The method of claim 44 wherein the set breakpoint encountered by one of the executing other target program threads during the execution of the target program is a conditional breakpoint that includes a condition, and wherein the halting of the execution of the thread includes requesting the debugger nub to evaluate the condition.

60. The method of claim 59 wherein the received indication from the debugger nub to resume execution is an indication that the condition is evaluated to be false, so that conditional breakpoints with false conditions are treated as if a break did not occur.

61. The method of claim 44 wherein the set breakpoint encountered by one of the executing other target program threads during the execution of the target program is a conditional breakpoint that includes a condition, and including, before the halting:

under control of that executing other target program thread,

evaluating the condition to determine if the condition is satisfied; and

when it is determined that the condition is not satisfied, resuming the execution of that executing other target program thread without halting the execution, such that the halting of the execution of that executing other target program thread occurs only when the condition is determined to be satisfied.

62. The method of claim 44 wherein a debugger in communication with the debugger nub is executing on a remote computer.

63. The method of claim 44 wherein the target program executes on a processor that has multiple hardware streams each able to execute at least one of the target program threads.

64. The method of claim 63 wherein the processor has multiple protection domains that are each able to execute a program, and wherein the target program executes in at least one of the protection domains.

65. A computer-readable medium whose contents cause a computing device to debug a target program having multiple software threads each able to execute without halting execution of the other threads, the executing target program having at least one breakpoint set in such a manner that encountering the breakpoint causes a break to occur, by:

executing a debugger nub using a thread of the target program;

executing the target program using the other target program threads; and

when one of the executing other target program threads encounters one of the set breakpoints and causes a break to occur, suspending execution of the thread until an indication is received from the debugger nub to resume execution; and

when the executing debugger nub thread encounters one of the set breakpoints and causes a break to occur, continuing execution of the thread without suspension of the execution.

66. The computer-readable medium of claim 65 wherein, when an executing thread causes a break to occur, control of execution for that thread is transferred to a breakpoint handler for that thread, and wherein the debugger nub thread has a breakpoint handler that is distinct from the breakpoint handlers of the other target program threads such that the distinct breakpoint handler causes the continuing of the execution of the debugger nub thread without suspension.

67. A computing device for debugging an executing target program having multiple software threads each able to execute without halting execution of the other threads, the executing target program having at least one breakpoint set in such a manner that encountering the breakpoint causes a break to occur, comprising:

a debugger nub execution component that is capable of executing the debugger nub using a thread of the target program, the executing debugger nub thread such that upon encountering one of the set breakpoints that causes a break to occur, execution of the thread continues; and

a target program execution component that is capable of executing the target program using the other multiple target program threads, each of the executing other target program threads such that upon encountering one of the set breakpoints that causes a break to occur, execution of the thread is halted until an indication is received from the debugger nub to resume execution.

68. The computing device of claim 67 wherein, when an executing thread causes a break to occur, control of execution for that thread is transferred to a breakpoint handler for that thread, and wherein the debugger nub thread has a breakpoint handler that is distinct from the breakpoint handlers of the other target program threads such that the distinct breakpoint handler causes the continuing of the execution of the debugger nub thread.

69. A computer-implemented method for debugging multiple executing tasks by using debugger nubs, the tasks each having multiple software threads executing on a first computer having multiple processors and each having at least one breakpoint set in such a manner that encountering the breakpoint causes a break to occur, the processors each having multiple protection domains that are each able to execute one of the multiple tasks and each having multiple hardware streams each able to execute at least one of the software threads, the method comprising:

for each of the multiple tasks,

executing a debugger nub for the task using a thread of the task, the debugger nub being of a debugger nub type that is selected based on a type of the task;

executing the task using the other task threads;

when one of the executing other task threads of the task encounters one of the breakpoints set for the task and causes a break to occur, halting execution of the thread until an indication is received from the debugger nub for the task to resume execution; and

under control of the executing debugger nub for the task,

responding to requests from an executing debugger to provide indicated information about a current state of the executing task, the executing debugger distinct from the executing debugger nub, the responding by obtaining the requested information from the executing task in a manner specific to the debugger nub type of the debugger nub for the task;

upon encountering one of the breakpoints set for the task, continuing execution without halting; and

providing indications to the executing other task threads of the task to control execution of those threads.