ikdebug - Integrated kernel debugger
The ikdebug debugger is the integrated interactive mode of the kdebug ker-
nel debugger. It provides interactive symbolic kernel debugging without the
need of a second host system to run dbx or kdbx. Its command syntax is
derived from the Mach microkernel debugger ddb.
This debugger is included as part of the kdebug kernel debugger module. It
is configured by enabling the kdebug module in your system configuration
file. When the kernel is booted with this debugger, the following message
will be displayed during boot:
Loading vmunix symbol table ... [XXX bytes]
If the kernel is booted with the k flag, the standard kdebug behavior is
If the kernel is booted with the e flag, the kernel enters the interactive
debugger. This allows breakpoints to be set early in the bootstrap process.
The normal bootstrap sequence is resumed using the debugger continue com-
If neither flag is specified, the kernel boots normally and the interactive
debugger is available through a console escape sequence described in the
The following kernel tunable parameters are available:
The console escape string is defined by this parameter. This string
defaults to "" (null) which disables the feature. It may be overridden
by adding the following to the /etc/sysconfigtab file:
kdebug_escape = fred
As this is the value with which you invoke the debugger, the string
should be easy to remember. Avoid strings that might cause you to
accidentally invoke the debugger.
Alternatively, the escape string can be changed at runtime by running
the following command as root:
# sysconfig -r kdebug kdebug_escape=ethyl
Also, the console escape can be disabled by setting the escape sequence
to the null string using the following command:
# sysconfig -r kdebug kdebug_escape=
Using this parameter, the debugger can be configured to stop automati-
cally when a kernel panic occurs. By default this feature is disabled.
This parameter controls whether the debugger uses the kernel console
driver or console firmware callbacks to perform I/O to the console. A
value of 1 selects the kernel console driver while value of 0 selects
console firmware callbacks. The debugger automatically selects the
appropriate value depending on the system type.
This parameter controls the debugger use of the console SAVETERM func-
tion. A value of 1 causes the SAVETERM function to be called on
debugger entry while a value of 0 disables the SAVETERM function call.
This tuneable is only meaningful when a graphics console is in use and
the parameter kdebug_unix_console is set to 0, enabling console
Entering and Exiting
The debugger is entered automatically during a kernel panic, after any sys-
tem dump is generated, and before a reboot. To continue with the reboot
operation, use the continue or quit commands. The current thread will be
the one that generated the panic. You may also invoke the debugger manu-
ally. If you are logged in to the system console on a serial line, typing
the kernel tuneable kdebug_escape string will drop you into the debugger.
The debugger escape sequence is recognized at all times.
You can exit the debugger by entering the continue command, or the command
Interacting With ikdebug
When you enter the debugger (or if it is invoked automatically), ikdebug
places you in the context of the task and thread that were active just
before the debugger assumed control. Note that all system activity on the
current CPU is suspended until you enter a command to the debugger's
Once invoked, the debugger's general command set allows you to examine or
modify register and address space contents. You can also call functions,
set breakpoints, execute instructions a step at a time, and perform stack
The debugger provides the abstraction of a current location, which, in
keeping with UNIX style, is referred to as dot (.). This dot represents
the address that the next command will use if you do not provide an alter-
The examine and write commands update dot to the address of the last line
examined or the last location modified, and set next to the address of the
next location to be examined or changed. Other commands do not change dot,
and set next to be the same as dot. See the Synopsis section for an expla-
nation of the command syntax.
Address Space Issues
No matter how the debugger is invoked, you always have access to the
kernel's address space. You also have access to a single user address
space. If the system was executing in the context of a user task just
before ikdebug assumed control, that task is the current task. Otherwise,
ikdebug sets the current task to the user task that was last executed. In
either case, the address space of the current task is the one available by
You may access a non-current user task's address space by naming a thread
belonging to that task. You can use the "show" facility to determine such a
In general, the accessibility of a given virtual address depends upon
whether or not the corresponding page resides in main memory. If the page
is not resident, it is likely that you will not be able to access the con-
Multipage Output Facility
The ikdebug debugger has a feature similar to the shell command more for
managing the display of multipaged output. If an output line exceeds the
number set in the $lines variable, it displays the message --db_more-- and
waits for a response. Valid responses are as follows:
+ Press the Spacebar to display one more page
+ Press the Return key to display one more line
+ Press the Q key to abort the current command, and return to the com-
mand input mode.
Any other character displays a single line, as for the Return key.
Commands and Parameters
The general command syntax is:
command [/modifier] address [,count]
Specifying address sets dot to the address. Omitting the address uses the
default address, dot. A missing count is taken to be 1 for printing com-
mands or infinity for stack traces. The string !! repeats the previous com-
mand, and the string "." repeats from the address next with the same count
Multi-threaded debugging is supported by ikdebug. Thread-specific break-
points can be set, and the address space and registers of a non-current
thread can be examined or modified if supported by machine dependent rou-
tines. For example, the command:
break/t mach_msg_trap $task11.0
sets a breakpoint at mach_msg_trap for the first thread of task 11 listed
by a show all threads command.
In the example, $task11.0 is translated to the corresponding thread
structure's address by the variable translation mechanism described later
in this reference page. If a default target thread is set in the variable
$thread, the $task11.0 argument can be omitted. In general, if t is speci-
fied as a modifier of a command line, the specified thread (or the current
default target thread) is used for the command instead of the current
thread -- the thread that was running when ikdebug was entered. The t
modifier in a command line is not valid in evaluating expressions in a com-
mand line. If you want to get a value indirectly from a specific thread's
address space or access to its registers within an expression, you have to
specify a default target thread in advance, and to use the :t modifier
immediately after the indirect access or the register reference as follows:
set $thread $task11.0
The following are the commands accepted by ikdebug. Each command may be
specified using the shortest unambiguous prefix of its name. In addition, a
few commands may be given as a single letter, even though this is not unam-
biguous. Commands specifiable with one letter are indicated by putting the
letter in parens after the command name, as in examine(x).
break[/tuTU] <addr>[,<count>] [ <thread1...]
Enables you to set a breakpoint at addr. If count is supplied, the
debugger continues count-1 times before stopping at the breakpoint. If
the breakpoint is set, a breakpoint number is printed with #. This
number can be used in deleting the breakpoint.
t Set a breakpoint only for a specific thread. The thread is speci-
fied by the thread parameter, or the default thread is used if the
parameter is omitted.
u Set a breakpoint at a user-space address. It may be combined with a
t or T. option (plus a thread argument) to specify an address space
other than the current one. This option is never required. Without
it, supplying a user-space address will produce a warning, but will
not cause an error. However, with the option, a kernel-space
address will cause an error. This option can be used only if it is
supported by machine dependent routines.
T Set a breakpoint only for threads in a specific task. It is like
the t option except that the breakpoint is valid for all threads
which belong to the same task as the specified target thread.
U Set a breakpoint in shared user space address. It is like the u
option, except that the breakpoint is valid for all threads which
share the same address space even if the t option is specified.
The t option is used only to specify the target shared space.
Without the t option, u and U have the same meanings. The U option
is useful for setting a user space breakpoint in non-current
address space with the t option such as in an emulation library
space. This option can be used only if it is supported by machine
Note that if user text is shadowed by a normal user-space debugger,
user-space breakpoints may not work correctly.
call func [ (arg-list) ]
Call the function named func passing it the parameters in arg-list.
After the function runs, the debugger will display its return value.
A single exclamation point may be used as a shorthand for call, for
Continue execution until a breakpoint. If c is given, count instruc-
tions while executing. Some systems will also count loads and stores.
Note that, when counting, the debugger is really silently single-
stepping. This means that the c option can easily cause unexpected
behavior, by single-stepping through low-level code.
On an SMP system, switch the current CPU. This affects the identity of
the current context (task plus thread). A CPU must already be in the
debugger to become the current CPU. If the command targets a CPU on
which ikdebug is inactive, the debugger ignores the command and prints
an error message.
delete(d)[/tuTU] <addr> | #<number>
Delete the breakpoint. The target breakpoint can be specified by a
breakpoint number with #, or by addr as specified in the break command.
examine(x) [/modifier] addr[,count] [thread]
Display the addressed locations according to the formats in the modif-
ier. Multiple modifier formats display multiple locations. If no format
is specified, the last formats specified for this command is used.
Address space other than that of the current thread can be specified
with the t option in the modifier and thread parameter. The format
b examine by bytes (8 bits)
h examine by half words (16 bits)
l examine by long words (32 bits)
q examine by quad words (64 bits)
a print the location being displayed, symbolically if possible; any
symbol close enough to the location will be used
A print the location being displayed, symbolically if possible; only
procedure names will be used, and if possible the source-level file
name and line number of the definition will be output
x display in unsigned hex
o display in unsigned octal
d display in signed decimal
U display in unsigned decimal
r display in current output radix ($oradix), signed
c display low 8 bits as a character. Non-printing characters are
displayed as an octal escape code (\000).
s display the null-terminated string at the location. Non-printing
characters are displayed as octal escapes.
m display in unsigned hex with character dump at the end of each
line. The location is also displayed in hex at the beginning of
i display as an instruction
I display as an instruction including register contents
help [/lsu] <command>
Displays detailed help, a summary or usage for the specified command.
If no command is specified, all commands are displayed.
/l Display detailed help for the specified command
/s Display a command summary for the specified command
/u Display a command usage for the specified command
print[/aAxodurc] <addr1[ <addr2... ]
Print addrs according to the modifier character. Valid formats are: a
A x o d u r c. (These are the same formats as for the examine command,
except that the u format here corresponds to the u format there.) If no
modifier is specified, the last one specified to it is used. Each addr
argument can be a string, which is printed literally, with printf(3)
style escape sequences. For example,
print/x "pc = " $pc "\nra = " $ra "\n"
will print the line
pc = xxxxxx
ra = yyyyyy
search[/bhlq] <addr<value[<mask>] [,<count>]
Search memory for a value. This command might fail in interesting ways
if it does not find the searched-for value. This is because ikdebug
does not always recover from touching bad memory. The optional count
argument limits the search.
set $<variable[=] <expr>
Set the named variable or register with the value of expr. Valid vari-
able names are described below.
step(s) [/p] [,<count>]
Single step count times. If -p option is specified, print each instruc-
tion at each step. Otherwise, only print the last instruction.
Depending on machine type, it may not be possible to single-step
through some low-level code paths or user space code. On machines
with software-emulated single-stepping (i860), stepping through code
executed by interrupt handlers is not recommended.
trace[/tuT] [ <frame_addr>|<thread_addr][,<count>]
Stack trace. The /u option causes traces to include stack frames in
user space; if omitted, traces include only frames in kernel space. If
the /t option is specified, it shows the stack trace of the specified
thread or a default target thread. Otherwise, it shows the stack trace
of the current thread from the frame address specified by a parameter
or from the current frame. The count is the number of frames to be
traced. If the count is omitted, all frames are printed. The /T option
outputs a trace for all threads within a task. In this case only,
<thread_addr> may refer either to a thread or to a task. To begin (or
resume) tracing the threads of a task at a thread other than thread 0,
just name the desired thread (as in $task4.27).
If the target thread's stack is not in the main memory at that time,
the stack trace will fail. User space stack trace is valid only if
the machine dependent code supports it.
write(w)[/bhlqtu] <addr[ <thread] <expr1[ <expr2... ]
Write the expressions at succeeding locations. The write unit size can
be specified in the modifier with a letter b (byte), h (half word), l
(long word), or q (quad word) respectively. If omitted, long word is
assumed. Target address space can also be specified with /t option in
the modifier and thread parameter. If the target is a user task, you
must include the /u option in the command.
Since there is no delimiter between expressions, strange things may
happen. It is best to enclose each expression in parentheses.
xf Examine forward. It executes an examine command with the last specified
parameters, updated so that the location following the last location
previously displayed becomes the start address for the current command.
xb Examine backward. It executes an examine command with the last speci-
fied parameters, updated so that the start address of the previous com-
mand, minus the number of items displayed by that command, becomes the
start address for the current command. (This displays a "window" con-
sisting of the items immediately preceding the last group of items out-
The debugger will display a number of interesting kernel-level data struc-
tures. The commands to do so all have the form show <name>, where <name>
is a tag related to the data structure in question. For some common cases
where it is useful to inspect all instances of a given structure with a
single command, the debugger provides commands of the form:
show all <<name>>
Unless otherwise indicated, each show command simply prints the interesting
members of the structure indicated. Each member is labeled by a name that
is at least related to its C-language name.
show thread[/l] [ <addr]
Display information for the thread specified by addr. If addr is omit-
ted, information on the current thread is displayed. If no modifier is
given, the information displayed for the thread is its ID, the address
of the corresponding thread structure, and the state of the thread. The
state is output as some combination of these letters:
R Thread is runnable.
W Thread is waiting (has blocked itself); following the state output
will appear the address of the event on which it is waiting, or
zero, if no event was asserted.
S Thread is suspended.
O Thread is swapped out.
N Thread is waiting uninterruptibly; if W appears without N, the wait
F Thread has used the floating-point unit.
If /l is given, more information is displayed for the thread:
+ the thread's ID
+ the address of the thread structure
+ the state of the thread as explained above
+ the swap state of the thread, indicated by one of the letters:
U Activation is unswappable.
I Activation is swapped in.
G Activation is being swapped out.
W Activation is being swapped in.
O Activation is swapped out.
The letter will be followed by a T if the thread's task is being
swapped, otherwise a hyphen will follow.
+ the base of the thread's kernel stack
+ the thread's suspend count
+ the priority of the thread, together with its scheduling policy,
FF for first-in/first-out (no pre-emption)
RR for round-robin (pre-empt only within priority level)
TS for timesharing (degrade priority as thread executes)
+ the event, if any, on which the thread is waiting
show all threads[/ul]
Display information on all tasks and threads. In all cases, this com-
mand shows task and thread identification numbers, which can be used to
specify a task or an thread symbolically in other commands (via task
variables). The numbers are valid only in the current debugger session;
if system execution is resumed, they may change. The currently-
executing thread is distinguished from others by a # after its id
instead of :.
With no options given, this command displays the same information for
each task and thread as show task and show thread would do, when
they're given with no arguments. The /u option expands the output shown
for each task just as it does for show task. The /l option expands the
output shown for each thread just as it does for show thread or show
show all tasks[/ul]
For each task, perform show task.
Show all breakpoints currently set. The information displayed for each
+ the ID (number) of the breakpoint
+ the address space (kernel, user-space or taskn) in which the
breakpoint is set
+ the scope of the breakpoint, one of all (if it is global), taskn
(if it is specific to task id n), or taskn.m (if it is specific to
thread m of task id n)
+ the count specified when the breakpoint was established
+ the address at which the breakpoint is set
show registers[/tu] [ <thread]
Display the register set. Target thread can be specified with t option
and thread parameter. If u option is specified, it displays user regis-
ters instead of kernel or currently saved one.
The support of t and u option depends on the machine. If not sup-
ported, incorrect information will be displayed.
show runq <addr>
For each processor set defined in the system, displays each occupied
run queue (i.e., each run queue where at least one thread resides).
For each run queue, the queue priority is output, followed by the task
plus thread ID of each thread on that queue.
show task/ul [ <addr]
Display information for the task specified by <addr>, or for the
current task if <addr> not given. Information is also displayed for all
threads in the task. If no options are given, the information displayed
for the task is its ID, the address of the corresponding task struc-
ture, and the number of threads the task contains; information
displayed for each thread is the same as for show threads with no
If the /l option is given, this command shows more information for each
thread -- the same information as show threads will with /l given.
If the /u option is given, this command shows more information for the
+ the task ID
+ the address of the task structure
+ the address of the task's map (vm_map_t)
+ the total number of threads in the task
+ the number of resident (swapped-in) threads in the task
+ the task's suspend count
+ the task's priority
+ the task's swap state, indicated by one of the letters:
I Task is swapped in.
O Task is swapped out.
G Task is being swapped out.
C Task is being swapped in.
U Task is unswappable.
+ any UNIX process information available for the task
Display the current values of all debugger variables.
The debugger accesses registers and variables as <name>. Register names are
as in the show registers command. Some variables are suffixed with numbers,
and may have some modifier following a colon immediately after the variable
name. For example, register variables can have u and t modifier to indicate
user register and that of a default target thread instead of that of the
current thread, such as $eax:tu.
Built-in variables currently supported are:
Set if the console device will "autowrap" (i.e., insert a CR/LF pair)
when its output width is exceeded.
Controls automatic completion of symbol names (see below). If zero,
completion is disabled. Otherwise, controls at what point all the
current alternatives for completion are displayed. (If more than this
number of alternatives exist, they won't be displayed.)
The number of lines. It is used by "more" feature.
Addresses are printed as 'symbol'+offset unless offset is greater than
The width of the displayed line.
Tab stop width.
Task or thread structure address. The xx and yy are task and thread
identification numbers printed by a show all threads command respec-
tively. This variable is read only. The value of xx can be 1 to 10.
The default target thread. The value is used when t option is speci-
fied without explicit thread structure address parameter in command
lines or expression evaluation.
Work variable. The value of xx can be 0 to 31.
Almost all expression operators in C are supported except ~, ^, and unary
&&. However, operator precedence is decidedly different. From lowest to
highest, precedence levels are:
+ logical or (||)
+ logical and (&&&&)
+ relational ops (==, !=, <<, <<=, >>=, >>)
+ bitwise shift (<<<<, >>>>)
+ "additive" (+, -, |)
+ "multiplicative" (*, /, %, #, &&)
+ unary (+, -, !)
Special rules in ikdebug are:
name of a symbol. It is translated to the address (or value) of it.
The characters fullpoint (.) and colon (:) can be used in the identif-
If supported by an object format dependent routine:
can all be accepted as a symbol.
By default, symbols may be specified using a unique (or near-unique)
prefix. When a symbol name is being entered, the debugger will attempt
to complete it on receipt of a <<tabcharacter. Any unique substring of
the name that follows the portion so far specified will be completed
immediately. If the name is not totally specified after that comple-
tion, then the debugger will display all the symbols that match the
name so far completed, so long as the count of matching symbols does
not exceed the value of the $completion variable.
radix is determined by the first two letters: 0x: hex, 0o: octal, 0t:
decimal, otherwise, follow current radix.
.. address of the start of the last line examined. Unlike dot or next,
this is only changed by examine or write command.
' last address explicitly specified.
register name or variable. It is translated to the value of it. It may
be followed by a : and modifiers as described above.
# a binary operator which rounds up the left hand side to the next multi-
ple of right hand side.
indirection. It may be followed by a ':' and modifiers as described
Commands: dbx(1), kdbx(8)