Using GDB to examine processes or core files resulting from binaries installed from RPMs presents additional challenges because the binaries usually have the symbol information stripped out. In addition, it is useful to be able to capture core files from an operational system and examine them on another system.

Of course, many of the details of these processes will vary from system to system, but the overall structure of the process should as it is described here.

Debugging information for processes or core files

In most RPMs, the symbol table information has been stripped from the executables and libraries and saved in seperate debugging information files. The debugging information files have names like

sipauthproxy.debug

. When GDB is examining a process or core file, it automatically uses the debugging information file corresponding to the executable.

To install the debugging information files, install the

debug

or

debuginfo

RPMs. These have names like

sipxacd-debug-3.6.3-008212.i386.rpm

, and can be obtained from where you obtain the sipX RPMs.

The debugging information files are installed into directories below

/usr/lib/debug

corresponding to the install location of the binaries and libraries. E.g., the debugging information file for

/usr/bin/sipregistrar

is

/usr/lib/debug/usr/bin/sipregistrar.debug

.

To get GDB to use the debugging information files, you have to tell it the root of the debugging information files:

set debug-file-directory /usr/lib/debug

Capturing core files for later debugging

It is often desirable to save a core file from an operational system to be examined with GDB later on another system. This process works much better than rumor has it, but you have to set GDB up correctly.

To capture the core file, the corresponding binary, and the needed dynamic libraries, use

sipx-snapshot

:

sipx-snapshot --core <core-file-name>

The snapshot file will include the specified core file, its binary, and the dynamic libraries.

Unpack the snapshot file starting at a directory $ROOT. Then use commands like the following to examine the core file:

$ gdb
GNU gdb Red Hat Linux (6.5-15.fc6rh)
(gdb) set solib-absolute-prefix $ROOT
(gdb) file $ROOT/usr/bin/sipregistrar
(gdb) core $ROOT/var/log/sipxpbx/core.1234

However, this won't give GDB access to the debugging information that corresponds to the binary.

Debugging information for core files from other systems

In order to give GDB access to the debugging information to match a captured core file, the appropriate debugging information files need to be installed on the debugging system, which will run GDB.

One method is to install the

debug

or

debuginfo

RPMs on the debugging system. In that case, the root of the debugging files is

/usr/lib/debug

.

A second method is to install or unpack (using

rpm2cpio xxx.rpm | cpio -i --make-directories

) the RPMs into some directory, in which case, the root of the debugging files is

usr/lib/debug

relative to the root of the install or unpack.

In either of these cases, remember that the RPMs need to be the ones corresponding to what is installed on the operational system, not RPMs for the debugging system.

A third method, if the debugging files are already installed on the operational system, is to add

--file /usr/lib/debug

to the invocation of

sipx-snapshot

:

sipx-snapshot --core <core-file-name> --file /usr/lib/debug

That option includes all the contents of

/usr/lib/debug

into the snapshot file, and once it is unpacked, the root of the debugging files is

$ROOT/usr/lib/debug

.

Once the debugging files have been installed, symbolic links need to be added to the unpacked snapshot tree to allow GDB to find the debugging information files. Calling the root of the debugging files $DEBUG, execute:

ln -s $DEBUG/usr/bin $ROOT/usr/bin/.debug
ln -s $DEBUG/usr/lib $ROOT/usr/lib/.debug

With these links installed, GDB does not need a

set debug-file-directory

command to locate the debugging information files, but rather uses the links.