Using SystemTap
short intro, contents of chapter
Using SystemTap
This chapter documents how to install SystemTap in the system and explains how to use the stap utility to run SystemTap scripts.
Running SystemTap Scripts
- basic commands (e.g. stap), useful options per command (e.g. stap -vv), tool references (man pages, related kernel-doc), references within book (i.e. errors chapter)
- running systemtap scripts
Usagerunning SystemTap scriptsrunning SystemTap scriptsUsageUsagestapstapUsageUsagestaprunstaprunUsage
SystemTap is distributed with a number of command line tools that allow you to monitor the activities of the system. The stap command reads probing instructions from a SystemTap script, translates these instructions into C code, builds a kernel module, and loads it into the running Linux kernel. The staprun command runs SystemTap instrumentation, that is, a kernel module built from SystemTap scripts during a cross-instrumentation.
SystemTap scripts, how to runRunning stap and staprun requires elevated privileges to the system. Because not all users can be granted root access just to run SystemTap, you can allow a non-privileged user to run SystemTap instrumentation on their machine by adding them to one of the following user groups:stapdevUsagestapdevstapdevUsage
Members of this group can use the stap command to run SystemTap scripts, or staprun to run SystemTap instrumentation modules.
Running the stap command involves compiling SystemTap scripts into kernel modules and
loading them into the kernel. This operation requires elevated privileges to the system, which are granted
to stapdev members. Unfortunately, such privileges also grant effective root
access to stapdev members. As a consequence, only grant
stapdev group membership to users whom you can trust with root access.
stapusrUsagestapusrstapusrUsage
Members of this group can only use the staprun command to run SystemTap instrumentation modules. In addition, they can only run modules from the /lib/modules/kernel_version/systemtap/ directory. Note that this directory must be owned only by the root user, and must only be writable by the root user.
The stap command reads a SystemTap script either from a file, or from standard input. To tell stap to read a SystemTap script from a file, specify the file name on the command line:
stap file_nameUsagestandard input, running scripts fromstandard input, running scripts fromUsagerunning scripts from standard input
To instruct stap to read a SystemTap script from standard input, use the switch instead of the file name. Note that any command-line options you wish to use must be inserted before the switch. For example, to make the output of the stap command more verbose, type:
echo "probe timer.s(1) {exit()}" | stap -v - Below is a list of commonly used stap options: Usageoptions, stapoptions, stapUsagestap options-vMakes the output of the SystemTap session more
verbose. You can repeat this option multiple times to provide more details on the script's execution,
for example:stap -vvv script.stpThis option is particularly useful if you encounter any errors in running the script.
For more information about common SystemTap script errors, refer to .-o file_nameSends the standard output to a file named file_name.-S size,countLimits the maximum size of output files to size megabytes and
the maximum number of stored files to
count. This option implements logrotate operations for SystemTap
and the resulting file names have a sequence number suffix.-x process_idSets the SystemTap handler function target() to the specified process ID. For more information about target(), refer to .-c 'command'Sets the SystemTap handler function
target() to the specified
command and runs the SystemTap instrumentation for the duration
of this command.
For more information about
target(), refer to .-e 'script'Uses script rather than a file as input for the SystemTap translator.-FUses SystemTap's flight recorder mode and makes the script a
background process. For more information about flight
recorder mode, refer to .any other useful options worth noting here for beginners?For more information about the stap command, refer to the stap1 man page. For more information about the staprun command and cross-instrumentation, refer to or the staprun8 man page.SystemTap Flight Recorder Modeflight recorder mode
SystemTap's flight recorder mode allows you to run a SystemTap script
for long periods of time and just focus on recent output. The flight
recorder mode limits the amount of output
generated.
There are two variations of the flight recorder mode:
in-memory and file mode.
In both cases, the SystemTap script runs as a
background process.
In-memory Flight Recorderflight recorder modein-memory mode
When flight recorder mode is used without a
file name, SystemTap uses a buffer in kernel memory to store the output of the
script. Once the SystemTap instrumentation module is loaded and the probes start
running, the instrumentation detaches and is put in the background. When
the interesting event occurs, you can reattach to the instrumentation to see
the recent output in the memory buffer and any continuing output.
To run a SystemTap script by using the flight recorder in-memory mode, run the stap command with the command line option:
stap -F iotime.stp
Once the script starts, stap prints a message similar to
the following to provide you with the command to reconnect to the running
script:
Disconnecting from systemtap module.
To reconnect, type "staprun -A stap_5dd0073edcb1f13f7565d8c343063e68_19556"
When the interesting event occurs, run the following command to connect
to the currently running script, output the recent data in the memory buffer,
and get continuing output:
staprun -A stap_5dd0073edcb1f13f7565d8c343063e68_19556
By default, the kernel buffer is 1MB in size. You can increase this value by
using the option with the size in megabytes (rounded up to the
next power over 2) for the buffer. For example, on the
SystemTap command line would specify 2MB for the buffer.
File Flight Recorderflight recorder modefile mode
The flight recorder mode can also store data to files. You can control the number and size of
the files kept by using the option followed by two
numerical arguments separated by a comma: the first argument is the maximum size
in megabytes for the each output file, the second argument is the number of
recent files to keep. To specify the file name, use the
option followed by the name. SystemTap automatically adds a number suffix to the file name
to indicate the order of the files.
The following command starts SystemTap in file
flight recorder mode with the output going to files named
/tmp/pfaults.log.[0-9]+, each
file 1MB or smaller, and keeping latest two files:
stap -F -o /tmp/pfaults.log -S 1,2 pfaults.stp
The command prints the process ID to standard output. Sending a SIGTERM to
the process terminates the SystemTap script and stops the data collection. For
example, if the previous command listed 7590 as the process ID, the following
command would stop the SystemTap script:
kill -s SIGTERM 7590
In this example, only the most recent two files generated by the script are
kept: SystemTap automatically removes older files. As a result, the
ls -sh /tmp/pfaults.log.* command lists two files:
1020K /tmp/pfaults.log.5 44K /tmp/pfaults.log.6
To examine the latest data, read the file with the highest number, in this case
/tmp/pfaults.log.6.