| 1 |
// Copyright (C) IBM Corp. 2007 |
| 2 |
// |
| 3 |
// This file is part of systemtap, and is free software. You can |
| 4 |
// redistribute it and/or modify it under the terms of the GNU General |
| 5 |
// Public License (GPL); either version 2, or (at your option) any |
| 6 |
// later version. |
| 7 |
|
| 8 |
TAPSET DEVELOPER'S GUIDE |
| 9 |
------------------------ |
| 10 |
|
| 11 |
Tapsets are scripts that encapsulate knowledge about a kernel subsystem |
| 12 |
into pre-written probes and functions that can be used by other scripts. |
| 13 |
Tapsets are analogous to libraries for C programs. They hide the |
| 14 |
underlying details of a kernel area while exposing the key information |
| 15 |
needed to manage and monitor that aspect of the kernel. They are typically |
| 16 |
developed by kernel subject-matter experts. |
| 17 |
|
| 18 |
This document assumes you are already familiar with SystemTap and the |
| 19 |
basics of writing a SystemTap script. The REFERENCE MATERIAL section |
| 20 |
below lists other sources of SystemTap information. At a minimum, you |
| 21 |
should study the SystemTap Tutorial, the HACKING file and some of the |
| 22 |
existing tapsets before attempting to write a tapset yourself. |
| 23 |
|
| 24 |
|
| 25 |
WHAT SHOULD A TAPSET CONTAIN? |
| 26 |
|
| 27 |
A tapset should expose the high-level data and state transitions of a |
| 28 |
subsystem. Assume the audience knows little to nothing about the |
| 29 |
subsystem's low-level details and probably doesn't care. Users who need |
| 30 |
low-level data typically bypass the tapsets and write custom scripts |
| 31 |
targeted to a specific problem. |
| 32 |
|
| 33 |
The first step is to create a simple model of your subject area. For |
| 34 |
example, a model of the process subsystem might include the following: |
| 35 |
|
| 36 |
Key data: |
| 37 |
|
| 38 |
* process ID |
| 39 |
* parent process ID |
| 40 |
* process group ID |
| 41 |
|
| 42 |
State transitions: |
| 43 |
|
| 44 |
* forked |
| 45 |
* execed |
| 46 |
* running |
| 47 |
* stopped |
| 48 |
* terminated |
| 49 |
|
| 50 |
NOTE: This is a simple example and not meant to be a complete list. |
| 51 |
|
| 52 |
Use your subsystem expertise to find probe points (function entries and |
| 53 |
exits) that expose the elements of the model, then define probe aliases |
| 54 |
for those points. Be aware that some state transitions can occur in more |
| 55 |
than one place. In those cases, an alias can place a probe in multiple |
| 56 |
locations. |
| 57 |
|
| 58 |
For example, process execs can occur in either the do_execve() or the |
| 59 |
compat_do_execve() functions. The following alias inserts probes at the |
| 60 |
beginning of those functions: |
| 61 |
|
| 62 |
probe kprocess.exec = kernel.function("do_execve"), |
| 63 |
kernel.function("compat_do_execve") { |
| 64 |
< probe body > |
| 65 |
} |
| 66 |
|
| 67 |
Try to place probes on stable interfaces whenever possible (i.e., functions |
| 68 |
that are unlikely to change at the interface level). This makes it less |
| 69 |
likely that the tapset will break due to changes in the kernel. Where |
| 70 |
kernel version or architecture dependencies are unavoidable, use |
| 71 |
preprocessor conditionals (see the stap(1) man page for details). |
| 72 |
|
| 73 |
Fill in the probe bodies with the key data available at the probe points. |
| 74 |
Function entry probes can access the entry parameters specified to |
| 75 |
the function, while exit probes can access the entry parameters and the |
| 76 |
return value. Convert the data into meaningful forms where appropriate |
| 77 |
(e.g., bytes to kilobytes, state values to strings, etc). You may need to |
| 78 |
use auxiliary functions to access or convert some of the data. Auxiliary |
| 79 |
functions often use embedded C to do things that cannot be done in the |
| 80 |
SystemTap language, like access structure fields in some contexts, follow |
| 81 |
linked lists, etc. You can use auxiliary functions defined in other tapsets |
| 82 |
or write your own. |
| 83 |
|
| 84 |
Take care when target variables may have invalid, unavailable, or |
| 85 |
esoteric values, or are passed via pointers. The a $pointer->field |
| 86 |
dereference can fail, so consider wrapping a try { } catch { } around |
| 87 |
it. For a char* being converted to a string, use {kernel,user}_string |
| 88 |
only if the pointer is guaranteed valid. Use {kernel,user}_string2($ptr,"") |
| 89 |
if the pointer might be NULL but should otherwise be good. Use |
| 90 |
{kernel,user}_string_quoted($ptr) to handle any possible pointer. |
| 91 |
|
| 92 |
In the following example, copy_process() returns a pointer to the |
| 93 |
task_struct for the new process. Note that the process ID of the new |
| 94 |
process is retrieved by calling task_pid() and passing it the task_struct |
| 95 |
pointer. In this case, the auxiliary function is an embedded C function |
| 96 |
that's defined in the task tapset (task.stp). |
| 97 |
|
| 98 |
probe kprocess.create = kernel.function("copy_process").return { |
| 99 |
task = $return |
| 100 |
new_pid = task_pid(task) |
| 101 |
} |
| 102 |
|
| 103 |
Avoid the temptation to write probes for every function. Most SystemTap |
| 104 |
users won't need or understand them. Keep your tapset simple and high- |
| 105 |
level. |
| 106 |
|
| 107 |
|
| 108 |
ELEMENTS OF A TAPSET |
| 109 |
|
| 110 |
The following sections describe the most important aspects of writing a |
| 111 |
tapset and contributing it to the project. If you're writing a tapset for |
| 112 |
personal use, most of these sections can be ignored, with the exception |
| 113 |
of "Tapset files", "Namespace" and "Embedded C & Safety." |
| 114 |
|
| 115 |
Tapset files |
| 116 |
------------ |
| 117 |
Tapset files are stored in src/tapset in the SystemTap GIT directory. |
| 118 |
Most are kept at that level. If you have code that only works on a specific |
| 119 |
architecture or kernel-version, you may choose to put that in the |
| 120 |
corresponding subdirectories. |
| 121 |
|
| 122 |
Installed tapsets are located in /usr/share/systemtap/tapset/ |
| 123 |
or /usr/local/share/systemtap/tapset. |
| 124 |
|
| 125 |
Personal tapsets can be stored anywhere, but you must use |
| 126 |
"-I <personal_dir> to specify their location when invoking the stap |
| 127 |
command. |
| 128 |
|
| 129 |
Namespace |
| 130 |
--------- |
| 131 |
Probe alias names should take the form <tapset_name>.<probe_name>. For |
| 132 |
example, the probe for sending a signal could be named "signal.send". |
| 133 |
|
| 134 |
Global symbol names (probes, functions and variables) should be unique |
| 135 |
across all tapsets. This helps avoid namespace collisions in scripts |
| 136 |
that use multiple tapsets. To ensure this, use tapset-specific prefixes |
| 137 |
in your global symbols. |
| 138 |
|
| 139 |
Internal symbol names should be prefixed with "_". |
| 140 |
|
| 141 |
Comments |
| 142 |
-------- |
| 143 |
All probes and functions should include comment blocks that describe |
| 144 |
their purpose, the data they provide and the context in which they run |
| 145 |
(e.g., interrupt, process, etc.). Also use comments in areas where your |
| 146 |
intent may not be clear from reading the code. The comments preceding |
| 147 |
the functions and probes should be written in the format described in |
| 148 |
the Documentation section to allow automatic generation of the tapset |
| 149 |
documentation. |
| 150 |
|
| 151 |
Documentation |
| 152 |
------------- |
| 153 |
SystemTap now documents the tapset functions and probe points in a |
| 154 |
manner similar to the Linux kernel. The tapset files have comments |
| 155 |
before each function and probe point that are processed by a modified |
| 156 |
kernel-doc script and pulled into an XML document. HTML, PDF, and man |
| 157 |
pages are generated from that document during the build process. |
| 158 |
|
| 159 |
For systemtap functions the structure of the documentation comment is: |
| 160 |
|
| 161 |
/** |
| 162 |
* sfunction function_name(:)? (- short description)? |
| 163 |
(* @parameter(space)*: (description of function parameter x)?)* |
| 164 |
(* a blank line)? |
| 165 |
* (Description:)? (Description of function)? |
| 166 |
* (section header: (section description)? )* |
| 167 |
(*)?*/ |
| 168 |
|
| 169 |
For sytemtap probes the structure of the documentation comment is the |
| 170 |
following: |
| 171 |
|
| 172 |
/** |
| 173 |
* probe probe_name(:)? (- short description)? |
| 174 |
(* @variable(space)*: (description of probe variable x)?)* |
| 175 |
(* a blank line)? |
| 176 |
* (Description:)? (Description of probe)? |
| 177 |
* (section header: (section description)? )* |
| 178 |
(*)?*/ |
| 179 |
|
| 180 |
The document extraction keys off the "/**" at the start of the |
| 181 |
comment; normal comments starting with "/*" are ignored. The next |
| 182 |
line of the comment describes whether the following code is for a |
| 183 |
SystemTap function or a probe and title descript. Following lines that |
| 184 |
have an '@' preceding the first word describe the parameters or |
| 185 |
variables available in the function or probe. There can be zero or |
| 186 |
more "section headers" after the variables or parameters. After each |
| 187 |
section there can be multiple lines of text. The comment is closed |
| 188 |
with "*/". |
| 189 |
|
| 190 |
The xml tapsets.tmpl in src/doc/SystemTap_Tapset_Reference makes use |
| 191 |
of those special comments. This xml file has place holders to extract |
| 192 |
the documentation from the tapsets like the following for |
| 193 |
ioscheduler.stp: |
| 194 |
|
| 195 |
!Itapset/ioscheduler.stp |
| 196 |
|
| 197 |
When a new tapset file is created you will need make a similar addition |
| 198 |
to tapsets.tmpl to pull the documentation in from the new tapset file. |
| 199 |
If configured to build reference documentation (--enable-refdocs) and |
| 200 |
the needed software is available, the build process will automatic |
| 201 |
generated PDF, HTML, and man pages. |
| 202 |
|
| 203 |
The creation of new man pages for the tapsets and functions is |
| 204 |
depricated and all the documentation should be migrated over to the |
| 205 |
tapsets.tmpl. |
| 206 |
|
| 207 |
Config & Makefiles |
| 208 |
------------------ |
| 209 |
Add your tapset man page to the AC_CONFIG_FILES line in |
| 210 |
src/configure.ac, then regenerate the src/configure script by running |
| 211 |
autoconf. |
| 212 |
|
| 213 |
Add your tapset man page to dist_man_MANS line in src/Makefile.am, then |
| 214 |
regenerate src/Makefile.in by running automake. |
| 215 |
|
| 216 |
Update other Makefiles as necessary. |
| 217 |
|
| 218 |
Test cases |
| 219 |
---------- |
| 220 |
All tapsets should be accompanied by test scripts. The tests are kept |
| 221 |
in src/testsuite in GIT and based on dejagnu. You must have dejagnu and |
| 222 |
expect installed on your system to run the tests. |
| 223 |
|
| 224 |
Your tests should validate that: |
| 225 |
|
| 226 |
- the tapset can be parsed and built |
| 227 |
- all probes and functions work as expected |
| 228 |
- all potential errors are handled appropriately |
| 229 |
|
| 230 |
See the "test suites" section of the HACKING file and the existing tests |
| 231 |
for details. |
| 232 |
|
| 233 |
Example Scripts |
| 234 |
--------------- |
| 235 |
Provide at least one example script that uses the probe aliases and |
| 236 |
functions in your tapset. This serves two purposes. First, it shows |
| 237 |
script writers how you envisioned the tapset being used. Second, and |
| 238 |
most important, it validates that the tapset can actually be used for |
| 239 |
something useful. If you can't write a script that uses the tapset in a |
| 240 |
meaningful way, perhaps you should rethink what the tapset provides. |
| 241 |
|
| 242 |
Example scripts are stored in testsuite/systemtap.examples/ in GIT. |
| 243 |
|
| 244 |
Embedded C & Safety |
| 245 |
------------------- |
| 246 |
As mentioned previously, you can use embedded C (raw C code) to do |
| 247 |
things not supported by the SystemTap language. Please do this |
| 248 |
carefully and sparingly, as embedded C bypasses all the safety |
| 249 |
features built into SystemTap. There are a few style conventions that |
| 250 |
make writing robust embedded C code easier: |
| 251 |
|
| 252 |
- Be especially careful when dereferencing pointers. Use the kread() |
| 253 |
macro to dereference any pointers that could potentially be invalid. |
| 254 |
If you're not sure, err on the side of caution. The cost of using |
| 255 |
kread() is small compared to the cost of your tapset inadvertently |
| 256 |
crashing a system! |
| 257 |
|
| 258 |
- Use the macros STAP_RETVALUE and STAP_ARG_* to access the function's |
| 259 |
arguments and return slot. This ensures that the function will not |
| 260 |
break if the internal naming scheme for local variables changes |
| 261 |
between SystemTap versions. |
| 262 |
|
| 263 |
- It is absolutely necessary to rigorously test embedded-C functions |
| 264 |
in the testsuite, particularly to ensure that they fail gracefully |
| 265 |
in unexpected situations instead of causing a kernel panic. |
| 266 |
|
| 267 |
Add the string |
| 268 |
/* pure */ |
| 269 |
into the body of the embedded-C function if it has no side-effects |
| 270 |
such as changing external state, so that systemtap could elide |
| 271 |
(optimize away) a call to the function if its results are unused. |
| 272 |
|
| 273 |
Add the string |
| 274 |
/* unprivileged */ |
| 275 |
into the body of the embedded-C function, only if it is safe for use |
| 276 |
by unprivileged users. In general, this requires the function to be |
| 277 |
absolutely robust with respect to its inputs, and expose/modify no |
| 278 |
information except that belonging to the user's own processes. |
| 279 |
(The assert_is_myproc() macro may help enforce this.) Roughly |
| 280 |
speaking, it should only perform operations that the same user |
| 281 |
could already do from ordinary userspace interfaces. |
| 282 |
|
| 283 |
Add the string |
| 284 |
/* myproc-unprivileged */ |
| 285 |
into the body of the embedded-C function, to mark it as /* unprivileged */ |
| 286 |
(see above) and to have the systemtap translator automatically add a call to |
| 287 |
assert_is_myproc() to the beginning of the function. |
| 288 |
|
| 289 |
Add the string |
| 290 |
/* unmodified-fnargs */ |
| 291 |
into the body of the embedded-C function to inform the translator |
| 292 |
that any string inputs and return values may be passed by reference |
| 293 |
rather than copy. This enables optimization. |
| 294 |
|
| 295 |
Add the string |
| 296 |
/* unmangled */ |
| 297 |
into a legacy (pre-SystemTap 1.8) embedded-C function that needs to |
| 298 |
use the old argument / return slot access syntax (namely, THIS->foo |
| 299 |
and THIS->__retvalue). This is useful for migrating tapsets to work |
| 300 |
with the new syntax. |
| 301 |
|
| 302 |
|
| 303 |
Review & Submission |
| 304 |
------------------- |
| 305 |
All new tapsets and major changes should be reviewed "early and often" |
| 306 |
by the SystemTap community. You can sign up for the systemtap mailing |
| 307 |
list at http://sourceware.org/systemtap/getinvolved.html. The |
| 308 |
mailing list archive is found at http://sourceware.org/ml/systemtap/. |
| 309 |
The systemtap-cvs mailing list archive is at |
| 310 |
http://sourceware.org/ml/systemtap-cvs/. |
| 311 |
|
| 312 |
You can request GIT write access at |
| 313 |
http://sourceware.org/cgi-bin/pdw/ps_form.cgi. |
| 314 |
|
| 315 |
|
| 316 |
REFERENCE MATERIAL |
| 317 |
|
| 318 |
The following documents, web sites and mailing lists will familiarize |
| 319 |
you with SystemTap: |
| 320 |
|
| 321 |
- SystemTap Tutorial. A good introduction to SystemTap. |
| 322 |
(html format: http://sourceware.org/systemtap/tutorial/, |
| 323 |
PDF format: http://sourceware.org/systemtap/tutorial.pdf) |
| 324 |
|
| 325 |
- SystemTap project home page |
| 326 |
(http://sourceware.org/systemtap/index.html) |
| 327 |
|
| 328 |
- SystemTap mailing lists, IRC channels and GIT instructions |
| 329 |
(http://sourceware.org/systemtap/getinvolved.html) |
| 330 |
|
| 331 |
- GIT repository |
| 332 |
(http://sourceware.org/git/?p=systemtap.git;a=summary) |
| 333 |
|
| 334 |
- HACKING file in the source directory. This file outlines what's |
| 335 |
expected of project contributors. |
| 336 |
|
| 337 |
- SystemTap Wiki. Contains papers and presentations, setup instructions |
| 338 |
for various distributions, and a growing set of example scripts. |
| 339 |
(http://sourceware.org/systemtap/wiki) |
| 340 |
|
| 341 |
- Existing tapsets |
| 342 |
|
| 343 |
- SystemTap Language Reference. |
| 344 |
(html format: http://sourceware.org/systemtap/langref/, |
| 345 |
PDF format: http://sourceware.org/systemtap/langref.pdf) |
| 346 |
|
| 347 |
- SystemTap Man Pages (use "man -k stap" to print a list) |