| 1 |
= Trouble-Shooting GNU APL |
| 2 |
// vim: et:ts=4:sw=4:tw=78 |
| 3 |
|
| 4 |
:Author: Dr. Jürgen Sauermann, GNU APL |
| 5 |
|
| 6 |
:page-width: 120em |
| 7 |
:toc: |
| 8 |
:toclevels: 4 |
| 9 |
:iconsdir: /usr/share/asciidoc/icons |
| 10 |
:icons: |
| 11 |
|
| 12 |
== Abstract |
| 13 |
|
| 14 |
This document contains hints for *GNU APL* users who would like to report |
| 15 |
*GNU APL* problems and who want to make the life of the *GNU APL* maintainer(s) |
| 16 |
a little easier by localizing the root cause of a problem. |
| 17 |
|
| 18 |
== Introduction |
| 19 |
|
| 20 |
*GNU APL* is, in terms of source code size, a fairly large program. As of |
| 21 |
this writing the source code for the interpreter, not counting code in |
| 22 |
sub-directories, has more than 110,000 lines of code. Finding faults in |
| 23 |
code of that quantity is not as trivial as finding faults in a short program |
| 24 |
of only a few thousand lines. Past experience shows, that after a new fault |
| 25 |
was observed, the time for reproducing and localizing the fault typically |
| 26 |
takes more than 90% of the total time to fix it. |
| 27 |
|
| 28 |
For this reason, *GNU APL* comes with a handful of built-in trouble-shooting |
| 29 |
capabilities that makes locating a fault easier. Some of these capabilities |
| 30 |
require that *GNU APL* is built and run in a specific way as described in |
| 31 |
this document. |
| 32 |
|
| 33 |
Almost every error that might occur in *GNU APL* falls into one of the |
| 34 |
following 4 categories of increasing severities: |
| 35 |
|
| 36 |
* Errors in User Programs (aka. APL errors), |
| 37 |
* Incorrect APL results, |
| 38 |
* Failed Assertions, and |
| 39 |
* Crashes of the *GNU APL* interpreter. |
| 40 |
|
| 41 |
Each category requires different means and preparations for locating the |
| 42 |
root cause of the error. |
| 43 |
|
| 44 |
== Errors in User Programs (aka. APL errors), |
| 45 |
|
| 46 |
Errors in user programs are reported by the APL intepreter when APL code |
| 47 |
is being executed. Before the interpreter starts its internal computation(s) |
| 48 |
it checks the arguments. These checks depend on the computation required |
| 49 |
and usually occur in the following order: |
| 50 |
|
| 51 |
. Syntax Check (may raise SYNTAX ERROR) |
| 52 |
. Valence Check (may raise VALENCE ERROR or AXIS ERROR) |
| 53 |
. Value Check (may raise VALUE ERROR) |
| 54 |
. Rank Check (may raise RANK ERROR) |
| 55 |
. Shape Check (may raise LENGTH ERROR) |
| 56 |
. Domain Checks (may raise DOMAIN ERROR) |
| 57 |
|
| 58 |
That is, a VALUE error can normally only occur in the absence of SYNTAX |
| 59 |
and VALENCE ERRORs, a LENGTH ERROR can only occur in the absence of SYNTAX, |
| 60 |
VALENCE, AXIS, VALUE, and RANK ERRORs, and so on. |
| 61 |
|
| 62 |
An APL error is displayed immediately and (provided that output coloring |
| 63 |
is enabled) in a different color than the APL output. The first line of the |
| 64 |
error display is the type of error, possibly followed by +. The +, if |
| 65 |
present, indicates that there is more information available about the error |
| 66 |
(and that the additional information can be obtained with command )MORE. |
| 67 |
Example: |
| 68 |
|
| 69 |
+/[1;2] 3 ⍝ ; is not allowed in function axes. |
| 70 |
AXIS ERROR+ |
| 71 |
+/[1;2]3 |
| 72 |
^ ^ |
| 73 |
)MORE |
| 74 |
illegal ; in axis |
| 75 |
|
| 76 |
APL errors shall, of course, not be reported to the *GNU APL* maintainers, |
| 77 |
with the following exceptions: |
| 78 |
|
| 79 |
* An error was raised incorrectly, i.e. for valid APL input, or |
| 80 |
* the display of )MORE is misleading or could be improved. Many errors |
| 81 |
create no )MORE information and that is on purpose. If you believe, however, |
| 82 |
that a particular error case is difficult to grasp and would benefit from |
| 83 |
a )MORE information, then feel free to propose that in an email to |
| 84 |
[email protected] |
| 85 |
|
| 86 |
== Incorrect APL results |
| 87 |
|
| 88 |
If *GNU APL* produces an incorrect result, then you should report it in an |
| 89 |
email to [email protected]. People on that mailing list are gentle and helpful, |
| 90 |
so don't be afraid to report an error. |
| 91 |
|
| 92 |
However, please note the following: Even though APL as a language is |
| 93 |
standardized, (ISO Standard 13751 "Programming Language APL, Extended") |
| 94 |
there exist considerable differences (and also errors) between: |
| 95 |
|
| 96 |
* the APL standard, and |
| 97 |
* the APL documentation of different vendors, |
| 98 |
* and the interpreter implementations of different vendors (including *GNU |
| 99 |
APL*). |
| 100 |
|
| 101 |
In the case of such differences, *GNU APL* usually resolves the issue in the |
| 102 |
following order: |
| 103 |
|
| 104 |
* IBM APL2 PC implementation (highest priority) |
| 105 |
* IBM APL2 language reference manual |
| 106 |
* ISO Standard 13751 |
| 107 |
* Other vendors (lowest priority). |
| 108 |
|
| 109 |
Before sending an email to [email protected] it may make sense to check if the |
| 110 |
issue was reported already (and often caused by such differences). However, |
| 111 |
don't be afraid to report the same error twice if you are not sure. |
| 112 |
|
| 113 |
== Failed Assertions |
| 114 |
|
| 115 |
An assertion is an expression in the source code that the programmer assumes |
| 116 |
to be *true* (aka. ≠ 0 in C/Cpp). It can be very hard to find the origin |
| 117 |
(root cause) of an error, in particular when many CPU instructions were |
| 118 |
performed between the root cause and error being detected. |
| 119 |
|
| 120 |
By making assertions all over place in the source code, the programmer |
| 121 |
introduces checks along the execution path in order to reduce the number of |
| 122 |
CPU instructions between a root cause and the point in time where it is |
| 123 |
being detected. In the C/Cpp code, assertions are typically made in the |
| 124 |
following places: |
| 125 |
|
| 126 |
* near the beginning of a function (also known as pre-condition) to check |
| 127 |
the arguments passed to the function, |
| 128 |
* after calling a reasonably complex sub-function, and |
| 129 |
* near the end of a function (post-condition) to check the result returned by |
| 130 |
the function. |
| 131 |
|
| 132 |
Assertions are implemented as one or the other of two C/Cpp macros called |
| 133 |
Assert() and Assert1(). As of this writing there are more than 530 Assert() |
| 134 |
and more than 110 Assert1() macros spread across the *GNU APL* source code. |
| 135 |
The reason for having different Assert macros is their impact on performance. |
| 136 |
|
| 137 |
Assert1() macros are intended to test more trivial conditions while Assert() |
| 138 |
macros test more complex conditions. Every test performed by a single Assert() |
| 139 |
or Assert1() macro has a (relatively mild) performance impact. However, the |
| 140 |
large number of Assert() macros increases the total performance impact which |
| 141 |
may be undesirable. To deal with this, the user may *./configure* one of 3 |
| 142 |
different assertion levels as follows (see also *README-2-configure*): |
| 143 |
|
| 144 |
* *ASSERT_LEVEL_WANTED = 0*: this disables both the Assert1() and the Assert() |
| 145 |
macro so that the interpreter will have the maximal performance. |
| 146 |
* *ASSERT_LEVEL_WANTED = 1* (the default): this disables the Assert1() macro |
| 147 |
and enables and the Assert() macro. |
| 148 |
* *ASSERT_LEVEL_WANTED = 2*: this enables both the Assert1() and the Assert() |
| 149 |
macro. |
| 150 |
|
| 151 |
We kindly ask the user to: |
| 152 |
|
| 153 |
make develop |
| 154 |
|
| 155 |
which, besides other settings, does *./configure* with ASSERT_LEVEL_WANTED=2, |
| 156 |
thus enabling both Assert macros and possibly making it easier to find the |
| 157 |
root cause of a fault. |
| 158 |
|
| 159 |
As confidence with the source code increases over time, Assert() macros will |
| 160 |
eventually be converted to Assert1() macros while new code will initially |
| 161 |
be protected with Assert() macros. |
| 162 |
|
| 163 |
=== Emulating Failed Assertions |
| 164 |
|
| 165 |
Fortunately we can emulate a failed Assert() or Assert1() macro with ⎕FIO: |
| 166 |
|
| 167 |
---- |
| 168 |
|
| 169 |
⎕FIO ¯16 ⍝ emulate Assert() |
| 170 |
|
| 171 |
⎕FIO ¯17 ⍝ emulate Assert1() |
| 172 |
---- |
| 173 |
|
| 174 |
If the respective macro is enabled with ASSERT_LEVEL_WANTED as explained |
| 175 |
above, then it produces a stack trace like this: |
| 176 |
|
| 177 |
---- |
| 178 |
⎕FIO ¯16 |
| 179 |
|
| 180 |
|
| 181 |
============================================================================== |
| 182 |
Assertion failed: 0 && "Simulated Assert() (aka. ⎕FIO ¯16)" |
| 183 |
in Function: eval_B |
| 184 |
in file: Quad_FIO.cc:1085 |
| 185 |
|
| 186 |
C/Cpp call stack: |
| 187 |
|
| 188 |
---------------------------------------- |
| 189 |
-- Stack trace at Assert.cc:72 |
| 190 |
---------------------------------------- |
| 191 |
0x7FBFD71CBBF7 __libc_start_main |
| 192 |
0x563D2320B04A main |
| 193 |
0x563D233E822F Workspace::immediate_execution(bool) |
| 194 |
0x563D2327411A Command::process_line() |
| 195 |
0x563D232742DC Command::process_line(UCS_string&) |
| 196 |
0x563D232768A2 Command::do_APL_expression(UCS_string&) |
| 197 |
0x563D23276C73 Command::finish_context() |
| 198 |
0x563D2329B5D2 Executable::execute_body() const |
| 199 |
0x563D2338199C StateIndicator::run() |
| 200 |
0x563D232DAF86 Prefix::reduce_statements() |
| 201 |
0x563D232DBE92 Prefix::reduce_MISC_F_B_() |
| 202 |
0x563D2332D0CF Quad_FIO::eval_B(Value_P) const |
| 203 |
0x563D2322C2A0 do_Assert(char const*, char const*, char const*, int) |
| 204 |
======================================== |
| 205 |
|
| 206 |
SI stack: |
| 207 |
|
| 208 |
Depth: 0 |
| 209 |
Exec: 0x563d23f9fde0 |
| 210 |
Safe exec: 0 |
| 211 |
Pmode: ◊ ⎕FIO ¯16 |
| 212 |
PC: 3 (4) RETURN_STATS |
| 213 |
Stat: ⎕FIO ¯16 |
| 214 |
err_code: 0x0 |
| 215 |
|
| 216 |
============================================================================== |
| 217 |
|
| 218 |
---- |
| 219 |
|
| 220 |
The C/Cpp stack above tells us which assertion has failed and where the |
| 221 |
assertion is located in the C/Cpp source code (i.e. file Quad_FIO.cc line 1085), |
| 222 |
while the )SI stack tells us where that location is in the APL code. |
| 223 |
|
| 224 |
This brings us closer to the root cause of a problem. Unfortunately the hex |
| 225 |
addresses numbers on the left side of the C/Cpp stack dump are process |
| 226 |
specific, i.e. the same *⎕FIO ¯16* will produce different numbers in a |
| 227 |
different process of the operating system. |
| 228 |
|
| 229 |
=== Line numbers in Stack Traces |
| 230 |
|
| 231 |
If the functions on the right side of the C/Cpp stack are relatively small (as |
| 232 |
most functions in the source code of *GNU APL* are) then we can usually find the |
| 233 |
exact source line for each stack entry rather easily using the function name |
| 234 |
displayed. For larger functions, the same function may be called from different |
| 235 |
source code lines in the same function. This can be improved by converting the |
| 236 |
hex addresses on the left into source line numbers as exlained in the |
| 237 |
following. |
| 238 |
|
| 239 |
First of all the Cpp compiler must be instructed to not discard line numbers. |
| 240 |
Older g++ versions would always keep line numbers in the object files, but |
| 241 |
newer versions do not. The trick is this: |
| 242 |
|
| 243 |
---- |
| 244 |
CXXFLAGS="-rdynamic -gdwarf-2" ./configure ... |
| 245 |
---- |
| 246 |
|
| 247 |
*make develop* also does that for you. The CXXFLAGS above tell the *g++* |
| 248 |
compiler to use an older object format in which line numbers are not discarded. |
| 249 |
If your compiler does not accept *-gdwarf-2* then it probably uses it anyway. |
| 250 |
|
| 251 |
The second step is to create a file named *apl.lines*. If *GNU APL* finds this |
| 252 |
file then it uses it to map hex addresses to line numbers. In the src |
| 253 |
directory, the make target *apl.lines* creates this file (which takes quite a |
| 254 |
while and is therefore not created in a standard build). *make apl.lines* |
| 255 |
essentially does: |
| 256 |
|
| 257 |
---- |
| 258 |
objdump --section=.text --line-numbers --disassemble apl > apl.lines |
| 259 |
---- |
| 260 |
|
| 261 |
which extracts the line numbers from the apl binary (provided that the |
| 262 |
compiler has inserted them - see above). If *GNU APL* finds apl.lines, then |
| 263 |
the stack dump looks a little different: |
| 264 |
|
| 265 |
---- |
| 266 |
⎕FIO ¯16 |
| 267 |
|
| 268 |
============================================================================== |
| 269 |
Assertion failed: 0 && "Simulated Assert() (aka. ⎕FIO ¯16)" |
| 270 |
in Function: eval_B |
| 271 |
in file: Quad_FIO.cc:1085 |
| 272 |
|
| 273 |
C/Cpp call stack: |
| 274 |
total_lines in apl.lines: 618610 |
| 275 |
assembler lines in apl.lines: 86495 |
| 276 |
source line numbers found: 86495 |
| 277 |
|
| 278 |
---------------------------------------- |
| 279 |
-- Stack trace at Assert.cc:72 |
| 280 |
---------------------------------------- |
| 281 |
0x29465EE76BF7 __libc_start_main |
| 282 |
0x14CCD2 main at main.cc:635 |
| 283 |
0x3166E1 Workspace::immediate_execution(bool) at Workspace.cc:273 |
| 284 |
0x1B1004 Command::process_line() at Command.cc:103 |
| 285 |
0x1B11C6 Command::process_line(UCS_string&) at Command.cc:139 |
| 286 |
0x1B378C Command::do_APL_expression(UCS_string&) at Command.cc:329 |
| 287 |
0x1B3B5D Command::finish_context() at Command.cc:347 |
| 288 |
0x1D6806 Executable::execute_body() const at Executable.cc:261 |
| 289 |
0x2B564F StateIndicator::run() at StateIndicator.cc:362 |
| 290 |
0x211DA4 Prefix::reduce_statements() at Prefix.cc:723 (discriminator 4) |
| 291 |
0x212A79 Prefix::reduce_MISC_F_B_() at Prefix.cc:990 |
| 292 |
0x261E7F Quad_FIO::eval_B(Value_P) const at Quad_FIO.cc:1090 |
| 293 |
0x16CC03 do_Assert(char const*, char const*, char const*, int) at Assert.cc:74 |
| 294 |
======================================== |
| 295 |
|
| 296 |
SI stack: |
| 297 |
|
| 298 |
Depth: 0 |
| 299 |
Exec: 0x561c865a4da0 |
| 300 |
Safe exec: 0 |
| 301 |
Pmode: ◊ ⎕FIO ¯16 |
| 302 |
PC: 3 (4) RETURN_STATS |
| 303 |
Stat: ⎕FIO ¯16 |
| 304 |
err_code: 0x0 |
| 305 |
|
| 306 |
|
| 307 |
============================================================================== |
| 308 |
---- |
| 309 |
|
| 310 |
The difference is twofold: |
| 311 |
|
| 312 |
. the hex numbers are now relative to the location of the *main()* function of |
| 313 |
*GNU APL*, which makes them not only smaller but also identical even in |
| 314 |
different processes. |
| 315 |
|
| 316 |
. The source line numbers are now displayed for most hex numbers. Some |
| 317 |
addresses can not be resolved, this could be caused by C/Cpp function |
| 318 |
inlining or calls through function pointers etc. |
| 319 |
|
| 320 |
Needless to say that *GNU APL* trouble-shooters prefer stack traces with hex |
| 321 |
numbers resolved to line numbers. |
| 322 |
|
| 323 |
Occasionally you may get this message: |
| 324 |
|
| 325 |
---- |
| 326 |
... |
| 327 |
C/Cpp call stack: |
| 328 |
|
| 329 |
Cannot show line numbers, since apl.lines is older than apl. |
| 330 |
Consider running 'make apl.lines' in directory |
| 331 |
/home/eedjsa/apl-1.8 to fix this |
| 332 |
... |
| 333 |
---- |
| 334 |
|
| 335 |
This happens if you do *make apl* without a subsequent *make apl.lines*. |
| 336 |
|
| 337 |
=== Summary concerning Assertions |
| 338 |
|
| 339 |
The *GNU APL* configuration for troubleshooting purposes is obtained like this: |
| 340 |
|
| 341 |
---- |
| 342 |
cd <top-level-directory> |
| 343 |
make develop |
| 344 |
cd src |
| 345 |
make clean |
| 346 |
make apl.lines |
| 347 |
---- |
| 348 |
|
| 349 |
where *top-level-directory* is the directory that contains the *README* files. |
| 350 |
Additional troubleshooting options may be used as well. Consider the *make* |
| 351 |
target *develop:* in file Makefile.incl (around line 41) only as a template |
| 352 |
that can be augmented by other troubleshooting options (see also |
| 353 |
*README-2-configure*). |
| 354 |
|
| 355 |
== The )CHECK command |
| 356 |
|
| 357 |
*GNU APL* provides a command *)CHECK* which is somewhat similar to IBM APL2's |
| 358 |
command *)CHECK WS*. This command performs an internal check of the internal |
| 359 |
data structures of the interpreter. |
| 360 |
|
| 361 |
Under normal circumstances, the output of the *)CHECK* command should be: |
| 362 |
|
| 363 |
---- |
| 364 |
OK - no stale functions |
| 365 |
OK - no stale values |
| 366 |
OK - no stale indices |
| 367 |
OK - no duplicate parents |
| 368 |
---- |
| 369 |
|
| 370 |
If not, then there were inconsistencies (primarily memory leaks) that need to |
| 371 |
be fixed and should therefore be reported to [email protected]. For example, |
| 372 |
a *stale value* is an APL value that has memory allocated but is not reachable |
| 373 |
any longer via either the symbol table or the interpreter (like an APL |
| 374 |
variable) or the body of a defined function (like an APL literal). |
| 375 |
|
| 376 |
If a )CHECK command fails, then it is important to remember, or better write |
| 377 |
down, which sequence of APL operations has led to the state of the workspace. |
| 378 |
*GNU APL* has a command *)HISTORY* which displays the last lines entered. |
| 379 |
|
| 380 |
An assertion checks the immediate state of the interpreter and can therefore |
| 381 |
not check the entire lifetime of an APL value or function. For this purpose, |
| 382 |
*GNU APL* povides a different debugging tool called *Value history*. |
| 383 |
|
| 384 |
The Value history tracks the locations in the *GNU APL* source code that have |
| 385 |
manipulated a value from its creation (memory allocation) to its destruction |
| 386 |
(memory deallocation). This tracking, when enabled, has performance impacts |
| 387 |
and is therefore disabled by default. |
| 388 |
|
| 389 |
If the )CHECK command indicates stale values, but also under some other error |
| 390 |
conditions, the history of a value at fault is being displayed and helps the |
| 391 |
*GNU APL* maintainers to locate the problem. The value history is enabled in |
| 392 |
*./configure*: |
| 393 |
|
| 394 |
---- |
| 395 |
./configure VALUE_HISTORY_WANTED=yes ... |
| 396 |
---- |
| 397 |
|
| 398 |
and therefore requires recompilation of the interpreter. |
| 399 |
|
| 400 |
|
| 401 |
== Logging facilities |
| 402 |
|
| 403 |
== Crashes of the GNU APL interpreter |
| 404 |
|
| 405 |
Some programming errors bypass all built-in debugging tools of GNU and |
| 406 |
crash the interpreter rather than returning to the interactive immediate |
| 407 |
execution mode of APL. |
| 408 |
|
| 409 |
The crash may be silent or accompanied by an error message and it may or may |
| 410 |
not create a core file. A core file is an important file if no other |
| 411 |
information is printed as to what kind of fault has occurred. |
| 412 |
|
| 413 |
In the good old times core files were always produced when a binary executable |
| 414 |
crashed. These days the creation of core files may need to be enabled before |
| 415 |
the crashing binary is started. In GNU/Linux with its *bash* shell that means: |
| 416 |
|
| 417 |
---- |
| 418 |
ulimit -c unlimited # enable core files |
| 419 |
---- |
| 420 |
|
| 421 |
Instead of *unlimited* you may also use a different file size limit. |
| 422 |
|
| 423 |
After a core file was produced (we always assume the interpreter was started |
| 424 |
in directory src below the top--level-directory) you can start the debugger |
| 425 |
*gdb* in the src directory with the core file as its second argument: |
| 426 |
|
| 427 |
---- |
| 428 |
gdb ./apl core |
| 429 |
---- |
| 430 |
|
| 431 |
After that, you can use the gdb command *bt* (for 'backtrace') to show a |
| 432 |
stack dump similar to the one discussed earlier. The *GNU APL* maintainer(s) |
| 433 |
will appreciate a copy of that stack dump for problems that are difficult to |
| 434 |
reproduce. |
| 435 |
|
| 436 |
Normally the file is named '*core*' and found in the same directory as the |
| 437 |
binary, but some GNU/Linux distributions use a different directory for storing |
| 438 |
core files. See also *man 5 core*. |
| 439 |
|
| 440 |
Finally, try to nail down the root cause with logging facilities (see below). |
| 441 |
|
| 442 |
== Non-reproducible Errors |
| 443 |
|
| 444 |
The most difficult to fix errors are those that occur on one machine (usually |
| 445 |
your's) and not on others. |
| 446 |
|
| 447 |
The first thing to locate their root cause it to *./configure* with either no |
| 448 |
options or via *make develop* to see if the error is related to specific |
| 449 |
*./configure* options. |
| 450 |
|
| 451 |
Also, look for compiler warnings when *GNU APL* is built. In the default build, |
| 452 |
compiler warnings can be overlooked easiy. *make develop* turns most warnings |
| 453 |
into errors so you will see at the end if the build was successful. |
| 454 |
|
| 455 |
=== Optimization |
| 456 |
|
| 457 |
Recently we have seen faults that may have been caused by an overly aggressive |
| 458 |
optimization of the *GNU APL* Cpp source code. These faults were observed for |
| 459 |
the same source code but only when compiled with newer versions of the |
| 460 |
same g++ compiler. |
| 461 |
|
| 462 |
By default *GNU APL* compiles with optimization level -O2. If a fault is |
| 463 |
suspected to be caused by optimization, then it could make sense to also |
| 464 |
test the same source code of *GNU APL* without any optimization. |
| 465 |
|
| 466 |
To do that: |
| 467 |
|
| 468 |
---- |
| 469 |
cd src |
| 470 |
find and remove or replace the string -O2 in the relevant Makefile(s) |
| 471 |
make clean all |
| 472 |
---- |
| 473 |
|
| 474 |
== Logging facilities |
| 475 |
|
| 476 |
The source code of *GNU APL* is populated with more than 380 Log() macros. |
| 477 |
A Log() macro may or may not print some additional information when the |
| 478 |
execution of the APL interpreter reaches them. |
| 479 |
|
| 480 |
If all these Log() macros would output something, then even a trivial |
| 481 |
APL statement would cause a lot of output in which the relevant information |
| 482 |
gets easily lost. For that reason, each Log () macro belongs to one of |
| 483 |
currently over 45 Groups called *logging facilities*. Each logging facility |
| 484 |
can be turned ON or OFF which causes Log() macros belonging the facility |
| 485 |
to print or not to print its information. |
| 486 |
|
| 487 |
The large (and increasing) number of logging facilities cause the tests, |
| 488 |
whether Log() macro or its respective logging facility is ON or OFF, to have |
| 489 |
significant performance impacts. For this reason, *GNU APL* provides two |
| 490 |
different methods to control how logging facilities are turned ON or OFF: |
| 491 |
|
| 492 |
. static logging |
| 493 |
. dynamic logging |
| 494 |
|
| 495 |
The method is selected via *./configure* at compile time (more precisely: at |
| 496 |
*./configure* time), see also *README-2-configure*. |
| 497 |
|
| 498 |
=== Static Logging |
| 499 |
|
| 500 |
Static logging is used with: |
| 501 |
|
| 502 |
---- |
| 503 |
./configure DYNAMIC_LOG_WANTED=no |
| 504 |
---- |
| 505 |
|
| 506 |
Static logging is also the default, therefore *DYNAMIC_LOG_WANTED=no* is the |
| 507 |
same as not using DYNAMIC_LOG_WANTED= in *./configure*. |
| 508 |
|
| 509 |
With Static logging, the selection of which logging facility shall be ON is |
| 510 |
made by the first argument of the *log_def()* macros in file *src/Logging.def. |
| 511 |
A value of 0 means OFF while 1 means ON for the respective logging facility. |
| 512 |
|
| 513 |
With static logging, the log_def() macro defines an *enum* value for each |
| 514 |
log_def() macro which is a compile-time constant. The compiler will therefore |
| 515 |
remove all code (including the ON/OFF for all log_def(0, ...) macros and |
| 516 |
also all tests for log_def(1, ...) in all Log() macros. That is: |
| 517 |
|
| 518 |
* Advantage: Static Logging has little if any performance impacts (none |
| 519 |
if all log_def macros are off, and |
| 520 |
* Disadvantage: changes of log_def() macros require re-compilation of GNU |
| 521 |
APL. |
| 522 |
|
| 523 |
=== Dynamic Logging |
| 524 |
|
| 525 |
Dynamic logging is used with: |
| 526 |
|
| 527 |
---- |
| 528 |
./configure DYNAMIC_LOG_WANTED=yes |
| 529 |
---- |
| 530 |
|
| 531 |
With dynamic logging, the log_def() macro defines the initial value of a |
| 532 |
variable (rather than an enum value) for each logging facility. After the |
| 533 |
interpreter was started, the value of each variable can be changed from |
| 534 |
OFF to ON (and vice versa) with the *GNU APL* debug command *]LOG* (which |
| 535 |
toggles the state of the respective variable). |
| 536 |
|
| 537 |
* Disadvantage: Dynamic Logging has a small performance impact (even |
| 538 |
if all variables are set to OFF) which will accumulate for all Log() macros, |
| 539 |
* Advantage: no re-compilation of *GNU APL* is required. |
| 540 |
|
| 541 |
==== The ]LOG command |
| 542 |
|
| 543 |
GNU APL has a few debug commands, which are like standard APL commands but |
| 544 |
start with ] instead of ). One of these debug commands is *]LOG* which |
| 545 |
controls the logging facilities. |
| 546 |
|
| 547 |
If static logging was *./configure*'d then *]LOG* displays an error message |
| 548 |
because changing the logging facilities at run-time requires dynamic logging. |
| 549 |
|
| 550 |
---- |
| 551 |
]LOG |
| 552 |
|
| 553 |
Command ]LOG is not available, since dynamic logging was not configured |
| 554 |
for this APL interpreter. To enable dynamic logging (which will slightly |
| 555 |
decrease performance), recompile the interpreter as follows: |
| 556 |
|
| 557 |
./configure DYNAMIC_LOG_WANTED=yes (... other configure options) |
| 558 |
make |
| 559 |
make install (or try: src/apl) |
| 560 |
|
| 561 |
above the src directory. |
| 562 |
---- |
| 563 |
|
| 564 |
If dynamic logging was ./configure'd then *]LOG* without an argument displays |
| 565 |
the current state (i.e. ON or OFF) of all logging facilities: |
| 566 |
|
| 567 |
---- |
| 568 |
]LOG |
| 569 |
1: (OFF) AV details |
| 570 |
2: (OFF) new and delete calls |
| 571 |
3: (OFF) input from user or testcase file |
| 572 |
4: (OFF) parser: parsing |
| 573 |
5: (OFF) ... function find_closing() |
| 574 |
6: (OFF) ... tokenization |
| 575 |
7: (OFF) ... function collect_constants() |
| 576 |
8: (OFF) ... create value() |
| 577 |
9: (OFF) defined function: fix() |
| 578 |
10: (OFF) ... set_line() |
| 579 |
11: (OFF) ... load() |
| 580 |
12: (OFF) ... execute() |
| 581 |
13: (OFF) ... enter/leave |
| 582 |
14: (OFF) State indicator: enter/leave |
| 583 |
15: (OFF) ... push/pop |
| 584 |
16: (OFF) Symbol: lookup |
| 585 |
17: (OFF) ... push/pop and )ERASE |
| 586 |
18: (OFF) ... resolve |
| 587 |
19: (OFF) Value: glue() |
| 588 |
20: (OFF) ... erase_stale() |
| 589 |
21: (OFF) APL primitive function format |
| 590 |
22: (OFF) character conversions |
| 591 |
23: (OFF) APL system function Quad-FX |
| 592 |
24: (OFF) commands )LOAD, )SAVE, )IN, and )OUT |
| 593 |
25: (OFF) more verbose errors |
| 594 |
26: (OFF) details of error throwing |
| 595 |
27: (OFF) nabla editor |
| 596 |
28: (OFF) execute(): state changes |
| 597 |
29: (OFF) PrintBuffer: align() function |
| 598 |
30: (OFF) Output: cork() functions |
| 599 |
31: (OFF) Details of test execution |
| 600 |
32: (OFF) Prefix parser |
| 601 |
33: (OFF) ... location information |
| 602 |
34: (OFF) FunOper1 and FunOper2 functions |
| 603 |
35: (OFF) Shared Variable operations |
| 604 |
36: (OFF) command line arguments (argc/argv) |
| 605 |
37: (OFF) interpreter start-up messages |
| 606 |
38: (OFF) optimization messages |
| 607 |
39: (OFF) )LOAD and )SAVE details |
| 608 |
40: (OFF) Svar_DB signals |
| 609 |
41: (OFF) Parallel (multi-core) execution |
| 610 |
42: (OFF) EOC handler functionality |
| 611 |
43: (OFF) ⎕DLX details |
| 612 |
44: (OFF) command ]DOXY |
| 613 |
45: (OFF) details of Value allocation |
| 614 |
46: (OFF) ⎕TF details |
| 615 |
47: (OFF) ⎕PLOT details |
| 616 |
---- |
| 617 |
|
| 618 |
Otherwise *]LOG N* toggles the state of logging facility N, e.g.: |
| 619 |
|
| 620 |
---- |
| 621 |
]log 25 |
| 622 |
Logging facility 'more verbose errors' is now ON |
| 623 |
|
| 624 |
]log 26 |
| 625 |
Logging facility 'details of error throwing' is now ON |
| 626 |
---- |
| 627 |
|
| 628 |
=== Logging Facilities of Interest |
| 629 |
|
| 630 |
Many of the currently over 40 logging facilities serve a particular purpose |
| 631 |
for the maintainers or have become obsolete and are therefore not too useful |
| 632 |
for the trouble-shooting performed by the user. The others mentioned below. |
| 633 |
|
| 634 |
==== ]Log 2 : new and delete calls |
| 635 |
|
| 636 |
For tracking the allocation and de-allocation of memory. |
| 637 |
To find memo leaks. |
| 638 |
|
| 639 |
==== ]Log 25 and 26 : APL Errors |
| 640 |
|
| 641 |
An APL ERROR provides enough information as to where an error has occurred |
| 642 |
in the APL code, but none as to where that error was detected on the C/Cpp |
| 643 |
code. These logging facilities add this information. |
| 644 |
|
| 645 |
==== ]Log 32 Prefix Parser |
| 646 |
|
| 647 |
Parsing of APL commands and Expressions. To analyze SYNTAX ERRORs. |
| 648 |
|
| 649 |
==== ]Log 37 : Interpreter Startup |
| 650 |
|
| 651 |
Information when the *GNU APL* interpreter starts. This facility can also be |
| 652 |
enabled from the command line because the output of interest occurs before |
| 653 |
the interpreter accepts commands like *]LOG*. |
| 654 |
|
| 655 |
== Intrusive Troubleshooting |
| 656 |
|
| 657 |
If all the above fails, then it is time for intrusive troubleshooting. |
| 658 |
By intrusive troubleshooting we mean that the source code is modified |
| 659 |
in such a way that it prints troubleshooting information at source code |
| 660 |
locations where no Log() macros have been inserted. The top-level algorithm |
| 661 |
for intrusive troubleshooting is this: |
| 662 |
|
| 663 |
---- |
| 664 |
loop: |
| 665 |
1. modify the source code by adding printouts |
| 666 |
2. recompile GNU APL |
| 667 |
3. run apl and reproduce the problem |
| 668 |
4. repeat until the error (or a least a source code file or function |
| 669 |
responsible for the error) was found. |
| 670 |
---- |
| 671 |
|
| 672 |
All this is usually done in the *src* directory where most of the source |
| 673 |
files are located and where the compile result will be stored. |
| 674 |
|
| 675 |
Alternatively one could use a debugger like gdb for the same purpose, |
| 676 |
setting breakpoints and printing data. However, the author prefers to modify |
| 677 |
the source code, in particular if more than one printout is needed to |
| 678 |
find a fault. |
| 679 |
|
| 680 |
To simplify the source code for printouts, *GNU APL* has defined two powerful |
| 681 |
macros *LOC* and *Q()*. The *Q()* macro is used exclusively for intrusive |
| 682 |
troubleshooting while the LOC macro is also used for other purposes in the |
| 683 |
source code (with more than 1700 invocations as of this writing). |
| 684 |
Understanding the LOC macro is therefore a prerequisite for understanding the |
| 685 |
*GNU APL* code. |
| 686 |
|
| 687 |
=== The LOC Macro |
| 688 |
|
| 689 |
The LOC macro is a short but somewhat tricky construction based on two |
| 690 |
other macros *Loc* and *STR*. The LOC macro takes no arguments and is |
| 691 |
therefore used without (), *STR()* takes one argument and *Loc(x) takes |
| 692 |
two arguments. All three macros *LOC*, *Loc()*, and *STR()* are *#defined* |
| 693 |
in file *Common.hh* which makes them available in practically every source |
| 694 |
file. The definitions are: |
| 695 |
|
| 696 |
---- |
| 697 |
#define STR(x) #x |
| 698 |
|
| 699 |
#define LOC Loc(__FILE__, __LINE__) |
| 700 |
|
| 701 |
#define Loc(f, l) f ":" STR(l) |
| 702 |
---- |
| 703 |
|
| 704 |
How does this work? Suppose we have a 13-line Cpp file *test.cc* like this: |
| 705 |
|
| 706 |
---- |
| 707 |
#include <iostream> // for cout |
| 708 |
|
| 709 |
#define LOC Loc(__FILE__, __LINE__) |
| 710 |
#define Loc(f, l) f ":" STR(l) |
| 711 |
#define STR(x) #x |
| 712 |
|
| 713 |
int |
| 714 |
main() |
| 715 |
{ |
| 716 |
std::cout << LOC << std::endl; // line 10 of test.cc |
| 717 |
std::cout << LOC << std::endl; // line 11 of test.cc |
| 718 |
std::cout << LOC << std::endl; // line 12 of test.cc |
| 719 |
} |
| 720 |
---- |
| 721 |
|
| 722 |
If we compile and run this file then it produces the following output: |
| 723 |
|
| 724 |
---- |
| 725 |
test.cc:10 |
| 726 |
test.cc:11 |
| 727 |
test.cc:12 |
| 728 |
---- |
| 729 |
|
| 730 |
That is, LOC expands to a string literal that consists of the filename in |
| 731 |
which LOC was used, followed by a colon, followed by the line number in which |
| 732 |
LOC was used. String literals have the advantage that they are produced |
| 733 |
at compile time and can be passed as function arguments at almost no |
| 734 |
performance cost. |
| 735 |
|
| 736 |
Now lets see how *LOC* works. Consider the first *LOC* on line *10* of |
| 737 |
file *test.cc*. Then: |
| 738 |
|
| 739 |
. macro *LOC* expands to *Loc(\_\_FILE\_\_, \_\_LINE__)* |
| 740 |
. *\_\_FILE__* expands to literal *"test.cc"* |
| 741 |
. *\_\_LINE__* expands to the line number *10*, so that (so far) |
| 742 |
. *LOC* expands to *Loc("test.cc", 10)* |
| 743 |
. *Loc("test.cc", 10)* expands to *"test.cc" ":" STR(10)* |
| 744 |
. *STR(10)* expands to *"10"* so that (so far) LOC expands to |
| 745 |
*"test.cc" ":" "10"* |
| 746 |
. the compiler combines the 3 adjacent string literals *"test.cc"*, *":"*, and |
| 747 |
*"10"* into the single string literal *"test.cc:10"* which is then printed. |
| 748 |
|
| 749 |
This is quite some work for the compiler, but the result is such that the |
| 750 |
run-time cost for *GNU APL* is only the loading of the string literal when |
| 751 |
*GNU APL* starts. |
| 752 |
|
| 753 |
=== The Q Macro |
| 754 |
|
| 755 |
The second trick is the Q() macro which is defined as |
| 756 |
|
| 757 |
---- |
| 758 |
#define Q(x) get_CERR() << std::left << setw(20) << #x ":" << " '" << x << "' at " LOC << endl; |
| 759 |
---- |
| 760 |
|
| 761 |
In this definition *get_CERR()* is a function which returns an output channel |
| 762 |
such as *std::cerr* (or *stderr* in C). The operator << is overloaded a lot |
| 763 |
and in such a way that commonly used data types in *GNU APL* are being printed |
| 764 |
nicely. A consequence of the overloading is that *Q()* macro can be used |
| 765 |
with many different data types without the burdon to properly format them |
| 766 |
for the debug output. |
| 767 |
|
| 768 |
*Q(x)* uses the *LOC* macro to get the source file name and line number of its |
| 769 |
invocation and essentially prints *x* in a suitable format and followed by |
| 770 |
its source code location. For example: |
| 771 |
|
| 772 |
---- |
| 773 |
#include <iostream> // for cout |
| 774 |
#include <iomanip> // for setw |
| 775 |
|
| 776 |
using namespace std; |
| 777 |
|
| 778 |
#define STR(x) #x |
| 779 |
#define LOC Loc(__FILE__, __LINE__) |
| 780 |
#define Loc(f, l) f ":" STR(l) |
| 781 |
|
| 782 |
#define Q(x) cerr << std::left << std::setw(20) << #x ":" << " '" << x << "' at " LOC << std::endl; |
| 783 |
|
| 784 |
int |
| 785 |
main() |
| 786 |
{ |
| 787 |
Q("Hello") // line 15 |
| 788 |
Q(10) |
| 789 |
Q(LOC) |
| 790 |
} |
| 791 |
---- |
| 792 |
|
| 793 |
which, when compiled, produces: |
| 794 |
|
| 795 |
---- |
| 796 |
"Hello": 'Hello' at test.cc:15 |
| 797 |
10: '10' at test.cc:16 |
| 798 |
LOC: 'test.cc:17' at test.cc:17 |
| 799 |
---- |
| 800 |
|
| 801 |
With these prerequisites intrusive troubleshooting boils down to spreading |
| 802 |
*Q()* macros in different source files until the fault is found. |
| 803 |
|
| 804 |
A frequent question is not the value of a Cpp variable, but rather if (and |
| 805 |
when) a particular source code line of code is reached. To answer this |
| 806 |
question just insert *Q(LOC)* or *Q(0)* on the line of interest. |
| 807 |
|
| 808 |
Another useful helper is *subversion* (svn). After having made multiple |
| 809 |
insertions of *Q()* macros in various source files (and hopefully having |
| 810 |
located the fault this way), simply do |
| 811 |
|
| 812 |
---- |
| 813 |
svn revert * |
| 814 |
---- |
| 815 |
|
| 816 |
in the *src* directory which undoes all changes you made. |
| 817 |
|
| 818 |
== Keep It Short and Simple |
| 819 |
|
| 820 |
The first thing to check before reporting an error is to make sure that you |
| 821 |
have compiled and installed the latest GNU APL version from subversion (SVN) |
| 822 |
or git. |
| 823 |
|
| 824 |
From a maintainer's perspective, the trouble reports that are appreciated the |
| 825 |
most are the short ones. Locating a fault usually requires multiple |
| 826 |
repetitions of the sequence that causes the fault. Consider two extremes: |
| 827 |
|
| 828 |
A. The fault can be reproduced by executing a handful of APL lines in |
| 829 |
immediate execution mode, possibly accompanied by one or two ∇-defined |
| 830 |
functions or lambdas. |
| 831 |
|
| 832 |
B. A tar file contains a workspace, possibly )COPYing other workspaces. |
| 833 |
In order to reproduce the fault, the workspace has to be )LOADed and |
| 834 |
started in a specific way in order to reproduce the error. |
| 835 |
|
| 836 |
In case A. the maintainer can simply copy the lines into a small apl script |
| 837 |
created for fixing the fault. That script can then be repeated again and again |
| 838 |
by starting the script from the command line of a shell. |
| 839 |
|
| 840 |
In case B. the maintainer must first install the workspaces that come with the |
| 841 |
fault. Then start apl, )LOAD the workspace and figure out: |
| 842 |
|
| 843 |
* how the workspace shall be started, |
| 844 |
* how the defined functions in the workspace appear, how they interact, |
| 845 |
and which ones are related to the fault, |
| 846 |
* and so on. |
| 847 |
|
| 848 |
Case B. creates a lot of overhead for the maintainer that does very little to |
| 849 |
find the fault. Although sometimes a complex workspace is needed to reproduce |
| 850 |
a fault, this is rather rare and almost all errors reported so far could be |
| 851 |
reduced to a small number of APL lines. |
| 852 |
|
| 853 |
Last but not least, please have a look here: |
| 854 |
|
| 855 |
https://www.gnu.org/software/libc/manual/html_node/Reporting-Bugs.html |
| 856 |
|