| 1 |
Testuite for Icecream |
| 2 |
===================== |
| 3 |
|
| 4 |
In this directory are tests for Icecream. |
| 5 |
|
| 6 |
These tests run Icecream binaries and test them in practice in a testing setup. |
| 7 |
This makes these tests a bit slower than unit tests, but it allows testing |
| 8 |
functionality that would be hard or impossible to test using unit tests. |
| 9 |
|
| 10 |
How to run the tests: |
| 11 |
===================== |
| 12 |
|
| 13 |
The tests do not conflict with system Icecream, so it is possible to compile |
| 14 |
and test while system Icecream is active. |
| 15 |
|
| 16 |
Since Icecream hardcodes locations of some binaries, it is necessary to |
| 17 |
install Icecream for the tests (really install, no $DESTDIR). It is however |
| 18 |
not necessary to do a system install, simply install somewhere. If you want |
| 19 |
to test also remote builds (recommended), it is necessary for the install to have |
| 20 |
the capability to chroot. This is most easily done by giving the CAP_SYS_CHROOT |
| 21 |
Linux capability to the iceccd binary (which is done is triggered to be done |
| 22 |
automatically by "make test" using sudo). |
| 23 |
|
| 24 |
An example of building Icecream and testing it: |
| 25 |
|
| 26 |
./configure --prefix=$HOME/iceinstall |
| 27 |
make test |
| 28 |
|
| 29 |
The 'make test' step roughly performs: |
| 30 |
|
| 31 |
make |
| 32 |
make install |
| 33 |
sudo /sbin/setcap cap_sys_chroot+ep ${prefix}/sbin/iceccd |
| 34 |
cd tests |
| 35 |
./test.sh ${prefix} ${builddir}/tests/results --builddir=${builddir}/tests |
| 36 |
|
| 37 |
This will build and install Icecream into $HOME/iceinstall prefix and all |
| 38 |
logs and temporary files will be in ${builddir}/tests/results. It is also possible to use |
| 39 |
'filecap' from cap-ng instead of 'setcap', but 'filecap' does not report failures. |
| 40 |
|
| 41 |
Note that the limitations (real install needed in a helper location, chroot capability) |
| 42 |
mean that these tests are not suitable to be run as a part of building binary |
| 43 |
packages, they are meant to be run by developers. |
| 44 |
|
| 45 |
If a test fails, all relevant files should be in the log and temporary directory |
| 46 |
passed to test.sh . The {daemon}.log files contain logs relevant only to the last run test, |
| 47 |
files {daemon}_all.log include complete logs (up to but not including the failed test, if any). |
| 48 |
|
| 49 |
Exit status of test.sh is 0 for all tests passing, 1 for all tests passing but some being |
| 50 |
skipped and 2 for errors. |
| 51 |
|
| 52 |
If you want to use icemon for the tests, use 'ICECC_SCHEDULER=localhost:8767 icemon'. Note that |
| 53 |
icemon needs to be built with recent (Feb 2014) libicecc library for this to work. |
| 54 |
|
| 55 |
|
| 56 |
Valgrind: |
| 57 |
========= |
| 58 |
|
| 59 |
It is possible to pass --valgrind to test.sh in order to test (almost) everything with Valgrind. |
| 60 |
Valgrind may be also automatically invoked using: |
| 61 |
|
| 62 |
make test VALGRIND=1 |
| 63 |
|
| 64 |
Note that in order to be able to chroot, it is necessary to give the capability to the actual |
| 65 |
Valgrind binary instead of iceccd (this is because this binary is what actually runs as |
| 66 |
the process): |
| 67 |
|
| 68 |
sudo /sbin/setcap cap_sys_chroot+ep /usr/lib/valgrind/memcheck-x86-linux |
| 69 |
|
| 70 |
Do not forget to reset it back when done: |
| 71 |
|
| 72 |
sudo /sbin/setcap -r /usr/lib/valgrind/memcheck-x86-linux |
| 73 |
|
| 74 |
|
| 75 |
Adding new tests: |
| 76 |
================= |
| 77 |
|
| 78 |
The test.sh script is hopefully straightforward and commented enough. |
| 79 |
If you want to test handling of specific flags, see the run_ice function. |
| 80 |
If you want more complex/specific tests, you can also check logs using |
| 81 |
the various check_log_* functions. |
| 82 |
|
| 83 |
Common log messages: |
| 84 |
(icecc.log) "Have to use host 127.0.0.1:" - will be built remotely (one one of the remote hosts) |
| 85 |
(icecc.log) "Have to use host 127.0.0.1:10246" - will be built remotely on remoteice1 |
| 86 |
(icecc.log) "Have to use host 127.0.0.1:10247" - will be built remotely on remoteice1 |
| 87 |
(icecc.log) "building myself, but telling localhost" - job could be built remotely, but |
| 88 |
the scheduler selected the local daemon for the build |
| 89 |
(icecc.log) "<building_local>" - the job is forced to be built locally (cannot be built remotely) |
| 90 |
|
| 91 |
|
| 92 |
Internals: |
| 93 |
========== |
| 94 |
|
| 95 |
For most tests, there is a new scheduler run and three daemons, one (localice) |
| 96 |
that serves as the "local" daemon that icecc uses, and two more (remoteice1/2) |
| 97 |
that serve as "remote" daemons that which remote jobs will be distributed. |
| 98 |
|
| 99 |
In order not to interfere with system Icecream, they all use different ports |
| 100 |
and a special socket is used for communication with localice daemon. |
| 101 |
All daemons are set with 2 jobs max, in order to have reliable results |
| 102 |
when testing paralellism. |
| 103 |
|
| 104 |
ICECC_PREFERRED_HOST is used when forcing whether a job should be built |
| 105 |
locally or remotely. The localice daemon also has ICECC_TEST_REMOTEBUILD=1 |
| 106 |
to avoid building locally when it in fact should forward to "remote" daemons |
| 107 |
even though they are technically local. |