Debian Codegraph @

tests/README

raw
1 Testuite for Icecream
2 =====================
3
4 In this directory are tests for Icecream.
5
6 These tests run Icecream binaries and test them in practice in a testing setup.
7 This makes these tests a bit slower than unit tests, but it allows testing
8 functionality that would be hard or impossible to test using unit tests.
9
10 How to run the tests:
11 =====================
12
13 The tests do not conflict with system Icecream, so it is possible to compile
14 and test while system Icecream is active.
15
16 Since Icecream hardcodes locations of some binaries, it is necessary to
17 install Icecream for the tests (really install, no $DESTDIR). It is however
18 not necessary to do a system install, simply install somewhere. If you want
19 to test also remote builds (recommended), it is necessary for the install to have
20 the capability to chroot. This is most easily done by giving the CAP_SYS_CHROOT
21 Linux capability to the iceccd binary (which is done is triggered to be done
22 automatically by "make test" using sudo).
23
24 An example of building Icecream and testing it:
25
26 ./configure --prefix=$HOME/iceinstall
27 make test
28
29 The 'make test' step roughly performs:
30
31 make
32 make install
33 sudo /sbin/setcap cap_sys_chroot+ep ${prefix}/sbin/iceccd
34 cd tests
35 ./test.sh ${prefix} ${builddir}/tests/results --builddir=${builddir}/tests
36
37 This will build and install Icecream into $HOME/iceinstall prefix and all
38 logs and temporary files will be in ${builddir}/tests/results. It is also possible to use
39 'filecap' from cap-ng instead of 'setcap', but 'filecap' does not report failures.
40
41 Note that the limitations (real install needed in a helper location, chroot capability)
42 mean that these tests are not suitable to be run as a part of building binary
43 packages, they are meant to be run by developers.
44
45 If a test fails, all relevant files should be in the log and temporary directory
46 passed to test.sh . The {daemon}.log files contain logs relevant only to the last run test,
47 files {daemon}_all.log include complete logs (up to but not including the failed test, if any).
48
49 Exit status of test.sh is 0 for all tests passing, 1 for all tests passing but some being
50 skipped and 2 for errors.
51
52 If you want to use icemon for the tests, use 'ICECC_SCHEDULER=localhost:8767 icemon'. Note that
53 icemon needs to be built with recent (Feb 2014) libicecc library for this to work.
54
55
56 Valgrind:
57 =========
58
59 It is possible to pass --valgrind to test.sh in order to test (almost) everything with Valgrind.
60 Valgrind may be also automatically invoked using:
61
62 make test VALGRIND=1
63
64 Note that in order to be able to chroot, it is necessary to give the capability to the actual
65 Valgrind binary instead of iceccd (this is because this binary is what actually runs as
66 the process):
67
68 sudo /sbin/setcap cap_sys_chroot+ep /usr/lib/valgrind/memcheck-x86-linux
69
70 Do not forget to reset it back when done:
71
72 sudo /sbin/setcap -r /usr/lib/valgrind/memcheck-x86-linux
73
74
75 Adding new tests:
76 =================
77
78 The test.sh script is hopefully straightforward and commented enough.
79 If you want to test handling of specific flags, see the run_ice function.
80 If you want more complex/specific tests, you can also check logs using
81 the various check_log_* functions.
82
83 Common log messages:
84 (icecc.log) "Have to use host 127.0.0.1:" - will be built remotely (one one of the remote hosts)
85 (icecc.log) "Have to use host 127.0.0.1:10246" - will be built remotely on remoteice1
86 (icecc.log) "Have to use host 127.0.0.1:10247" - will be built remotely on remoteice1
87 (icecc.log) "building myself, but telling localhost" - job could be built remotely, but
88 the scheduler selected the local daemon for the build
89 (icecc.log) "<building_local>" - the job is forced to be built locally (cannot be built remotely)
90
91
92 Internals:
93 ==========
94
95 For most tests, there is a new scheduler run and three daemons, one (localice)
96 that serves as the "local" daemon that icecc uses, and two more (remoteice1/2)
97 that serve as "remote" daemons that which remote jobs will be distributed.
98
99 In order not to interfere with system Icecream, they all use different ports
100 and a special socket is used for communication with localice daemon.
101 All daemons are set with 2 jobs max, in order to have reliable results
102 when testing paralellism.
103
104 ICECC_PREFERRED_HOST is used when forcing whether a job should be built
105 locally or remotely. The localice daemon also has ICECC_TEST_REMOTEBUILD=1
106 to avoid building locally when it in fact should forward to "remote" daemons
107 even though they are technically local.