| 1 |
There's a booth-test package which contains two types of tests. |
| 2 |
|
| 3 |
It installs the necessary files into `/usr/share/booth/tests`. |
| 4 |
|
| 5 |
=== Live tests (booth operation) |
| 6 |
|
| 7 |
BEWARE: Run this with _test_ clusters only! |
| 8 |
|
| 9 |
The live testing utility tests booth operation using the given |
| 10 |
`booth.conf`: |
| 11 |
|
| 12 |
$ /usr/share/booth/tests/test/live_test.sh booth.conf |
| 13 |
|
| 14 |
It is possible to run only specific tests. Run the script without |
| 15 |
arguments to see usage and the list of tests and netem network |
| 16 |
emulation functions. |
| 17 |
|
| 18 |
There are some restrictions on how booth.conf is formatted. |
| 19 |
There may be several tickets defined and all of them will be |
| 20 |
tested, one after another (they will be tested separately). |
| 21 |
The tickets must have expire and timeout parameters configured. |
| 22 |
|
| 23 |
Example booth.conf: |
| 24 |
|
| 25 |
------------ |
| 26 |
transport="UDP" |
| 27 |
port="9929" |
| 28 |
arbitrator="10.2.12.53" |
| 29 |
arbitrator="10.2.13.82" |
| 30 |
site="10.2.12.101" |
| 31 |
site="10.2.13.101" |
| 32 |
site="10.121.187.99" |
| 33 |
|
| 34 |
ticket="ticket-A" |
| 35 |
expire = 30 |
| 36 |
timeout = 3 |
| 37 |
retries = 3 |
| 38 |
before-acquire-handler = /usr/share/booth/service-runnable d-src1 |
| 39 |
------------ |
| 40 |
|
| 41 |
A split brain condition is also tested. For that to work, all |
| 42 |
sites need `iptables` installed. The supplied script `booth_path` |
| 43 |
is used to manipulate iptables rules. |
| 44 |
|
| 45 |
==== Pacemaker configuration |
| 46 |
|
| 47 |
This is a sample pacemaker configuration for a single-node |
| 48 |
cluster: |
| 49 |
|
| 50 |
primitive booth ocf:pacemaker:booth-site |
| 51 |
primitive d-src1 ocf:heartbeat:Dummy |
| 52 |
rsc_ticket global-d-src1 ticket-A: d-src1 |
| 53 |
|
| 54 |
Additionally, you may also add an ocf:booth:sharedrsc resource to |
| 55 |
also check that the ticket is granted always to only one site: |
| 56 |
|
| 57 |
primitive shared ocf:booth:sharedrsc \ |
| 58 |
params dir="10.2.13.82:/var/tmp/boothtestdir" |
| 59 |
rsc_ticket global-shared ticket-A: shared |
| 60 |
|
| 61 |
Please adjust to your environment. |
| 62 |
|
| 63 |
==== Network environment emulation |
| 64 |
|
| 65 |
To introduce packet loss or network delays, set the NETEM_ENV |
| 66 |
environment variable. There are currently three netem network |
| 67 |
emulation settings supported: |
| 68 |
|
| 69 |
- loss: all servers emulate packet loss (30% by default) |
| 70 |
|
| 71 |
- single_loss: the first site in the configuration emulates |
| 72 |
packet loss (30% by default) |
| 73 |
|
| 74 |
- net_delay: all servers emulate packet delay (100ms by default |
| 75 |
with random variation of 10%) |
| 76 |
|
| 77 |
The settings can be supplied by adding ':' to the emulator name. |
| 78 |
For instance: |
| 79 |
|
| 80 |
# NETEM_ENV=loss:50 /usr/share/booth/tests/test/live_test.sh booth.conf |
| 81 |
|
| 82 |
It is not necessary to run the test script on one of the sites. |
| 83 |
Just copy the script and make the test `booth.conf` available |
| 84 |
locally: |
| 85 |
|
| 86 |
$ scp testsite:/usr/share/booth/tests/test/live_test.sh . |
| 87 |
$ scp testsite:/etc/booth/booth.conf . |
| 88 |
$ sh live_test.sh booth.conf |
| 89 |
|
| 90 |
You need at least two sites and one arbitrator. |
| 91 |
|
| 92 |
The configuration can contain just one ticket. |
| 93 |
|
| 94 |
It is not necessary to configure the `before-acquire-handler`. |
| 95 |
|
| 96 |
Notes: |
| 97 |
|
| 98 |
- (BEWARE!) the supplied configuration files is copied to |
| 99 |
/etc/booth/booth.conf to all sites/arbitrators thus overwriting |
| 100 |
any existing configuration |
| 101 |
|
| 102 |
- the utility uses ssh to manage booth at all sites/arbitrators |
| 103 |
and logs in as user `root` |
| 104 |
|
| 105 |
- it is required that ssh public authentication works without |
| 106 |
providing the passphrase (otherwise it is impractical) |
| 107 |
|
| 108 |
- the log file is ./test_booth.log (it is actually a shell trace, |
| 109 |
with timestamps if you're running bash) |
| 110 |
|
| 111 |
- in case one of the tests fail, hb_report is created |
| 112 |
|
| 113 |
If you want to open a bug report, please attach all hb_reports |
| 114 |
and `test_booth.log`. |
| 115 |
|
| 116 |
|
| 117 |
|
| 118 |
|
| 119 |
=== Simple tests (commandline, config file) |
| 120 |
|
| 121 |
|
| 122 |
Run (as non-root) |
| 123 |
|
| 124 |
# make check |
| 125 |
|
| 126 |
or |
| 127 |
|
| 128 |
# make test/runtests.py |
| 129 |
# python test/runtests.py |
| 130 |
|
| 131 |
to run the tests written in python. |
| 132 |
|
| 133 |
It is also possible to run the tests as a root when |
| 134 |
"--allow-root-user" parameter is used or if |
| 135 |
the BOOTH_RUNTESTS_ROOT_USER environment variable is defined. |
| 136 |
|
| 137 |
By default tests uses TCP port based on current PID in range |
| 138 |
from 9929 to 10937 to allow running multiple instances in parallel. |
| 139 |
It is possible to use "--single-instance" parameter or define |
| 140 |
BOOTH_RUNTESTS_SINGLE_INSTANCE environment variable to make tests use |
| 141 |
only single port (9929), but parallel instances will fail. |
| 142 |
|
| 143 |
|
| 144 |
# vim: set ft=asciidoc : |