| 1 |
\chapter{\bacula{} Regression Testing} |
| 2 |
\label{blb:TheChapterStart8} |
| 3 |
\index{Testing!\bacula{} Regression} |
| 4 |
\index{\bacula{} Regression Testing} |
| 5 |
|
| 6 |
\section{Setting up Regession Testing} |
| 7 |
\index{Setting up Regession Testing} |
| 8 |
|
| 9 |
This document is intended mostly for developers who wish to ensure that their |
| 10 |
changes to \bacula{} don't introduce bugs in the base code. However, you |
| 11 |
don't need to be a developer to run the regression scripts, and we |
| 12 |
recommend them before putting your system into production, and before each |
| 13 |
upgrade, especially if you build from source code. They are |
| 14 |
simply shell scripts that drive \bacula{} through bconsole and then typically |
| 15 |
compare the input and output with \btool{diff}. |
| 16 |
|
| 17 |
You can find the existing regression scripts in the \bacula{} developer's |
| 18 |
\btool{git} repository on SourceForge. We strongly recommend that you |
| 19 |
\textbf{clone} the repository because afterwards, you can easily get pull the |
| 20 |
updates that have been made. |
| 21 |
|
| 22 |
To get started, we recommend that you create a directory named |
| 23 |
\bdirectoryname{bacula}, under which you will put the current source code and the current |
| 24 |
set of regression scripts. Below, we will describe how to set this up. |
| 25 |
|
| 26 |
The top level directory that we call \bdirectoryname{bacula} can be named anything you |
| 27 |
want. Note, all the standard regression scripts run as non-root and can be |
| 28 |
run on the same machine as a production \bacula{} system (the developers run |
| 29 |
it this way). |
| 30 |
|
| 31 |
To create the directory structure for the current trunk and to |
| 32 |
clone the repository, do the following (note, we assume you |
| 33 |
are working in your home directory in a non-root account): |
| 34 |
|
| 35 |
\footnotesize |
| 36 |
\begin{bVerbatim} |
| 37 |
git clone http://git.bacula.org/bacula.git bacula |
| 38 |
\end{bVerbatim} |
| 39 |
|
| 40 |
This will create the directory \bdirectoryname{bacula} and populate it with |
| 41 |
three directories: \bdirectoryname{bacula}, \bdirectoryname{gui}, and \bdirectoryname{regress}. |
| 42 |
\bdirectoryname{bacula} contains the \bacula{} source code; \bdirectoryname{gui} contains |
| 43 |
certain \acs{GUI} programs that you will not need, and \bdirectoryname{regress} contains |
| 44 |
all the regression scripts. The above should be needed only |
| 45 |
once. Thereafter to update to the latest code, you do: |
| 46 |
|
| 47 |
\begin{bVerbatim} |
| 48 |
cd bacula |
| 49 |
git pull |
| 50 |
\end{bVerbatim} |
| 51 |
|
| 52 |
\smallskip |
| 53 |
|
| 54 |
There are two different aspects of regression testing that this document will |
| 55 |
discuss: 1. Running the Regression Script, 2. Writing a Regression test. |
| 56 |
|
| 57 |
\section{Running the Regression Script} |
| 58 |
\index{Running the Regression Script} |
| 59 |
\index{Script!Running the Regression} |
| 60 |
|
| 61 |
There are a number of different tests that may be run, such as: the standard |
| 62 |
set that uses disk Volumes and runs under any userid; a small set of tests |
| 63 |
that write to tape; another set of tests where you must be root to run them. |
| 64 |
Normally, I run all my tests as non-root and very rarely run the root |
| 65 |
tests. The tests vary in length, and running the full tests including disk |
| 66 |
based testing, tape based testing, autochanger based testing, and multiple |
| 67 |
drive autochanger based testing can take 3 or 4 hours. |
| 68 |
|
| 69 |
\subsection{Setting the Configuration Parameters} |
| 70 |
\index{Setting the Configuration Parameters} |
| 71 |
\index{Parameters!Setting the Configuration} |
| 72 |
|
| 73 |
There is nothing you need to change in the source directory. |
| 74 |
|
| 75 |
To begin: |
| 76 |
|
| 77 |
\begin{bVerbatim} |
| 78 |
cd bacula/regress |
| 79 |
\end{bVerbatim} |
| 80 |
|
| 81 |
|
| 82 |
The |
| 83 |
very first time you are going to run the regression scripts, you will |
| 84 |
need to create a custom config file for your system. |
| 85 |
We suggest that you start by: |
| 86 |
|
| 87 |
\begin{bVerbatim} |
| 88 |
cp prototype.conf config |
| 89 |
\end{bVerbatim} |
| 90 |
|
| 91 |
Then you can edit the \bfilename{config} file directly. |
| 92 |
|
| 93 |
\begin{bVerbatim} |
| 94 |
# Where to get the source to be tested |
| 95 |
BACULA_SOURCE="${HOME}/bacula/bacula" |
| 96 |
|
| 97 |
# Where to send email !!!!! Change me !!!!!!! |
| 98 |
[email protected] |
| 99 |
SMTP_HOST="localhost" |
| 100 |
|
| 101 |
TAPE_DRIVE="/dev/nst0" |
| 102 |
# if you don't have an autochanger set AUTOCHANGER to /dev/null |
| 103 |
AUTOCHANGER="/dev/sg0" |
| 104 |
# For two drive tests -- set to /dev/null if you do not have it |
| 105 |
TAPE_DRIVE1="/dev/null" |
| 106 |
|
| 107 |
# This must be the path to the autochanger including its name |
| 108 |
AUTOCHANGER_PATH="/usr/sbin/mtx" |
| 109 |
|
| 110 |
# Set what backend to use "postresql" "mysql" or "sqlite3" |
| 111 |
DBTYPE="postgresql" |
| 112 |
|
| 113 |
# Set your database here |
| 114 |
#WHICHDB="--with-${DBTYPE}=${SQLITE3_DIR}" |
| 115 |
WHICHDB="--with-${DBTYPE}" |
| 116 |
|
| 117 |
# Set this to "--with-tcp-wrappers" or "--without-tcp-wrappers" |
| 118 |
TCPWRAPPERS="--with-tcp-wrappers" |
| 119 |
|
| 120 |
# Set this to "" to disable OpenSSL support, "--with-openssl=yes" |
| 121 |
# to enable it, or provide the path to the OpenSSL installation, |
| 122 |
# eg "--with-openssl=/usr/local" |
| 123 |
OPENSSL="--with-openssl" |
| 124 |
|
| 125 |
# You may put your real host name here, but localhost or 127.0.0.1 |
| 126 |
# is valid also and it has the advantage that it works on a |
| 127 |
# non-networked machine |
| 128 |
HOST="localhost" |
| 129 |
\end{bVerbatim} |
| 130 |
|
| 131 |
\begin{bdescription} |
| 132 |
\item [BACULA\_SOURCE] should be the full path to the \bacula{} source code |
| 133 |
that you wish to test. It will be loaded configured, compiled, and |
| 134 |
installed with the "make setup" command, which needs to be done only |
| 135 |
once each time you change the source code. |
| 136 |
|
| 137 |
\item [EMAIL] should be your email addres. Please remember to change this |
| 138 |
or I will get a flood of unwanted messages. You may or may not want to see |
| 139 |
these emails. In my case, I don't need them so I direct it to the bit bucket. |
| 140 |
|
| 141 |
\item [SMTP\_HOST] defines where your \acs{SMTP} server is. |
| 142 |
|
| 143 |
|
| 144 |
\item [TAPE\_DRIVE] is the full path to your tape drive. The base set of |
| 145 |
regression tests do not use a tape, so this is only important if you want to |
| 146 |
run the full tests. Set this to \bdirectoryname{/dev/null} if you do not have a tape drive. |
| 147 |
|
| 148 |
\item [TAPE\_DRIVE1] is the full path to your second tape drive, if |
| 149 |
have one. The base set of |
| 150 |
regression tests do not use a tape, so this is only important if you want to |
| 151 |
run the full two drive tests. Set this to \bdirectoryname{/dev/null} if you do not have a |
| 152 |
second tape drive. |
| 153 |
|
| 154 |
\item [AUTOCHANGER] is the name of your autochanger control device. Set this to |
| 155 |
\bdirectoryname{/dev/null} if you do not have an autochanger. |
| 156 |
|
| 157 |
\item [AUTOCHANGER\_PATH] is the full path including the program name for |
| 158 |
your autochanger program (normally \mtx{}). Leave the default value if you |
| 159 |
do not have one. |
| 160 |
|
| 161 |
\item [TCPWRAPPERS] defines whether or not you want the \btool{./configure} |
| 162 |
to be performed with \texttt{tcpwrappers} enabled. |
| 163 |
|
| 164 |
\item [OPENSSL] used to enable/disable \acs{SSL} support for \bacula{} |
| 165 |
communications and data encryption. |
| 166 |
|
| 167 |
\item [HOST] is the hostname that it will use when building the |
| 168 |
scripts. The \bacula{} daemons will be named \bbracket{HOST}-dir, \bbracket{HOST}-fd, |
| 169 |
\ldots{} It is also the name of the \texttt{HOST} machine that to connect to the |
| 170 |
daemons by the network. Hence the name should either be your real |
| 171 |
hostname (with an appropriate \acs{DNS} or \bfilename{/etc/hosts} entry) or \bfilename{localhost} as it is in the default file. |
| 172 |
|
| 173 |
\item [bin] is the binary location. |
| 174 |
|
| 175 |
\item [scripts] is the bacula scripts location (where we could find |
| 176 |
database creation script, autochanger handler, etc.) |
| 177 |
\end{bdescription} |
| 178 |
|
| 179 |
\subsection{Building the Test \bacula{}} |
| 180 |
\index{Building the Test \bacula{}} |
| 181 |
\index{\bacula{}!Building the Test} |
| 182 |
|
| 183 |
Once the above variables are set, you can build the setup by entering: |
| 184 |
|
| 185 |
\begin{bVerbatim} |
| 186 |
make setup |
| 187 |
\end{bVerbatim} |
| 188 |
|
| 189 |
This will setup the regression testing and you should not need to |
| 190 |
do this again unless you want to change the database or other regression |
| 191 |
configuration parameters. |
| 192 |
|
| 193 |
|
| 194 |
\subsection{Setting up your SQL engine} |
| 195 |
\index{Setting up your SQL engine} |
| 196 |
If you are using \sqlite{} or \sqlite{}3, there is nothing more to do; you can |
| 197 |
simply run the tests as described in the next section. |
| 198 |
|
| 199 |
If you are using \mysql{} or \postgresql{}, you will need to establish an |
| 200 |
account with your database engine for the user name \texttt{regress} and |
| 201 |
you will need to manually create a database named \textbf{regress} that can be |
| 202 |
used by user name \texttt{regress}, which means you will have to give the user |
| 203 |
regress sufficient permissions to use the database named \texttt{regress}. |
| 204 |
There is no password on the \texttt{regress} account. |
| 205 |
|
| 206 |
You have probably already done this procedure for the user name and |
| 207 |
database named \texttt{bacula}. If not, the manual describes roughly how to |
| 208 |
do it, and the scripts in \bdirectoryname{bacula/regress/build/src/cats} named |
| 209 |
\bfilename{create\_mysql\_database}, \bfilename{create\_postgresql\_database}, |
| 210 |
\bfilename{grant\_mysql\_privileges}, and \bfilename{grant\_postgresql\_privileges} |
| 211 |
may be of a help to you. |
| 212 |
|
| 213 |
Generally, to do the above, you will need to run under \texttt{root} to |
| 214 |
be able to create databases and modify permissions within \mysql{} and |
| 215 |
\postgresql{}. |
| 216 |
|
| 217 |
It is possible to configure \mysql{} access for database accounts that |
| 218 |
require a password to be supplied. This can be done by creating a \bfilename{~/.my.cnf} |
| 219 |
file which supplies the credentials by default to the \mysql{} commandline |
| 220 |
utilities. |
| 221 |
|
| 222 |
\begin{bVerbatim} |
| 223 |
[client] |
| 224 |
host = localhost |
| 225 |
user = regress |
| 226 |
password = asecret |
| 227 |
\end{bVerbatim} |
| 228 |
|
| 229 |
A similar technique can be used \postgresql{} regression testing where the |
| 230 |
database is configured to require a password. The \bfilename{~/.pgpass} file should |
| 231 |
contain a line with the database connection properties. |
| 232 |
|
| 233 |
\begin{bVerbatim} |
| 234 |
hostname:port:database:username:password |
| 235 |
\end{bVerbatim} |
| 236 |
|
| 237 |
\subsection{Running the Disk Only Regression} |
| 238 |
\index{Regression!Running the Disk Only} |
| 239 |
\index{Running the Disk Only Regression} |
| 240 |
|
| 241 |
The simplest way to copy the source code, configure it, compile it, link |
| 242 |
it, and run the tests is to use a helper script: |
| 243 |
|
| 244 |
\begin{bVerbatim} |
| 245 |
./do_disk |
| 246 |
\end{bVerbatim} |
| 247 |
|
| 248 |
|
| 249 |
|
| 250 |
|
| 251 |
This will run the base set of tests using disk Volumes. |
| 252 |
If you are testing on a |
| 253 |
non-Linux machine several of the of the tests may not be run. In any case, |
| 254 |
as we add new tests, the number will vary. It will take about 1 hour |
| 255 |
and you don't need to be \texttt{root} |
| 256 |
to run these tests (I run under my regular userid). The result should be |
| 257 |
something similar to: |
| 258 |
|
| 259 |
\begin{bVerbatim} |
| 260 |
Test results |
| 261 |
===== auto-label-test OK 12:31:33 ===== |
| 262 |
===== backup-bacula-test OK 12:32:32 ===== |
| 263 |
===== bextract-test OK 12:33:27 ===== |
| 264 |
===== bscan-test OK 12:34:47 ===== |
| 265 |
===== bsr-opt-test OK 12:35:46 ===== |
| 266 |
===== compressed-test OK 12:36:52 ===== |
| 267 |
===== compressed-encrypt-test OK 12:38:18 ===== |
| 268 |
===== concurrent-jobs-test OK 12:39:49 ===== |
| 269 |
===== data-encrypt-test OK 12:41:11 ===== |
| 270 |
===== encrypt-bug-test OK 12:42:00 ===== |
| 271 |
===== fifo-test OK 12:43:46 ===== |
| 272 |
===== backup-bacula-fifo OK 12:44:54 ===== |
| 273 |
===== differential-test OK 12:45:36 ===== |
| 274 |
===== four-concurrent-jobs-test OK 12:47:39 ===== |
| 275 |
===== four-jobs-test OK 12:49:22 ===== |
| 276 |
===== incremental-test OK 12:50:38 ===== |
| 277 |
===== query-test OK 12:51:37 ===== |
| 278 |
===== recycle-test OK 12:53:52 ===== |
| 279 |
===== restore2-by-file-test OK 12:54:53 ===== |
| 280 |
===== restore-by-file-test OK 12:55:40 ===== |
| 281 |
===== restore-disk-seek-test OK 12:56:29 ===== |
| 282 |
===== six-vol-test OK 12:57:44 ===== |
| 283 |
===== span-vol-test OK 12:58:52 ===== |
| 284 |
===== sparse-compressed-test OK 13:00:00 ===== |
| 285 |
===== sparse-test OK 13:01:04 ===== |
| 286 |
===== two-jobs-test OK 13:02:39 ===== |
| 287 |
===== two-vol-test OK 13:03:49 ===== |
| 288 |
===== verify-vol-test OK 13:04:56 ===== |
| 289 |
===== weird-files2-test OK 13:05:47 ===== |
| 290 |
===== weird-files-test OK 13:06:33 ===== |
| 291 |
===== migration-job-test OK 13:08:15 ===== |
| 292 |
===== migration-jobspan-test OK 13:09:33 ===== |
| 293 |
===== migration-volume-test OK 13:10:48 ===== |
| 294 |
===== migration-time-test OK 13:12:59 ===== |
| 295 |
===== hardlink-test OK 13:13:50 ===== |
| 296 |
===== two-pool-test OK 13:18:17 ===== |
| 297 |
===== fast-two-pool-test OK 13:24:02 ===== |
| 298 |
===== two-volume-test OK 13:25:06 ===== |
| 299 |
===== incremental-2disk OK 13:25:57 ===== |
| 300 |
===== 2drive-incremental-2disk OK 13:26:53 ===== |
| 301 |
===== scratch-pool-test OK 13:28:01 ===== |
| 302 |
Total time = 0:57:55 or 3475 secs |
| 303 |
\end{bVerbatim} |
| 304 |
|
| 305 |
and the working tape tests are run with |
| 306 |
|
| 307 |
\begin{bVerbatim} |
| 308 |
make full_test |
| 309 |
\end{bVerbatim} |
| 310 |
|
| 311 |
|
| 312 |
\begin{bVerbatim} |
| 313 |
Test results |
| 314 |
|
| 315 |
===== Bacula tape test OK ===== |
| 316 |
===== Small File Size test OK ===== |
| 317 |
===== restore-by-file-tape test OK ===== |
| 318 |
===== incremental-tape test OK ===== |
| 319 |
===== four-concurrent-jobs-tape OK ===== |
| 320 |
===== four-jobs-tape OK ===== |
| 321 |
\end{bVerbatim} |
| 322 |
|
| 323 |
Each separate test is self contained in that it initializes to run \bacula{} from |
| 324 |
scratch (i.e. newly created database). It will also kill any \bacula{} session |
| 325 |
that is currently running. In addition, it uses ports 8101, 8102, and 8103 so |
| 326 |
that it does not intefere with a production system. |
| 327 |
|
| 328 |
Alternatively, you can do the \bfilename{./do\_disk} work by hand with: |
| 329 |
|
| 330 |
\begin{bVerbatim} |
| 331 |
make setup |
| 332 |
\end{bVerbatim} |
| 333 |
|
| 334 |
The above will then copy the source code within |
| 335 |
the regression tree (in directory \bdirectoryname{regress/build}), configure it, and build it. |
| 336 |
There should be no errors. If there are, please correct them before |
| 337 |
continuing. From this point on, as long as you don't change the \bacula{} |
| 338 |
source code, you should not need to repeat any of the above steps. If |
| 339 |
you pull down a new version of the source code, simply run \btool{make setup} |
| 340 |
again. |
| 341 |
|
| 342 |
|
| 343 |
Once \bacula{} is built, you can run the basic disk only \texttt{non-root} regression test |
| 344 |
by entering: |
| 345 |
|
| 346 |
\begin{bVerbatim} |
| 347 |
make test |
| 348 |
\end{bVerbatim} |
| 349 |
|
| 350 |
|
| 351 |
\subsection{Other Tests} |
| 352 |
\index{Other Tests} |
| 353 |
\index{Tests!Other} |
| 354 |
|
| 355 |
There are a number of other tests that can be run as well. All the tests are a |
| 356 |
simply shell script keep in the regress directory. For example the \btool{make test} |
| 357 |
simply executes \btool{./all-non-root-tests}. The other tests, which |
| 358 |
are invoked by directly running the script are: |
| 359 |
|
| 360 |
\begin{bdescription} |
| 361 |
|
| 362 |
\item [all\_non-root-tests] |
| 363 |
\index{all\_non-root-tests} |
| 364 |
All non-tape tests not requiring \texttt{root}. This is the standard set of tests, |
| 365 |
that in general, backup some data, then restore it, and finally compares the |
| 366 |
restored data with the original data. |
| 367 |
|
| 368 |
\item [all-root-tests] |
| 369 |
\index{all-root-tests} |
| 370 |
All non-tape tests requiring \texttt{root} permission. These are a relatively small |
| 371 |
number of tests that require running as \texttt{root}. The amount of data backed up |
| 372 |
can be quite large. For example, one test backs up \bdirectoryname{/usr}, another backs up |
| 373 |
\bdirectoryname{/etc}. One or more of these tests reports an error -- I'll fix it one day. |
| 374 |
|
| 375 |
\item [all-non-root-tape-tests] |
| 376 |
\index{all-non-root-tape-tests} |
| 377 |
All tape test not requiring root. There are currently three tests, all run |
| 378 |
without being root, and backup to a tape. The first two tests use one volume, |
| 379 |
and the third test requires an autochanger, and uses two volumes. If you |
| 380 |
don't have an autochanger, then this script will probably produce an error. |
| 381 |
|
| 382 |
\item [all-tape-and-file-tests] |
| 383 |
\index{all-tape-and-file-tests} |
| 384 |
All tape and file tests not requiring \texttt{root}. This includes just about |
| 385 |
everything, and I don't run it very often. |
| 386 |
\end{bdescription} |
| 387 |
|
| 388 |
\subsection{If a Test Fails} |
| 389 |
\index{Fails!If a Test} |
| 390 |
\index{If a Test Fails} |
| 391 |
|
| 392 |
If you one or more tests fail, the line output will be similar to: |
| 393 |
|
| 394 |
\begin{bVerbatim} |
| 395 |
!!!!! concurrent-jobs-test failed!!! !!!!! |
| 396 |
\end{bVerbatim} |
| 397 |
|
| 398 |
If you want to determine why the test failed, you will need to rerun the |
| 399 |
script with the debug output turned on. You do so by defining the |
| 400 |
environment variable \textbf{\texttt{REGRESS\_DEBUG}} with commands such as: |
| 401 |
|
| 402 |
\begin{bVerbatim} |
| 403 |
REGRESS_DEBUG=1 |
| 404 |
export REGRESS_DEBUG |
| 405 |
\end{bVerbatim} |
| 406 |
|
| 407 |
Then from the \bdirectoryname{regress} directory (all regression scripts assume that |
| 408 |
you have \bdirectoryname{regress} as the current directory), enter: |
| 409 |
|
| 410 |
\begin{bVerbatim} |
| 411 |
tests/test-name |
| 412 |
\end{bVerbatim} |
| 413 |
|
| 414 |
where test-name should be the name of a test script -- for example: |
| 415 |
\btool{tests/backup-bacula-test}. |
| 416 |
|
| 417 |
\section{Testing a Binary Installation} |
| 418 |
\index{Test!Testing a Binary Installation} |
| 419 |
|
| 420 |
If you have installed your \bacula{} from a binary release such as (rpms or debs), |
| 421 |
you can still run regression tests on it. |
| 422 |
First, make sure that your regression \bfilename{config} file uses the same catalog backend as |
| 423 |
your installed binaries. Then define the variables \texttt{bin} and \texttt{scripts} variables |
| 424 |
in your config file. |
| 425 |
|
| 426 |
Example: |
| 427 |
\begin{bVerbatim} |
| 428 |
bin=/opt/bacula/bin |
| 429 |
scripts=/opt/bacula/scripts |
| 430 |
\end{bVerbatim} |
| 431 |
|
| 432 |
The \bfilename{./scripts/prepare-other-loc} will tweak the regress scripts to use |
| 433 |
your binary location. You will need to run it manually once before you run any |
| 434 |
regression tests. |
| 435 |
|
| 436 |
\begin{bVerbatim} |
| 437 |
$ ./scripts/prepare-other-loc |
| 438 |
$ ./tests/backup-bacula-test |
| 439 |
... |
| 440 |
\end{bVerbatim} |
| 441 |
|
| 442 |
All regression scripts must be run by hand or by calling the test scripts. |
| 443 |
These are principally scripts that begin with \textbf{all\_...} such as \btool{all\_disk\_tests}, |
| 444 |
\btool{./all\_test}, \ldots{} |
| 445 |
None of the |
| 446 |
\btool{./do\_disk}, \btool{./do\_all}, \btool{./nightly...} scripts will work. |
| 447 |
|
| 448 |
If you want to switch back to running the regression scripts from source, first |
| 449 |
remove the \texttt{bin} and \texttt{scripts} variables from your \bfilename{config} file and |
| 450 |
rerun the \btool{make setup} step. |
| 451 |
|
| 452 |
\section{Running a Single Test} |
| 453 |
\index{Running a Single Test} |
| 454 |
|
| 455 |
If you wish to run a single test, you can simply: |
| 456 |
|
| 457 |
\begin{bVerbatim} |
| 458 |
cd regress |
| 459 |
tests/<name-of-test> |
| 460 |
\end{bVerbatim} |
| 461 |
|
| 462 |
or, if the source code has been updated, you would do: |
| 463 |
|
| 464 |
\begin{bVerbatim} |
| 465 |
cd bacula |
| 466 |
git pull |
| 467 |
cd regress |
| 468 |
make setup |
| 469 |
tests/backup-to-null |
| 470 |
\end{bVerbatim} |
| 471 |
|
| 472 |
|
| 473 |
\section{Writing a Regression Test} |
| 474 |
\index{Test!Writing a Regression} |
| 475 |
\index{Writing a Regression Test} |
| 476 |
|
| 477 |
Any developer, who implements a major new feature, should write a regression |
| 478 |
test that exercises and validates the new feature. Each regression test is a |
| 479 |
complete test by itself. It terminates any running \bacula{}, initializes the |
| 480 |
database, starts \bacula{}, then runs the test by using the console program. |
| 481 |
|
| 482 |
\subsection{Running the Tests by Hand} |
| 483 |
\index{Hand!Running the Tests by} |
| 484 |
\index{Running the Tests by Hand} |
| 485 |
|
| 486 |
You can run any individual test by hand by cd'ing to the \bdirectoryname{regress} |
| 487 |
directory and entering: |
| 488 |
|
| 489 |
\begin{bVerbatim} |
| 490 |
tests/<test-name> |
| 491 |
\end{bVerbatim} |
| 492 |
|
| 493 |
\subsection{Directory Structure} |
| 494 |
\index{Structure!Directory} |
| 495 |
\index{Directory Structure} |
| 496 |
|
| 497 |
The directory structure of the regression tests is: |
| 498 |
|
| 499 |
\begin{bVerbatim} |
| 500 |
regress - Makefile, scripts to start tests |
| 501 |
|------ scripts - Scripts and conf files |
| 502 |
|-------tests - All test scripts are here |
| 503 |
| |
| 504 |
|------------------ -- All directories below this point are used |
| 505 |
| for testing, but are created from the |
| 506 |
| above directories and are removed with |
| 507 |
| "make distclean" |
| 508 |
| |
| 509 |
|------ bin - This is the install directory for |
| 510 |
| Bacula to be used testing |
| 511 |
|------ build - Where the Bacula source build tree is |
| 512 |
|------ tmp - Most temp files go here |
| 513 |
|------ working - Bacula working directory |
| 514 |
|------ weird-files - Weird files used in two of the tests. |
| 515 |
\end{bVerbatim} |
| 516 |
|
| 517 |
\subsection{Adding a New Test} |
| 518 |
\index{Adding a New Test} |
| 519 |
\index{Test!Adding a New} |
| 520 |
|
| 521 |
If you want to write a new regression test, it is best to start with one of |
| 522 |
the existing test scripts, and modify it to do the new test. |
| 523 |
|
| 524 |
When adding a new test, be extremely careful about adding anything to any of |
| 525 |
the daemons' configuration files. The reason is that it may change the prompts |
| 526 |
that are sent to the console. For example, adding a Pool means that the |
| 527 |
current scripts, which assume that \bacula{} automatically selects a Pool, will |
| 528 |
now be presented with a new prompt, so the test will fail. If you need to |
| 529 |
enhance the configuration files, consider making your own versions. |
| 530 |
|
| 531 |
Once written, if the test can be executed automatically, you can do a separated |
| 532 |
commit to schedule the new test via CDASH by editing the |
| 533 |
\texttt{regress/DartTestFile.txt.in} |
| 534 |
|
| 535 |
\subsection{Adding a Unittest} |
| 536 |
\index{Adding a Unittest} |
| 537 |
\index{Test!Adding a New} |
| 538 |
|
| 539 |
For low level operations, it is important to write unittests. They are usually |
| 540 |
directly included at the end of the file that has the code to test. It is possible |
| 541 |
to create a new file in \texttt{regress/src} or \texttt{bacula/src/tools}. |
| 542 |
|
| 543 |
The C code should be compiled with the appropriate Makefile rule (see \texttt{alist\_test}) |
| 544 |
for example. |
| 545 |
\begin{bVerbatim} |
| 546 |
mytest_test: Makefile libbac.la mytest.c unittests.o |
| 547 |
$(CXX) -DTEST_PROGRAM $(DEFS) $(DEBUG) -c $(CPPFLAGS) -I$(srcdir) -I$(basedir) $(DINCLUDE) $(CFLAGS) mytest.c -o mytest_test.o |
| 548 |
$(LIBTOOL_LINK) $(CXX) $(LDFLAGS) -L. -o $@ mytest_test.o unittests.o $(DLIB) -lbac -lm $(LIBS) $(OPENSSL_LIBS) |
| 549 |
$(LIBTOOL_INSTALL) $(INSTALL_PROGRAM) $@ $(DESTDIR)$(sbindir)/ |
| 550 |
\end{bVerbatim} |
| 551 |
|
| 552 |
The Bacula Unittest framework is a simple set of functions that helps to write |
| 553 |
a unittest. It has functions to |
| 554 |
\begin{itemize} |
| 555 |
\item Test variables |
| 556 |
\item Log messages |
| 557 |
\item Configure/Unconfigure Bacula tools such as mempool, lock manager |
| 558 |
\item Do some high level filesystem functions (mkdir, rmdir, stat) |
| 559 |
\item Ease some string operations |
| 560 |
\end{itemize} |
| 561 |
|
| 562 |
The following code explains how to use the Bacula Unittest framework: |
| 563 |
|
| 564 |
\begin{bVerbatim} |
| 565 |
int function_to_test(int parameter1, int parameter2) |
| 566 |
{ |
| 567 |
/* some interesting code */ |
| 568 |
return 1; |
| 569 |
} |
| 570 |
|
| 571 |
/* In this case, the test procedure is at the end of the file, |
| 572 |
* it can be also in a separated file |
| 573 |
*/ |
| 574 |
#ifdef TEST_PROGRAM |
| 575 |
#include "lib/unittests.h" |
| 576 |
|
| 577 |
int main(int argc, char **argv) |
| 578 |
{ |
| 579 |
Unittests tests("app_test"); |
| 580 |
tests.set_nb_tests(4); /* Plan the number of expected tests */ |
| 581 |
tests.configure(TEST_QUIET|TEST_PRINT_LOCAL); /* Configure */ |
| 582 |
|
| 583 |
log("Test a the function_to_test()"); |
| 584 |
is(function_to_test(1,2), 1, "Test function_to_test(1,2)"); |
| 585 |
ok(function_to_test(1,2) == 1, "Test function_to_test(1,2)"); |
| 586 |
isnt(function_to_test(3,4), 2, "Test function_to_test(3,4)"); |
| 587 |
nok(function_to_test(3,4) == 2, "Test function_to_test(3,4)"); |
| 588 |
|
| 589 |
return report(); |
| 590 |
} |
| 591 |
#endif /* TEST_PROGRAM */ |
| 592 |
\end{bVerbatim} |
| 593 |
|
| 594 |
The current options in the unittest library are: |
| 595 |
\begin{itemize} |
| 596 |
\item [TEST\_QUIET] Display only test in error |
| 597 |
\item [TEST\_PRINT\_LOCAL] Display the local variables after an error |
| 598 |
\end{itemize} |
| 599 |
|
| 600 |
These options can be set via an environment variable (UNITTEST\_TEST\_QUIET, UNITTEST\_TEST\_PRINT\_LOCAL). |
| 601 |
|
| 602 |
\subsection{Running a Test Under The Debugger} |
| 603 |
\index{Debugger} |
| 604 |
You can run a test under the debugger (actually run a \bacula{} daemon |
| 605 |
under the debugger) by first setting the environment variable |
| 606 |
\textbf{\texttt{REGRESS\_WAIT}} with commands such as: |
| 607 |
|
| 608 |
\begin{bVerbatim} |
| 609 |
REGRESS_WAIT=1 |
| 610 |
export REGRESS_WAIT |
| 611 |
\end{bVerbatim} |
| 612 |
|
| 613 |
Then executing the script. When the script prints the following line: |
| 614 |
|
| 615 |
\begin{bVerbatim} |
| 616 |
Start Bacula under debugger and enter anything when ready ... |
| 617 |
\end{bVerbatim} |
| 618 |
|
| 619 |
You start the \bacula{} component you want to run under the debugger in a |
| 620 |
different shell window. For example: |
| 621 |
|
| 622 |
\begin{bVerbatim} |
| 623 |
cd .../regress/bin |
| 624 |
gdb bacula-sd |
| 625 |
(possibly set breakpoints, ...) |
| 626 |
run -s -f |
| 627 |
\end{bVerbatim} |
| 628 |
|
| 629 |
Then enter any character in the window with the above message. |
| 630 |
An error message will appear saying that the daemon you are debugging |
| 631 |
is already running, which is the case. You can simply ignore the |
| 632 |
error message. |