| 1 |
.. _coverage: |
| 2 |
|
| 3 |
============= |
| 4 |
Test Coverage |
| 5 |
============= |
| 6 |
|
| 7 |
This assumes you have a single user development environment where you can |
| 8 |
already build and install Cyrus, and run the CUnit and Cassandane tests. It |
| 9 |
also assumes your Cyrus install and Cassandane setup do not use "destdir", |
| 10 |
and that your compiler is GCC. |
| 11 |
|
| 12 |
We'll be tinkering with group membership and file permissions, so proceed |
| 13 |
with caution on multi-user systems. |
| 14 |
|
| 15 |
When a coverage-enabled binary runs, it writes coverage data into ``foo.gcda`` |
| 16 |
files alongside the source files. CUnit runs as you, but Cassandane runs as |
| 17 |
cyrus, so we need to arrange for both these users to be able to write to the |
| 18 |
source directory. We'll do that using group memberships and the |
| 19 |
group-writeable file mode bit. |
| 20 |
|
| 21 |
One-time setup |
| 22 |
============== |
| 23 |
|
| 24 |
Group membership |
| 25 |
---------------- |
| 26 |
|
| 27 |
1. Add the "cyrus" user account to your own user group |
| 28 |
2. Perhaps: also add your user account to the "cyrus" user's group. This might |
| 29 |
be "cyrus" or "mail" depending on how you set your system up. I can't |
| 30 |
remember if this is actually necessary for coverage, or if I have it for |
| 31 |
something else, so skip it unless it becomes necessary (and update this |
| 32 |
doc!) |
| 33 |
|
| 34 |
File permissions |
| 35 |
---------------- |
| 36 |
|
| 37 |
1. Change into your cyrus-imapd directory: ``cd ~/path/to/cyrus-imapd`` |
| 38 |
2. Start from a clean state: ``git clean -xfd`` |
| 39 |
3. Set the group-writeable bit on everything: ``chmod -R g+w .`` |
| 40 |
4. Allow the group-writeable bit on new files you create: ``umask 0002`` |
| 41 |
5. Add that ``umask 0002`` line to your .bashrc or equivalent too, otherwise |
| 42 |
you'll have to remember to fix up file permissions every time you want to |
| 43 |
make a coverage report |
| 44 |
|
| 45 |
Dependencies |
| 46 |
------------ |
| 47 |
|
| 48 |
You'll need the ``lcov`` and ``genhtml`` tools for producing human-readable |
| 49 |
reports. On Debian, these are both found in the ``lcov`` package. |
| 50 |
|
| 51 |
Preparing a coverage report |
| 52 |
=========================== |
| 53 |
|
| 54 |
Compile Cyrus and run CUnit tests |
| 55 |
--------------------------------- |
| 56 |
|
| 57 |
The collection of coverage data slows things down, and it might also log a lot |
| 58 |
of complaints about overwriting old coverage data, or being unable to. So I |
| 59 |
do not recommend routinely compiling with coverage enabled -- only do this when |
| 60 |
you're preparing a coverage report. |
| 61 |
|
| 62 |
1. Change into your cyrus-imapd directory: ``cd ~/path/to/cyrus-imapd`` |
| 63 |
2. Start from a clean state: ``git clean -xfd`` |
| 64 |
3. Configure Cyrus, using your usual configure options, plus |
| 65 |
``--enable-coverage`` |
| 66 |
4. Compile Cyrus: ``make -j4`` |
| 67 |
5. Run the CUnit tests: ``make -j4 check`` |
| 68 |
6. Install Cyrus (might need sudo): ``make install`` |
| 69 |
|
| 70 |
Run Cassandane |
| 71 |
-------------- |
| 72 |
|
| 73 |
1. Run Cassandane on the installed Cyrus as you usually would |
| 74 |
|
| 75 |
Generate report |
| 76 |
--------------- |
| 77 |
|
| 78 |
I'd suggest making a script to automate this part. I use one like `this |
| 79 |
<https://github.com/elliefm/cyrus-build-tools/blob/master/cyrus-coverage>`_ |
| 80 |
|
| 81 |
1. Change into your cyrus-imapd directory: ``cd ~/path/to/cyrus-imapd`` |
| 82 |
2. Some of the ``foo.gcda`` files will be owned by your user (from the CUnit |
| 83 |
run), some will be owned by the cyrus user (from the Cassandane run). |
| 84 |
You can use something like this to reclaim the ownership (if your user:group |
| 85 |
is ellie:ellie):: |
| 86 |
|
| 87 |
find . -name \*.gcda -not -user ellie -execdir sudo chown ellie:ellie "{}" + |
| 88 |
|
| 89 |
3. If you want to keep accumulating results, you'll need to ensure the Cyrus |
| 90 |
user can still write to those files. I don't know if this is useful, but |
| 91 |
something like this will do it: |
| 92 |
``find . -name \*.gcda -execdir chmod g+rw "{}" +`` |
| 93 |
4. Process all those ``foo.gcda`` files into an intermediate form: |
| 94 |
``lcov --directory . -c -o coverage.info`` |
| 95 |
5. Strip out unit test and external library clutter: |
| 96 |
``lcov --remove coverage.info "cunit/*" "/usr/*" -o coverage.info`` |
| 97 |
6. Generate HTML: |
| 98 |
``genhtml -o coverage coverage.info`` |
| 99 |
7. You can now open that report in your browser. Something like this will |
| 100 |
give you a link to copy and paste: |
| 101 |
``echo file://$PWD/coverage/index.html`` |