| 1 |
.. _test-suite: |
| 2 |
|
| 3 |
============================================================================== |
| 4 |
libinput test suite |
| 5 |
============================================================================== |
| 6 |
|
| 7 |
libinput's primary test suite can be invoked with |
| 8 |
|
| 9 |
:: |
| 10 |
|
| 11 |
$ sudo ./builddir/libinput-test-suite |
| 12 |
|
| 13 |
When developing libinput, the ``libinput-test-suite`` should always be |
| 14 |
run to check for behavior changes and/or regressions. For quick iteration, |
| 15 |
the number of tests to run can be filtered, see :ref:`test-filtering`. |
| 16 |
This allows for developers to verify a subset of tests (e.g. |
| 17 |
touchpad tap-to-click) while hacking on that specific feature and only run |
| 18 |
the full suite when development is done finished. |
| 19 |
|
| 20 |
.. note:: The test suite relies on udev and the kernel, specifically uinput. |
| 21 |
It creates virtual input devices and replays the events. This may |
| 22 |
interfere with your running session. The test suite is not suitable |
| 23 |
for running inside containers. |
| 24 |
|
| 25 |
In addition, libinput ships with a set of (primarily janitorial) tests that |
| 26 |
must pass for any merge request. These tests are invoked by calling |
| 27 |
``meson test -C builddir`` (or ``ninja test``). The ``libinput-test-suite`` is |
| 28 |
part of that test set by default. |
| 29 |
|
| 30 |
The upstream CI runs all these tests and is run for every merge request. |
| 31 |
|
| 32 |
.. _test-job-control: |
| 33 |
|
| 34 |
------------------------------------------------------------------------------ |
| 35 |
Job control in the test suite |
| 36 |
------------------------------------------------------------------------------ |
| 37 |
|
| 38 |
The test suite runner has a make-like job control enabled by the ``-j`` or |
| 39 |
``--jobs`` flag and will fork off as many parallel processes as given by this |
| 40 |
flag. The default if unspecified is 8. When debugging a specific test case |
| 41 |
failure it is recommended to employ test filtures (see :ref:`test-filtering`) |
| 42 |
and disable parallel tests. The test suite automatically disables parallel |
| 43 |
make when run in gdb. |
| 44 |
|
| 45 |
.. _test-config: |
| 46 |
|
| 47 |
------------------------------------------------------------------------------ |
| 48 |
X.Org config to avoid interference |
| 49 |
------------------------------------------------------------------------------ |
| 50 |
|
| 51 |
uinput devices created by the test suite are usually recognised by X as |
| 52 |
input devices. All events sent through these devices will generate X events |
| 53 |
and interfere with your desktop. |
| 54 |
|
| 55 |
Copy the file ``$srcdir/test/50-litest.conf`` into your ``/etc/X11/xorg.conf.d`` |
| 56 |
and restart X. This will ignore any litest devices and thus not interfere |
| 57 |
with your desktop. |
| 58 |
|
| 59 |
.. _test-root: |
| 60 |
|
| 61 |
------------------------------------------------------------------------------ |
| 62 |
Permissions required to run tests |
| 63 |
------------------------------------------------------------------------------ |
| 64 |
|
| 65 |
Most tests require the creation of uinput devices and access to the |
| 66 |
resulting ``/dev/input/eventX`` nodes. Some tests require temporary udev rules. |
| 67 |
**This usually requires the tests to be run as root**. If not run as |
| 68 |
root, the test suite runner will exit with status 77, an exit status |
| 69 |
interpreted as "skipped". |
| 70 |
|
| 71 |
.. _test-filtering: |
| 72 |
|
| 73 |
------------------------------------------------------------------------------ |
| 74 |
Selective running of tests |
| 75 |
------------------------------------------------------------------------------ |
| 76 |
|
| 77 |
litest's tests are grouped into test groups, test names and devices. A test |
| 78 |
group is e.g. "touchpad:tap" and incorporates all tapping-related tests for |
| 79 |
touchpads. Each test function is (usually) run with one or more specific |
| 80 |
devices. The ``--list`` commandline argument shows the list of suites and |
| 81 |
tests. This is useful when trying to figure out if a specific test is |
| 82 |
run for a device. |
| 83 |
|
| 84 |
|
| 85 |
:: |
| 86 |
|
| 87 |
$ ./builddir/libinput-test-suite --list |
| 88 |
... |
| 89 |
pointer:left-handed: |
| 90 |
pointer_left_handed_during_click_multiple_buttons: |
| 91 |
trackpoint |
| 92 |
ms-surface-cover |
| 93 |
mouse-wheelclickcount |
| 94 |
mouse-wheelclickangle |
| 95 |
low-dpi-mouse |
| 96 |
mouse-roccat |
| 97 |
mouse-wheel-tilt |
| 98 |
mouse |
| 99 |
logitech-trackball |
| 100 |
cyborg-rat |
| 101 |
magicmouse |
| 102 |
pointer_left_handed_during_click: |
| 103 |
trackpoint |
| 104 |
ms-surface-cover |
| 105 |
mouse-wheelclickcount |
| 106 |
mouse-wheelclickangle |
| 107 |
low-dpi-mouse |
| 108 |
mouse-roccat |
| 109 |
mouse-wheel-tilt |
| 110 |
mouse |
| 111 |
logitech-trackball |
| 112 |
cyborg-rat |
| 113 |
litest-magicmouse-device |
| 114 |
pointer_left_handed: |
| 115 |
trackpoint |
| 116 |
ms-surface-cover |
| 117 |
mouse-wheelclickcount |
| 118 |
mouse-wheelclickangle |
| 119 |
low-dpi-mouse |
| 120 |
mouse-roccat |
| 121 |
mouse-wheel-tilt |
| 122 |
mouse |
| 123 |
... |
| 124 |
|
| 125 |
|
| 126 |
In the above example, the "pointer:left-handed" suite contains multiple |
| 127 |
tests, e.g. "pointer_left_handed_during_click" (this is also the function |
| 128 |
name of the test, making it easy to grep for). This particular test is run |
| 129 |
for various devices including the trackpoint device and the magic mouse |
| 130 |
device. |
| 131 |
|
| 132 |
The "no device" entry signals that litest does not instantiate a uinput |
| 133 |
device for a specific test (though the test itself may |
| 134 |
instantiate one). |
| 135 |
|
| 136 |
The ``--filter-test`` argument enables selective running of tests through |
| 137 |
basic shell-style function name matching. For example: |
| 138 |
|
| 139 |
|
| 140 |
:: |
| 141 |
|
| 142 |
$ ./builddir/libinput-test-suite --filter-test="*1fg_tap*" |
| 143 |
|
| 144 |
|
| 145 |
The ``--filter-device`` argument enables selective running of tests through |
| 146 |
basic shell-style device name matching. The device names matched are the |
| 147 |
litest-specific shortnames, see the output of ``--list``. For example: |
| 148 |
|
| 149 |
|
| 150 |
:: |
| 151 |
|
| 152 |
$ ./builddir/libinput-test-suite --filter-device="synaptics*" |
| 153 |
|
| 154 |
|
| 155 |
The ``--filter-group`` argument enables selective running of test groups |
| 156 |
through basic shell-style test group matching. The test groups matched are |
| 157 |
litest-specific test groups, see the output of ``--list``. For example: |
| 158 |
|
| 159 |
|
| 160 |
:: |
| 161 |
|
| 162 |
$ ./builddir/libinput-test-suite --filter-group="touchpad:*hover*" |
| 163 |
|
| 164 |
|
| 165 |
The ``--filter-device`` and ``--filter-group`` arguments can be combined with |
| 166 |
``--list`` to show which groups and devices will be affected. |
| 167 |
|
| 168 |
.. _test-verbosity: |
| 169 |
|
| 170 |
------------------------------------------------------------------------------ |
| 171 |
Controlling test output |
| 172 |
------------------------------------------------------------------------------ |
| 173 |
|
| 174 |
Each test supports the ``--verbose`` commandline option to enable debugging |
| 175 |
output, see **libinput_log_set_priority()** for details. The ``LITEST_VERBOSE`` |
| 176 |
environment variable, if set, also enables verbose mode. |
| 177 |
|
| 178 |
|
| 179 |
:: |
| 180 |
|
| 181 |
$ ./builddir/libinput-test-suite --verbose |
| 182 |
$ LITEST_VERBOSE=1 meson test -C builddir |
| 183 |
|
| 184 |
.. _test-installed: |
| 185 |
|
| 186 |
------------------------------------------------------------------------------ |
| 187 |
Installing the test suite |
| 188 |
------------------------------------------------------------------------------ |
| 189 |
|
| 190 |
If libinput is configured to install the tests, the test suite is available |
| 191 |
as the ``libinput test-suite`` command. When run as installed binary, the |
| 192 |
behavior of the test suite changes: |
| 193 |
|
| 194 |
- the ``libinput.so`` used is the one in the library lookup paths |
| 195 |
- no system-wide quirks are installed by the test suite, only those specific |
| 196 |
to the test devices |
| 197 |
- test device-specific quirks are installed in the system-wide quirks |
| 198 |
directory, usually ``/usr/share/libinput/``. |
| 199 |
|
| 200 |
It is not advisable to run ``libinput test-suite`` on a production machine. |
| 201 |
Data loss may occur. The primary use-case for the installed test suite is |
| 202 |
verification of distribution composes. |
| 203 |
|
| 204 |
.. note:: The ``prefix`` is still used by the test suite. For verification |
| 205 |
of a system package, the test suite must be configured with the same prefix. |
| 206 |
|
| 207 |
To configure libinput to install the tests, use the ``-Dinstall-tests=true`` |
| 208 |
meson option:: |
| 209 |
|
| 210 |
$ meson setup builddir -Dtests=true -Dinstall-tests=true <other options> |
| 211 |
|
| 212 |
.. _test-meson-suites: |
| 213 |
|
| 214 |
------------------------------------------------------------------------------ |
| 215 |
Meson test suites |
| 216 |
------------------------------------------------------------------------------ |
| 217 |
|
| 218 |
This section is primarily of interest to distributors that want to run test |
| 219 |
or developers working on libinput's CI. |
| 220 |
|
| 221 |
Tests invoked by ``meson test`` are grouped into test suites, the test suite |
| 222 |
names identify when the respective test can be run: |
| 223 |
|
| 224 |
- ``valgrind``: tests that can be run under valgrind (in addition to a |
| 225 |
normal run) |
| 226 |
- ``root``: tests that must be run as root |
| 227 |
- ``hardware``: tests that require a VM or physical machine |
| 228 |
- ``all``: all tests, only needed because of |
| 229 |
`meson bug 5340 <https://github.com/mesonbuild/meson/issues/5340>`_ |
| 230 |
|
| 231 |
The suite names can be provided as filters to ``meson test |
| 232 |
--suite=<suitename>`` or ``meson test --no-suite=<suitename>``. |
| 233 |
For example, if running a container-based CI, you may specify the test |
| 234 |
suites as: |
| 235 |
|
| 236 |
:: |
| 237 |
|
| 238 |
$ meson test --no-suite=machine # only run container-friendly tests |
| 239 |
$ meson test --suite=valgrind --setup=valgrind # run all valgrind-compatible tests |
| 240 |
$ meson test --no-suite=root # run all tests not requiring root |
| 241 |
|
| 242 |
These suites are subject to change at any time. |