Debian Codegraph @

doc/user/test-suite.rst

raw
1 .. _test-suite:
2
3 ==============================================================================
4 libinput test suite
5 ==============================================================================
6
7 libinput's primary test suite can be invoked with
8
9 ::
10
11 $ sudo ./builddir/libinput-test-suite
12
13 When developing libinput, the ``libinput-test-suite`` should always be
14 run to check for behavior changes and/or regressions. For quick iteration,
15 the number of tests to run can be filtered, see :ref:`test-filtering`.
16 This allows for developers to verify a subset of tests (e.g.
17 touchpad tap-to-click) while hacking on that specific feature and only run
18 the full suite when development is done finished.
19
20 .. note:: The test suite relies on udev and the kernel, specifically uinput.
21 It creates virtual input devices and replays the events. This may
22 interfere with your running session. The test suite is not suitable
23 for running inside containers.
24
25 In addition, libinput ships with a set of (primarily janitorial) tests that
26 must pass for any merge request. These tests are invoked by calling
27 ``meson test -C builddir`` (or ``ninja test``). The ``libinput-test-suite`` is
28 part of that test set by default.
29
30 The upstream CI runs all these tests and is run for every merge request.
31
32 .. _test-job-control:
33
34 ------------------------------------------------------------------------------
35 Job control in the test suite
36 ------------------------------------------------------------------------------
37
38 The test suite runner has a make-like job control enabled by the ``-j`` or
39 ``--jobs`` flag and will fork off as many parallel processes as given by this
40 flag. The default if unspecified is 8. When debugging a specific test case
41 failure it is recommended to employ test filtures (see :ref:`test-filtering`)
42 and disable parallel tests. The test suite automatically disables parallel
43 make when run in gdb.
44
45 .. _test-config:
46
47 ------------------------------------------------------------------------------
48 X.Org config to avoid interference
49 ------------------------------------------------------------------------------
50
51 uinput devices created by the test suite are usually recognised by X as
52 input devices. All events sent through these devices will generate X events
53 and interfere with your desktop.
54
55 Copy the file ``$srcdir/test/50-litest.conf`` into your ``/etc/X11/xorg.conf.d``
56 and restart X. This will ignore any litest devices and thus not interfere
57 with your desktop.
58
59 .. _test-root:
60
61 ------------------------------------------------------------------------------
62 Permissions required to run tests
63 ------------------------------------------------------------------------------
64
65 Most tests require the creation of uinput devices and access to the
66 resulting ``/dev/input/eventX`` nodes. Some tests require temporary udev rules.
67 **This usually requires the tests to be run as root**. If not run as
68 root, the test suite runner will exit with status 77, an exit status
69 interpreted as "skipped".
70
71 .. _test-filtering:
72
73 ------------------------------------------------------------------------------
74 Selective running of tests
75 ------------------------------------------------------------------------------
76
77 litest's tests are grouped into test groups, test names and devices. A test
78 group is e.g. "touchpad:tap" and incorporates all tapping-related tests for
79 touchpads. Each test function is (usually) run with one or more specific
80 devices. The ``--list`` commandline argument shows the list of suites and
81 tests. This is useful when trying to figure out if a specific test is
82 run for a device.
83
84
85 ::
86
87 $ ./builddir/libinput-test-suite --list
88 ...
89 pointer:left-handed:
90 pointer_left_handed_during_click_multiple_buttons:
91 trackpoint
92 ms-surface-cover
93 mouse-wheelclickcount
94 mouse-wheelclickangle
95 low-dpi-mouse
96 mouse-roccat
97 mouse-wheel-tilt
98 mouse
99 logitech-trackball
100 cyborg-rat
101 magicmouse
102 pointer_left_handed_during_click:
103 trackpoint
104 ms-surface-cover
105 mouse-wheelclickcount
106 mouse-wheelclickangle
107 low-dpi-mouse
108 mouse-roccat
109 mouse-wheel-tilt
110 mouse
111 logitech-trackball
112 cyborg-rat
113 litest-magicmouse-device
114 pointer_left_handed:
115 trackpoint
116 ms-surface-cover
117 mouse-wheelclickcount
118 mouse-wheelclickangle
119 low-dpi-mouse
120 mouse-roccat
121 mouse-wheel-tilt
122 mouse
123 ...
124
125
126 In the above example, the "pointer:left-handed" suite contains multiple
127 tests, e.g. "pointer_left_handed_during_click" (this is also the function
128 name of the test, making it easy to grep for). This particular test is run
129 for various devices including the trackpoint device and the magic mouse
130 device.
131
132 The "no device" entry signals that litest does not instantiate a uinput
133 device for a specific test (though the test itself may
134 instantiate one).
135
136 The ``--filter-test`` argument enables selective running of tests through
137 basic shell-style function name matching. For example:
138
139
140 ::
141
142 $ ./builddir/libinput-test-suite --filter-test="*1fg_tap*"
143
144
145 The ``--filter-device`` argument enables selective running of tests through
146 basic shell-style device name matching. The device names matched are the
147 litest-specific shortnames, see the output of ``--list``. For example:
148
149
150 ::
151
152 $ ./builddir/libinput-test-suite --filter-device="synaptics*"
153
154
155 The ``--filter-group`` argument enables selective running of test groups
156 through basic shell-style test group matching. The test groups matched are
157 litest-specific test groups, see the output of ``--list``. For example:
158
159
160 ::
161
162 $ ./builddir/libinput-test-suite --filter-group="touchpad:*hover*"
163
164
165 The ``--filter-device`` and ``--filter-group`` arguments can be combined with
166 ``--list`` to show which groups and devices will be affected.
167
168 .. _test-verbosity:
169
170 ------------------------------------------------------------------------------
171 Controlling test output
172 ------------------------------------------------------------------------------
173
174 Each test supports the ``--verbose`` commandline option to enable debugging
175 output, see **libinput_log_set_priority()** for details. The ``LITEST_VERBOSE``
176 environment variable, if set, also enables verbose mode.
177
178
179 ::
180
181 $ ./builddir/libinput-test-suite --verbose
182 $ LITEST_VERBOSE=1 meson test -C builddir
183
184 .. _test-installed:
185
186 ------------------------------------------------------------------------------
187 Installing the test suite
188 ------------------------------------------------------------------------------
189
190 If libinput is configured to install the tests, the test suite is available
191 as the ``libinput test-suite`` command. When run as installed binary, the
192 behavior of the test suite changes:
193
194 - the ``libinput.so`` used is the one in the library lookup paths
195 - no system-wide quirks are installed by the test suite, only those specific
196 to the test devices
197 - test device-specific quirks are installed in the system-wide quirks
198 directory, usually ``/usr/share/libinput/``.
199
200 It is not advisable to run ``libinput test-suite`` on a production machine.
201 Data loss may occur. The primary use-case for the installed test suite is
202 verification of distribution composes.
203
204 .. note:: The ``prefix`` is still used by the test suite. For verification
205 of a system package, the test suite must be configured with the same prefix.
206
207 To configure libinput to install the tests, use the ``-Dinstall-tests=true``
208 meson option::
209
210 $ meson setup builddir -Dtests=true -Dinstall-tests=true <other options>
211
212 .. _test-meson-suites:
213
214 ------------------------------------------------------------------------------
215 Meson test suites
216 ------------------------------------------------------------------------------
217
218 This section is primarily of interest to distributors that want to run test
219 or developers working on libinput's CI.
220
221 Tests invoked by ``meson test`` are grouped into test suites, the test suite
222 names identify when the respective test can be run:
223
224 - ``valgrind``: tests that can be run under valgrind (in addition to a
225 normal run)
226 - ``root``: tests that must be run as root
227 - ``hardware``: tests that require a VM or physical machine
228 - ``all``: all tests, only needed because of
229 `meson bug 5340 <https://github.com/mesonbuild/meson/issues/5340>`_
230
231 The suite names can be provided as filters to ``meson test
232 --suite=<suitename>`` or ``meson test --no-suite=<suitename>``.
233 For example, if running a container-based CI, you may specify the test
234 suites as:
235
236 ::
237
238 $ meson test --no-suite=machine # only run container-friendly tests
239 $ meson test --suite=valgrind --setup=valgrind # run all valgrind-compatible tests
240 $ meson test --no-suite=root # run all tests not requiring root
241
242 These suites are subject to change at any time.