Debian Codegraph @

podofo/test/common/cmake/Catch.cmake

raw
1 # Distributed under the OSI-approved BSD 3-Clause License. See accompanying
2 # file Copyright.txt or https://cmake.org/licensing for details.
3
4 #[=======================================================================[.rst:
5 Catch
6 -----
7
8 This module defines a function to help use the Catch test framework.
9
10 The :command:`catch_discover_tests` discovers tests by asking the compiled test
11 executable to enumerate its tests. This does not require CMake to be re-run
12 when tests change. However, it may not work in a cross-compiling environment,
13 and setting test properties is less convenient.
14
15 This command is intended to replace use of :command:`add_test` to register
16 tests, and will create a separate CTest test for each Catch test case. Note
17 that this is in some cases less efficient, as common set-up and tear-down logic
18 cannot be shared by multiple test cases executing in the same instance.
19 However, it provides more fine-grained pass/fail information to CTest, which is
20 usually considered as more beneficial. By default, the CTest test name is the
21 same as the Catch name; see also ``TEST_PREFIX`` and ``TEST_SUFFIX``.
22
23 .. command:: catch_discover_tests
24
25 Automatically add tests with CTest by querying the compiled test executable
26 for available tests::
27
28 catch_discover_tests(target
29 [TEST_SPEC arg1...]
30 [EXTRA_ARGS arg1...]
31 [WORKING_DIRECTORY dir]
32 [TEST_PREFIX prefix]
33 [TEST_SUFFIX suffix]
34 [PROPERTIES name1 value1...]
35 [TEST_LIST var]
36 [REPORTER reporter]
37 [OUTPUT_DIR dir]
38 [OUTPUT_PREFIX prefix}
39 [OUTPUT_SUFFIX suffix]
40 )
41
42 ``catch_discover_tests`` sets up a post-build command on the test executable
43 that generates the list of tests by parsing the output from running the test
44 with the ``--list-test-names-only`` argument. This ensures that the full
45 list of tests is obtained. Since test discovery occurs at build time, it is
46 not necessary to re-run CMake when the list of tests changes.
47 However, it requires that :prop_tgt:`CROSSCOMPILING_EMULATOR` is properly set
48 in order to function in a cross-compiling environment.
49
50 Additionally, setting properties on tests is somewhat less convenient, since
51 the tests are not available at CMake time. Additional test properties may be
52 assigned to the set of tests as a whole using the ``PROPERTIES`` option. If
53 more fine-grained test control is needed, custom content may be provided
54 through an external CTest script using the :prop_dir:`TEST_INCLUDE_FILES`
55 directory property. The set of discovered tests is made accessible to such a
56 script via the ``<target>_TESTS`` variable.
57
58 The options are:
59
60 ``target``
61 Specifies the Catch executable, which must be a known CMake executable
62 target. CMake will substitute the location of the built executable when
63 running the test.
64
65 ``TEST_SPEC arg1...``
66 Specifies test cases, wildcarded test cases, tags and tag expressions to
67 pass to the Catch executable with the ``--list-test-names-only`` argument.
68
69 ``EXTRA_ARGS arg1...``
70 Any extra arguments to pass on the command line to each test case.
71
72 ``WORKING_DIRECTORY dir``
73 Specifies the directory in which to run the discovered test cases. If this
74 option is not provided, the current binary directory is used.
75
76 ``TEST_PREFIX prefix``
77 Specifies a ``prefix`` to be prepended to the name of each discovered test
78 case. This can be useful when the same test executable is being used in
79 multiple calls to ``catch_discover_tests()`` but with different
80 ``TEST_SPEC`` or ``EXTRA_ARGS``.
81
82 ``TEST_SUFFIX suffix``
83 Similar to ``TEST_PREFIX`` except the ``suffix`` is appended to the name of
84 every discovered test case. Both ``TEST_PREFIX`` and ``TEST_SUFFIX`` may
85 be specified.
86
87 ``PROPERTIES name1 value1...``
88 Specifies additional properties to be set on all tests discovered by this
89 invocation of ``catch_discover_tests``.
90
91 ``TEST_LIST var``
92 Make the list of tests available in the variable ``var``, rather than the
93 default ``<target>_TESTS``. This can be useful when the same test
94 executable is being used in multiple calls to ``catch_discover_tests()``.
95 Note that this variable is only available in CTest.
96
97 ``REPORTER reporter``
98 Use the specified reporter when running the test case. The reporter will
99 be passed to the Catch executable as ``--reporter reporter``.
100
101 ``OUTPUT_DIR dir``
102 If specified, the parameter is passed along as
103 ``--out dir/<test_name>`` to Catch executable. The actual file name is the
104 same as the test name. This should be used instead of
105 ``EXTRA_ARGS --out foo`` to avoid race conditions writing the result output
106 when using parallel test execution.
107
108 ``OUTPUT_PREFIX prefix``
109 May be used in conjunction with ``OUTPUT_DIR``.
110 If specified, ``prefix`` is added to each output file name, like so
111 ``--out dir/prefix<test_name>``.
112
113 ``OUTPUT_SUFFIX suffix``
114 May be used in conjunction with ``OUTPUT_DIR``.
115 If specified, ``suffix`` is added to each output file name, like so
116 ``--out dir/<test_name>suffix``. This can be used to add a file extension to
117 the output e.g. ".xml".
118
119 #]=======================================================================]
120
121 #------------------------------------------------------------------------------
122 function(catch_discover_tests TARGET)
123 cmake_parse_arguments(
124 ""
125 ""
126 "TEST_PREFIX;TEST_SUFFIX;WORKING_DIRECTORY;TEST_LIST;REPORTER;OUTPUT_DIR;OUTPUT_PREFIX;OUTPUT_SUFFIX"
127 "TEST_SPEC;EXTRA_ARGS;PROPERTIES"
128 ${ARGN}
129 )
130
131 if(NOT _WORKING_DIRECTORY)
132 set(_WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
133 endif()
134 if(NOT _TEST_LIST)
135 set(_TEST_LIST ${TARGET}_TESTS)
136 endif()
137
138 ## Generate a unique name based on the extra arguments
139 string(SHA1 args_hash "${_TEST_SPEC} ${_EXTRA_ARGS} ${_REPORTER} ${_OUTPUT_DIR} ${_OUTPUT_PREFIX} ${_OUTPUT_SUFFIX}")
140 string(SUBSTRING ${args_hash} 0 7 args_hash)
141
142 # Define rule to generate test list for aforementioned test executable
143 set(ctest_include_file "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}_include-${args_hash}.cmake")
144 set(ctest_tests_file "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}_tests-${args_hash}.cmake")
145 get_property(crosscompiling_emulator
146 TARGET ${TARGET}
147 PROPERTY CROSSCOMPILING_EMULATOR
148 )
149 add_custom_command(
150 TARGET ${TARGET} POST_BUILD
151 BYPRODUCTS "${ctest_tests_file}"
152 COMMAND "${CMAKE_COMMAND}"
153 -D "TEST_TARGET=${TARGET}"
154 -D "TEST_EXECUTABLE=$<TARGET_FILE:${TARGET}>"
155 -D "TEST_EXECUTOR=${crosscompiling_emulator}"
156 -D "TEST_WORKING_DIR=${_WORKING_DIRECTORY}"
157 -D "TEST_SPEC=${_TEST_SPEC}"
158 -D "TEST_EXTRA_ARGS=${_EXTRA_ARGS}"
159 -D "TEST_PROPERTIES=${_PROPERTIES}"
160 -D "TEST_PREFIX=${_TEST_PREFIX}"
161 -D "TEST_SUFFIX=${_TEST_SUFFIX}"
162 -D "TEST_LIST=${_TEST_LIST}"
163 -D "TEST_REPORTER=${_REPORTER}"
164 -D "TEST_OUTPUT_DIR=${_OUTPUT_DIR}"
165 -D "TEST_OUTPUT_PREFIX=${_OUTPUT_PREFIX}"
166 -D "TEST_OUTPUT_SUFFIX=${_OUTPUT_SUFFIX}"
167 -D "CTEST_FILE=${ctest_tests_file}"
168 -P "${_CATCH_DISCOVER_TESTS_SCRIPT}"
169 VERBATIM
170 )
171
172 file(WRITE "${ctest_include_file}"
173 "if(EXISTS \"${ctest_tests_file}\")\n"
174 " include(\"${ctest_tests_file}\")\n"
175 "else()\n"
176 " add_test(${TARGET}_NOT_BUILT-${args_hash} ${TARGET}_NOT_BUILT-${args_hash})\n"
177 "endif()\n"
178 )
179
180 if(NOT ${CMAKE_VERSION} VERSION_LESS "3.10.0")
181 # Add discovered tests to directory TEST_INCLUDE_FILES
182 set_property(DIRECTORY
183 APPEND PROPERTY TEST_INCLUDE_FILES "${ctest_include_file}"
184 )
185 else()
186 # Add discovered tests as directory TEST_INCLUDE_FILE if possible
187 get_property(test_include_file_set DIRECTORY PROPERTY TEST_INCLUDE_FILE SET)
188 if (NOT ${test_include_file_set})
189 set_property(DIRECTORY
190 PROPERTY TEST_INCLUDE_FILE "${ctest_include_file}"
191 )
192 else()
193 message(FATAL_ERROR
194 "Cannot set more than one TEST_INCLUDE_FILE"
195 )
196 endif()
197 endif()
198
199 endfunction()
200
201 ###############################################################################
202
203 set(_CATCH_DISCOVER_TESTS_SCRIPT
204 ${CMAKE_CURRENT_LIST_DIR}/CatchAddTests.cmake
205 CACHE INTERNAL "Catch2 full path to CatchAddTests.cmake helper file"
206 )