| 1 |
Background |
| 2 |
========== |
| 3 |
|
| 4 |
In order to easily find the most appropriate documentation, see the `TriBITS |
| 5 |
Developer and User Roles`_ guide. This guide describes the different roles |
| 6 |
that users of TriBITS may play and offers links to relevant sections of the |
| 7 |
documentation. Additionally, the reader may wish to review the `CMake Language |
| 8 |
Overview and Gotchas`_ section which is meant for users that are new to both |
| 9 |
CMake and TriBITS. This section gives a brief overview of getting started with |
| 10 |
CMake and provides some warnings about non-obvious CMake behavior that often |
| 11 |
trips up new users of TriBITS. |
| 12 |
|
| 13 |
|
| 14 |
TriBITS Developer and User Roles |
| 15 |
-------------------------------- |
| 16 |
|
| 17 |
There are approximately five different types roles related to TriBITS. These |
| 18 |
different roles require different levels of expertise and knowledge of CMake |
| 19 |
and knowledge of the TriBITS system. The primary roles are 1) `TriBITS |
| 20 |
Project User`_, 2) `TriBITS Project Developer`_, 3) `TriBITS Project |
| 21 |
Architect`_, 4) `TriBITS System Developer`_, and 5) `TriBITS System |
| 22 |
Architect`_. Each of these roles builds on the necessary knowledge of the |
| 23 |
lower-level roles. |
| 24 |
|
| 25 |
.. _TriBITS Project User: |
| 26 |
.. _TriBITS Project Users: |
| 27 |
|
| 28 |
The first role is that of a **TriBITS Project User** who only needs to be able |
| 29 |
to configure, build, and test a project that uses TriBITS as its build system. |
| 30 |
A person acting in this role needs to know little about CMake other than |
| 31 |
basics about how to run the ``cmake`` and ``ctest`` executables, how to set |
| 32 |
CMake cache variables, and the basics of building software by typing ``make`` |
| 33 |
and running tests with ``ctest``. The proper reference for a TriBITS Project |
| 34 |
User is the `Project-Specific Build Reference`_. The `TriBITS |
| 35 |
Overview`_ document may also be of some help. A TriBITS project user may also |
| 36 |
need to consult `Package Dependencies and Enable/Disable Logic`_. |
| 37 |
|
| 38 |
.. _TriBITS Project Developer: |
| 39 |
.. _TriBITS Project Developers: |
| 40 |
|
| 41 |
A **TriBITS Project Developer** is someone who contributes to a software |
| 42 |
project that uses TriBITS. They will add source files, libraries and |
| 43 |
executables , test executables and define tests run with ``ctest``. They have |
| 44 |
to configure and build the project code in order to be able to develop and run |
| 45 |
tests and therefore this role includes all of the necessary knowledge and |
| 46 |
functions of a TriBITS Project User. A casual TriBITS Project Developer |
| 47 |
typically does not need to know a lot about CMake and really only needs to |
| 48 |
know a subset of the `TriBITS Macros and Functions`_ defined in the document |
| 49 |
`TriBITS Users Guide and Reference`_ in addition to the genetic `TriBITS Build |
| 50 |
Reference <TribitsBuildReference.html>`_ document. A slightly more |
| 51 |
sophisticated TriBITS Project Developer will also add new packages, add new |
| 52 |
package dependencies, and define new external packages/TPLs. The `TriBITS |
| 53 |
Users Guide and Reference`_ should supply everything such a developer needs to |
| 54 |
know and more. However, only a smaller part of that document needs to be |
| 55 |
understood and accessed by people assuming this role. |
| 56 |
|
| 57 |
.. jfrye: Add links to parts of document above? |
| 58 |
|
| 59 |
.. _TriBITS Project Architect: |
| 60 |
.. _TriBITS Project Architects: |
| 61 |
|
| 62 |
The next level of roles is a **TriBITS Project Architect**. This is someone |
| 63 |
(perhaps only one person on a project development team) that knows the usage |
| 64 |
and functioning of TriBITS in great detail. They understand how to set up a |
| 65 |
TriBITS project from scratch, how to set up automated testing using the |
| 66 |
TriBITS system, and know how to use TriBITS to implement the overall software |
| 67 |
development process. A person in this role is also likely to be the one who |
| 68 |
makes the initial technical decision for their project to adopt TriBITS for |
| 69 |
its native build and test system. The document `TriBITS Users Guide and |
| 70 |
Reference`_, detailed CMake/CTest/CDash documentation provided by Kitware, and |
| 71 |
great books like `Professional CMake`_ should provide most of what a person in |
| 72 |
this role needs to know. A person assuming this role is the primary audience |
| 73 |
for a lot of the more advanced material in that document. |
| 74 |
|
| 75 |
.. _TriBITS System Developer: |
| 76 |
.. _TriBITS System Developers: |
| 77 |
.. _TriBITS System Architect: |
| 78 |
.. _TriBITS System Architects: |
| 79 |
|
| 80 |
The last two roles **TriBITS System Developer** and **TriBITS System |
| 81 |
Architect** are for those individuals that actually extend and modify the |
| 82 |
TriBITS system itself. A TriBITS System Developer needs to know how to add |
| 83 |
new TriBITS functionality while maintaining backward compatibility, know how |
| 84 |
to add new unit tests for the TriBITS system (see `The TriBITS Test |
| 85 |
Package`_), and perform other related tasks. Such a developer needs to be |
| 86 |
very knowledgeable of the basic functioning of CMake and know how TriBITS is |
| 87 |
implemented in the CMake language. A TriBITS System Architect is someone who |
| 88 |
must be consulted on almost all non-trivial changes or additions to the |
| 89 |
TriBITS system. A *TriBITS System Architect* in addition needs to know the |
| 90 |
entire TriBITS system, the design philosophy that provides the foundation for |
| 91 |
TriBITS and be an expert in CMake, CTest, and CDash. Much of what needs to be |
| 92 |
known by a TriBITS System Developer and a TriBITS System Architect is |
| 93 |
contained in the document `TriBITS Maintainers Guide and Reference`_. The |
| 94 |
rest of the primary documentation for these roles will be in the TriBITS |
| 95 |
CMake source code and various unit tests itself defined in `The TriBITS Test |
| 96 |
Package`_. At the time of this writing, there is currently there is only one |
| 97 |
TriBITS System Architect (who also happens to be the primary author of this |
| 98 |
document). |
| 99 |
|
| 100 |
An explicit goal of the document `TriBITS Users Guide and Reference`_ is to |
| 101 |
foster the creation of new `TriBITS Project Architects`_ (i.e. those who would |
| 102 |
make the decision to adopt TriBITS for their projects). An explicit goal of |
| 103 |
the document `TriBITS Maintainers Guide and Reference`_ is to foster the |
| 104 |
creation of new `TriBITS System Developers`_ to help extend and maintain the |
| 105 |
TriBITS package itself. |
| 106 |
|
| 107 |
Depending on the particular role that a reader falls into, this document |
| 108 |
may not be necessary and the `TriBITS Overview`_ or the |
| 109 |
`<Project>BuildReference`_ documents may be more appropriate. Hopefully the |
| 110 |
above roles and discussion help the reader select the right document to |
| 111 |
start with. |
| 112 |
|
| 113 |
NOTE: Before getting started with TriBITS, if a reader is unfamiliar with |
| 114 |
CMake, please review the `CMake Language Overview and Gotchas`_. Once those |
| 115 |
CMake basics and common gotchas have been reviewed, we now get into the meat |
| 116 |
of TriBITS starting with software engineering principles that lie at the |
| 117 |
foundation of TriBITS. That is followed by with and overall of the structure |
| 118 |
of a TriBITS project. |
| 119 |
|
| 120 |
|
| 121 |
Software Engineering Packaging Principles |
| 122 |
----------------------------------------- |
| 123 |
|
| 124 |
The design of TriBITS takes into account standard software engineering |
| 125 |
packaging principles. In his book [`Agile Software Development, 2003`_], |
| 126 |
Robert Martin defines several software engineering principles related to |
| 127 |
packaging software which are listed below: |
| 128 |
|
| 129 |
* *Package Cohesion OO Principles*: |
| 130 |
|
| 131 |
1) *REP (Release-Reuse Equivalency Principle)*: The granule of reuse is the |
| 132 |
granule of release. |
| 133 |
|
| 134 |
2) *CCP (Common Closure Principle)*: The classes in a package should be |
| 135 |
closed together against the same kinds of changes. A change that affects |
| 136 |
a closed package affects all the classes in that package and no other |
| 137 |
packages. |
| 138 |
|
| 139 |
3) *CRP (Common Reuse Principle)*: The classes in a package are used |
| 140 |
together. If you reuse one of the classes in a package, you reuse them |
| 141 |
all. |
| 142 |
|
| 143 |
* *Package Coupling OO Principles*: |
| 144 |
|
| 145 |
4) *ADP (Acyclic Dependencies Principle)*: Allow no cycles in the package |
| 146 |
dependency graph. |
| 147 |
|
| 148 |
5) *SDP (Stable Dependencies Principle)*: Depend in the direction of |
| 149 |
stability. |
| 150 |
|
| 151 |
6) SAP (Stable Abstractions Principle): A package should be as abstract as |
| 152 |
it is stable. |
| 153 |
|
| 154 |
Any of these six OO packaging principles (and other issues) may be considered |
| 155 |
when deciding how to partition software into different `TriBITS Packages`_. |
| 156 |
|
| 157 |
NOTE: The purpose of this TriBITS Developers Guide document is not teach |
| 158 |
basic software engineering so these various principles will not be expanded on |
| 159 |
further. However, interested readers are strongly encouraged to read |
| 160 |
[`Agile Software Development, 2003`_] as one of the better software |
| 161 |
engineering books out there (see https://bartlettroscoe.github.io/reading-list/#most_recommended_se_books). |
| 162 |
|
| 163 |
|
| 164 |
TriBITS Project Structure |
| 165 |
========================= |
| 166 |
|
| 167 |
TriBITS is a Framework, implemented in CMake, to create CMake projects. As a |
| 168 |
`Software Framework`_, TriBITS defines the overall structure of a CMake build |
| 169 |
system for a project and it processes the various project-, repository-, and |
| 170 |
package-specific files in a specified order. Almost all of this processing |
| 171 |
takes place in the `tribits_project()`_ macro (or macros and functions it |
| 172 |
calls). The following subsections define the essence of the TriBITS framework |
| 173 |
in some detail. Later sections cover specific topics and the various sections |
| 174 |
link to each other. Within this section, the subsection `TriBITS Structural |
| 175 |
Units`_ defines the basic units `TriBITS Project`_, `TriBITS Repository`_, |
| 176 |
`TriBITS Package`_, `TriBITS External Package/TPL`_ and other related |
| 177 |
structural units. The subsection `Processing of TriBITS Files: Ordering and |
| 178 |
Details`_ defines exactly what files TriBITS processes and in what order. It |
| 179 |
also shows how to get TriBITS to show exactly what files it is processing to |
| 180 |
help in debugging issues. The subsection `Coexisting Projects, Repositories, |
| 181 |
and Packages`_ gives some of the rules and constrains for how the different |
| 182 |
structure units can co-exist in the same directories. The last two |
| 183 |
subsections in this section cover `Standard TriBITS TPLs`_ and `Common TriBITS |
| 184 |
TPLs`_. |
| 185 |
|
| 186 |
TriBITS Structural Units |
| 187 |
------------------------ |
| 188 |
|
| 189 |
A CMake project that uses TriBITS as its build and test system is composed of |
| 190 |
a single `TriBITS Project`_, one or more `TriBITS Repositories`_ and one or |
| 191 |
more `TriBITS Packages`_. In addition, a TriBITS Package can be broken up |
| 192 |
into `TriBITS Subpackages`_. Together, the collection of TriBITS Packages and |
| 193 |
TriBITS Subpackages are called *TriBITS Software Engineering Packages*, or |
| 194 |
`TriBITS Packages`_ for short. |
| 195 |
|
| 196 |
First, to better establish the basic nomenclature, the key structural TriBITS |
| 197 |
units are: |
| 198 |
|
| 199 |
.. _TriBITS Top-Level Package: |
| 200 |
.. _TriBITS Top-Level Packages: |
| 201 |
|
| 202 |
* `TriBITS Top-Level Package`_: A collection of related software that |
| 203 |
typically includes one or more source files built into one or more libraries |
| 204 |
and has associated tests to help define and protect the functionality |
| 205 |
provided by the software. A top-level package (or just "package" if no |
| 206 |
ambiguity) also typically defines a single integrated unit of documentation |
| 207 |
and testing (see `TriBITS Automated Testing`_). A top-level TriBITS package |
| 208 |
may or may not be broken down into multiple subpackages. Examples of TriBITS |
| 209 |
packages in `TribitsExampleProject`_ include ``SimpleCXX``, ``MixedLang`` |
| 210 |
and ``WithSubpackages``. (Don't confuse a TriBITS "Package" with a raw |
| 211 |
CMake "Package". A raw CMake "Package" actually maps to a `TriBITS External |
| 212 |
Package/TPL`_; see `Why a TriBITS Package is not a CMake Package`_.) |
| 213 |
|
| 214 |
.. _TriBITS Subpackages: |
| 215 |
|
| 216 |
* `TriBITS Subpackage`_: A part of a parent `TriBITS Package`_ that also |
| 217 |
typically has source files built into libraries and tests but is documented |
| 218 |
and tested along with the other subpackages in the parent package. The primary |
| 219 |
purpose for supporting subpackages is to provide finer-grained control of |
| 220 |
software dependencies. In `TribitsExampleProject`_, ``WithSubpackages`` is |
| 221 |
an example of a package with subpackages ``'A'``, ``'B'``, and ``'C'``. The |
| 222 |
full subpackage name is prefixed by the parent package name (e.g. the full |
| 223 |
name for subpackage ``'A'`` is ``WithSubpackagesA``). The parent package is |
| 224 |
always implicitly dependent on its subpackages (e.g. parent package |
| 225 |
``WithSubpackages`` depends on its subpackages ``WithSubpackagesA``, |
| 226 |
``WithSubpackagesB``, and ``WithSubpackagesC``). |
| 227 |
|
| 228 |
.. _TriBITS TPL: |
| 229 |
.. _TriBITS TPLs: |
| 230 |
.. _TriBITS External Package/TPL: |
| 231 |
.. _TriBITS External Packages/TPLs: |
| 232 |
|
| 233 |
* `TriBITS External Package/TPL`_: The specification for a particular external |
| 234 |
dependency that is required or can be used in one or more `TriBITS |
| 235 |
Packages`_. A modern TriBITS external package/TPL (Third Party Library) is |
| 236 |
typically just a small file ``FindTPL<tplName>.cmake`` that calls |
| 237 |
``find_package(<externalPkg>)`` and defines the ``<tplName>::all_libs`` |
| 238 |
target. More generally, an external package/TPL can be specified as a list |
| 239 |
list of libraries and/or include directories for header files. Examples of |
| 240 |
basic external packages/TPLs include ``BLAS``, ``LAPACK``, and ``Boost``. |
| 241 |
|
| 242 |
.. _TriBITS Package: |
| 243 |
.. _TriBITS Packages: |
| 244 |
|
| 245 |
* **TriBITS Package**: The combined set of `TriBITS Top-Level Packages`_, |
| 246 |
`TriBITS Subpackages`_, and `TriBITS External Packages/TPLs`_ that |
| 247 |
constitute the basic *Software Engineering* packages of a TriBITS project |
| 248 |
(see `Software Engineering Packaging Principles`_): Packages are the basis |
| 249 |
for setting dependencies in the TriBITS system. For example, the Packages |
| 250 |
provided by the example top-level package ``WithSubpackages`` (in order of |
| 251 |
increasing dependencies) are ``WithSubpackagesA``, ``WithSubpackagesB``, |
| 252 |
``WithSubpackagesC``, and ``WithSubpackages`` (see |
| 253 |
`TribitsExampleProject`_). |
| 254 |
|
| 255 |
.. _TriBITS Repositories: |
| 256 |
|
| 257 |
* `TriBITS Repository`_: A collection of one or more `TriBITS Packages`_ |
| 258 |
specified in a `<repoDir>/PackagesList.cmake`_ file and zero or more |
| 259 |
external package/TPL declarations specified in a `<repoDir>/TPLsList.cmake`_ |
| 260 |
file. As discussed below, Repositories can include Native Repositories and |
| 261 |
Version Control (VC) Repositories. |
| 262 |
|
| 263 |
.. _TriBITS Projects: |
| 264 |
|
| 265 |
* `TriBITS Project`_: A collection of `TriBITS Repositories`_ and `TriBITS |
| 266 |
Packages`_ that defines a complete CMake ``PROJECT`` defining software which can be |
| 267 |
directly configured with ``cmake`` and then be built, tested, installed, |
| 268 |
etc. |
| 269 |
|
| 270 |
.. _TriBITS Meta-Project: |
| 271 |
.. _TriBITS Meta-Projects: |
| 272 |
|
| 273 |
* **TriBITS Meta-Project**: A `TriBITS Project`_ that contains no native |
| 274 |
`TriBITS packages`_ or `TriBITS external packages/TPLs`_ but is composed out |
| 275 |
packages from other `TriBITS Repositories`_. |
| 276 |
|
| 277 |
In this document, dependencies are described as either being *upstream* or |
| 278 |
*downstream/forward* defined as: |
| 279 |
|
| 280 |
.. _upstream: |
| 281 |
.. _upstream dependency: |
| 282 |
.. _upstream dependencies: |
| 283 |
|
| 284 |
* If unit "B" requires (or can use, or comes before) unit "A", then "A" is an |
| 285 |
**upstream dependency** of "B". |
| 286 |
|
| 287 |
.. _downstream: |
| 288 |
.. _downstream dependency: |
| 289 |
.. _downstream dependencies: |
| 290 |
|
| 291 |
* If unit "B" requires (or can use, or comes before) unit "A", then "B" is a |
| 292 |
**downstream dependency** or a **forward dependency** of "A". |
| 293 |
|
| 294 |
The following subsections define the major structural units of a TriBITS |
| 295 |
project in more detail. Each structural unit is described along with the |
| 296 |
files and directories associated with each. In addition, a key set of TriBITS |
| 297 |
CMake variables for each are defined as well. |
| 298 |
|
| 299 |
In the next major section following this one, some `Example TriBITS Projects`_ |
| 300 |
are described. For those who just want to jump in and learn best by example, |
| 301 |
these example projects are a good way to start. These example projects will |
| 302 |
be referenced in the more detailed descriptions given in this document. |
| 303 |
|
| 304 |
The last issue to touch on before getting into the detailed descriptions of |
| 305 |
the different TriBITS structural units is the issue of how CMake variables are |
| 306 |
defined and used by TriBITS. The CMake variables described in the TriBITS |
| 307 |
structural units below fall into one of two major types: |
| 308 |
|
| 309 |
* *Local Fixed-Name Variables* are used temporarily in the processing of a |
| 310 |
TriBITS unit. These include variables such as ``PROJECT_NAME``, |
| 311 |
``REPOSITORY_NAME``, ``PACKAGE_NAME``, and ``PARENT_PACKAGE_NAME``. These |
| 312 |
are distinguished by having a non-namespaced fixed/constant name. They are |
| 313 |
typically part of TriBITS reflection system, allowing subordinate units to |
| 314 |
determine the encapsulating unit in which they are participating. For, |
| 315 |
example, a TriBITS subpackage can determine its name, its parent package's |
| 316 |
name and directories, its parent repository name and directories, and the |
| 317 |
enclosing project's name and directories without having to refer to any |
| 318 |
specific names. |
| 319 |
|
| 320 |
* *Globally Scoped Namespaced Variables* are used to refer to properties of a |
| 321 |
named TriBITS unit that are seen globally by the entire TriBITS project. |
| 322 |
These include variables such as ``${REPOSITORY_NAME}_SOURCE_DIR`` |
| 323 |
(e.g. ``TribitsExProj_SOURCE_DIR``) and ``${PACKAGE_NAME}_BINARY_DIR`` |
| 324 |
(e.g. ``SimpleCXX_BINARY_DIR``). They are available after processing the |
| 325 |
unit, for use by `downstream`_ or subordinate units. They are part of the |
| 326 |
TriBITS dependency system, allowing downstream units to access properties of |
| 327 |
their known `upstream dependencies`_. |
| 328 |
|
| 329 |
More information about these various files is described in section `Processing |
| 330 |
of TriBITS Files: Ordering and Details`_. |
| 331 |
|
| 332 |
|
| 333 |
TriBITS Project |
| 334 |
+++++++++++++++ |
| 335 |
|
| 336 |
A TriBITS Project: |
| 337 |
|
| 338 |
* Defines the ``PROJECT_NAME`` CMake variable (defined in |
| 339 |
`<projectDir>/ProjectName.cmake`_) |
| 340 |
* Defines a complete CMake project which calls ``project(${PROJECT_NAME} |
| 341 |
...)`` and can be directly configured, built, tested, installed, etc. |
| 342 |
* Consists of one or more `TriBITS Repositories`_ (and may itself be a |
| 343 |
`TriBITS Repository`_) which can include native and extra repositories. |
| 344 |
* Allows for extra Repositories to be added on before or after the set of |
| 345 |
native Repositories (specified in |
| 346 |
`<projectDir>/cmake/ExtraRepositoriesList.cmake`_ or by CMake cache |
| 347 |
variables) |
| 348 |
* Defines a default CDash server and default project name on the server (the |
| 349 |
project name on the CDash server must be the same as ``${PROJECT_NAME}``). |
| 350 |
* Defines pre-push testing standard builds (using `checkin-test.py`_) |
| 351 |
|
| 352 |
For more details on the definition of a TriBITS Project, see: |
| 353 |
|
| 354 |
* `TriBITS Project Core Files`_ |
| 355 |
* `TriBITS Project Core Variables`_ |
| 356 |
|
| 357 |
|
| 358 |
TriBITS Project Core Files |
| 359 |
.......................... |
| 360 |
|
| 361 |
The core files making up a TriBITS Project (where ``<projectDir> = |
| 362 |
${PROJECT_SOURCE_DIR}``) are:: |
| 363 |
|
| 364 |
<projectDir>/ |
| 365 |
ProjectName.cmake # Defines PACKAGE_NAME |
| 366 |
CMakeLists.txt # Base project CMakeLists.txt file |
| 367 |
CTestConfig.cmake # [Optional] Needed for CDash submits |
| 368 |
Version.cmake # [Optional] Dev mode, Project version, VC branch |
| 369 |
project-checkin-test-config.py # [Optional] checkin-test.py config |
| 370 |
cmake/ |
| 371 |
NativeRepositoriesList.cmake # [Optional] Rarely used |
| 372 |
ExtraRepositoriesList.cmake # [Optional] Lists repos and VC URLs |
| 373 |
ProjectCiFileChangeLogic.py # [Optional] CI global change/test logic |
| 374 |
ProjectCompilerPostConfig.cmake # [Optional] Override/tweak build flags |
| 375 |
ProjectDependenciesSetup.cmake # [Optional] Project deps overrides |
| 376 |
CallbackDefineProjectPackaging.cmake # [Optional] CPack settings |
| 377 |
tribits/ # [Optional] Or provide ${PROJECT_NAME}_TRIBITS_DIR |
| 378 |
ctest/ |
| 379 |
CTestCustom.cmake.in # [Optional] Custom ctest settings |
| 380 |
|
| 381 |
These TriBITS Project files are documented in more detail below: |
| 382 |
|
| 383 |
* `<projectDir>/ProjectName.cmake`_ |
| 384 |
* `<projectDir>/CMakeLists.txt`_ |
| 385 |
* `<projectDir>/CTestConfig.cmake`_ |
| 386 |
* `<projectDir>/Version.cmake`_ |
| 387 |
* `<projectDir>/project-checkin-test-config.py`_ |
| 388 |
* `<projectDir>/cmake/NativeRepositoriesList.cmake`_ |
| 389 |
* `<projectDir>/cmake/ExtraRepositoriesList.cmake`_ |
| 390 |
* `<projectDir>/cmake/ProjectCiFileChangeLogic.py`_ |
| 391 |
* `<projectDir>/cmake/ProjectCompilerPostConfig.cmake`_ |
| 392 |
* `<projectDir>/cmake/ProjectDependenciesSetup.cmake`_ |
| 393 |
* `<projectDir>/cmake/CallbackDefineProjectPackaging.cmake`_ |
| 394 |
* `<projectDir>/cmake/tribits/`_ |
| 395 |
* `<projectDir>/cmake/ctest/CTestCustom.cmake.in`_ |
| 396 |
|
| 397 |
|
| 398 |
.. _<projectDir>/ProjectName.cmake: |
| 399 |
|
| 400 |
**<projectDir>/ProjectName.cmake**: [Required] At a minimum provides a |
| 401 |
``set()`` statement to set the local variable ``PROJECT_NAME``. This file is |
| 402 |
the first file that is read by a number of tools in order to get the TriBITS |
| 403 |
project's name. This file is read first in every context that involves |
| 404 |
processing the TriBITS project's files, including processes and tools that |
| 405 |
just need to build the package dependency tree (see `Reduced Package |
| 406 |
Dependency Processing`_). Being this is the first file read in for a TriBITS |
| 407 |
project and that it is read in first at the top level scope in every context, |
| 408 |
this is a good file to put in other universal static project options. Note |
| 409 |
that this is a project, not a repository file so no general |
| 410 |
repository-specific settings should go in this file. A simple example of this |
| 411 |
file is `TribitsExampleProject`_/``ProjectName.cmake``: |
| 412 |
|
| 413 |
.. include:: ../../examples/TribitsExampleProject/ProjectName.cmake |
| 414 |
:literal: |
| 415 |
|
| 416 |
A meta-project's ``ProjectName.cmake`` file might have a number of other |
| 417 |
variables set that define how the various different TriBITS repos are cobbled |
| 418 |
together into a single TriBITS meta-project (usually because the different |
| 419 |
TriBITS repos and packages are a bit messy and have other issues). For |
| 420 |
example, the CASL VERA TriBITS meta-project at one point had a very extensive |
| 421 |
collection of set statements in this file. |
| 422 |
|
| 423 |
|
| 424 |
.. _<projectDir>/CMakeLists.txt: |
| 425 |
|
| 426 |
**<projectDir>/CMakeLists.txt**: [Required] The top-level CMake project file. |
| 427 |
This is the first file that the ``cmake`` executable processes that starts |
| 428 |
everything off and is the base-level scope for local (non-cache) CMake |
| 429 |
variables. Due to a few CMake limitations and quirks, a project's top-level |
| 430 |
``CMakeLists.txt`` file is not quit as clean as one might otherwise hope would |
| 431 |
be but it is not too bad. A simple, but representative, example is |
| 432 |
`TribitsExampleProject`_/``CMakeLists.txt``: |
| 433 |
|
| 434 |
.. include:: ../../examples/TribitsExampleProject/CMakeLists.txt |
| 435 |
:literal: |
| 436 |
|
| 437 |
A couple of CMake and TriBITS quirks that the above example |
| 438 |
``CMakeLists.txt`` addresses are worth some discussion. First, to avoid |
| 439 |
duplication, the project's ``ProjectName.cmake`` file is read in with an |
| 440 |
``include()`` that defines the local variable ``PROJECT_NAME``. Right after |
| 441 |
this initial include, the built-in CMake command ``project(${PROJECT_NAME} |
| 442 |
NONE)`` is run. This command must be explicitly called with ``NONE`` so as to |
| 443 |
avoid default CMake behavior for defining compilers. The definition of |
| 444 |
compilers comes later as part of the TriBITS system inside of the |
| 445 |
`tribits_project()`_ command (see `Full Processing of TriBITS Project |
| 446 |
Files`_). |
| 447 |
|
| 448 |
As noted in the above example file, the only project defaults that should be |
| 449 |
set in this top-level ``CMakeLists.txt`` file are those that do not impact the |
| 450 |
list of package enables/disables. The latter type of defaults should set in |
| 451 |
other files (see below). |
| 452 |
|
| 453 |
In this example project, a CMake cache variable `${PROJECT_NAME}_TRIBITS_DIR`_ |
| 454 |
must be set by the user to define where the base ``tribits`` source directory |
| 455 |
is located. With this variable set (i.e. passed into ``cmake`` command-line |
| 456 |
use ``-DTribitsExProj_TRIBITS_DIR=<someDir>``), one just includes a single |
| 457 |
file to pull in the TriBITS system:: |
| 458 |
|
| 459 |
include("${${PROJECT_NAME}_TRIBITS_DIR}/TriBITS.cmake") |
| 460 |
|
| 461 |
With the ``TriBITS.cmake`` file included, the configuration of the project |
| 462 |
using TriBITS occurs with a single call to `tribits_project()`_. |
| 463 |
|
| 464 |
Some projects, like Trilinos, actually snapshot the `TriBITS/tribits/`_ |
| 465 |
directory into their source tree `<projectDir>/cmake/tribits/`_ and therefore |
| 466 |
don't need to have this variable set. In Trilinos, the include line is just:: |
| 467 |
|
| 468 |
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/tribits/TriBITS.cmake) |
| 469 |
|
| 470 |
The minimum CMake version must also be declared in the top-level |
| 471 |
``CMakeLists.txt`` file as shown. Explicitly setting the minimum CMake |
| 472 |
version avoids strange errors that can occur when someone tries to build the |
| 473 |
project using a version of CMake that is too old. The project should set the |
| 474 |
minimum CMake version based on the CMake features used in that project's own |
| 475 |
CMake files. The minimum CMake version required by TriBITS is defined in |
| 476 |
the variable ``TRIBITS_CMAKE_MINIMUM_REQUIRED`` (the current minimum version |
| 477 |
of CMake required by TriBITS is given at in `Getting set up to use CMake`_) . |
| 478 |
For example, the ``VERA/CMakeLists.txt`` file lists as its first line:: |
| 479 |
|
| 480 |
set(VERA_TRIBITS_CMAKE_MINIMUM_REQUIRED 3.23.0) |
| 481 |
cmake_minimum_required(VERSION ${VERA_TRIBITS_CMAKE_MINIMUM_REQUIRED} |
| 482 |
FATAL_ERROR) |
| 483 |
|
| 484 |
|
| 485 |
.. _<projectDir>/CTestConfig.cmake: |
| 486 |
|
| 487 |
**<projectDir>/CTestConfig.cmake**: [Optional] Specifies the CDash site and |
| 488 |
project to submit results to when doing an automated build driven by the CTest |
| 489 |
driver function `tribits_ctest_driver()`_ (see `TriBITS CTest/CDash Driver`_). |
| 490 |
This file is also required to use the TriBITS-generated ``dashboard`` target |
| 491 |
(see `Dashboard Submissions`_). An example of this file is |
| 492 |
`TribitsExampleProject`_/``CTestConfig.cmake``: |
| 493 |
|
| 494 |
.. include:: ../../examples/TribitsExampleProject/CTestConfig.cmake |
| 495 |
:literal: |
| 496 |
|
| 497 |
Most of the variables set in this file are directly understood by raw |
| 498 |
``ctest`` and those variables not be explained here further (see documentation |
| 499 |
for the standard CMake module ``CTest``). The usage of the function |
| 500 |
`set_default_and_from_env()`_ allows the variables to be overridden both as |
| 501 |
CMake cache variables and in the environment. The latter is needed when |
| 502 |
running using ``ctest`` as the driver (since older versions of ``ctest`` did |
| 503 |
not support ``-D<var-name>:<type>=<value>`` command-line arguments like for |
| 504 |
``cmake``). Given that all of these variables are nicely namespaced, |
| 505 |
overriding them in the shell environment is not as dangerous as might |
| 506 |
otherwise be the case but this is what had to be done to get around |
| 507 |
limitations for older versions of CMake/CTest. |
| 508 |
|
| 509 |
NOTE: One can also set:: |
| 510 |
|
| 511 |
set_default_and_from_env(TRIBITS_2ND_CTEST_DROP_SITE ...) |
| 512 |
set_default_and_from_env(TRIBITS_2ND_CTEST_DROP_LOCATION ...) |
| 513 |
|
| 514 |
in this file in order to submit to a second CDash site/location. For details, |
| 515 |
see `Dashboard Submissions`_. This is useful when considering a CDash upgrade |
| 516 |
and/or implementing new CDash features or tweaks. |
| 517 |
|
| 518 |
|
| 519 |
.. _<projectDir>/Version.cmake: |
| 520 |
|
| 521 |
**<projectDir>/Version.cmake**: If defined, gives the project's version and |
| 522 |
determines development/release mode (see `Project and Repository Versioning |
| 523 |
and Release Mode`_). This file is read in (using ``include()``) in the |
| 524 |
project's base-level ``<projectDir>/CMakeLists.txt`` file scope so local |
| 525 |
variables set in this file are seen by the entire CMake project. For example, |
| 526 |
`TribitsExampleProject`_/``Version.cmake``, looks like: |
| 527 |
|
| 528 |
.. include:: ../../examples/TribitsExampleProject/Version.cmake |
| 529 |
:literal: |
| 530 |
|
| 531 |
When this file exists in the base project, these will be used to create |
| 532 |
standard SOVERSION symlinks to shared libs. For example, on Linux, in |
| 533 |
addition to the real shared lib ``lib<libname>.so``, the standard SOVERSION |
| 534 |
symlinks are created like:: |
| 535 |
|
| 536 |
lib<libname>.so.01 |
| 537 |
lib<libname>.so.1.1 |
| 538 |
|
| 539 |
When this file exists at the repository level, the prefix |
| 540 |
``${REPOSITORY_NAME}_`` is used instead of hard-coding the project name. This |
| 541 |
is so that the same ``Version.txt`` file can be used as the |
| 542 |
`<repoDir>/Version.cmake`_ file and have the repository name be flexible. |
| 543 |
TriBITS sets ``REPOSITORY_NAME = ${PROJECT_NAME}`` when it reads in this file |
| 544 |
at the project-level scope. |
| 545 |
|
| 546 |
It is strongly recommended that every TriBITS project contain a |
| 547 |
``Version.cmake`` file, even if a release has never occurred. Otherwise, the |
| 548 |
project needs to define the variable |
| 549 |
`${PROJECT_NAME}_ENABLE_DEVELOPMENT_MODE_DEFAULT`_ at the global project scope |
| 550 |
(perhaps in ``<projectDir>/ProjectName.cmake``) to get right development mode |
| 551 |
behavior. |
| 552 |
|
| 553 |
|
| 554 |
.. _<projectDir>/project-checkin-test-config.py: |
| 555 |
|
| 556 |
**<projectDir>/project-checkin-test-config.py**: [Optional] Used to define the |
| 557 |
``--default-builds`` and other project-level configuration options for the |
| 558 |
project's usage of the `checkin-test.py`_ tool. Machine or package-specific |
| 559 |
options should **not** be placed in this file. An example of this file for |
| 560 |
`TribitsExampleProject`_/``project-checkin-test-config.py`` is shown below: |
| 561 |
|
| 562 |
.. include:: ../../examples/TribitsExampleProject/project-checkin-test-config.py |
| 563 |
:literal: |
| 564 |
|
| 565 |
The contents of the file ``project-checkin-test-config.py`` show above are |
| 566 |
pretty self explanatory. This file defines a single python dictionary |
| 567 |
data-structure called ``configuration`` which gives some default arguments in |
| 568 |
``defaults``, and then cmake options that define the projects |
| 569 |
``--default-builds``. For more details, see the section `Pre-push Testing |
| 570 |
using checkin-test.py`_. |
| 571 |
|
| 572 |
|
| 573 |
.. _<projectDir>/cmake/NativeRepositoriesList.cmake: |
| 574 |
|
| 575 |
**<projectDir>/cmake/NativeRepositoriesList.cmake**: [Deprecated] If present, |
| 576 |
this file gives the list of native repositories for the TriBITS project. The |
| 577 |
file must contain a ``set()`` statement defining the variable |
| 578 |
``${PROJECT_NAME}_NATIVE_REPOSITORIES`` which is just a flat list of repository |
| 579 |
names that must also be directory names under ``<projectDir>/``. For example, |
| 580 |
if this file contains:: |
| 581 |
|
| 582 |
set(${PROJECT_NAME}_NATIVE_REPOSITORIES Repo0 Repo1) |
| 583 |
|
| 584 |
then the directories ``<projectDir>/Repo0/`` and ``<projectDir>/Repo1/`` must |
| 585 |
exist and must be valid TriBITS repositories (see `TriBITS Repository`_). |
| 586 |
|
| 587 |
There are no examples for the usage of this file in any of the TriBITS |
| 588 |
examples or test projects. However, support for this file is maintained for |
| 589 |
backward compatibility since there may be some TriBITS projects that still use |
| 590 |
it. It is recommended instead to define multiple repositories using the |
| 591 |
`<projectDir>/cmake/ExtraRepositoriesList.cmake`_ file as it allows for more |
| 592 |
flexibility in how extra repositories are specified and how they are accessed. |
| 593 |
The latter file allows the various tools to perform version control (VC) |
| 594 |
activities with these repos while "native repositories" do not. |
| 595 |
|
| 596 |
If this file ``NativeRepositoriesList.cmake`` does not exist, then TriBITS |
| 597 |
sets ``${PROJECT_NAME}_NATIVE_REPOSITORIES`` equal to ".", or the base project |
| 598 |
directory (i.e. ``<projectDir>/.``). In this case, the file |
| 599 |
``<projectDir>/PackagesList.cmake`` and ``<projectDir>/TPLsList.cmake`` must |
| 600 |
exist. However, if the project has no native packages or external |
| 601 |
packages/TPLs, then these files can be set up with empty lists. This is the |
| 602 |
case for meta-projects like CASL VERA that have only extra repositories |
| 603 |
specified in the file `<projectDir>/cmake/ExtraRepositoriesList.cmake`_. |
| 604 |
|
| 605 |
.. _<projectDir>/cmake/ExtraRepositoriesList.cmake: |
| 606 |
|
| 607 |
**<projectDir>/cmake/ExtraRepositoriesList.cmake**: [Optional] If present, |
| 608 |
this file defines a list of extra repositories that are added on to the |
| 609 |
project's native repositories. The list of repositories is defined using the |
| 610 |
macro `tribits_project_define_extra_repositories()`_. For example, the extra |
| 611 |
repos file: |
| 612 |
|
| 613 |
.. include:: ExtraReposList.cmake |
| 614 |
:literal: |
| 615 |
|
| 616 |
shows the specification of both TriBITS Repositories and non-TriBITS VC |
| 617 |
Repositories. In the above file, the repositories ``ExtraRepo1``, |
| 618 |
``ExtraRepo3``, and ``ExtraRepo4`` are both TriBITS and VC repositories that |
| 619 |
are cloned into directories under ``<projectDir>`` of the same names from the |
| 620 |
URLs ``someurl.com:/ExtraRepo1``, ``someurl3.com:/ExtraRepo3``, and |
| 621 |
``someurl4.com:/ExtraRepo4``, respectively. However, the repository |
| 622 |
``ExtraRepo2`` is **not** a `TriBITS Repository`_ because it is marked as |
| 623 |
``NOPACKAGES``. In this case, it gets cloned as the directory:: |
| 624 |
|
| 625 |
<projectDir>/packages/SomePackage/Blah |
| 626 |
|
| 627 |
However, the code in the tools `checkin-test.py`_ and |
| 628 |
`tribits_ctest_driver()`_ will consider non-TriBITS VC repos like |
| 629 |
``ExtraRepo2`` and any changes to this repository will be listed as changes to |
| 630 |
``somePackage`` (see `Pre-push Testing using checkin-test.py`_). |
| 631 |
|
| 632 |
NOTE: This file can be overridden by setting the cache variable |
| 633 |
`<Project>_EXTRAREPOS_FILE`_. |
| 634 |
|
| 635 |
.. ToDO: I am not really sure what repos get cloned the first time based on |
| 636 |
.. the list of extra repos. From looking at TribitsCTestDriverCore.cmake, it |
| 637 |
.. looks like only the selected repos will be cloned. I need to add some unit |
| 638 |
.. tests that really show what the real behavior is and then document that |
| 639 |
.. behavior here. |
| 640 |
|
| 641 |
|
| 642 |
.. _<projectDir>/cmake/ProjectCiFileChangeLogic.py: |
| 643 |
|
| 644 |
**<projectDir>/cmake/ProjectCiFileChangeLogic.py**: [Optional] If present, |
| 645 |
then this Python module is imported and the Python class defined there |
| 646 |
ProjectCiFileChangeLogic there is used to determine which files need to |
| 647 |
trigger a global rebuild of the project enabling all packages. |
| 648 |
|
| 649 |
An example of this given in the file |
| 650 |
``TribitsExampleProject/cmake/ProjectCiFileChangeLogic.py``: |
| 651 |
|
| 652 |
.. include:: ../../examples/TribitsExampleProject/cmake/ProjectCiFileChangeLogic.py |
| 653 |
:literal: |
| 654 |
|
| 655 |
This logic is used in all code that is used in CI testing including |
| 656 |
`checkin-test.py`_, `tribits_ctest_driver()`_ and |
| 657 |
`get-tribits-packages-from-files-list.py`_. If this file does not exist, then |
| 658 |
TriBITS has some default logic which may or may not be sufficient for the |
| 659 |
needs of a given project. |
| 660 |
|
| 661 |
|
| 662 |
.. _<projectDir>/cmake/ProjectCompilerPostConfig.cmake: |
| 663 |
|
| 664 |
**<projectDir>/cmake/ProjectCompilerPostConfig.cmake**: [Optional] If present, |
| 665 |
then this file is read using ``include()`` at the top-level CMakeLists.txt |
| 666 |
file scope right after the compilers for the languages ``<LANG>`` = ``C``, |
| 667 |
``CXX``, and ``Fortran`` are determined and checked using |
| 668 |
``enable_language(<LANG>)`` but before any other checks are performed. This |
| 669 |
file can contain logic for the project to adjust the flags set in |
| 670 |
``CMAKE_<LANG>_FLAGS`` and changes to other aspects of the build flags |
| 671 |
(including link flags, etc.). |
| 672 |
|
| 673 |
One example of the usage of this file is the Trilinos project where this file |
| 674 |
is (or was) used to apply specialized logic implemented in the Kokkos build |
| 675 |
system to select compiler options and to determine how C++11 and OpenMP flags |
| 676 |
are set. This file in Trilinos looked like:: |
| 677 |
|
| 678 |
if (${Trilinos_ENABLE_Kokkos}) |
| 679 |
|
| 680 |
... |
| 681 |
|
| 682 |
include(${Kokkos_GEN_DIR}/kokkos_generated_settings.cmake) |
| 683 |
|
| 684 |
if (NOT KOKKOS_ARCH STREQUAL "None") |
| 685 |
|
| 686 |
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${KOKKOS_CXX_FLAGS}") |
| 687 |
|
| 688 |
message("-- " "Skip adding flags for OpenMP because Kokkos flags does that ...") |
| 689 |
set(OpenMP_CXX_FLAGS_OVERRIDE " ") |
| 690 |
|
| 691 |
endif() |
| 692 |
|
| 693 |
endif() |
| 694 |
|
| 695 |
The exact context where this file is processed (if it exists) is described in |
| 696 |
`Full Processing of TriBITS Project Files`_ and `TriBITS Environment Probing |
| 697 |
and Setup`_. |
| 698 |
|
| 699 |
|
| 700 |
.. _<projectDir>/cmake/ProjectDependenciesSetup.cmake: |
| 701 |
|
| 702 |
**<projectDir>/cmake/ProjectDependenciesSetup.cmake**: [Optional] If present, |
| 703 |
this file is included a single time as part of the generation of the project's |
| 704 |
dependency data-structure (see `Reduced Package Dependency Processing`_). It |
| 705 |
gets included at the top project level scope after all of the |
| 706 |
`<repoDir>/cmake/RepositoryDependenciesSetup.cmake`_ files have been included |
| 707 |
but before all of the package `<packageDir>/cmake/Dependencies.cmake`_ files |
| 708 |
are included. Any local variables set in this file have project-wide scope. |
| 709 |
The primary purpose for this file is to set variables that will impact the |
| 710 |
processing of project's package ``Dependencies.cmake`` files. |
| 711 |
|
| 712 |
The typical usage of this file is to set the default CDash email address for |
| 713 |
all packages or override the email addresses for all of a repository's package |
| 714 |
CDash regression email addresses (see `CDash regression email addresses`_). |
| 715 |
For example, to set the default email address for all of the packages, one |
| 716 |
would set in this file:: |
| 717 |
|
| 718 |
set_default(${PROJECT_NAME}_PROJECT_MASTER_EMAIL_ADDRESS |
| 719 |
[email protected]) |
| 720 |
|
| 721 |
The repository email address variables |
| 722 |
`${REPOSITORY_NAME}_REPOSITORY_EMAIL_URL_ADDRESS_BASE`_ and |
| 723 |
`${REPOSITORY_NAME}_REPOSITORY_MASTER_EMAIL_ADDRESS`_ possibly set in the just |
| 724 |
processed `<repoDir>/cmake/RepositoryDependenciesSetup.cmake`_ files can also |
| 725 |
be overridden in this file. The CASL VERA meta-project uses this file to |
| 726 |
override several of the repository-specific email addresses for its |
| 727 |
constituent repositories. |
| 728 |
|
| 729 |
In general, variables that affect how package dependencies are defined or |
| 730 |
affect package enable/disable logic for *only this particular project* should |
| 731 |
be defined in this file. |
| 732 |
|
| 733 |
.. _<projectDir>/cmake/CallbackDefineProjectPackaging.cmake: |
| 734 |
|
| 735 |
**<projectDir>/cmake/CallbackDefineProjectPackaging.cmake**: [Optional] If |
| 736 |
exists, defines the CPack settings for the project (see `Official CPack |
| 737 |
Documentation <http://www.cmake.org/cmake/help/documentation.html>`_ and |
| 738 |
`Online CPack Wiki <http://www.cmake.org/Wiki/CMake:Packaging_With_CPack>`_). |
| 739 |
This file must define a macro called ``tribits_project_define_packaging()`` |
| 740 |
which is then invoked by TriBITS. The file: |
| 741 |
|
| 742 |
`TribitsExampleProject`_/``cmake/CallbackDefineProjectPackaging.cmake`` |
| 743 |
|
| 744 |
provides a good example which is: |
| 745 |
|
| 746 |
.. include:: ../../examples/TribitsExampleProject/cmake/CallbackDefineProjectPackaging.cmake |
| 747 |
:literal: |
| 748 |
|
| 749 |
The CPack variables show above that should be defined at the project-level are |
| 750 |
described in the `Official CPack Documentation`_. |
| 751 |
|
| 752 |
Settings that are general for all distributions (like non-package repository |
| 753 |
files to exclude from the tarball) should be set at the in the file |
| 754 |
`<repoDir>/cmake/CallbackDefineRepositoryPackaging.cmake`_. See `Creating |
| 755 |
Source Distributions` for more details. |
| 756 |
|
| 757 |
.. _<projectDir>/cmake/tribits/: |
| 758 |
|
| 759 |
**<projectDir>/cmake/tribits/**: [Optional] This is the typical location of |
| 760 |
the `TriBITS/tribits/`_ source tree for projects that choose to snapshot |
| 761 |
TriBITS into their source tree. In fact, TriBITS assumes this is the default |
| 762 |
location for the TriBITS source tree if ``${PROJECT_NAME}_TRIBITS_DIR`` is not |
| 763 |
otherwise specified. Trilinos, for example, currently snapshots the TriBITS |
| 764 |
source tree into this directory. See `TriBITS directory snapshotting`_ for |
| 765 |
more details. |
| 766 |
|
| 767 |
.. _<projectDir>/cmake/ctest/CTestCustom.cmake.in: |
| 768 |
|
| 769 |
**<projectDir>/cmake/ctest/CTestCustom.cmake.in**: [Optional] If this file |
| 770 |
exists, it is processed using a ``configure_file()`` command to write the file |
| 771 |
``CTestCustom.cmake`` in the project base build directory |
| 772 |
```${PROJECT_BINARY_DIR}/``. This file is picked up automatically by |
| 773 |
``ctest`` (see `CTest documentation`_). This file is typically used to change |
| 774 |
the maximum size of test output. For example, the |
| 775 |
`TribitsExampleProject`_/``cmake/ctest/CTestCustom.cmake.in`` looks like: |
| 776 |
|
| 777 |
.. include:: ../../examples/TribitsExampleProject/cmake/ctest/CTestCustom.cmake.in |
| 778 |
:literal: |
| 779 |
|
| 780 |
which sets the output size for each test submitted to CDash be unlimited |
| 781 |
(which is not really recommended). These variables used by Trilinos at one |
| 782 |
time were:: |
| 783 |
|
| 784 |
set(CTEST_CUSTOM_MAXIMUM_PASSED_TEST_OUTPUT_SIZE 50000) |
| 785 |
set(CTEST_CUSTOM_MAXIMUM_FAILED_TEST_OUTPUT_SIZE 5000000) |
| 786 |
|
| 787 |
which sets the max output for passed and failed tests to 50000k and 5000000k, |
| 788 |
respectively. |
| 789 |
|
| 790 |
For documentation of the options one can change for CTest, see online `CTest |
| 791 |
documentation`_. |
| 792 |
|
| 793 |
|
| 794 |
TriBITS Project Core Variables |
| 795 |
.............................. |
| 796 |
|
| 797 |
The following `local variables` are defined in the top-level Project |
| 798 |
``CMakeLists.txt`` file scope and are therefore accessible by all files |
| 799 |
processed by TriBITS: |
| 800 |
|
| 801 |
.. _PROJECT_NAME: |
| 802 |
|
| 803 |
``PROJECT_NAME`` |
| 804 |
|
| 805 |
The name of the TriBITS Project. This exists to support, among other |
| 806 |
things, the ability for subordinate units (Repositories and Packages) to |
| 807 |
determine the Project in which is participating. This is typically read |
| 808 |
from a ``set()`` statement in the project's |
| 809 |
`<projectDir>/ProjectName.cmake`_ file. |
| 810 |
|
| 811 |
.. _PROJECT_SOURCE_DIR: |
| 812 |
|
| 813 |
``PROJECT_SOURCE_DIR`` |
| 814 |
|
| 815 |
The absolute path to the base Project source directory. This is set |
| 816 |
automatically by TriBITS given the directory passed into ``cmake`` at |
| 817 |
configure time at the beginning of the `tribits_project()`_ macro. |
| 818 |
|
| 819 |
.. _PROJECT_BINARY_DIR: |
| 820 |
|
| 821 |
``PROJECT_BINARY_DIR`` |
| 822 |
|
| 823 |
The absolute path to the base Project binary/build directory. This is set |
| 824 |
automatically by TriBITS and is the directory where ``cmake`` is run from |
| 825 |
and is set at the beginning of the `tribits_project()`_ macro. |
| 826 |
|
| 827 |
.. _${PROJECT_NAME}_SOURCE_DIR: |
| 828 |
|
| 829 |
``${PROJECT_NAME}_SOURCE_DIR`` |
| 830 |
|
| 831 |
Set to the same directory as ``${PROJECT_SOURCE_DIR}`` automatically by |
| 832 |
the built-in ``project()`` command called in the top-level |
| 833 |
`<projectDir>/CMakeLists.txt`_ file.. |
| 834 |
|
| 835 |
.. _${PROJECT_NAME}_BINARY_DIR: |
| 836 |
|
| 837 |
``${PROJECT_NAME}_BINARY_DIR`` |
| 838 |
|
| 839 |
Set to the same directory as ``${PROJECT_BINARY_DIR}`` automatically by |
| 840 |
the built-in ``project()`` command called in the top-level |
| 841 |
`<projectDir>/CMakeLists.txt`_ file.. |
| 842 |
|
| 843 |
The following `cache variables` are defined for every TriBITS project: |
| 844 |
|
| 845 |
.. _${PROJECT_NAME}_TRIBITS_DIR: |
| 846 |
|
| 847 |
``${PROJECT_NAME}_TRIBITS_DIR`` |
| 848 |
|
| 849 |
CMake cache variable that gives the path to the TriBITS implementation |
| 850 |
directory. When set to a relative path (set as type ``STRING``, see |
| 851 |
below), this is taken relative to ``${CMAKE_CURRENT_SOURCE_DIR}/`` (the |
| 852 |
project base source dir). When an absolute path is given, it is used |
| 853 |
without modification. If this variable is not set in the |
| 854 |
`<projectDir>/CMakeLists.txt`_ file, then it will be automatically set as |
| 855 |
a ``PATH`` cache variable by the include of ``TriBITS.cmake`` by the |
| 856 |
statement :: |
| 857 |
|
| 858 |
set( ${PROJECT_NAME}_TRIBITS_DIR |
| 859 |
"${CMAKE_CURRENT_SOURCE_DIR}/cmake/tribits" CACHE PATH "...") |
| 860 |
|
| 861 |
Therefore, projects that snapshot TriBITS into |
| 862 |
`<projectDir>/cmake/tribits/`_ don't need to explicitly set |
| 863 |
``${PROJECT_NAME}_TRIBITS_DIR``. In addition, one can also point to a |
| 864 |
different TriBITS implementation just by setting the absolute path:: |
| 865 |
|
| 866 |
-D <Project>_TRIBITS_DIR=<some-abs-dir> |
| 867 |
|
| 868 |
or to a relative path using, for example:: |
| 869 |
|
| 870 |
-D <Project>_TRIBITS_DIR:STRING=TriBITS/tribits |
| 871 |
|
| 872 |
Note that when the ``TriBITS`` git repo itself is cloned by a TriBITS |
| 873 |
project, then ``${PROJECT_NAME}_TRIBITS_DIR`` should be set to the |
| 874 |
directory ``TriBITS/tribits`` (see `TriBITS/tribits/`_) as shown above. |
| 875 |
|
| 876 |
.. _${PROJECT_NAME}_ENABLE_TESTS: |
| 877 |
|
| 878 |
``${PROJECT_NAME}_ENABLE_TESTS`` |
| 879 |
|
| 880 |
CMake cache variable that if set to ``ON``, then tests for all explicitly |
| 881 |
enabled packages will be turned on. This has a default value of ``OFF``. |
| 882 |
This is used in logic to enable individual package tests (see |
| 883 |
`<Project>_ENABLE_TESTS only enables explicitly enabled package |
| 884 |
tests`_). |
| 885 |
|
| 886 |
``${PACKAGE_NAME}_ENABLE_EXAMPLES`` |
| 887 |
|
| 888 |
CMake cache variable that if set to ``ON``, then examples for all |
| 889 |
explicitly enabled packages will be turned on. This has a default value |
| 890 |
of ``OFF``. |
| 891 |
|
| 892 |
The following `internal project-scope local` (non-cache) CMake variables are |
| 893 |
defined by TriBITS giving the project's TriBITS repositories.: |
| 894 |
|
| 895 |
.. _${PROJECT_NAME}_NATIVE_REPOSITORIES: |
| 896 |
|
| 897 |
``${PROJECT_NAME}_NATIVE_REPOSITORIES`` |
| 898 |
|
| 899 |
The list of Native Repositories for a given TriBITS project |
| 900 |
(i.e. Repositories that are always present when configuring the Project |
| 901 |
and are managed in the same VC repo typically). This variable is set in |
| 902 |
the file `<projectDir>/cmake/NativeRepositoriesList.cmake`_ if it exists. |
| 903 |
If the file ``NativeRepositoriesList.cmake`` does not exist, then the |
| 904 |
project is assumed to also be a repository and the list of native |
| 905 |
repositories is just the local project directory |
| 906 |
``${PROJECT_SOURCE_DIR}/.``. In this case, the |
| 907 |
``${PROJECT_SOURCE_DIR}/`` must contain at a minimum a |
| 908 |
``PackagesList.cmake`` file, and a ``TPLsList.cmake`` file (see `TriBITS |
| 909 |
Repository`_). |
| 910 |
|
| 911 |
.. _${PROJECT_NAME}_EXTRA_REPOSITORIES: |
| 912 |
|
| 913 |
``${PROJECT_NAME}_EXTRA_REPOSITORIES`` |
| 914 |
|
| 915 |
The list of Extra Repositories that the project is being configured with. |
| 916 |
This list of repositories either comes from processing the project's |
| 917 |
`<projectDir>/cmake/ExtraRepositoriesList.cmake`_ file or comes from the |
| 918 |
CMake cache variable ``${PROJECT_NAME}_EXTRA_REPOSITORIES``. See |
| 919 |
`Enabling extra repositories with add-on packages`_ for details. |
| 920 |
|
| 921 |
.. _${PROJECT_NAME}_ALL_REPOSITORIES: |
| 922 |
|
| 923 |
``${PROJECT_NAME}_ALL_REPOSITORIES`` |
| 924 |
|
| 925 |
Concatenation of all the repos listed in |
| 926 |
`${PROJECT_NAME}_NATIVE_REPOSITORIES`_ and |
| 927 |
`${PROJECT_NAME}_EXTRA_REPOSITORIES`_ in the order they are processed. |
| 928 |
|
| 929 |
|
| 930 |
TriBITS Repository |
| 931 |
++++++++++++++++++ |
| 932 |
|
| 933 |
A TriBITS Repository is the basic unit of ready-made composition between |
| 934 |
different collections of software that use the TriBITS CMake build and system. |
| 935 |
|
| 936 |
In short, a TriBITS Repository: |
| 937 |
|
| 938 |
* Is a named collection of related TriBITS Packages and external packages/TPLs |
| 939 |
(defined in `<repoDir>/PackagesList.cmake`_ and `<repoDir>/TPLsList.cmake`_ |
| 940 |
respectively) |
| 941 |
* Defines the base source and binary directories for the Repository |
| 942 |
``${REPOSITORY_NAME}_SOURCE_DIR`` and ``${REPOSITORY_NAME}_BINARY_DIR``. |
| 943 |
* Defines a common set of initializations and other hooks for all the |
| 944 |
packages in the repository. |
| 945 |
* Typically maps to a VC (i.e. git) repository and therefore represents a unit |
| 946 |
of integration, versioning and reuse. (But core TriBITS has no dependency |
| 947 |
on any VC tool.) |
| 948 |
|
| 949 |
For more details on the definition of a TriBITS Repository, see: |
| 950 |
|
| 951 |
* `TriBITS Repository Core Files`_ |
| 952 |
* `TriBITS Repository Core Variables`_ |
| 953 |
|
| 954 |
TriBITS Repository Core Files |
| 955 |
............................. |
| 956 |
|
| 957 |
The core files making up a TriBITS Repository (where ``<repoDir> = |
| 958 |
${${REPOSITORY_NAME}_SOURCE_DIR}``) are:: |
| 959 |
|
| 960 |
<repoDir>/ |
| 961 |
PackagesList.cmake |
| 962 |
TPLsList.cmake |
| 963 |
Copyright.txt # [Optional] Only needed if creating version header file |
| 964 |
Version.cmake # [Optional] Info inserted into ${REPO_NAME}_version.h |
| 965 |
cmake/ |
| 966 |
RepositoryDependenciesSetup.cmake # [Optional] CDash email addresses? |
| 967 |
CallbackSetupExtraOptions.cmake # [Optional] Called after main options |
| 968 |
CallbackDefineRepositoryPackaging.cmake # [Optional] CPack packaging |
| 969 |
|
| 970 |
These TriBITS Repository files are documented in more detail below: |
| 971 |
|
| 972 |
* `<repoDir>/PackagesList.cmake`_ |
| 973 |
* `<repoDir>/TPLsList.cmake`_ |
| 974 |
* `<repoDir>/Copyright.txt`_ |
| 975 |
* `<repoDir>/Version.cmake`_ |
| 976 |
* `<repoDir>/cmake/RepositoryDependenciesSetup.cmake`_ |
| 977 |
* `<repoDir>/cmake/CallbackSetupExtraOptions.cmake`_ |
| 978 |
* `<repoDir>/cmake/CallbackDefineRepositoryPackaging.cmake`_ |
| 979 |
|
| 980 |
|
| 981 |
.. _<repoDir>/PackagesList.cmake: |
| 982 |
|
| 983 |
**<repoDir>/PackagesList.cmake**: [Required] Provides the list of top-level |
| 984 |
packages defined by the repository. This file typically just calls the macro |
| 985 |
`tribits_repository_define_packages()`_ to define the list of packages along |
| 986 |
with their directories and other properties. For example, the file |
| 987 |
`TribitsExampleProject`_/``PackagesList.cmake`` looks like: |
| 988 |
|
| 989 |
.. include:: ../../examples/TribitsExampleProject/PackagesList.cmake |
| 990 |
:literal: |
| 991 |
|
| 992 |
Other commands that are appropriate to use in this file include |
| 993 |
`tribits_disable_package_on_platforms()`_ and |
| 994 |
`tribits_allow_missing_external_packages()`_. Also, if the binary directory |
| 995 |
for any package ``<packageName>`` needs to be changed from the default, then |
| 996 |
the variable ``<packageName>_SPECIFIED_BINARY_DIR`` can be set. (see `TriBITS |
| 997 |
Package == TriBITS Repository == TriBITS Project`_). |
| 998 |
|
| 999 |
It is perfectly legal for a TriBITS repository to define no packages at all |
| 1000 |
with:: |
| 1001 |
|
| 1002 |
tribits_repository_define_packages() |
| 1003 |
|
| 1004 |
and this would be the case for a TriBITS meta-project that has no native |
| 1005 |
packages, only extra repositories. |
| 1006 |
|
| 1007 |
.. ToDo: Point to example meta-project. |
| 1008 |
|
| 1009 |
|
| 1010 |
.. _<repoDir>/TPLsList.cmake: |
| 1011 |
|
| 1012 |
**<repoDir>/TPLsList.cmake**: [Required] Provides the list of external |
| 1013 |
packages/TPLs that are referenced as dependencies in the repository's |
| 1014 |
package's `<packageDir>/cmake/Dependencies.cmake`_ files (see `TriBITS |
| 1015 |
External Package/TPL`_). This file typically just calls the macro |
| 1016 |
`tribits_repository_define_tpls()`_ to define the TPLs along with their find |
| 1017 |
modules and other properties. An example is |
| 1018 |
`ReducedMockTrilinos`_/``TPLsList.cmake`` which shows: |
| 1019 |
|
| 1020 |
.. include:: ../../examples/ReducedMockTrilinos/TPLsList.cmake |
| 1021 |
:literal: |
| 1022 |
|
| 1023 |
See `TriBITS External Package/TPL`_ for details on what gets defined for each |
| 1024 |
TriBITS TPL once this file is processed. |
| 1025 |
|
| 1026 |
It is perfectly fine to specify no TPLs at all for a repository with:: |
| 1027 |
|
| 1028 |
tribits_repository_define_tpls() |
| 1029 |
|
| 1030 |
but the macro ``tribits_repository_define_tpls()`` has to be called, even if |
| 1031 |
there are no TPLs. See `tribits_repository_define_tpls()`_ for further |
| 1032 |
details and constraints. |
| 1033 |
|
| 1034 |
|
| 1035 |
.. _<repoDir>/Copyright.txt: |
| 1036 |
|
| 1037 |
**<repoDir>/Copyright.txt**: [Optional] Gives the default copyright and |
| 1038 |
license declaration for all of the software in the TriBITS repository |
| 1039 |
directory ``<repoDir>/``. This file is read into a string and then used to |
| 1040 |
configure the repository's version header file (see `Project and Repository |
| 1041 |
Versioning and Release Mode`_). Even if a repository version header file is |
| 1042 |
not produced, it is a good idea for every TriBITS repository to define this |
| 1043 |
file, just for legal purposes. For a good open-source license, one should |
| 1044 |
consider copying the ``TriBITS/Copyright.txt`` file which is a simple 3-clause |
| 1045 |
BSD-like license like: |
| 1046 |
|
| 1047 |
.. include:: ../../Copyright.txt |
| 1048 |
:literal: |
| 1049 |
|
| 1050 |
.. _<repoDir>/Version.cmake: |
| 1051 |
|
| 1052 |
**<repoDir>/Version.cmake**: [Optional] Contains version information for the |
| 1053 |
repository (and the project also if this is also the base project). For |
| 1054 |
example, `TribitsExampleProject`_/``Version.cmake``, this looks like: |
| 1055 |
|
| 1056 |
.. include:: ../../examples/TribitsExampleProject/Version.cmake |
| 1057 |
:literal: |
| 1058 |
|
| 1059 |
Note that the prefix ``${REPOSITORY_NAME}_`` is used instead of hard-coding |
| 1060 |
the repository's name to allow flexibility in what a meta-project names a |
| 1061 |
given TriBITS repository. |
| 1062 |
|
| 1063 |
The local variables in these set statements are processed in the base project |
| 1064 |
directory's local scope and are therefore seen by the entire CMake project. |
| 1065 |
When this file is read in repository mode, the variable |
| 1066 |
``${REPOSITORY_NAME}_ENABLE_DEVELOPMENT_MODE_DEFAULT`` is ignored. |
| 1067 |
|
| 1068 |
.. _<repoDir>/cmake/RepositoryDependenciesSetup.cmake: |
| 1069 |
|
| 1070 |
**<repoDir>/cmake/RepositoryDependenciesSetup.cmake**: [Optional] If present, |
| 1071 |
this file is included a single time as part of the generation of the project |
| 1072 |
dependency data-structure (see `Reduced Package Dependency Processing`_). It |
| 1073 |
gets included in the order listed in `${PROJECT_NAME}_ALL_REPOSITORIES`_. Any |
| 1074 |
local variables set in this file have project-wide scope. The primary purpose |
| 1075 |
for this file is to set variables that will impact the processing of project's |
| 1076 |
package's ``Dependencies.cmake`` files and take care of other enable/disable |
| 1077 |
issues that are not otherwise cleanly handled by the TriBITS system |
| 1078 |
automatically. |
| 1079 |
|
| 1080 |
The typical usage of this file is to set the default CDash email address for |
| 1081 |
all of the defined packages (see `CDash regression email addresses`_). For |
| 1082 |
example, to set the default email address for all of the packages in this |
| 1083 |
repository, one would set in this file:: |
| 1084 |
|
| 1085 |
set_default(${REPOSITORY_NAME}_REPOSITORY_MASTER_EMAIL_ADDRESS |
| 1086 |
[email protected]) |
| 1087 |
|
| 1088 |
Note that the prefix ``${REPOSITORY_NAME}_`` is used instead of hard-coding |
| 1089 |
the repo name to allow greater flexibility in how meta-projects refer to this |
| 1090 |
TriBITS repo. |
| 1091 |
|
| 1092 |
.. _<repoDir>/cmake/CallbackSetupExtraOptions.cmake: |
| 1093 |
|
| 1094 |
**<repoDir>/cmake/CallbackSetupExtraOptions.cmake**: [Optional] If defined, |
| 1095 |
this file is processed (included) for each repo in order right after the basic |
| 1096 |
TriBITS options are defined in the macro |
| 1097 |
``tribits_define_global_options_and_define_extra_repos()``. This file must |
| 1098 |
define the macro ``tribits_repository_setup_extra_options()`` which is then |
| 1099 |
called by the TriBITS system. This file is only processed when doing a basic |
| 1100 |
configuration of the project and **not** when it is just building up the |
| 1101 |
dependency data-structures (i.e. it is **not** processed in the `Reduced |
| 1102 |
Package Dependency Processing`_). Any local variables set have project-wide |
| 1103 |
scope. |
| 1104 |
|
| 1105 |
A few additional variables are defined by the time this file is processed and |
| 1106 |
can be used in the logic in these files. Some of the variables that should |
| 1107 |
already be defined (in addition to all of the basic user TriBITS cache |
| 1108 |
variables set in ``tribits_define_global_options_and_define_extra_repos()``) |
| 1109 |
include ``CMAKE_HOST_SYSTEM_NAME``, ``${PROJECT_NAME}_HOSTNAME``, and |
| 1110 |
``Python3_EXECUTABLE`` (see `Python Support`_). The types of commands and |
| 1111 |
logic to put in this file include: |
| 1112 |
|
| 1113 |
* Setting additional user cache variable options that are used by multiple |
| 1114 |
packages in the TriBITS Repository. For example, Trilinos defines a |
| 1115 |
``Trilinos_DATA_DIR`` user cache variable that several Trilinos packages use |
| 1116 |
to get extra test data to define extra tests. |
| 1117 |
* Disabling packages in the TriBITS Repository when conditions will not allow |
| 1118 |
them to be enabled. For example, Trilinos disables the package |
| 1119 |
``ForTrilinos`` when ``Fortran`` is disabled and disables the package |
| 1120 |
``PyTrilinos`` when Python support is disabled. |
| 1121 |
|
| 1122 |
An example of this file is: |
| 1123 |
|
| 1124 |
`TribitsExampleProject`_/``/cmake/CallbackSetupExtraOptions.cmake`` |
| 1125 |
|
| 1126 |
which currently looks like: |
| 1127 |
|
| 1128 |
.. include:: ../../examples/TribitsExampleProject/cmake/CallbackSetupExtraOptions.cmake |
| 1129 |
:literal: |
| 1130 |
|
| 1131 |
.. _<repoDir>/cmake/CallbackDefineRepositoryPackaging.cmake: |
| 1132 |
|
| 1133 |
**<repoDir>/cmake/CallbackDefineRepositoryPackaging.cmake**: [Optional] If |
| 1134 |
this file exists, then it defines extra CPack-related options that are |
| 1135 |
specific to this TriBITS Repository. This file must define the macro |
| 1136 |
``tribits_repository_define_packaging()`` which is called by TriBITS. This |
| 1137 |
file is processed as the top project-level scope so any local variables set |
| 1138 |
have project-wide effect. This file is processed after the project's |
| 1139 |
`<projectDir>/cmake/CallbackDefineProjectPackaging.cmake`_ file so any project |
| 1140 |
CPACK variables are defined for the repository-level options and commands are |
| 1141 |
created. This file typically just sets extra excludes to remove files from |
| 1142 |
the tarball. The file: |
| 1143 |
|
| 1144 |
`TribitsExampleProject`_/``cmake/CallbackDefineRepositoryPackaging.cmake`` |
| 1145 |
|
| 1146 |
provides a good example which is: |
| 1147 |
|
| 1148 |
.. include:: ../../examples/TribitsExampleProject/cmake/CallbackDefineRepositoryPackaging.cmake |
| 1149 |
:literal: |
| 1150 |
|
| 1151 |
As shown in the above example, it is important to prefix the excluded files |
| 1152 |
and directories with the repository base directory |
| 1153 |
``${${REPOSITORY_NAME}_SOURCE_DIR}/`` since these are interpreted by CPack as |
| 1154 |
regular-expressions. |
| 1155 |
|
| 1156 |
|
| 1157 |
TriBITS Repository Core Variables |
| 1158 |
................................. |
| 1159 |
|
| 1160 |
The following temporary local variables are defined automatically by TriBITS |
| 1161 |
before processing a given TriBITS repository's files |
| 1162 |
(e.g. ``PackagesList.cmake``, ``TPLsList.cmake``, etc.): |
| 1163 |
|
| 1164 |
.. _REPOSITORY_NAME: |
| 1165 |
|
| 1166 |
``REPOSITORY_NAME`` |
| 1167 |
|
| 1168 |
The name of the current TriBITS repository. This name will be the |
| 1169 |
repository name listed in |
| 1170 |
`<projectDir>/cmake/ExtraRepositoriesList.cmake`_ file or if this |
| 1171 |
repository directory is the project base directory, ``REPOSITORY_NAME`` |
| 1172 |
will be set to ``${PROJECT_NAME}``. |
| 1173 |
|
| 1174 |
``REPOSITORY_DIR`` |
| 1175 |
|
| 1176 |
Path of the current Repository *relative* to the Project's base source |
| 1177 |
directory `${PROJECT_NAME}_SOURCE_DIR`_.. This is typically just the |
| 1178 |
repository name but can be an arbitrary directory if specified through the |
| 1179 |
`<projectDir>/cmake/ExtraRepositoriesList.cmake`_ file. |
| 1180 |
|
| 1181 |
The following project-scope (non-cache) local variables are set once the list |
| 1182 |
of TriBITS repositories is processed and before any of the repository's files |
| 1183 |
are processed: |
| 1184 |
|
| 1185 |
``${REPOSITORY_NAME}_SOURCE_DIR`` |
| 1186 |
|
| 1187 |
The absolute path to the base of a given TriBITS Repository's source |
| 1188 |
directory. CMake code, for example in a packages' ``CMakeLists.txt`` |
| 1189 |
file, typically refers to this by the raw name like ``RepoX_SOURCE_DIR``. |
| 1190 |
This makes such CMake code independent of where the various TriBITS repos |
| 1191 |
are in relation to each other or the TriBITS Project (but does hard-code |
| 1192 |
the repository name which is not ideal). |
| 1193 |
|
| 1194 |
``${REPOSITORY_NAME}_BINARY_DIR`` |
| 1195 |
|
| 1196 |
The absolute path to the base of a given TriBITS Repository's binary |
| 1197 |
directory. CMake code, for example in packages, refer to this by the raw |
| 1198 |
name like ``RepoX_SOURCE_DIR``. This makes such CMake code independent of |
| 1199 |
where the various TriBITS repos are in relation to each other or the |
| 1200 |
Project. |
| 1201 |
|
| 1202 |
The following project-level local variables can be defined by the project or |
| 1203 |
the user to help define the what packages from the repository |
| 1204 |
``${REPOSITORY_NAME}`` contribute to the primary meta-project packages (PMPP): |
| 1205 |
|
| 1206 |
.. _${REPOSITORY_NAME}_NO_PRIMARY_META_PROJECT_PACKAGES: |
| 1207 |
|
| 1208 |
``${REPOSITORY_NAME}_NO_PRIMARY_META_PROJECT_PACKAGES`` |
| 1209 |
|
| 1210 |
If set to ``TRUE``, then the package's in the TriBITS repository are not |
| 1211 |
considered to be part of the primary meta-project packages. This affects |
| 1212 |
what packages get enabled by default when enabling all packages with |
| 1213 |
setting ``${PROJECT_NAME}_ENABLE_ALL_PACKAGES=ON`` and what tests and |
| 1214 |
examples get enabled by default when setting |
| 1215 |
``${PROJECT_NAME}_ENABLE_TESTS=ON``. See `TriBITS Dependency Handling |
| 1216 |
Behaviors`_ for more details. |
| 1217 |
|
| 1218 |
.. _${REPOSITORY_NAME}_NO_PRIMARY_META_PROJECT_PACKAGES_EXCEPT: |
| 1219 |
|
| 1220 |
``${REPOSITORY_NAME}_NO_PRIMARY_META_PROJECT_PACKAGES_EXCEPT`` |
| 1221 |
|
| 1222 |
When the above variable is set to ``TRUE``, this variable is read by |
| 1223 |
TriBITS to find the list of TriBITS packages selected packages in the |
| 1224 |
repository ``${REPOSITORY_NAME}`` which are considered to be part of the |
| 1225 |
set of the project's primary meta-project package when the above variable |
| 1226 |
is set to ``ON``. NOTE: It is not necessary to list all of the |
| 1227 |
subpackages in a given parent package. Only the parent package need be |
| 1228 |
listed and it will be equivalent to listing all of the subpackages. See |
| 1229 |
`TriBITS Dependency Handling Behaviors`_ for more details. |
| 1230 |
|
| 1231 |
The above primary meta-project variables should be set in the meta-project's |
| 1232 |
`<projectDir>/ProjectName.cmake`_ file so that they will be set in all |
| 1233 |
situations. |
| 1234 |
|
| 1235 |
|
| 1236 |
TriBITS Package |
| 1237 |
+++++++++++++++ |
| 1238 |
|
| 1239 |
A TriBITS Package: |
| 1240 |
|
| 1241 |
* Is the fundamental TriBITS structural unit of software partitioning and |
| 1242 |
aggregation. |
| 1243 |
|
| 1244 |
* Must have a unique package name (``PACKAGE_NAME``) that is globally unique |
| 1245 |
(see `Globally unique TriBITS package names`_). |
| 1246 |
|
| 1247 |
* Typically defines a set of libraries and/or header files and/or executables |
| 1248 |
and/or tests with CMake build targets for building these for which TriBITS |
| 1249 |
exports the list of include directories, libraries, and targets that are |
| 1250 |
created (along with CMake dependencies). |
| 1251 |
|
| 1252 |
* Is declared in its parent repository's `<repoDir>/PackagesList.cmake`_ file. |
| 1253 |
|
| 1254 |
* Declares dependencies on `upstream`_ packages by just naming the |
| 1255 |
dependencies in the file `<packageDir>/cmake/Dependencies.cmake`_. |
| 1256 |
|
| 1257 |
* Can optionally have subpackages listed in the argument |
| 1258 |
`SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS`_ to the macro call |
| 1259 |
`tribits_package_define_dependencies()`_. |
| 1260 |
|
| 1261 |
* Is the unit of testing as driven by `tribits_ctest_driver()`_ and displayed |
| 1262 |
on CDash. |
| 1263 |
|
| 1264 |
.. _Globally unique TriBITS package names: |
| 1265 |
|
| 1266 |
**WARNING:** As noted above, one must be very careful to pick **globally |
| 1267 |
unique TriBITS package names**. This name must be unique not only within its |
| 1268 |
defined TriBITS repository but also across all packages in all TriBITS |
| 1269 |
repositories that ever might be cobbled together into a single TriBITS (meta) |
| 1270 |
project! Choosing a good package name is the single most important decision |
| 1271 |
when it comes to defining a TriBITS package. One must be careful **not** to |
| 1272 |
pick names like "Debug" or "StandardUtils" that have a high chance of clashing |
| 1273 |
with poorly named TriBITS packages from other TriBITS repositories. |
| 1274 |
|
| 1275 |
For more details on the definition of a TriBITS Package (or subpackage), see: |
| 1276 |
|
| 1277 |
* `TriBITS Package Core Files`_ |
| 1278 |
* `TriBITS Package Core Variables`_ |
| 1279 |
|
| 1280 |
TriBITS Package Core Files |
| 1281 |
.......................... |
| 1282 |
|
| 1283 |
The core files that make up a *TriBITS Package* (where ``<packageDir> = |
| 1284 |
${${PACKAGE_NAME}_SOURCE_DIR}``) are:: |
| 1285 |
|
| 1286 |
<packageDir>/ |
| 1287 |
CMakeLists.txt # Only processed if the package is enabled |
| 1288 |
cmake/ |
| 1289 |
Dependencies.cmake # Always processed if its repo is processed |
| 1290 |
<packageName>_config.h.in # [Optional], name is not fixed |
| 1291 |
|
| 1292 |
There are a few simple rules for the location and the contents of the |
| 1293 |
``<packageDir>/`` directory: |
| 1294 |
|
| 1295 |
* The directory ``<packageDir>/`` must not be a subdirectory of the package |
| 1296 |
directory of any other package (e.g. not ``pkga/pkgb``). |
| 1297 |
* All of the source files, test files, etc. for the package should be included |
| 1298 |
under ``<packageDir>/``. |
| 1299 |
|
| 1300 |
The above rules are not needed for basic building and testing but are needed |
| 1301 |
for extended features like automatically detecting when a package has changed |
| 1302 |
by looking at what files have changed (see `Pre-push Testing using |
| 1303 |
checkin-test.py`_) and for creating source tarballs correctly (see `Creating |
| 1304 |
Source Distributions`_). Therefore, it would be wise to abide by the above |
| 1305 |
rules when defining packages. |
| 1306 |
|
| 1307 |
The following TriBITS Package files are documented in more detail below: |
| 1308 |
|
| 1309 |
* `<packageDir>/cmake/Dependencies.cmake`_ |
| 1310 |
* `<packageDir>/cmake/<packageName>_config.h.in`_ |
| 1311 |
* `<packageDir>/CMakeLists.txt`_ |
| 1312 |
|
| 1313 |
.. _<packageDir>/cmake/Dependencies.cmake: |
| 1314 |
|
| 1315 |
**<packageDir>/cmake/Dependencies.cmake**: [Required] Defines the dependencies |
| 1316 |
for a given TriBITS package using the macro |
| 1317 |
`tribits_package_define_dependencies()`_. This file is processed at the |
| 1318 |
top-level project scope (using an ``include()``) so any local variables set |
| 1319 |
will be seen by the entire project. This file is always processed, including |
| 1320 |
when just building the project's dependency data-structure (see `Reduced |
| 1321 |
Package Dependency Processing`_). |
| 1322 |
|
| 1323 |
An example of a ``Dependencies.cmake`` file for a package with optional and |
| 1324 |
required dependencies is for the mock ``Panzer`` package in `MockTrilinos`_: |
| 1325 |
|
| 1326 |
.. include:: ../../examples/MockTrilinos/packages/panzer/cmake/Dependencies.cmake |
| 1327 |
:literal: |
| 1328 |
|
| 1329 |
.. _with_subpackages/cmake/Dependencies.cmake: |
| 1330 |
|
| 1331 |
An example of a package with subpackages is ``WithSubpackages`` which has the |
| 1332 |
dependencies file: |
| 1333 |
|
| 1334 |
`TribitsExampleProject`_/``packages/with_subpackages/cmake/Dependencies.cmake`` |
| 1335 |
|
| 1336 |
which is: |
| 1337 |
|
| 1338 |
.. include:: ../../examples/TribitsExampleProject/packages/with_subpackages/cmake/Dependencies.cmake |
| 1339 |
:literal: |
| 1340 |
|
| 1341 |
``WithSubpackages`` defines three subpackages which creates three new packages |
| 1342 |
with names ``WithSubpackagesA``, ``WithSubpackagesB``, and |
| 1343 |
``WithSubpackagesC``. |
| 1344 |
|
| 1345 |
if a TriBITS Package or Subpackage has no dependencies, it still has to call |
| 1346 |
``tribits_package_define_dependencies()`` but it is called with no arguments |
| 1347 |
such as with: |
| 1348 |
|
| 1349 |
`TribitsHelloWorld`_/``hello_world/cmake/Dependencies.cmake:`` |
| 1350 |
|
| 1351 |
which contains: |
| 1352 |
|
| 1353 |
.. include:: ../../examples/TribitsHelloWorld/hello_world/cmake/Dependencies.cmake |
| 1354 |
:literal: |
| 1355 |
|
| 1356 |
Other TriBITS macros/functions that can be called in this file include |
| 1357 |
`tribits_tpl_tentatively_enable()`_ and |
| 1358 |
`tribits_allow_missing_external_packages()`_. |
| 1359 |
|
| 1360 |
.. _<packageName>_config.h.in: |
| 1361 |
.. _<packageDir>/cmake/<packageName>_config.h.in: |
| 1362 |
|
| 1363 |
**<packageDir>/cmake/<packageName>_config.h.in**: [Optional] The package's |
| 1364 |
configured header file. This file will contain placeholders for variables |
| 1365 |
that will be substitute at configure time with `tribits_configure_file()`_. |
| 1366 |
This includes usage of ``#cmakedefine <varName>`` and other standard CMake |
| 1367 |
file configuration features used by CMake's ``configure_file()`` command. |
| 1368 |
|
| 1369 |
An example of this file is shown in: |
| 1370 |
|
| 1371 |
`TribitsExampleProject`_/``packages/simple_cxx/cmake/SimpleCxx_config.h.in`` |
| 1372 |
|
| 1373 |
which is: |
| 1374 |
|
| 1375 |
.. include:: ../../examples/TribitsExampleProject/packages/simple_cxx/cmake/SimpleCxx_config.h.in |
| 1376 |
:literal: |
| 1377 |
|
| 1378 |
The variable ``HAVE_SIMPLECXX___INT64`` is set up in the base file |
| 1379 |
``SimpleCxx/CMakeLists.txt`` (see `<packageDir>/CMakeLists.txt`_ below). For |
| 1380 |
an explanation of ``HAVE_SIMPLECXX_DEBUG``, see `tribits_add_debug_option()`_. |
| 1381 |
For an explanation of ``HAVE_SIMPLECXX_SIMPLETPL``, see `How to add a new |
| 1382 |
TriBITS Package dependency`_. For an explanation of |
| 1383 |
``@SIMPLECXX_DEPRECATED_DECLARATIONS@``, see `Setting up support for |
| 1384 |
deprecated code handling`_. |
| 1385 |
|
| 1386 |
**NOTE:** The file name ``<packageName>_config.h.in`` is not at all fixed and |
| 1387 |
the package can call this file anything it wants. Also, a package can |
| 1388 |
configure multiple header files in different directories for different |
| 1389 |
purposes using `tribits_configure_file()`_ or even calls to the raw CMake |
| 1390 |
function `configure_file()`_. |
| 1391 |
|
| 1392 |
.. _<packageDir>/CMakeLists.txt: |
| 1393 |
|
| 1394 |
**<packageDir>/CMakeLists.txt**: [Required] The package's top-level |
| 1395 |
``CMakeLists.txt`` file that defines the libraries, include directories, and |
| 1396 |
contains the tests for the package. |
| 1397 |
|
| 1398 |
The basic structure of this file for a **package without subpackages** is |
| 1399 |
shown in: |
| 1400 |
|
| 1401 |
`TribitsExampleProject`_/``packages/simple_cxx/CMakeLists.txt`` |
| 1402 |
|
| 1403 |
which is: |
| 1404 |
|
| 1405 |
.. include:: ../../examples/TribitsExampleProject/packages/simple_cxx/CMakeLists.txt |
| 1406 |
:literal: |
| 1407 |
|
| 1408 |
The first command at the top of the file is a call to `tribits_package()`_ |
| 1409 |
which takes the package name (``SimpleCxx`` in this case) in addition to a few |
| 1410 |
other options. While TriBITS obviously already knows the package name (since |
| 1411 |
it read it from the `<repoDir>/PackagesList.cmake`_ file), the purpose for |
| 1412 |
repeating it in this call is as documentation for the developer's sake (and |
| 1413 |
this name is checked against the expected package name). Then a set of |
| 1414 |
configure-time tests are typically performed (if the package needs any of |
| 1415 |
these). In this example, the existence of the C++ ``__int64`` data-type is |
| 1416 |
checked using the module ``CheckFor__int64.cmake`` (which is in the ``cmake/`` |
| 1417 |
directory of this package. (CMake has great support `Configure-time System |
| 1418 |
Tests`_.) This is followed by package-specific options. In this case, the |
| 1419 |
standard TriBITS options for debug checking and deprecated warnings are added |
| 1420 |
using the standard macros `tribits_add_debug_option()`_ and |
| 1421 |
`tribits_add_show_deprecated_warnings_option()`_. After all of this up-front |
| 1422 |
stuff is complete (which will be present in any moderately complex |
| 1423 |
CMake-configured project) the source and the test sub-directories are added |
| 1424 |
that actually define the library and the tests. In this case, the standard |
| 1425 |
`tribits_add_test_directories()`_ macro is used which only conditionally adds |
| 1426 |
the tests for the package. |
| 1427 |
|
| 1428 |
The final command in the package's base ``CMakeLists.txt`` file must always be |
| 1429 |
`tribits_package_postprocess()`_. This is needed in order to perform some |
| 1430 |
necessary post-processing by TriBITS. |
| 1431 |
|
| 1432 |
It is also possible for the package's top-level ``CMakeLists.txt`` to be the |
| 1433 |
only ``CMakeLists.txt`` file for a package. Such an example can be seen in the |
| 1434 |
example project `TribitsHelloWorld`_ in the ``HelloWorld`` package. |
| 1435 |
|
| 1436 |
When a TriBITS package is broken up into subpackages (see `TriBITS |
| 1437 |
Subpackage`_), its ``CMakeLists.txt`` file looks a little different from a |
| 1438 |
package with no subpackages as shown above. The basic structure of this file |
| 1439 |
for a **package with subpackages** is shown in: |
| 1440 |
|
| 1441 |
`TribitsExampleProject`_/``packages/with_subpackages/CMakeLists.txt`` |
| 1442 |
|
| 1443 |
which contains: |
| 1444 |
|
| 1445 |
.. include:: ../../examples/TribitsExampleProject/packages/with_subpackages/CMakeLists.txt |
| 1446 |
:literal: |
| 1447 |
|
| 1448 |
What is different about ``CMakeLists.txt`` files for packages without |
| 1449 |
subpackages is that the `tribits_package()`_ command is broken up into two |
| 1450 |
parts `tribits_package_decl()`_ and `tribits_package_def()`_. In between |
| 1451 |
these two commands, the parent package can define the common package options |
| 1452 |
and then calls the command `tribits_process_subpackages()`_ which fully |
| 1453 |
processes the packages. If the parent package has libraries and/or |
| 1454 |
tests/example of its own, it can define those after calling |
| 1455 |
`tribits_package_def()`_, just like with a regular package. However, it is |
| 1456 |
rare for a package broken up into subpackages to have its own libraries and/or |
| 1457 |
tests and examples. As always, the final command called inside of a package's |
| 1458 |
base ``CMakeLists.txt`` file is `tribits_package_postprocess()`_. |
| 1459 |
|
| 1460 |
NOTE: The package's base ``CMakeLists.txt`` file only gets processed if the |
| 1461 |
package is actually enabled |
| 1462 |
(i.e. ``${PROJECT_NAME}_ENABLE_${PACKAGE_NAME}=ON``). This is an important |
| 1463 |
design feature of TriBITS in that the contents of non-enabled packages can't |
| 1464 |
damage the configure, build, and test of the enabled packages based on errors |
| 1465 |
in non-enabled packages. This is critical to allow experimental `EX`_ |
| 1466 |
test-group packages and lower-maturity packages to exist in the same source |
| 1467 |
repositories safely with higher-maturity and more important packages. |
| 1468 |
|
| 1469 |
|
| 1470 |
TriBITS Package Core Variables |
| 1471 |
.............................. |
| 1472 |
|
| 1473 |
A packages' core variables are broken down into the following categories: |
| 1474 |
|
| 1475 |
* `TriBITS Package Local Variables`_ |
| 1476 |
* `TriBITS Package Top-Level Local Variables`_ |
| 1477 |
* `TriBITS Package Cache Variables`_ |
| 1478 |
* `TriBITS Package Optional Dependency Macro Variables`_ |
| 1479 |
|
| 1480 |
.. _TriBITS Package Local Variables: |
| 1481 |
|
| 1482 |
The following locally scoped **TriBITS Package Local Variables** are defined |
| 1483 |
when the files for a given TriBITS Package (or any package for that matter) |
| 1484 |
are being processed: |
| 1485 |
|
| 1486 |
``PACKAGE_NAME`` |
| 1487 |
|
| 1488 |
The name of the current TriBITS package. This is set automatically by |
| 1489 |
TriBITS before the packages' ``CMakeLists.txt`` file is processed. |
| 1490 |
**WARNING:** This name must be globally unique across the entire project |
| 1491 |
(see `Globally unique TriBITS package names`_). |
| 1492 |
|
| 1493 |
``PACKAGE_SOURCE_DIR`` |
| 1494 |
|
| 1495 |
The absolute path to the package's base source directory. This is |
| 1496 |
set automatically by TriBITS in the macro `tribits_package()`_. |
| 1497 |
|
| 1498 |
``PACKAGE_BINARY_DIR`` |
| 1499 |
|
| 1500 |
The absolute path to the package's base binary/build directory. This is |
| 1501 |
set automatically by TriBITS in the macro `tribits_package()`_. |
| 1502 |
|
| 1503 |
``PACKAGE_NAME_UC`` |
| 1504 |
|
| 1505 |
This is set to the upper-case version of ``${PACKAGE_NAME}``. This is set |
| 1506 |
automatically by TriBITS in the macro `tribits_package()`_. |
| 1507 |
|
| 1508 |
.. _TriBITS Package Top-Level Local Variables: |
| 1509 |
|
| 1510 |
Once all of the TriBITS package's ``Dependencies.cmake`` files have been |
| 1511 |
processed, the following **TriBITS Package Top-Level Local Variables** are |
| 1512 |
defined: |
| 1513 |
|
| 1514 |
.. _${PACKAGE_NAME}_SOURCE_DIR: |
| 1515 |
|
| 1516 |
``${PACKAGE_NAME}_SOURCE_DIR`` |
| 1517 |
|
| 1518 |
The absolute path to the package's base source directory. CMake code, for |
| 1519 |
example in other packages, refer to this by the raw name like |
| 1520 |
``PackageX_SOURCE_DIR``. This makes such CMake code independent of where |
| 1521 |
the package is in relation to other packages. NOTE: This variable is |
| 1522 |
defined for all declared packages that exist, independent of whether they |
| 1523 |
are enabled or not. This variable is set as soon as it is known if the |
| 1524 |
given package exists or not. |
| 1525 |
|
| 1526 |
.. _${PACKAGE_NAME}_REL_SOURCE_DIR: |
| 1527 |
|
| 1528 |
``${PACKAGE_NAME}_REL_SOURCE_DIR`` |
| 1529 |
|
| 1530 |
The **relative path** to the package's base source directory, relative to |
| 1531 |
the projects base source directory `${PROJECT_NAME}_SOURCE_DIR`_. This is |
| 1532 |
used in various contexts such as processing the packages |
| 1533 |
`<packageDir>/CMakeLists.txt`_ file and generating the projects |
| 1534 |
`<Project>PackageDependencies.xml`_ file where relative paths are needed. |
| 1535 |
|
| 1536 |
.. _${PACKAGE_NAME}_BINARY_DIR: |
| 1537 |
|
| 1538 |
``${PACKAGE_NAME}_BINARY_DIR`` |
| 1539 |
|
| 1540 |
The absolute path to the package's base binary directory. CMake code, for |
| 1541 |
example in other packages, refer to this by the raw name like |
| 1542 |
``PackageX_BINARY_DIR``. This makes such CMake code independent of where |
| 1543 |
the package is in relation to other packages. NOTE: This variable is |
| 1544 |
**only** defined if the package is actually enabled! |
| 1545 |
|
| 1546 |
.. _${PACKAGE_NAME}_PARENT_REPOSITORY: |
| 1547 |
|
| 1548 |
``${PACKAGE_NAME}_PARENT_REPOSITORY`` |
| 1549 |
|
| 1550 |
The name of the package's parent repository. This can be used by a |
| 1551 |
package to access information about its parent repository. For example, |
| 1552 |
the variable ``${${PACKAGE_NAME}_PARENT_REPOSITORY}_SOURCE_DIR`` can be |
| 1553 |
dereferenced and read of needed (but it is not recommended that packages |
| 1554 |
be aware of their parent repository in general).. |
| 1555 |
|
| 1556 |
.. _${PACKAGE_NAME}_TESTGROUP: |
| 1557 |
|
| 1558 |
``${PACKAGE_NAME}_TESTGROUP`` |
| 1559 |
|
| 1560 |
Defines the `Package Test Group`_ for the package. This determines in |
| 1561 |
what contexts the package is enabled or not for testing-related purposes |
| 1562 |
(see `Nested Layers of TriBITS Project Testing`_) |
| 1563 |
|
| 1564 |
.. _${PACKAGE_NAME}_SUBPACKAGES: |
| 1565 |
|
| 1566 |
``${PACKAGE_NAME}_SUBPACKAGES`` |
| 1567 |
|
| 1568 |
Defines the list of subpackage names for a top-level parent package. This |
| 1569 |
gives the unique subpackage name without the parent package prefix. For |
| 1570 |
example, the `ReducedMockTrilinos`_ package ``Thyra`` has the subpackages |
| 1571 |
``CoreLibs``, ``GoodStuff``, etc. (which forms the full package names |
| 1572 |
``ThyraCoreLibs``, ``ThyraGoodStuff``, etc.). If a top-level package is |
| 1573 |
not broken down into subpackages, then this list is empty. |
| 1574 |
|
| 1575 |
.. _TriBITS Package Cache Variables: |
| 1576 |
|
| 1577 |
In addition, the following user-settable **TriBITS Package Cache Variables** |
| 1578 |
are defined before a Package's ``CMakeLists.txt`` file is processed: |
| 1579 |
|
| 1580 |
.. _${PROJECT_NAME}_ENABLE_${PACKAGE_NAME}: |
| 1581 |
|
| 1582 |
``${PROJECT_NAME}_ENABLE_${PACKAGE_NAME}`` |
| 1583 |
|
| 1584 |
Set to ``ON`` if the package is enabled and is to be processed or will be |
| 1585 |
set to ``ON`` or ``OFF`` automatically during enable/disable logic. For a |
| 1586 |
parent package that is not directly enabled but where one of its |
| 1587 |
subpackages is enabled, this will get set to ``ON`` (but that is not the |
| 1588 |
same as the parent package being directly enabled and therefore does not |
| 1589 |
imply that all of the required subpackages will be enabled, only that the |
| 1590 |
parent package will be processed). |
| 1591 |
|
| 1592 |
.. _${PACKAGE_NAME}_ENABLE_${UPSTREAM_PACKAGE_NAME}: |
| 1593 |
|
| 1594 |
``${PACKAGE_NAME}_ENABLE_${UPSTREAM_PACKAGE_NAME}`` |
| 1595 |
|
| 1596 |
Set to ``ON`` if support for the optional `upstream`_ dependent package |
| 1597 |
``${UPSTREAM_PACKAGE_NAME}`` is enabled in package |
| 1598 |
``${PACKAGE_NAME}``. Here ``${UPSTREAM_PACKAGE_NAME}`` corresponds to |
| 1599 |
each optional upstream package listed in the ``LIB_OPTIONAL_PACKAGES`` |
| 1600 |
and ``TEST_OPTIONAL_PACKAGES`` arguments to the |
| 1601 |
`tribits_package_define_dependencies()`_ macro. |
| 1602 |
|
| 1603 |
**NOTE:** It is important that the CMake code in the package's |
| 1604 |
``CMakeLists.txt`` files key off of this variable and **not** the |
| 1605 |
project-level variable |
| 1606 |
``${PROJECT_NAME}_ENABLE_${UPSTREAM_PACKAGE_NAME}`` because the |
| 1607 |
package-level variable |
| 1608 |
``${PACKAGE_NAME}_ENABLE_${UPSTREAM_PACKAGE_NAME}`` can be explicitly |
| 1609 |
turned off by the user even through the packages ``${PACKAGE_NAME}`` and |
| 1610 |
``${UPSTREAM_PACKAGE_NAME}`` are both enabled at the project level! |
| 1611 |
See `Support for optional package can be explicitly disabled`_. |
| 1612 |
|
| 1613 |
**NOTE:** This variable will also be set for required dependencies as well |
| 1614 |
to allow for uniform processing such as when looping over the items in |
| 1615 |
`${PACKAGE_NAME}_LIB_DEFINED_DEPENDENCIES`_ or |
| 1616 |
`${PACKAGE_NAME}_TEST_DEFINED_DEPENDENCIES`_. |
| 1617 |
|
| 1618 |
**NOTE:** The value of this variable also determines the value of the |
| 1619 |
macro define variable name |
| 1620 |
`HAVE_<PACKAGE_NAME_UC>_<UPSTREAM_PACKAGE_NAME_UC>`_. |
| 1621 |
|
| 1622 |
.. _${PACKAGE_NAME}_ENABLE_TESTS: |
| 1623 |
|
| 1624 |
``${PACKAGE_NAME}_ENABLE_TESTS`` |
| 1625 |
|
| 1626 |
Set to ``ON`` if the package's tests are to be enabled. This will enable |
| 1627 |
a package's tests and all of its subpackage's tests. |
| 1628 |
|
| 1629 |
.. _${PACKAGE_NAME}_ENABLE_EXAMPLES: |
| 1630 |
|
| 1631 |
``${PACKAGE_NAME}_ENABLE_EXAMPLES`` |
| 1632 |
|
| 1633 |
Set to ``ON`` if the package's examples are to be enabled. This will |
| 1634 |
enable a package's examples and all of its subpackage's examples. |
| 1635 |
|
| 1636 |
The above global cache variables can be explicitly set by the user or may be |
| 1637 |
set automatically as part of the `Package Dependencies and Enable/Disable |
| 1638 |
Logic`_. |
| 1639 |
|
| 1640 |
.. _TriBITS Package Optional Dependency Macro Variables: |
| 1641 |
|
| 1642 |
The following local **TriBITS Package Optional Dependency Macro Variables** |
| 1643 |
are defined in the top-level project scope before a Package's |
| 1644 |
``CMakeLists.txt`` file is processed: |
| 1645 |
|
| 1646 |
.. _HAVE_<PACKAGE_NAME_UC>_<UPSTREAM_PACKAGE_NAME_UC>: |
| 1647 |
|
| 1648 |
``HAVE_<PACKAGE_NAME_UC>_<UPSTREAM_PACKAGE_NAME_UC>`` |
| 1649 |
|
| 1650 |
Set to ``ON`` if support for optional upstream package |
| 1651 |
``${UPSTREAM_PACKAGE_NAME`` is enabled in downstream package |
| 1652 |
``${PACKAGE_NAME}`` |
| 1653 |
(i.e. `${PACKAGE_NAME}_ENABLE_${UPSTREAM_PACKAGE_NAME}`_ = ``ON``) and is |
| 1654 |
set to ``FALSE`` otherwise. Here, ``<PACKAGE_NAME_UC>`` and |
| 1655 |
``<UPSTREAM_PACKAGE_NAME_UC>`` are the upper-case names for the packages |
| 1656 |
``${PACKAGE_NAME}`` and ``${UPSTREAM_PACKAGE_NAME}``, respectively. |
| 1657 |
For example, if optional support for upstream package ``Triutils`` is |
| 1658 |
enabled in downstream package ``EpetraExt`` in `ReducedMockTrilinos`_, |
| 1659 |
then ``EpetraExt_ENABLE_TriUtils=ON`` and ``HAVE_EPETRAEXT_TRIUTILS=ON``. |
| 1660 |
This variable is meant to be used in:: |
| 1661 |
|
| 1662 |
#cmakedefine HAVE_<PACKAGE_NAME_UC>_<UPSTREAM_PACKAGE_NAME_UC> |
| 1663 |
|
| 1664 |
in configured header files |
| 1665 |
(e.g. `<packageDir>/cmake/<packageName>_config.h.in`_). For example, for |
| 1666 |
the ``EpetraExt`` and ``Triutils`` example, this would be:: |
| 1667 |
|
| 1668 |
#cmakedefine HAVE_EPETRAEXT_TRIUTILS |
| 1669 |
|
| 1670 |
NOTE: TriBITS automatically sets this variable depending on the value of |
| 1671 |
`${PACKAGE_NAME}_ENABLE_${UPSTREAM_PACKAGE_NAME}`_ during the step |
| 1672 |
"Adjust package and TPLs enables and disables" in `Full Processing of |
| 1673 |
TriBITS Project Files`_. And tweaking this variable after that must be |
| 1674 |
done carefully as described in `How to tweak downstream TriBITS "ENABLE" |
| 1675 |
variables during package configuration`_. |
| 1676 |
|
| 1677 |
Currently, a Package can refer to its containing Repository and refer to its |
| 1678 |
source and binary directories. This is so that it can refer to |
| 1679 |
repository-level resources (e.g. like the ``Trilinos_version.h`` file for |
| 1680 |
Trilinos packages). However, this may be undesirable because it will make it |
| 1681 |
hard to pull a package out of one TriBITS repository and place it in another |
| 1682 |
repository for a different use. However, a package can indirectly refer to |
| 1683 |
its own repository without loss of generality by reading the variable |
| 1684 |
``${PACKAGE_NAME}_PARENT_REPOSITORY``. The problem is referring to other |
| 1685 |
TriBITS repositories explicitly. |
| 1686 |
|
| 1687 |
|
| 1688 |
TriBITS Subpackage |
| 1689 |
++++++++++++++++++ |
| 1690 |
|
| 1691 |
A TriBITS Subpackage: |
| 1692 |
|
| 1693 |
* Is a compartmentalization of a parent `TriBITS Package`_ according to |
| 1694 |
`Software Engineering Packaging Principles`_. |
| 1695 |
* Typically defines a set of libraries and/or header files and/or executables |
| 1696 |
and/or tests with CMake build targets for building these for which TriBITS |
| 1697 |
exports the list of include directories, libraries, and targets that are |
| 1698 |
created (along with CMake dependencies). |
| 1699 |
* Is declared in its parent package's `<packageDir>/cmake/Dependencies.cmake`_ |
| 1700 |
file in a call to `tribits_package_define_dependencies()`_ using the |
| 1701 |
argument `SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS`_. |
| 1702 |
* Defines dependencies on `upstream`_ packages by just naming them in the file |
| 1703 |
`<packageDir>/<spkgDir>/cmake/Dependencies.cmake`_ using the macro |
| 1704 |
`tribits_package_define_dependencies()`_. |
| 1705 |
* Can **NOT** have its own subpackages defined (only top-level packages can |
| 1706 |
have subpackages). |
| 1707 |
* Is enabled or disabled along with all other subpackages in the parent |
| 1708 |
package automatically if it's parent package is enabled or disabled with |
| 1709 |
``${PROJECT_NAME}_ENABLE_${PARENT_PACKAGE_NAME}`` set to ``ON`` or ``OFF`` |
| 1710 |
respectively (see `Enable/disable of parent package is enable/disable for |
| 1711 |
subpackages`_ and `PARENT_PACKAGE_NAME`_). |
| 1712 |
* Has tests turned on automatically if |
| 1713 |
``${PARENT_PACKAGE_NAME}_ENABLE_TESTS=ON``. |
| 1714 |
|
| 1715 |
The contents of a TriBITS Subpackage are almost identical to those of a |
| 1716 |
TriBITS Package. The differences are described below and in `How is a TriBITS |
| 1717 |
Subpackage different from a TriBITS Package?`_. |
| 1718 |
|
| 1719 |
For more details on the definition of a TriBITS Package (or subpackage), see: |
| 1720 |
|
| 1721 |
* `TriBITS Subpackage Core Files`_ |
| 1722 |
* `TriBITS Subpackage Core Variables`_ |
| 1723 |
|
| 1724 |
|
| 1725 |
TriBITS Subpackage Core Files |
| 1726 |
.............................. |
| 1727 |
|
| 1728 |
The set of core files for a subpackage are identical to the `TriBITS Package |
| 1729 |
Core Files`_ and are:: |
| 1730 |
|
| 1731 |
<packageDir>/<spkgDir>/ |
| 1732 |
CMakeLists.txt # Only processed if this subpackage is enabled |
| 1733 |
cmake/ |
| 1734 |
Dependencies.cmake # Always processed if the parent package |
| 1735 |
# is listed in the enclosing Repository |
| 1736 |
|
| 1737 |
(where ``<packageDir> = ${${PARENT_PACKAGE_NAME}_SOURCE_DIR}`` and |
| 1738 |
``<spkgDir>`` is the subpackage directory listed in the |
| 1739 |
`SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS`_ to |
| 1740 |
`tribits_package_define_dependencies()`_). |
| 1741 |
|
| 1742 |
There are a few simple rules for the location and the contents of the |
| 1743 |
``<spkgDir>/`` directory: |
| 1744 |
|
| 1745 |
* The relative directory ``<spkgDir>/`` should be a strict subdirectory of |
| 1746 |
``<packageDir>/`` (e.g. not ``../../somewhereelse``). |
| 1747 |
* The directory ``<spkgDir>/`` must not be a subdirectory of the package |
| 1748 |
directory of any other subpackage (e.g. not ``spkga/spkgb``). |
| 1749 |
* All of the source files, test files, etc. for the subpackage should be |
| 1750 |
included under ``<spkgDir>/``. |
| 1751 |
|
| 1752 |
The above rules are not needed for basic building and testing but are needed |
| 1753 |
for extended features like automatically detecting when a package has changed |
| 1754 |
by looking at what files have changed (see `Pre-push Testing using |
| 1755 |
checkin-test.py`_) and for creating source tarballs correctly (see `Creating |
| 1756 |
Source Distributions`_). Therefore, it would be wise to abide by the above |
| 1757 |
rules when defining subpackages. |
| 1758 |
|
| 1759 |
These TriBITS Subpackage files are documented in more detail below: |
| 1760 |
|
| 1761 |
* `<packageDir>/<spkgDir>/cmake/Dependencies.cmake`_ |
| 1762 |
* `<packageDir>/<spkgDir>/CMakeLists.txt`_ |
| 1763 |
|
| 1764 |
.. _<packageDir>/<spkgDir>/cmake/Dependencies.cmake: |
| 1765 |
|
| 1766 |
**<packageDir>/<spkgDir>/cmake/Dependencies.cmake**: [Required] The contents |
| 1767 |
of this file for subpackages are identical as for top-level packages. It just |
| 1768 |
contains a call to the macro `tribits_package_define_dependencies()`_ to |
| 1769 |
define this package's `upstream`_ package dependencies. A simple example is |
| 1770 |
for the example subpackage ``WithSubpackagesB`` (declared in |
| 1771 |
`with_subpackages/cmake/Dependencies.cmake`_) with the file: |
| 1772 |
|
| 1773 |
`TribitsExampleProject`_/``packages/with_subpackages/b/cmake/Dependencies.cmake`` |
| 1774 |
|
| 1775 |
which is: |
| 1776 |
|
| 1777 |
.. include:: ../../examples/TribitsExampleProject/packages/with_subpackages/b/cmake/Dependencies.cmake |
| 1778 |
:literal: |
| 1779 |
|
| 1780 |
What this shows is that subpackages must list their dependencies on each other |
| 1781 |
(if such dependencies exist) using the full package name |
| 1782 |
``${PARENT_PACKAGE_NAME}${SUBPACKAGE_NAME}`` or in this case:: |
| 1783 |
|
| 1784 |
'WithSubpackagesA' = 'WithSubpackages' + 'A' |
| 1785 |
|
| 1786 |
Note that the parent package depends on its subpackages, not the other way |
| 1787 |
around. For example, the ``WithSubpackages`` parent package automatically |
| 1788 |
depends its subpackages ``WithSubpackagesA``, ``WithSubpackagesC``, and |
| 1789 |
``WithSubpackagesC``. As such all (direct) dependencies for a subpackage must |
| 1790 |
be listed in its own ``Dependencies.cmake`` file. For example, the |
| 1791 |
``WithSubpackages`` subpackage ``A`` depends on the ``SimpleCxx`` package and |
| 1792 |
is declared as such as shown in: |
| 1793 |
|
| 1794 |
`TribitsExampleProject`_/``packages/with_subpackages/a/cmake/Dependencies.cmake`` |
| 1795 |
|
| 1796 |
which is: |
| 1797 |
|
| 1798 |
.. include:: ../../examples/TribitsExampleProject/packages/with_subpackages/a/cmake/Dependencies.cmake |
| 1799 |
:literal: |
| 1800 |
|
| 1801 |
What this means is that any package dependencies listed in the parent |
| 1802 |
package's `<packageDir>/cmake/Dependencies.cmake`_ file are **NOT** |
| 1803 |
dependencies of its subpackages. For example, if |
| 1804 |
`with_subpackages/cmake/Dependencies.cmake`_ where changed to be:: |
| 1805 |
|
| 1806 |
tribits_package_define_dependencies( |
| 1807 |
LIB_REQUIRED_TPLS Boost |
| 1808 |
SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS |
| 1809 |
A A PT REQUIRED |
| 1810 |
... |
| 1811 |
) |
| 1812 |
|
| 1813 |
then the ``Boost`` TPL would **NOT** be a dependency of the package |
| 1814 |
``WithSubpackagesA`` but instead would be listed as a |
| 1815 |
dependency of the parent package ``WithSubpackages``. (And in this |
| 1816 |
case, this TPL dependency is pretty worthless since the package |
| 1817 |
``WithSubpackages`` does not even define any libraries or tests of its |
| 1818 |
own.) |
| 1819 |
|
| 1820 |
.. _<packageDir>/<spkgDir>/CMakeLists.txt: |
| 1821 |
|
| 1822 |
**<packageDir>/<spkgDir>/CMakeLists.txt**: [Required] The subpackage's |
| 1823 |
top-level ``CMakeLists.txt`` file that defines the libraries, include |
| 1824 |
directories, and contains the tests for the subpackage. The contents of a |
| 1825 |
subpackage's top-level ``CMakeLists.txt`` file are almost identical to a |
| 1826 |
top-level package's `<packageDir>/CMakeLists.txt`_ file. The primary |
| 1827 |
difference is that the commands `tribits_package()`_ and |
| 1828 |
`tribits_package_postprocess()`_ and replaced with `tribits_subpackage()`_ and |
| 1829 |
`tribits_subpackage_postprocess()`_ as shown in the file: |
| 1830 |
|
| 1831 |
|
| 1832 |
`TribitsExampleProject`_/``packages/with_subpackages/a/CMakeLists.txt`` |
| 1833 |
|
| 1834 |
which contains: |
| 1835 |
|
| 1836 |
.. include:: ../../examples/TribitsExampleProject/packages/with_subpackages/a/CMakeLists.txt |
| 1837 |
:literal: |
| 1838 |
|
| 1839 |
Unlike `tribits_package()`_, `tribits_subpackage()`_ does not take any extra |
| 1840 |
arguments. Those extra settings are assumed to be defined by the top-level |
| 1841 |
parent package. Like top-level packages, subpackages are free to define |
| 1842 |
user-settable options and configure-time tests but typically don't. The idea |
| 1843 |
is that subpackages should be lighter weight than top-level packages. Other |
| 1844 |
than using `tribits_subpackage()`_ and `tribits_subpackage_postprocess()`_, a |
| 1845 |
subpackage can be laid out just like any other package and can call on any |
| 1846 |
other commands to add libraries, add executables, add test, etc. |
| 1847 |
|
| 1848 |
|
| 1849 |
TriBITS Subpackage Core Variables |
| 1850 |
................................. |
| 1851 |
|
| 1852 |
The core variables associated with a subpackage are identical to the `TriBITS |
| 1853 |
Package Core Variables`_. In addition, a subpackage may need to refer to its |
| 1854 |
top-level parent package where a top-level package does not have a parent |
| 1855 |
package. These additional variables that are defined for subpackages are |
| 1856 |
broken down into the following categories: |
| 1857 |
|
| 1858 |
* `TriBITS Subpackage Local Variables`_ |
| 1859 |
* `TriBITS Subpackage Top-Level Local Variables`_ |
| 1860 |
|
| 1861 |
.. _TriBITS Subpackage Local Variables: |
| 1862 |
|
| 1863 |
In addition to the `TriBITS Package Local Variables`_, the following locally |
| 1864 |
scoped **TriBITS Subpackage Local Variables** are defined when the files for a |
| 1865 |
given TriBITS Subpackage are being processed: |
| 1866 |
|
| 1867 |
.. _PARENT_PACKAGE_NAME: |
| 1868 |
|
| 1869 |
``PARENT_PACKAGE_NAME`` |
| 1870 |
|
| 1871 |
The name of the parent package. |
| 1872 |
|
| 1873 |
``PARENT_PACKAGE_SOURCE_DIR`` |
| 1874 |
|
| 1875 |
The absolute path to the parent package's base source directory. |
| 1876 |
|
| 1877 |
``PARENT_PACKAGE_BINARY_DIR`` |
| 1878 |
|
| 1879 |
The absolute path to the parent package's base binary directory. |
| 1880 |
|
| 1881 |
|
| 1882 |
.. _TriBITS Subpackage Top-Level Local Variables: |
| 1883 |
|
| 1884 |
In addition to the `TriBITS Package Top-Level Local Variables`_, once all of a |
| 1885 |
TriBITS subpackage's ``Dependencies.cmake`` files have been processed, the |
| 1886 |
following **TriBITS Subpackage Top-Level Local Variables** are defined: |
| 1887 |
|
| 1888 |
.. _${PACKAGE_NAME}_PARENT_PACKAGE: |
| 1889 |
|
| 1890 |
``${PACKAGE_NAME}_PARENT_PACKAGE`` |
| 1891 |
|
| 1892 |
The name of the parent package. (NOTE: If this is empty "", then |
| 1893 |
``${PACKAGE_NAME}`` is actually a parent package and not a subpackage.) |
| 1894 |
|
| 1895 |
|
| 1896 |
How is a TriBITS Subpackage different from a TriBITS Package? |
| 1897 |
............................................................. |
| 1898 |
|
| 1899 |
A common question this is natural to ask is how a TriBITS Subpackage is |
| 1900 |
different from a TriBITS Package? They contain the same basic files (i.e. a |
| 1901 |
``cmake/Dependencies.cmake`` file, a top-level ``CMakeLists.txt`` file, source |
| 1902 |
files, test files, etc.). They both are included in the list of `TriBITS |
| 1903 |
Packages`_ and therefore can both be enabled/disabled by the user or in |
| 1904 |
automatic dependency logic (see `Package Dependencies and Enable/Disable |
| 1905 |
Logic`_). The primary difference is that a subpackage is meant to involve |
| 1906 |
less overhead and is to be used to partition the parent package's software |
| 1907 |
into chunks according to `Software Engineering Packaging Principles`_. Also, |
| 1908 |
the dependency logic treats a parent package's subpackages as part of itself |
| 1909 |
so when the parent package is explicitly enabled or disabled, it is identical |
| 1910 |
to explicitly enabling or disabling all of its subpackages (see |
| 1911 |
`Enable/disable of parent package is enable/disable for subpackages`_). Also, |
| 1912 |
subpackages are tested along with their peer subpackages with the parent |
| 1913 |
package as part of `TriBITS CTest/CDash Driver`_. This effectively means that |
| 1914 |
if a build failure is detected in any subpackage, then that will effectively |
| 1915 |
disable the parent package and all of its other subpackages in downstream |
| 1916 |
testing. This is a type of "all for one and one for all" when it comes to the |
| 1917 |
relationship between the subpackages within a single parent package. These |
| 1918 |
are some of the issues to consider when breaking up software into packages and |
| 1919 |
subpackages that will be mentioned in other sections as well. |
| 1920 |
|
| 1921 |
|
| 1922 |
TriBITS External Package/TPL |
| 1923 |
++++++++++++++++++++++++++++ |
| 1924 |
|
| 1925 |
A *TriBITS External Package/TPL*: |
| 1926 |
|
| 1927 |
* Provides access to a set of pre-built/installed libraries and/or header |
| 1928 |
files and/or executables and/or some other resources through a standard |
| 1929 |
interface to one or more downstream TriBITS Packages. |
| 1930 |
* Has a globally unique name ``<tplName>`` (see `Globally unique TriBITS TPL |
| 1931 |
names`_) that is declared in a `<repoDir>/TPLsList.cmake`_ file. |
| 1932 |
* Has as `FindTPL<tplName>.cmake`_ module (for non-`TriBITS-Compliant External |
| 1933 |
Packages`_) that finds the pieces of an external package/TPL and provides |
| 1934 |
them to downstream packages through a required INTERFACE target |
| 1935 |
``<tplName>::all_libs`` (which gives the libraries, include directories, and |
| 1936 |
other usage requirements, see `TriBITS-Compliant External Package`_). |
| 1937 |
* Is listed as an explicit optional or required dependency in one or more |
| 1938 |
downstream TriBITS packages' `<packageDir>/cmake/Dependencies.cmake`_ files. |
| 1939 |
* Can be enabled automatically or can trigger the disable of dependent |
| 1940 |
downstream packages (see `TriBITS Dependency Handling Behaviors`_). |
| 1941 |
|
| 1942 |
The TriBITS external package/TPL mechanism provides a uniform way to find and |
| 1943 |
provide access to any type of external resource no matter how it might be |
| 1944 |
installed or ways to provide access to it. Using a TriBITS external |
| 1945 |
package/TPL is to be preferred over using raw CMake |
| 1946 |
``find_package(<externalPkg>)`` call because the TriBITS system guarantees |
| 1947 |
that only a single unique version of an external package/TPL of the same |
| 1948 |
version will be used all of the downstream packages that uses it. Also, by |
| 1949 |
defining a TriBITS TPL, automatic enable/disable logic will be applied as |
| 1950 |
described in `Package Dependencies and Enable/Disable Logic`_. For example, |
| 1951 |
if an external package/TPL is explicitly disabled, all of the downstream |
| 1952 |
packages that depend on it will be automatically disabled as well (see |
| 1953 |
`Package disable triggers auto-disables of downstream dependencies`_). |
| 1954 |
|
| 1955 |
NOTE: The TriBITS TPL system implements a mechanism to turn external |
| 1956 |
dependencies into both `TriBITS-compliant packages`_ for consumption by |
| 1957 |
downstream TriBITS internal packages and also writes ``<tplName>Config.cmake`` |
| 1958 |
files that are `TriBITS-compliant external packages`_ for consumption by |
| 1959 |
downstream ``<Package>Config.cmake`` files (which are `TriBITS-compliant |
| 1960 |
external packages`_) generated by `TriBITS-compliant internal packages`_. |
| 1961 |
|
| 1962 |
|
| 1963 |
.. _Globally unique TriBITS TPL names: |
| 1964 |
|
| 1965 |
**WARNING:** One must be very careful to pick **Globally unique TriBITS |
| 1966 |
External Package/TPL names** ``<tplName>`` across all TPLs in all TriBITS |
| 1967 |
repositories that ever might be cobbled together into a single TriBITS (meta) |
| 1968 |
project! However, choosing TPL names is usually much easier and less risky |
| 1969 |
than choosing `Globally unique TriBITS package names`_ because widely used |
| 1970 |
TPLs tend to already be uniquely named. For example, the external package/TPL |
| 1971 |
names ``BLAS`` and ``LAPACK`` are well defined in the applied math and |
| 1972 |
computational science community and are not likely to clash. |
| 1973 |
|
| 1974 |
|
| 1975 |
TriBITS External Package/TPL Core Files |
| 1976 |
....................................... |
| 1977 |
|
| 1978 |
The core files that define a *TriBITS External Package/TPL* are:: |
| 1979 |
|
| 1980 |
<tplDefsDir>/ |
| 1981 |
FindTPL<tplName>.cmake # The name is not fixed (see <tplName>_FINDMOD) |
| 1982 |
FindTPL<tplName>Dependencies.cmake # [Optional], defines upstream dependencies |
| 1983 |
|
| 1984 |
Above, ``<tplDefsDir>/`` can be a subdirectory under a parent TriBITS |
| 1985 |
repository ``<repoDir>/`` (e.g. ``<repoDir>/cmake/tpls/``) or can be under a |
| 1986 |
TriBITS package directory ``<packageDir>/`` |
| 1987 |
(e.g. ``<packageDir>/cmake/tpls/``). |
| 1988 |
|
| 1989 |
The following TriBITS External Package/TPL files are documented in more detail |
| 1990 |
below: |
| 1991 |
|
| 1992 |
* `<tplDefsDir>/FindTPL<tplName>.cmake`_ |
| 1993 |
* `<tplDefsDir>/FindTPL<tplName>Dependencies.cmake`_ |
| 1994 |
|
| 1995 |
|
| 1996 |
.. _FindTPL<tplName>.cmake: |
| 1997 |
.. _<tplDefsDir>/FindTPL<tplName>.cmake: |
| 1998 |
|
| 1999 |
**<tplDefsDir>/FindTPL<tplName>.cmake**: [Required] *TriBITS TPL find module* |
| 2000 |
that defines how a TriBITS external package/TPL is found and provided for |
| 2001 |
usage by a downstream TriBITS package. This module must provide the |
| 2002 |
``<tplName>::all_libs`` target and must create a `TriBITS-compliant external |
| 2003 |
package`_ wrapper package config file ``<tplName>Config.cmake``. (See the |
| 2004 |
requirements for a ``FindTPL<tplName>.cmake`` file in `Requirements for |
| 2005 |
FindTPL<tplName>.cmake modules`_). |
| 2006 |
|
| 2007 |
The form of a simple ``FindTPL<tplName>.cmake`` file that uses an internal |
| 2008 |
call to ``find_package(<externalPkg>)`` which provides modern IMPORTED CMake |
| 2009 |
targets can use the |
| 2010 |
`tribits_extpkg_create_imported_all_libs_target_and_config_file()`_ function |
| 2011 |
and looks like:: |
| 2012 |
|
| 2013 |
find_package(<externalPkg> REQUIRED) |
| 2014 |
tribits_extpkg_create_imported_all_libs_target_and_config_file( |
| 2015 |
<tplName> |
| 2016 |
INNER_FIND_PACKAGE_NAME <externalPkg> |
| 2017 |
IMPORTED_TARGETS_FOR_ALL_LIBS <importedTarget0> <importedTarget1> ... ) |
| 2018 |
|
| 2019 |
In this case, the purpose for the ``FindTPL<tplName>.cmake`` file (as opposed |
| 2020 |
to a direct call to ``find_package(<externalPkg>)``) is to ensure the |
| 2021 |
definition of the complete target ``<tplName>::all_libs`` which contains all |
| 2022 |
usage requirements for the external package/TPL (i.e. all of the libraries, |
| 2023 |
include directories, etc.) and this also generates the wrapper package config |
| 2024 |
file ``<tplName>Config.cmake``. |
| 2025 |
|
| 2026 |
The form of a simple ``FindTPL<tplName>.cmake`` file that just provides a list |
| 2027 |
of required header files and libraries that does **not** use an internal call |
| 2028 |
to ``find_package()`` and instead uses the function |
| 2029 |
`tribits_tpl_find_include_dirs_and_libraries()`_ looks like:: |
| 2030 |
|
| 2031 |
tribits_tpl_find_include_dirs_and_libraries( <tplName> |
| 2032 |
REQUIRED_HEADERS <header0> <header1> ... |
| 2033 |
REQUIRED_LIBS_NAMES <libname0> <libname1> ... |
| 2034 |
MUST_FIND_ALL_LIBS |
| 2035 |
) |
| 2036 |
|
| 2037 |
An example concrete file is ``tribits/common_tpls/FindTPLPETSC.cmake``: |
| 2038 |
|
| 2039 |
.. include:: ../../common_tpls/FindTPLPETSC.cmake |
| 2040 |
:literal: |
| 2041 |
|
| 2042 |
For complete details, see `Creating the FindTPL<tplName>.cmake file`_. |
| 2043 |
|
| 2044 |
|
| 2045 |
.. _FindTPL<tplName>Dependencies.cmake: |
| 2046 |
.. _<tplDefsDir>/FindTPL<tplName>Dependencies.cmake: |
| 2047 |
|
| 2048 |
**<tplDefsDir>/FindTPL<tplName>Dependencies.cmake**: [Optional] |
| 2049 |
Declares dependencies on upstream external packages/TPLs for the external |
| 2050 |
package/TPL ``<tplName>``. Many external packages/TPLs defined with a |
| 2051 |
`FindTPL<tplName>.cmake`_ file do not have any upstream dependencies or have |
| 2052 |
internal mechanisms to get those (such as when using |
| 2053 |
``find_package(<externalPkg>)`` where the ``<externalPkg>Config.cmake`` file |
| 2054 |
which recursively uses ``find_dependency()`` to get its upstream |
| 2055 |
dependencies). But for ``FindTPL<tplName>.cmake`` files that just use |
| 2056 |
`tribits_tpl_find_include_dirs_and_libraries()`_ (see `Creating a |
| 2057 |
FindTPL<tplName>.cmake module without find_package()`_), TriBITS needs to be told |
| 2058 |
about any upstream external packages/TPLs that it may depend on so it can add |
| 2059 |
the dependencies between the created IMPORTED target libraries. |
| 2060 |
|
| 2061 |
The file ``FindTPL<tplName>Dependencies.cmake`` is typically just a single |
| 2062 |
call to `tribits_extpkg_define_dependencies()`_ and takes the form:: |
| 2063 |
|
| 2064 |
tribits_extpkg_define_dependencies( <tplName> |
| 2065 |
DEPENDENCIES <upstreamTpl_0> <upstreamTpl_1> ... ) |
| 2066 |
|
| 2067 |
This defines all of the TPLs that ``<tplName>`` could directly depends on but |
| 2068 |
only dependencies for enabled upstream TPLs will be added to the IMPORTED |
| 2069 |
targets. |
| 2070 |
|
| 2071 |
NOTE: TPL-to-TPL dependencies are optional. Therefore, in the above example, |
| 2072 |
enabling the TPL ``<tplName>`` will not auto-enable a dependent upstream TPL |
| 2073 |
``<upstreamTpl_i>``. Likewise, disabling an upstream TPL ``<upstreamTpl_i>`` |
| 2074 |
will not auto-disable a dependent downstream TPL ``<tplName>``. |
| 2075 |
|
| 2076 |
|
| 2077 |
TriBITS External Package/TPL Core Variables |
| 2078 |
........................................... |
| 2079 |
|
| 2080 |
Once the `<repoDir>/TPLsList.cmake`_ files are all processed, then each |
| 2081 |
defined TPL ``TPL_NAME`` is assigned the following global non-cache variables: |
| 2082 |
|
| 2083 |
.. _${PACKAGE_NAME}_FINDMOD: |
| 2084 |
.. _<tplName>_FINDMOD: |
| 2085 |
.. _${TPL_NAME}_FINDMOD: |
| 2086 |
|
| 2087 |
``${TPL_NAME}_FINDMOD`` |
| 2088 |
|
| 2089 |
For a **non-** `TriBITS-compliant external package`_, this is the relative |
| 2090 |
path (w.r.t. ``<projectDir>``) or absolute path for the *TriBITS TPL find |
| 2091 |
module* (typically named `FindTPL<tplName>.cmake`_). This is set using the |
| 2092 |
``FINDMOD`` field in the call to `tribits_repository_define_tpls()`_. The |
| 2093 |
final value of the variable is defined by the **last** |
| 2094 |
`<repoDir>/TPLsList.cmake`_ file that is processed that declares the TPL |
| 2095 |
``TPL_NAME``. For example, if ``Repo1/TPLsList.cmake`` and |
| 2096 |
``Repo2/TPLsList.cmake`` both list the TPL ``SomeTpl``, then if ``Repo2`` |
| 2097 |
is processed after ``Repo1``, then ``SomeTpl_FINDMOD`` is determined by |
| 2098 |
``Repo2/TPLsList.cmake`` and the find module listed in |
| 2099 |
``Repo1/TPLsList.cmake`` is ignored. NOTE: for a `TriBITS-compliant |
| 2100 |
external package`_, the special value ``TRIBITS_PKG`` is also recognized. |
| 2101 |
(Any pre-installed TriBITS package is a `TriBITS-compliant external |
| 2102 |
package`_.) |
| 2103 |
|
| 2104 |
.. _<tplName>_DEPENDENCIES_FILE: |
| 2105 |
.. _${TPL_NAME}_DEPENDENCIES_FILE: |
| 2106 |
|
| 2107 |
``${TPL_NAME}_DEPENDENCIES_FILE`` |
| 2108 |
|
| 2109 |
Relative path (w.r.t. ``<projectDir>``) or absolute path for the external |
| 2110 |
package/TPL's dependencies file (typically named |
| 2111 |
`FindTPL<tplName>Dependencies.cmake`_). This is always beside the find |
| 2112 |
module `${TPL_NAME}_FINDMOD`_. (In fact, for a **non-** |
| 2113 |
`TriBITS-compliant external package`_, ``${TPL_NAME}_DEPENDENCIES_FILE`` |
| 2114 |
is constructed from ``${TPL_NAME}_FINDMOD``). NOTE: A `TriBITS-compliant |
| 2115 |
external package`_ with dependencies will also have this file set and the |
| 2116 |
path will be specified independent of the path to the non-existent |
| 2117 |
``FindTPL<tplName>.cmake`` file (see the ``FINDMOD`` field in the call to |
| 2118 |
`tribits_repository_define_tpls()`_). |
| 2119 |
|
| 2120 |
.. _<tplName>_TESTGROUP: |
| 2121 |
.. _${TPL_NAME}_TESTGROUP: |
| 2122 |
|
| 2123 |
``${TPL_NAME}_TESTGROUP`` |
| 2124 |
|
| 2125 |
TPL's `Package Test Group`_: This is set using the ``CLASSIFICATION`` |
| 2126 |
field in the call to `tribits_repository_define_tpls()`_. If multiple |
| 2127 |
repos define a given TPL, then the *first* `<repoDir>/TPLsList.cmake`_ |
| 2128 |
file that is processed that declares the TPL ``TPL_NAME`` specifies the |
| 2129 |
test group. For example, if ``Repo1/TPLsList.cmake`` and |
| 2130 |
``Repo2/TPLsList.cmake`` both list the TPL ``SomeTpl``, then if ``Repo2`` |
| 2131 |
is processed after ``Repo1``, then ``SomeTpl_TESTGROUP`` is determined by |
| 2132 |
``Repo1/TPLsList.cmake`` and the test group in ``Repo2/TPLsList.cmake`` is |
| 2133 |
ignored. However, if ``${TPL_NAME}_TESTGROUP`` is already set before the |
| 2134 |
`<repoDir>/TPLsList.cmake`_ files are processed, then that test group will |
| 2135 |
be used. Therefore, the project can override the test group for a given |
| 2136 |
TPL if desired by setting ``${TPL_NAME}_TESTGROUP`` before the first |
| 2137 |
`<repoDir>/TPLsList.cmake`_ file gets processed. |
| 2138 |
|
| 2139 |
.. _${TPL_NAME}_TPLS_LIST_FILE: |
| 2140 |
|
| 2141 |
``${TPL_NAME}_TPLS_LIST_FILE`` |
| 2142 |
|
| 2143 |
Absolute path of the (last) `<repoDir>/TPLsList.cmake`_ file that declared |
| 2144 |
this external package/TPL. |
| 2145 |
|
| 2146 |
Note, the ``<findmod>`` field path in the call to |
| 2147 |
`tribits_repository_define_tpls()`_ is relative to the TriBITS repository dir |
| 2148 |
``<repoDir>`` but a relative path in for the variable `<tplName>_FINDMOD`_ is |
| 2149 |
relative to the project dir ``<projectDir>``. There is a translation of the |
| 2150 |
``<findmod>`` field to the variable ``<tplName>_FINDMOD`` that takes place |
| 2151 |
when the `<repoDir>/TPLsList.cmake`_ file is processed to make this so. |
| 2152 |
|
| 2153 |
As noted above, it is allowed for the same TPL to be listed in multiple |
| 2154 |
`<repoDir>/TPLsList.cmake`_ files. In this case, the rules for overrides of |
| 2155 |
the find module and the test group as described above. |
| 2156 |
|
| 2157 |
The specification given in `Enabling support for an optional Third-Party |
| 2158 |
Library (TPL)`_ and `Creating the FindTPL<tplName>.cmake file`_ describe how |
| 2159 |
to create a ``FindTPL<tplName>.cmake`` module. However, all that is required |
| 2160 |
is that some CMake file fragment exist such that, once included, will define |
| 2161 |
the target ``<tplName>::all_libs`` and create the ``<tplName>Config.cmake`` |
| 2162 |
file in the correct location (see `Requirements for FindTPL<tplName>.cmake modules`_). |
| 2163 |
|
| 2164 |
|
| 2165 |
Processing of TriBITS Files: Ordering and Details |
| 2166 |
-------------------------------------------------- |
| 2167 |
|
| 2168 |
One of the most important things to know about TriBITS is what files it |
| 2169 |
processes, in what order, and in what context. This is critical to being able |
| 2170 |
to understand what impact (if any) setting a variable or otherwise changing |
| 2171 |
the CMake run-time state will have on configuring a CMake project which uses |
| 2172 |
TriBITS. While the different files that make up a `TriBITS Project`_, |
| 2173 |
`TriBITS Repository`_, `TriBITS Package`_, `TriBITS Subpackage`_, and `TriBITS |
| 2174 |
TPL`_ were defined in the section `TriBITS Structural Units`_, that material |
| 2175 |
did not fully describe the context and in what order these files are processed |
| 2176 |
by the TriBITS framework. |
| 2177 |
|
| 2178 |
The TriBITS system processes the project's files in one of two general use |
| 2179 |
cases. The first use case is in the basic configuration of the project with a |
| 2180 |
standard ``cmake`` command invocation in order to set up the build files in |
| 2181 |
the binary directory (see `Full TriBITS Project Configuration`_). The second |
| 2182 |
use case is in reading the project's dependency-related files in order to |
| 2183 |
build the package dependency data-structure (e.g. the |
| 2184 |
`<Project>PackageDependencies.xml`_ file, see `Reduced Package Dependency |
| 2185 |
Processing`_). The second use case of reading the project's dependency files |
| 2186 |
is largely a subset of the first. |
| 2187 |
|
| 2188 |
Another factor that is important to understand is the scoping in which the |
| 2189 |
various files are processed (with ``include()`` or ``add_subdirectory()``). |
| 2190 |
This scoping has a large impact on the configuration of the project and what |
| 2191 |
effect the processing of files and setting variables have on the project as a |
| 2192 |
whole. Some of the strange scoping rules for CMake are discussed in `CMake |
| 2193 |
Language Overview and Gotchas`_ and should be understood before trying to |
| 2194 |
debug issues with processing. Many of the basic files are processed |
| 2195 |
(included) in the base project `<projectDir>/CMakeLists.txt`_ scope and |
| 2196 |
therefore any local variables set in these files are accessible to the entire |
| 2197 |
CMake project (after the file is processed, of course). Other files get |
| 2198 |
processed inside of functions which have their own local scope and therefore |
| 2199 |
only impact the rest of the project in more purposeful ways. And of course |
| 2200 |
all of the package `<packageDir>/CMakeLists.txt`_ files that are processed |
| 2201 |
using ``add_subdirectory()`` create a new local scope for that given package. |
| 2202 |
|
| 2203 |
|
| 2204 |
Full TriBITS Project Configuration |
| 2205 |
++++++++++++++++++++++++++++++++++ |
| 2206 |
|
| 2207 |
The first use case to describe is the full processing of all of the TriBITS |
| 2208 |
project's files starting with the base `<projectDir>/CMakeLists.txt`_ file. |
| 2209 |
This begins with the invocation of the following CMake command to generate the |
| 2210 |
project's build files:: |
| 2211 |
|
| 2212 |
$ cmake [options] <projectDir> |
| 2213 |
|
| 2214 |
Below, is a short pseudo-code algorithm for the TriBITS framework processing |
| 2215 |
and callbacks that begins in the `<projectDir>/CMakeLists.txt`_ file and |
| 2216 |
proceeds through the call to `tribits_project()`_. |
| 2217 |
|
| 2218 |
.. _Full Processing of TriBITS Project Files: |
| 2219 |
|
| 2220 |
**Full Processing of TriBITS Project Files:** |
| 2221 |
|
| 2222 |
| 1. Read `<projectDir>/ProjectName.cmake`_ (sets `PROJECT_NAME`_) |
| 2223 |
| 2. Call ``project(${PROJECT_NAME} NONE)`` (sets `${PROJECT_NAME}_SOURCE_DIR`_ |
| 2224 |
| and `${PROJECT_NAME}_BINARY_DIR`_) |
| 2225 |
| 3. Call `tribits_project()`_: |
| 2226 |
| 1) Set `PROJECT_SOURCE_DIR`_ and `PROJECT_BINARY_DIR`_ |
| 2227 |
| 2) For each ``<optFile>`` in ${`${PROJECT_NAME}_CONFIGURE_OPTIONS_FILE`_} |
| 2228 |
| then in ${`${PROJECT_NAME}_CONFIGURE_OPTIONS_FILE_APPEND`_} |
| 2229 |
: |
| 2230 |
| * ``include(<optFile>)`` |
| 2231 |
| 3) Set variables ``CMAKE_HOST_SYSTEM_NAME`` and ``${PROJECT_NAME}_HOSTNAME`` |
| 2232 |
| (both of these can be overridden in the cache by the user) |
| 2233 |
| 4) Find some optional command-line tools: |
| 2234 |
| a) Find Python (sets ``Python3_EXECUTABLE``, see `Python Support`_) |
| 2235 |
| b) Find Git (sets ``GIT_EXECUTABLE`` and ``GIT_VERSION_STRING``) |
| 2236 |
| 5) ``include(`` `<projectDir>/Version.cmake`_ ``)`` |
| 2237 |
| 6) Define primary TriBITS options and read in the list of extra repositories |
| 2238 |
| (calls ``tribits_define_global_options_and_define_extra_repos()``) |
| 2239 |
| * ``include(`` `<projectDir>/cmake/ExtraRepositoriesList.cmake`_ ``)`` |
| 2240 |
| 7) For each ``<repoDir>`` in all defined TriBITS repositories: |
| 2241 |
| * ``include(`` `<repoDir>/cmake/CallbackSetupExtraOptions.cmake`_ ``)`` |
| 2242 |
| * Call macro ``tribits_repository_setup_extra_options()`` |
| 2243 |
| 9) Call `tribits_read_all_project_deps_files_create_deps_graph()`_: |
| 2244 |
| a) For each ``<repoDir>`` in all defined TriBITS repositories: |
| 2245 |
| * ``include(`` `<repoDir>/TPLsList.cmake`_ ``)`` and process list |
| 2246 |
| * ``include(`` `<repoDir>/PackagesList.cmake`_ ``)`` and process list |
| 2247 |
| b) For each ``<repoDir>`` in all defined TriBITS repositories: |
| 2248 |
| * ``include(`` `<repoDir>/cmake/RepositoryDependenciesSetup.cmake`_ ``)`` |
| 2249 |
| c) ``include(`` `<projectDir>/cmake/ProjectDependenciesSetup.cmake`_ ``)`` |
| 2250 |
| d) For each ``<packageDir>`` in all defined top-level packages: |
| 2251 |
| * ``include(`` `<packageDir>/cmake/Dependencies.cmake`_ ``)`` |
| 2252 |
| - Sets all package-specific options (see `TriBITS Package Cache Variables`_) |
| 2253 |
| * For each ``<spkgDir>`` in all subpackages for this package: |
| 2254 |
| * ``include(`` `<packageDir>/<spkgDir>/cmake/Dependencies.cmake`_ ``)`` |
| 2255 |
| - Sets all subpackage-specific options |
| 2256 |
| 10) Adjust package and TPLs enables and disables |
| 2257 |
| (see `Package Dependencies and Enable/Disable Logic`_) |
| 2258 |
| 11) `Probe and set up the environment`_ (finds MPI, compilers, etc.) |
| 2259 |
| (see `TriBITS Environment Probing and Setup`_) |
| 2260 |
| * ``include(`` `<projectDir>/cmake/ProjectCompilerPostConfig.cmake`_ ``)`` |
| 2261 |
| 12) For each ``<tplName>`` in the set of enabled TPLs: |
| 2262 |
| * ``include(${<tplName>_FINDMOD})`` (see `TriBITS TPL`_) |
| 2263 |
| 13) For each ``<repoDir>`` in all defined TriBITS repositories: |
| 2264 |
| * Read `<repoDir>/Copyright.txt`_ |
| 2265 |
| * ``include(`` `<repoDir>/Version.cmake`_ ``)`` |
| 2266 |
| (see `Project and Repository Versioning and Release Mode`_) |
| 2267 |
| 14) For each ``<packageDir>`` in all enabled top-level packages |
| 2268 |
| * ``add_subdirectory(`` `<packageDir>/CMakeLists.txt`_ ``)`` |
| 2269 |
| * For each ``<spkgDir>`` in all enabled subpackages for this package: |
| 2270 |
| * ``add_subdirectory(`` `<packageDir>/<spkgDir>/CMakeLists.txt`_ ``)`` |
| 2271 |
| 16) ``include(`` `<projectDir>/cmake/CallbackDefineProjectPackaging.cmake`_ ``)`` |
| 2272 |
| * Call ``tribits_project_define_packaging()`` |
| 2273 |
| 16) For each ``<repoDir>`` in all defined TriBITS repositories: |
| 2274 |
| * ``include(`` `<repoDir>/cmake/CallbackDefineRepositoryPackaging.cmake`_ ``)`` |
| 2275 |
| * Call ``tribits_repository_define_packaging()`` |
| 2276 |
|
| 2277 |
The TriBITS Framework obviously does a lot more than what is described above |
| 2278 |
but the basic trace of major operations and ordering and the processing of |
| 2279 |
project, repository, package, and subpackage files should be clear. All of |
| 2280 |
this information should also be clear when enabling `File Processing |
| 2281 |
Tracing`_ and watching the output from the ``cmake`` configure STDOUT. |
| 2282 |
|
| 2283 |
Reduced Package Dependency Processing |
| 2284 |
++++++++++++++++++++++++++++++++++++++ |
| 2285 |
|
| 2286 |
In addition to the full processing that occurs as part of the `Full TriBITS |
| 2287 |
Project Configuration`_, there are also TriBITS tools that only process as |
| 2288 |
subset of project's files. This reduced processing is performed in order to |
| 2289 |
build up the project's package dependencies data-structure and to write the |
| 2290 |
file `<Project>PackageDependencies.xml`_. For example, the tool |
| 2291 |
`checkin-test.py`_ and the function `tribits_ctest_driver()`_ both drive this |
| 2292 |
type of processing. In particular, the CMake -P script |
| 2293 |
`TribitsDumpDepsXmlScript.cmake`_ reads all of the project's |
| 2294 |
dependency-related files and dumps out the `<Project>PackageDependencies.xml`_ |
| 2295 |
file (see `TriBITS Project Dependencies XML file and tools`_). This reduced |
| 2296 |
processing (e.g. as executed in ``cmake -P TribitsDumpDepsXmlScript.cmake``) |
| 2297 |
is described below. |
| 2298 |
|
| 2299 |
.. _Reduced Dependency Processing of TriBITS Project Files: |
| 2300 |
|
| 2301 |
**Reduced Dependency Processing of TriBITS Project:** |
| 2302 |
|
| 2303 |
| 1. Read `<projectDir>/ProjectName.cmake`_ (sets `PROJECT_NAME`_) |
| 2304 |
| 2. ``include(`` `<projectDir>/cmake/ExtraRepositoriesList.cmake`_ ``)`` |
| 2305 |
| 3. Call ``tribits_read_all_project_deps_files_create_deps_graph()``: |
| 2306 |
| a) For each ``<repoDir>`` in all defined TriBITS repositories: |
| 2307 |
| * ``include(`` `<repoDir>/TPLsList.cmake`_ ``)`` |
| 2308 |
| * ``include(`` `<repoDir>/PackagesList.cmake`_ ``)`` |
| 2309 |
| b) For each ``<repoDir>`` in all defined TriBITS repositories: |
| 2310 |
| * ``include(`` `<repoDir>/cmake/RepositoryDependenciesSetup.cmake`_ ``)`` |
| 2311 |
| c) ``include(`` `<projectDir>/cmake/ProjectDependenciesSetup.cmake`_ ``)`` |
| 2312 |
| d) For each ``<packageDir>`` in all defined top-level packages: |
| 2313 |
| * ``include(`` `<packageDir>/cmake/Dependencies.cmake`_ ``)`` |
| 2314 |
| - Sets all package-specific options (see `TriBITS Package Cache Variables`_) |
| 2315 |
| * For each ``<spkgDir>`` in all subpackages for this package: |
| 2316 |
| * ``include(`` `<packageDir>/<spkgDir>/cmake/Dependencies.cmake`_ ``)`` |
| 2317 |
| - Sets all subpackage-specific options |
| 2318 |
| e) Write the file `<Project>PackageDependencies.xml`_ |
| 2319 |
| (specified using `${PROJECT_NAME}_DEPS_XML_OUTPUT_FILE`_) |
| 2320 |
|
| 2321 |
When comparing the above reduced dependency processing to the `Full Processing |
| 2322 |
of TriBITS Project Files`_ it is important to note that that several files are |
| 2323 |
**not** processed in the reduced algorithm shown above. The files that are |
| 2324 |
**not** processed include `<projectDir>/Version.cmake`_, |
| 2325 |
`<repoDir>/Version.cmake`_ and |
| 2326 |
`<repoDir>/cmake/CallbackSetupExtraOptions.cmake`_ (in addition to not |
| 2327 |
processing any of the ``CMakeLists.txt`` files obviously). Therefore, one |
| 2328 |
cannot put anything in these non-processed files that would impact the |
| 2329 |
definition of TriBITS repositories, packages, TPLs, etc. Anything that would |
| 2330 |
affect the dependencies data-structure that gets written out as |
| 2331 |
`<Project>PackageDependencies.xml`_ must be contained in the files that are |
| 2332 |
processed shown in the reduced processing above. |
| 2333 |
|
| 2334 |
Debugging issues with `Reduced Dependency Processing of TriBITS Project |
| 2335 |
Files`_ is more difficult because one cannot easily turn on `File Processing |
| 2336 |
Tracing`_ like one can when doing the full CMake configure. However, options |
| 2337 |
may be added to the various tools to show this file processing and help debug |
| 2338 |
problems. |
| 2339 |
|
| 2340 |
File Processing Tracing |
| 2341 |
+++++++++++++++++++++++ |
| 2342 |
|
| 2343 |
In order to aid in debugging problems with `Full TriBITS Project |
| 2344 |
Configuration`_ and `Reduced Package Dependency Processing`_, TriBITS defines |
| 2345 |
the CMake cache option `${PROJECT_NAME}_TRACE_FILE_PROCESSING`_. When |
| 2346 |
enabled, TriBITS will print out when any of the project-related, |
| 2347 |
repository-related, or package-related file is being processed by TriBITS. |
| 2348 |
When ``${PROJECT_NAME}_TRACE_FILE_PROCESSING=ON``, lines starting with ``"-- |
| 2349 |
File Trace:"`` are printed to ``cmake`` stdout for files that TriBITS |
| 2350 |
automatically processes where there may be any confusion about what files are |
| 2351 |
processed and when. |
| 2352 |
|
| 2353 |
For example, for `TribitsExampleProject`_, the configure file trace for the |
| 2354 |
configure command:: |
| 2355 |
|
| 2356 |
$ cmake \ |
| 2357 |
-DTribitsExProj_TRIBITS_DIR=<tribitsDir> \ |
| 2358 |
-DTribitsExProj_ENABLE_MPI=ON \ |
| 2359 |
-DTribitsExProj_ENABLE_ALL_PACKAGES=ON \ |
| 2360 |
-DTribitsExProj_ENABLE_TESTS=ON \ |
| 2361 |
-DTribitsExProj_TRACE_FILE_PROCESSING=ON \ |
| 2362 |
-DTribitsExProj_ENABLE_CPACK_PACKAGING=ON \ |
| 2363 |
-DTribitsExProj_DUMP_CPACK_SOURCE_IGNORE_FILES=ON \ |
| 2364 |
<tribitsDir>/doc/TribitsExampleProject \ |
| 2365 |
| grep "^-- File Trace:" |
| 2366 |
|
| 2367 |
looks something like:: |
| 2368 |
|
| 2369 |
-- File Trace: PROJECT INCLUDE [...]/Version.cmake |
| 2370 |
-- File Trace: REPOSITORY INCLUDE [...]/cmake/CallbackSetupExtraOptions.cmake |
| 2371 |
-- File Trace: REPOSITORY INCLUDE [...]/PackagesList.cmake |
| 2372 |
-- File Trace: REPOSITORY INCLUDE [...]/TPLsList.cmake |
| 2373 |
-- File Trace: PACKAGE INCLUDE [...]/packages/simple_cxx/cmake/Dependencies.cmake |
| 2374 |
-- File Trace: PACKAGE INCLUDE [...]/packages/mixed_lang/cmake/Dependencies.cmake |
| 2375 |
-- File Trace: PACKAGE INCLUDE [...]/packages/with_subpackages/cmake/Dependencies.cmake |
| 2376 |
-- File Trace: PACKAGE INCLUDE [...]/packages/with_subpackages/a/cmake/Dependencies.cmake |
| 2377 |
-- File Trace: PACKAGE INCLUDE [...]/packages/with_subpackages/b/cmake/Dependencies.cmake |
| 2378 |
-- File Trace: PACKAGE INCLUDE [...]/packages/with_subpackages/c/cmake/Dependencies.cmake |
| 2379 |
-- File Trace: PACKAGE INCLUDE [...]/packages/wrap_external/cmake/Dependencies.cmake |
| 2380 |
-- File Trace: PROJECT CONFIGURE [...]/cmake/ctest/CTestCustom.cmake.in |
| 2381 |
-- File Trace: REPOSITORY READ [...]/Copyright.txt |
| 2382 |
-- File Trace: REPOSITORY INCLUDE [...]/Version.cmake |
| 2383 |
-- File Trace: PACKAGE ADD_SUBDIR [...]/packages/simple_cxx/CMakeLists.txt |
| 2384 |
-- File Trace: PACKAGE ADD_SUBDIR [...]/packages/simple_cxx/test/CMakeLists.txt |
| 2385 |
-- File Trace: PACKAGE ADD_SUBDIR [...]/packages/mixed_lang/CMakeLists.txt |
| 2386 |
-- File Trace: PACKAGE ADD_SUBDIR [...]/packages/mixed_lang/test/CMakeLists.txt |
| 2387 |
-- File Trace: PACKAGE ADD_SUBDIR [...]/packages/with_subpackages/CMakeLists.txt |
| 2388 |
-- File Trace: PACKAGE ADD_SUBDIR [...]/packages/with_subpackages/a/CMakeLists.txt |
| 2389 |
-- File Trace: PACKAGE ADD_SUBDIR [...]/packages/with_subpackages/a/tests/CMakeLists.txt |
| 2390 |
-- File Trace: PACKAGE ADD_SUBDIR [...]/packages/with_subpackages/b/CMakeLists.txt |
| 2391 |
-- File Trace: PACKAGE ADD_SUBDIR [...]/packages/with_subpackages/b/tests/CMakeLists.txt |
| 2392 |
-- File Trace: PACKAGE ADD_SUBDIR [...]/packages/with_subpackages/c/CMakeLists.txt |
| 2393 |
-- File Trace: PACKAGE ADD_SUBDIR [...]/packages/with_subpackages/c/tests/CMakeLists.txt |
| 2394 |
-- File Trace: PROJECT INCLUDE [...]/cmake/CallbackDefineProjectPackaging.cmake |
| 2395 |
-- File Trace: REPOSITORY INCLUDE [...]/cmake/CallbackDefineRepositoryPackaging.cmake |
| 2396 |
|
| 2397 |
However, every file that TriBITS processes is not printed in this file trace |
| 2398 |
if it should be obvious that the file is being processed. For example, the |
| 2399 |
package's configured header file created using `tribits_configure_file()`_ |
| 2400 |
does not result in a file trace print statement because this is an |
| 2401 |
unconditional command that is explicitly called in one of the package's |
| 2402 |
``CMakeLists.txt`` files so it should be clear that this file is being |
| 2403 |
processed and exactly when it is processed. |
| 2404 |
|
| 2405 |
|
| 2406 |
Coexisting Projects, Repositories, and Packages |
| 2407 |
----------------------------------------------- |
| 2408 |
|
| 2409 |
Certain simplifications are allowed when defining TriBITS projects, |
| 2410 |
repositories and packages. The known allowed simplifications are described |
| 2411 |
below. |
| 2412 |
|
| 2413 |
**TriBITS Repository Dir == TriBITS Project Dir**: It is allowed for a TriBITS |
| 2414 |
Project and a TriBITS Repository to be the same source directory and in fact |
| 2415 |
this is the default for every TriBITS project (unless the |
| 2416 |
`<projectDir>/cmake/NativeRepositoriesList.cmake`_ is defined). In this case, |
| 2417 |
the repository name `REPOSITORY_NAME`_ and the project name `PROJECT_NAME`_ |
| 2418 |
are the same as well. This is quite common and is in fact the default that |
| 2419 |
every TriBITS Project is also a TriBITS repository (and therefore must contain |
| 2420 |
`<repoDir>/PackagesList.cmake`_ and `<repoDir>/TPLsList.cmake`_ files). This |
| 2421 |
is the case, for example, with the `Trilinos`_ and the |
| 2422 |
`TribitsExampleProject`_ projects and repositories. In this case, the |
| 2423 |
Project's and the Repository's ``Version.cmake`` and ``Copyright.txt`` files |
| 2424 |
are also one and the same, as they should be (see `Project and Repository |
| 2425 |
Versioning and Release Mode`_). |
| 2426 |
|
| 2427 |
**TriBITS Package Dir == TriBITS Repository Dir**: It is also allowed for a |
| 2428 |
TriBITS Repository to have only one package and to have that package be the |
| 2429 |
base repository directory. The TriBITS Repository and the single TriBITS |
| 2430 |
Package would typically have the same name in this case (but that is actually |
| 2431 |
not required but it is confusing if they are not the same). For example, in |
| 2432 |
the TriBITS test project `MockTrilinos`_, the repository and package |
| 2433 |
``extraRepoOnePackage`` are the same directory. In this case, the file |
| 2434 |
``extraRepoOnePackage/PackagesList.cmake`` looks like: |
| 2435 |
|
| 2436 |
.. include:: ../../examples/MockTrilinos/extraRepoOnePackage/PackagesList.cmake |
| 2437 |
:literal: |
| 2438 |
|
| 2439 |
(Note the dot ``'.'`` for the package directory.) |
| 2440 |
|
| 2441 |
This is also how the real TriBITS repository and package `DataTransferKit`_ is |
| 2442 |
set up (at least that is the way it was when this document was first written). |
| 2443 |
|
| 2444 |
.. _DataTransferKit: https://github.com/CNERG/DataTransferKit |
| 2445 |
|
| 2446 |
However, to maximize flexibility, it is recommended that a TriBITS package and |
| 2447 |
its TriBITS repository **not** share the same directory or the same name. |
| 2448 |
This allows a TriBITS repository to define more packages later. |
| 2449 |
|
| 2450 |
.. _TriBITS Package == TriBITS Repository == TriBITS Project: |
| 2451 |
|
| 2452 |
**TriBITS Package Dir == TriBITS Repository Dir == TriBITS Project Dir**: In |
| 2453 |
the extreme, it is possible to collapse a single TriBITS package, repository, |
| 2454 |
and project into the same base source directory. They can also share the same |
| 2455 |
name for the package, repository and package. One example of this is the |
| 2456 |
``TriBITS`` project and `The TriBITS Test Package`_ themselves, which are both |
| 2457 |
rooted in the base ``TriBITS/`` source directory of the stand-alone TriBITS |
| 2458 |
repository. There are a few restrictions and modifications needed to get this |
| 2459 |
to work: |
| 2460 |
|
| 2461 |
* The base ``CMakeLists.txt`` file must be modified to allow it to be |
| 2462 |
processed both as the base project ``CMakeLists.txt`` file and as the |
| 2463 |
package's base ``CMakeLists.txt`` file. In the case of |
| 2464 |
``TriBITS/CMakeLists.txt``, a big if statement is used. |
| 2465 |
|
| 2466 |
.. ToDo: Create a new macro called tribits_project_and_package() that will |
| 2467 |
.. automatically take care of the details of a TriBITS package also being a |
| 2468 |
.. TriBITS project. |
| 2469 |
|
| 2470 |
Other than that simple modification to the top-level ``CMakeLists.txt`` file, |
| 2471 |
a TriBITS project, repository, and package can all be rooted in the same |
| 2472 |
source directory. |
| 2473 |
|
| 2474 |
The primary use case for collapsing a project, repository, and package into a |
| 2475 |
single base source directory would be to support the stand-alone build of a |
| 2476 |
TriBITS package as its own entity that uses an independent installation of the |
| 2477 |
TriBITS (or a minimal snapshot of TriBITS). If a given TriBITS package has no |
| 2478 |
required `upstream`_ TriBITS package dependencies and minimal external |
| 2479 |
package/TPL dependencies (or only uses `Standard TriBITS TPLs`_ or `Common |
| 2480 |
TriBITS TPLs`_ already defined in the ``tribits/core/std_tpls/`` or |
| 2481 |
``tribits/common_tpls/`` directories), then creating a stand-alone project |
| 2482 |
build of a single TriBITS package requires fairly little extra overhead or |
| 2483 |
duplication. |
| 2484 |
|
| 2485 |
|
| 2486 |
Standard and Common TPLs |
| 2487 |
------------------------ |
| 2488 |
|
| 2489 |
While a TriBITS Repository can define their own external packages/TPLs and |
| 2490 |
their own TPL find modules (see `TriBITS External Package/TPL`_), the TriBITS |
| 2491 |
source tree contains TriBITS find modules for a few different standard TPLs |
| 2492 |
and common TPLs. `Standard TriBITS TPLs`_ are integral to the TriBITS system |
| 2493 |
itself while `Common TriBITS TPLs`_ are TPL that are used in several different |
| 2494 |
TriBITS Repositories and are contained in TriBITS for convenience and |
| 2495 |
uniformity. |
| 2496 |
|
| 2497 |
|
| 2498 |
Standard TriBITS TPLs |
| 2499 |
+++++++++++++++++++++ |
| 2500 |
|
| 2501 |
TriBITS contains find modules for a few standard TPLs integral to the TriBITS |
| 2502 |
system. The standard TriBITS TPLs are contained under the directory:: |
| 2503 |
|
| 2504 |
tribits/core/std_tpls/ |
| 2505 |
|
| 2506 |
The current list of standard TriBITS TPL find modules is: |
| 2507 |
|
| 2508 |
.. include:: TribitsStandardTPLsList.txt |
| 2509 |
:literal: |
| 2510 |
|
| 2511 |
The TPLs ``MPI`` and ``CUDA`` are standard because they are special in that |
| 2512 |
they define compilers and other special tools that are used in |
| 2513 |
`tribits_add_library()`_, `tribits_add_executable()`_, `tribits_add_test()`_ |
| 2514 |
and other commands. |
| 2515 |
|
| 2516 |
These standard TPLs are used in a `<repoDir>/TPLsList.cmake`_ file as:: |
| 2517 |
|
| 2518 |
tribits_repository_define_tpls( |
| 2519 |
MPI "${${PROJECT_NAME}_TRIBITS_DIR}/core/std_tpls/" PT |
| 2520 |
CUDA "${${PROJECT_NAME}_TRIBITS_DIR}/core/std_tpls/" ST |
| 2521 |
... |
| 2522 |
) |
| 2523 |
|
| 2524 |
|
| 2525 |
Common TriBITS TPLs |
| 2526 |
+++++++++++++++++++ |
| 2527 |
|
| 2528 |
TriBITS also contains find modules for several TPLs that are used across many |
| 2529 |
independent TriBITS repositories. The goal of maintaining these under TriBITS |
| 2530 |
is to enforce conformity in case these independent repositories are combined |
| 2531 |
into a single meta-project. |
| 2532 |
|
| 2533 |
The common TriBITS TPLs are contained under the directory:: |
| 2534 |
|
| 2535 |
tribits/common_tpls/ |
| 2536 |
|
| 2537 |
The current list of common TriBITS TPL find modules is: |
| 2538 |
|
| 2539 |
.. include:: TribitsCommonTPLsList.txt |
| 2540 |
:literal: |
| 2541 |
|
| 2542 |
Common TPLs are used in a `<repoDir>/TPLsList.cmake`_ file as:: |
| 2543 |
|
| 2544 |
tribits_repository_define_tpls( |
| 2545 |
BLAS "${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/" PT |
| 2546 |
LAPACK "${${PROJECT_NAME}_TRIBITS_DIR}/common_tpls/" PT |
| 2547 |
... |
| 2548 |
) |
| 2549 |
|
| 2550 |
By using a standard TPL definition, it is guaranteed that the TPL used will be |
| 2551 |
consistent with all of the TriBITS packages that depend on these TPLs in case |
| 2552 |
they are combined into a single project. |
| 2553 |
|
| 2554 |
Note that just because packages in two different TriBITS repositories |
| 2555 |
reference the same TPL does not necessarily mean that it needs to be moved |
| 2556 |
into the TriBITS source tree under ``tribits/common_tpls/``. For example, if |
| 2557 |
the TPL ``QT`` is defined in an `upstream`_ repository (e.g. Trilinos), then a |
| 2558 |
package in a `downstream`_ repository can list a dependency on the TPL ``QT`` |
| 2559 |
without having to define its own ``QT`` TPL in its repository's |
| 2560 |
`<repoDir>/TPLsList.cmake`_ file. For more details, see `TriBITS TPL`_. |
| 2561 |
|
| 2562 |
.. Where to set variables? |
| 2563 |
.. ----------------------- |
| 2564 |
|
| 2565 |
.. A TriBITS project, repository, package have a number of files in which |
| 2566 |
variables can be set and commands can be called in order to affect how the |
| 2567 |
project is defined and configured. With so many files that can be included by |
| 2568 |
TriBITS, it can be difficult to know where the right place is to set a given |
| 2569 |
set of variables. The primary considerations for where to set a variable |
| 2570 |
depend on: |
| 2571 |
|
| 2572 |
.. ToDo: Describe considerations on where to set variables ... |
| 2573 |
|
| 2574 |
|
| 2575 |
.. _TriBITS-Compliant Package: |
| 2576 |
|
| 2577 |
TriBITS-Compliant Packages |
| 2578 |
========================== |
| 2579 |
|
| 2580 |
At the CMake build-system level, there are just a few key requirements that a |
| 2581 |
TriBITS package has for its upstream dependent packages when it is being |
| 2582 |
configured to be built. These requirements apply whether the upstream package |
| 2583 |
is defined internally in the current CMake project or provided externally and |
| 2584 |
pulled in through ``find_package(<Package>)``. |
| 2585 |
|
| 2586 |
The common requirements for both internal and external **TriBITS-compliant |
| 2587 |
packages** as imposed by downstream TriBITS internal packages are: |
| 2588 |
|
| 2589 |
* Provides the (INTERFACE) target ``<Package>::all_libs`` which provides all |
| 2590 |
usage requirements for the libraries of ``<Package>`` through the target |
| 2591 |
properties: |
| 2592 |
|
| 2593 |
* ``INTERFACE_LINK_LIBRARIES``: The library files needed link against (or |
| 2594 |
upstream library targets including ``<UpstreamPackage>::all_libs`` for all |
| 2595 |
its upstream packages) |
| 2596 |
|
| 2597 |
* ``INTERFACE_INCLUDE_DIRECTORIES``: Include directories to all public header files |
| 2598 |
|
| 2599 |
* ``INTERFACE_COMPILE_OPTIONS``: Required compiler options |
| 2600 |
|
| 2601 |
* ``INTERFACE_COMPILE_DEFINITIONS``: Required compiler/macro definitions |
| 2602 |
|
| 2603 |
* ``INTERFACE_LINK_OPTIONS``: Required compiler/macro definitions |
| 2604 |
|
| 2605 |
* Any other ``INTERFACE_XXX`` or ``IMPORTED_XXX`` target property needed to |
| 2606 |
correctly use the libraries for package ``<Package>``. |
| 2607 |
|
| 2608 |
* Provides namespaced variables ``<Package>_ENABLE_<UpstreamPackage>`` set to |
| 2609 |
``TRUE`` or ``FALSE`` for all of the upstream required and optional |
| 2610 |
dependencies for the package ``<Package>``. |
| 2611 |
|
| 2612 |
* [Optional] Provides namespaced variables of the form |
| 2613 |
``<Package>_<SOME_INFO>`` for any other information about the configuration |
| 2614 |
of package ``<Package>`` that may need to be known by a downstream TriBITS |
| 2615 |
package. |
| 2616 |
|
| 2617 |
* [Optional] Provides any (namespaced by ``<package>_`` or ``<Package>_``) |
| 2618 |
CMake macros or functions that downstream CMake packages may need to use the |
| 2619 |
upstream package ``<Package>``. |
| 2620 |
|
| 2621 |
* [Optional] All of the upstream dependencies (listed in the |
| 2622 |
``INTERFACE_LINK_LIBRARIES`` property recursively) are also |
| 2623 |
`TriBITS-compliant packages`_ |
| 2624 |
|
| 2625 |
The TriBITS system will also set the variable: |
| 2626 |
|
| 2627 |
* ``<Package>_IS_TRIBITS_COMPLIANT``: Set to ``TRUE`` |
| 2628 |
|
| 2629 |
for all packages that are determined to be TriBITS-compliant packages and |
| 2630 |
satisfy the above criteria. |
| 2631 |
|
| 2632 |
The above are all that is needed by downstream TriBITS packages to build and |
| 2633 |
link against their upstream dependencies. |
| 2634 |
|
| 2635 |
Additional requirements are placed on TriBITS-compliant packages depending on |
| 2636 |
if they are defined as internal CMake packages (i.e. `TriBITS-compliant |
| 2637 |
internal packages`_) or are pulled in as external pre-built/pre-installed |
| 2638 |
packages (i.e. `TriBITS-compliant external packages`_). |
| 2639 |
|
| 2640 |
|
| 2641 |
.. _TriBITS-Compliant Internal Packages: |
| 2642 |
|
| 2643 |
TriBITS-Compliant Internal Packages |
| 2644 |
----------------------------------- |
| 2645 |
|
| 2646 |
For TriBITS packages that are defined, built, and installed from a TriBITS |
| 2647 |
CMake project, there are an additional set of requirements for them to |
| 2648 |
behavior correctly with respect to other TriBITS packages. |
| 2649 |
|
| 2650 |
The requirements for **TriBITS-compliant internal packages** are: |
| 2651 |
|
| 2652 |
* All of the requirements for a `TriBITS-Compliant Package`_. |
| 2653 |
|
| 2654 |
* At the end of configuration and generation, writes out a `TriBITS-Compliant |
| 2655 |
External Package`_ file ``<Package>Config.cmake`` and supporting files under |
| 2656 |
the build directory ``<buildDir>/cmake_packages/<Package>/`` allowing the |
| 2657 |
built (but not installed) package to be used by downstream CMake |
| 2658 |
packages/projects. |
| 2659 |
|
| 2660 |
* Provides an install target to create a `TriBITS-Compliant External Package`_ |
| 2661 |
file ``<Package>Config.cmake`` and supporting files under the install |
| 2662 |
directory ``<installDir>/lib/cmake/<Package>/`` allowing the installed |
| 2663 |
package to be used by downstream CMake packages/projects. |
| 2664 |
|
| 2665 |
* [Optional] All of the upstream dependencies (recursively) are also |
| 2666 |
`TriBITS-compliant packages`_. |
| 2667 |
|
| 2668 |
If a TriBITS package provides any CTest tests/examples, then it must also |
| 2669 |
satisfy the following requirements: |
| 2670 |
|
| 2671 |
* Test names must be prefixed with the package name ``<Package>_``. |
| 2672 |
|
| 2673 |
* The package name ``<Package>`` must be appended to the ``LABELS`` test |
| 2674 |
property list. |
| 2675 |
|
| 2676 |
* Tests should only be added if the variable ``<Package>_ENABLE_TESTS`` is |
| 2677 |
true. |
| 2678 |
|
| 2679 |
* Examples (that run as CTest tests) should only be added if the variable |
| 2680 |
``<Package>_ENABLE_EXAMPLES`` is true. |
| 2681 |
|
| 2682 |
* The ``PROCESSORS`` test property and other test properties must be set in a |
| 2683 |
way consistent with `tribits_add_test()`_ so as to run in parallel with |
| 2684 |
other tests and not overwhelm the computing resources on the machine. |
| 2685 |
|
| 2686 |
* The test ``<fullTestName>`` must not be added if the cache variable |
| 2687 |
``<fullTestName>_DISABLE`` is set to ``TRUE`` or if the cache variable |
| 2688 |
``<fullTestName>_SET_DISABLED_AND_MSG`` is set to non-empty (and the message |
| 2689 |
string should be printed to STDOUT). |
| 2690 |
|
| 2691 |
TriBITS internal packages that are defined using the TriBITS framework using |
| 2692 |
the TriBITS-provided macros and functions such as `tribits_add_library()`_ and |
| 2693 |
have tests defined using the functions `tribits_add_test()`_ and |
| 2694 |
`tribits_add_advanced_test()`_ are automatically `TriBITS-compliant internal |
| 2695 |
packages`_. And when these TriBITS-implemented internal packages are |
| 2696 |
installed, they automatically provide `TriBITS-compliant external packages`_. |
| 2697 |
But it is possible for a CMake package to write its own raw CMake code to |
| 2698 |
satisfy these basic requirements for both internal and (installed) external |
| 2699 |
packages. |
| 2700 |
|
| 2701 |
|
| 2702 |
.. _TriBITS-Compliant External Package: |
| 2703 |
|
| 2704 |
TriBITS-Compliant External Packages |
| 2705 |
----------------------------------- |
| 2706 |
|
| 2707 |
For packages that are installed on the system and not built in the current |
| 2708 |
CMake project, a streamlined type of `TriBITS External Package/TPL`_ is a |
| 2709 |
*TriBITS-compliant external package*. These special types of external |
| 2710 |
package's don't need to provide a `FindTPL<tplName>.cmake`_ find module. |
| 2711 |
Instead, they are fully defined by calling ``find_package(<Package>)`` or |
| 2712 |
``include(<someBaseDir>/<Package>Config.cmake)`` to load their |
| 2713 |
``<Package>Config.cmake`` package config file. |
| 2714 |
|
| 2715 |
The requirements for **TriBITS-compliant external packages** are: |
| 2716 |
|
| 2717 |
* All of the requirements for a `TriBITS-Compliant Package`_. |
| 2718 |
|
| 2719 |
* Defined by an installed ``<Package>Config.cmake`` file that provides |
| 2720 |
IMPORTED targets and ``set()`` statements for all of the needed variables. |
| 2721 |
|
| 2722 |
* Provides CMake variables: |
| 2723 |
|
| 2724 |
* ``<Package>_CONFIG`` or |
| 2725 |
``<Package>_TRIBITS_COMPLIANT_PACKAGE_CONFIG_FILE``: Points to the file |
| 2726 |
``<Package>Config.cmake`` (i.e. ``${CMAKE_CURRENT_LIST_FILE}``) |
| 2727 |
|
| 2728 |
* [Optional] All of the upstream dependencies (recursively) are also provided |
| 2729 |
as `TriBITS-compliant external packages`_ with |
| 2730 |
``<UpstreamPackage>Config.cmake`` files (see above) and all of the targets |
| 2731 |
and variables for a TriBITS-compliant external package are defined when the |
| 2732 |
``<Package>Config.cmake`` file is included (or pulled in with |
| 2733 |
``find_package()`` or ``find_dependency()``). |
| 2734 |
|
| 2735 |
NOTE: TriBITS-compliant external packages that provide TriBITS-compliant |
| 2736 |
external packages for all of their upstream dependencies are said to be *fully |
| 2737 |
TriBITS-compliant external packages* while those that support the minimal |
| 2738 |
requirements are said to be *minimally TriBITS-compliant external packages*. |
| 2739 |
The TriBITS external package/TPL system is robust enough to deal with |
| 2740 |
minimally TriBITS-compliant external packages. Any TriBITS external |
| 2741 |
packages/TPLs upstream from a minimally TriBITS-compliant external package |
| 2742 |
will be found again in the current TriBITS project. (In these cases, it is up |
| 2743 |
to the user to make sure that the same upstream packages are found.) |
| 2744 |
|
| 2745 |
|
| 2746 |
Example TriBITS Projects |
| 2747 |
========================= |
| 2748 |
|
| 2749 |
In this section, a few different example TriBITS projects and packages are |
| 2750 |
previewed. Most of these examples exist in the TriBITS source directory |
| 2751 |
``tribits`` itself so they are available to all users of TriBITS. These |
| 2752 |
examples also provide a means to test the TriBITS system itself (see `The |
| 2753 |
TriBITS Test Package`_). |
| 2754 |
|
| 2755 |
The first example covered is the bare bones `TribitsHelloWorld`_ example |
| 2756 |
project. The second example covered in detail is `TribitsExampleProject`_. |
| 2757 |
This example covers all the basics for setting up a simple multi-package |
| 2758 |
TriBITS project. The third example outlined is `MockTrilinos`_ which mostly |
| 2759 |
exists to test the TriBITS system itself but also contains some nice examples |
| 2760 |
of a few different TriBITS features and behaviors. The forth example is the |
| 2761 |
`ReducedMockTrilinos`_ project which is used to demonstrate TriBITS behavior |
| 2762 |
in this document. Also mentioned is the `Trilinos`_ project itself which can |
| 2763 |
be a useful example of the usage of TriBITS (see disclaimers in the section |
| 2764 |
`Trilinos`_). The last example mentioned is `The TriBITS Test Package`_ |
| 2765 |
itself which allows the TriBITS system to be tested and installed from any |
| 2766 |
TriBITS project that lists it, including the ``TriBITS`` project itself |
| 2767 |
(see `Coexisting Projects, Repositories, and Packages`_). |
| 2768 |
|
| 2769 |
The directory ``tribits/examples/`` contains some other example TriBITS |
| 2770 |
projects and repositories as well that are referred to in this and other |
| 2771 |
documents. |
| 2772 |
|
| 2773 |
|
| 2774 |
TribitsHelloWorld |
| 2775 |
----------------- |
| 2776 |
|
| 2777 |
``TribitsHelloWorld`` is about the simplest possible TriBITS project that you |
| 2778 |
can imagine and is contained under the directory:: |
| 2779 |
|
| 2780 |
tribits/examples/TribitsHelloWorld/ |
| 2781 |
|
| 2782 |
This example project contains only a single TriBITS package and no frills at |
| 2783 |
all (does not support MPI or Fortran). However, it does show how minimal a |
| 2784 |
`TriBITS Project`_ (which is also a `TriBITS Repository`_) and a `TriBITS |
| 2785 |
Package`_ can be and still demonstrates some of the value of TriBITS over raw |
| 2786 |
CMake. The simple ``HelloWorld`` package is used to compare with the raw |
| 2787 |
CMakeLists.txt file in the ``RawHeloWorld`` example project in the `TriBITS |
| 2788 |
Overview`_ document. |
| 2789 |
|
| 2790 |
The directory structure for this example his shown below: |
| 2791 |
|
| 2792 |
.. include:: TribitsHelloWorldDirAndFiles.txt |
| 2793 |
:literal: |
| 2794 |
|
| 2795 |
This has all of the required `TriBITS Project Core Files`_, `TriBITS |
| 2796 |
Repository Core Files`_, and `TriBITS Package Core Files`_. It just builds a |
| 2797 |
simple library, a simple executable, a test executable, and the tests them as |
| 2798 |
shown by the file ``TribitsHelloWorld/hello_world/CMakeLists.txt`` which is: |
| 2799 |
|
| 2800 |
.. include:: ../../examples/TribitsHelloWorld/hello_world/CMakeLists.txt |
| 2801 |
:literal: |
| 2802 |
|
| 2803 |
The build and test of this simple project is tested in the `The TriBITS Test |
| 2804 |
Package`_ file:: |
| 2805 |
|
| 2806 |
TriBITS/test/core/ExamplesUnitTests/CMakeLists.txt |
| 2807 |
|
| 2808 |
Note that this little example is a fully functional `TriBITS Repository`_ and |
| 2809 |
can be embedded in to a larger TriBITS meta-project and be seamlessly built |
| 2810 |
along with any other such TriBITS-based software. |
| 2811 |
|
| 2812 |
.. ToDo: Put in reference to the example meta-project. |
| 2813 |
|
| 2814 |
|
| 2815 |
TribitsExampleProject |
| 2816 |
---------------------- |
| 2817 |
|
| 2818 |
``TribitsExampleProject`` in an example `TriBITS Project`_ and `TriBITS |
| 2819 |
Repository`_ contained in the TriBITS source tree under:: |
| 2820 |
|
| 2821 |
tribits/examples/TribitsExampleProject/ |
| 2822 |
|
| 2823 |
When this is used as the base TriBITS project, this is the directory |
| 2824 |
corresponds to ``<projectDir>`` and ``<repoDir>`` referenced in `TriBITS |
| 2825 |
Project Core Files`_ and `TriBITS Repository Core Files`_, respectively. |
| 2826 |
|
| 2827 |
Several files from this project are used as examples in the section `TriBITS |
| 2828 |
Project Structure`_. Here, a fuller description is given of this project and |
| 2829 |
a demonstration of how TriBITS works. From this simple example project, one |
| 2830 |
can quickly see how the basic structural elements of a TriBITS project, |
| 2831 |
repository, and package (and subpackage) are pulled together. |
| 2832 |
|
| 2833 |
This simple project shows how what is listed in files: |
| 2834 |
|
| 2835 |
* `<repoDir>/PackagesList.cmake`_, |
| 2836 |
* `<packageDir>/cmake/Dependencies.cmake`_, and |
| 2837 |
* `<packageDir>/<spkgDir>/cmake/Dependencies.cmake`_ |
| 2838 |
|
| 2839 |
are used to specify the packages and packages in a TriBITS project and |
| 2840 |
repository. More details about the contents of the ``Dependencies.cmake`` |
| 2841 |
files is described in the section `Package Dependencies and Enable/Disable |
| 2842 |
Logic`_. |
| 2843 |
|
| 2844 |
The name of this project ``PROJECT_NAME`` is given in its |
| 2845 |
``TribitsExampleProject/ProjectName.cmake`` file: |
| 2846 |
|
| 2847 |
.. include:: ../../examples/TribitsExampleProject/ProjectName.cmake |
| 2848 |
:literal: |
| 2849 |
|
| 2850 |
The variable ``PROJECT_NAME=TribitsExProj`` is used to prefix (using |
| 2851 |
``"${PROJECT_NAME}_"``) all of the project's global TriBITS variables like |
| 2852 |
``TribitsExProj_ENABLE_TESTS``, ``TribitsExProj_ENABLE_ALL_PACKAGES``, etc. |
| 2853 |
|
| 2854 |
.. _TribitsExampleProject Files and Directories: |
| 2855 |
|
| 2856 |
The directory structure and key files for this example project are shown in |
| 2857 |
the partial list of `TribitsExampleProject Files and Directories`_ below:: |
| 2858 |
|
| 2859 |
TribitsExampleProject/ |
| 2860 |
CMakeLists.txt |
| 2861 |
Copyright.txt |
| 2862 |
PackagesList.cmake |
| 2863 |
ProjectName.cmake |
| 2864 |
project-checkin-test-config.py |
| 2865 |
TPLsList.cmake |
| 2866 |
Version.cmake |
| 2867 |
... |
| 2868 |
cmake/ |
| 2869 |
CallbackDefineProjectPackaging.cmake |
| 2870 |
CallbackDefineRepositoryPackaging.cmake |
| 2871 |
CallbackSetupExtraOptions.cmake |
| 2872 |
packages/ |
| 2873 |
simple_cxx/ |
| 2874 |
CMakeLists.txt |
| 2875 |
cmake/ |
| 2876 |
CheckFor__int64.cmake |
| 2877 |
Dependencies.cmake |
| 2878 |
SimpleCxx_config.h.in |
| 2879 |
src/ |
| 2880 |
CMakeLists.txt |
| 2881 |
SimpleCxx_HelloWorld.cpp |
| 2882 |
SimpleCxx_HelloWorld.hpp |
| 2883 |
test/ |
| 2884 |
CMakeLists.txt |
| 2885 |
SimpleCxx_HelloWorld_Tests.cpp |
| 2886 |
mixed_lang/ ... |
| 2887 |
with_subpackages/ |
| 2888 |
CMakeLists.txt |
| 2889 |
cmake/ |
| 2890 |
Dependencies.cmake |
| 2891 |
A/ |
| 2892 |
CMakeLists.txt |
| 2893 |
cmake/ |
| 2894 |
Dependencies.cmake |
| 2895 |
... |
| 2896 |
B/ ... |
| 2897 |
C/ ... |
| 2898 |
wrap_external/ ... |
| 2899 |
|
| 2900 |
Above, the sub-directories under ``packages/`` are sorted according to the |
| 2901 |
order listed in the ``TribitsExampleProject/PackagesList.cmake`` file: |
| 2902 |
|
| 2903 |
.. include:: ../../examples/TribitsExampleProject/PackagesList.cmake |
| 2904 |
:literal: |
| 2905 |
|
| 2906 |
From this file, we get the list of top-level packages ``SimpleCxx``, |
| 2907 |
``MixedLang``, ``WithSubpackages``, and ``WrapExternal`` (and their base |
| 2908 |
package directories and testing group, see `<repoDir>/PackagesList.cmake`_). |
| 2909 |
(NOTE: By default the package ``InsertedPkg`` is not defined because its |
| 2910 |
directory is missing, see `How to insert a package into an upstream repo`_.) |
| 2911 |
|
| 2912 |
A full listing of package files in `TribitsExampleProject Files and |
| 2913 |
Directories`_ is only shown for the ``SimpleCxx`` package directory |
| 2914 |
``packages/simple_cxx/``. For this package, ``<packageDir> = |
| 2915 |
<repoDir>/packages/simple_cxx`` and ``PACKAGE_NAME = SimpleCxx``. As |
| 2916 |
explained in `TriBITS Package Core Files`_, the files |
| 2917 |
`<packageDir>/cmake/Dependencies.cmake`_ and `<packageDir>/CMakeLists.txt`_ |
| 2918 |
must exist for every package directory listed in |
| 2919 |
`<repoDir>/PackagesList.cmake`_ and we see these files under in the directory |
| 2920 |
``packages/simple_cxx/``. The package ``SimpleCxx`` does not have any |
| 2921 |
`upstream`_ package dependencies. |
| 2922 |
|
| 2923 |
Now consider the example top-level package ``WithSubpackages`` which, |
| 2924 |
as the name suggests, is broken down into subpackages. The |
| 2925 |
``WithSubpackages`` dependencies file:: |
| 2926 |
|
| 2927 |
TribitsExampleProject/packages/with_subpackages/cmake/Dependencies.cmake |
| 2928 |
|
| 2929 |
with contents: |
| 2930 |
|
| 2931 |
.. include:: ../../examples/TribitsExampleProject/packages/with_subpackages/cmake/Dependencies.cmake |
| 2932 |
:literal: |
| 2933 |
|
| 2934 |
references the three subpackages with sub-directories ``<spkgDir>`` = ``A``, |
| 2935 |
``B``, and ``C`` under the parent package directory |
| 2936 |
``packages/package_with_packages/`` which are shown in `TribitsExampleProject |
| 2937 |
Files and Directories`_. This gives another set of three packages |
| 2938 |
``WithSubpackagesA``, ``WithSubpackagesB``, |
| 2939 |
and ``WithSubpackagesC``. Combining ``<packageDir> = |
| 2940 |
packages/package_with_packages`` and ``<spkgDir>`` for each subpackage gives |
| 2941 |
the subpackage directories:: |
| 2942 |
|
| 2943 |
TribitsExampleProject/packages/with_subpackages/a/ |
| 2944 |
TribitsExampleProject/packages/with_subpackages/b/ |
| 2945 |
TribitsExampleProject/packages/with_subpackages/c/ |
| 2946 |
|
| 2947 |
Together with the top-level parent package ``WithSubpackages`` |
| 2948 |
itself, this top-level package provides four `TriBITS Packages`_ giving the |
| 2949 |
final list of packages provided by this TriBITS repo as:: |
| 2950 |
|
| 2951 |
SimpleCxx MixedLang WithSubpackagesA WithSubpackagesB WithSubpackagesC \ |
| 2952 |
WithSubpackages WrapExternal 7 |
| 2953 |
|
| 2954 |
The above list of packages is printed (with the number of packages printed |
| 2955 |
at the end) by TriBITS to the ``cmake`` stdout on the line starting with |
| 2956 |
``"Final set of non-enabled packages:"`` when no packages are enabled (see |
| 2957 |
`Selecting the list of packages to enable`_). (Note that TriBITS does not put |
| 2958 |
in line-brakes with continuation characters ``"\"`` as shown above.) TriBITS |
| 2959 |
defines enable/disable cache variables for each of these defined packages |
| 2960 |
like ``TribitsExProj_ENABLE_SimpleCxx`` and |
| 2961 |
``TribitsExProj_ENABLE_WithSubpackagesA``, and defines all |
| 2962 |
the variables listed in `TriBITS Package Cache Variables`_ that are settable |
| 2963 |
by the users or by the dependency logic described in section `Package |
| 2964 |
Dependencies and Enable/Disable Logic`_. |
| 2965 |
|
| 2966 |
When starting a new TriBITS project, repository, or package, one should |
| 2967 |
consider basing these on the examples in ``TribitsExampleProject``. In fact, |
| 2968 |
the skeletons for any of the |
| 2969 |
|
| 2970 |
* `TriBITS Project Core Files`_, |
| 2971 |
* `TriBITS Repository Core Files`_, |
| 2972 |
* `TriBITS Package Core Files`_, or |
| 2973 |
* `TriBITS Subpackage Core Files`_ |
| 2974 |
|
| 2975 |
should be copied from this example project as they represent best practice |
| 2976 |
when using TriBITS for the typical use cases. |
| 2977 |
|
| 2978 |
|
| 2979 |
TribitsExampleProject2 |
| 2980 |
---------------------- |
| 2981 |
|
| 2982 |
``TribitsExampleProject2`` in an example `TriBITS Project`_ and `TriBITS |
| 2983 |
Repository`_ contained in the TriBITS source tree under:: |
| 2984 |
|
| 2985 |
tribits/examples/TribitsExampleProject2/ |
| 2986 |
|
| 2987 |
This example TriBITS project provides some examples for a few other features |
| 2988 |
and testing scenarios. It contains three internal packages ``Package1``, |
| 2989 |
``Package2``, and ``Package3`` as shown in its ``PackagesList.cmake`` file: |
| 2990 |
|
| 2991 |
.. include:: ../../examples/TribitsExampleProject2/PackagesList.cmake |
| 2992 |
:literal: |
| 2993 |
|
| 2994 |
and supports four external packages/TPLs ``Tpl1``, ``Tpl2``, ``Tpl3``, and |
| 2995 |
``Tpl4`` as shown in its ``TPLsList.cmake`` file: |
| 2996 |
|
| 2997 |
.. include:: ../../examples/TribitsExampleProject2/TPLsList.cmake |
| 2998 |
:literal: |
| 2999 |
|
| 3000 |
|
| 3001 |
MockTrilinos |
| 3002 |
------------- |
| 3003 |
|
| 3004 |
The TriBITS project ``MockTrilinos`` is contained under the directory:: |
| 3005 |
|
| 3006 |
tribits/examples/MockTrilinos/ |
| 3007 |
|
| 3008 |
This TriBITS project is not a full TriBITS project (i.e. it does not build |
| 3009 |
anything). Instead, it is primarily used to test the TriBITS system using |
| 3010 |
tests defined in the `The TriBITS Test Package`_. The ``MockTrilinos`` |
| 3011 |
project is actually given the name ``PROJECT_NAME = Trilinos`` and contains a |
| 3012 |
subset of Trilinos packages with slightly modified dependencies from a real |
| 3013 |
version of the Trilinos project from May 2009. The list of packages in:: |
| 3014 |
|
| 3015 |
tribits/examples/MockTrilinos/PackagesList.cmake |
| 3016 |
|
| 3017 |
is: |
| 3018 |
|
| 3019 |
.. include:: ../../examples/MockTrilinos/PackagesList.cmake |
| 3020 |
:literal: |
| 3021 |
|
| 3022 |
All of the package directories listed above have |
| 3023 |
`<packageDir>/cmake/Dependencies.cmake`_ files but generally do not have |
| 3024 |
`<packageDir>/CMakeLists.txt`_ files since most of usage of ``MockTrilinos`` |
| 3025 |
just involves the testing of the algorithms and behaviors described in the |
| 3026 |
section `Package Dependencies and Enable/Disable Logic`_. |
| 3027 |
|
| 3028 |
``MockTrilinos`` also contains a number of extra TriBITS repositories used in |
| 3029 |
various tests. These extra repositories offer examples of different types of |
| 3030 |
TriBITS repositories like: |
| 3031 |
|
| 3032 |
* ``extraRepoOnePackage``: Contains just the single package |
| 3033 |
``extraRepoOnePackage`` which is defined in the base repository directory. |
| 3034 |
|
| 3035 |
* ``extraRepoOnePackageThreeSubpackages``: Contains just the single package |
| 3036 |
``extraRepoOnePackageThreeSubpackages`` which is defined in the base |
| 3037 |
repository directory but is broken up into subpackages. |
| 3038 |
|
| 3039 |
* ``extraRepoTwoPackages``: Contains just two packages but provides an example |
| 3040 |
of defining multiple repositories with possible missing required and |
| 3041 |
optional `upstream`_ packages (see `Multi-Repository Support`_). |
| 3042 |
|
| 3043 |
* ``extraTrilinosRepo``: Just a typical extra repo with add-on packages and |
| 3044 |
external packages/TPLs that depends on a few upstream ``MockTrilinos`` |
| 3045 |
packages. |
| 3046 |
|
| 3047 |
New test extra repositories are added when new types of tests are needed that |
| 3048 |
would require new package and TPL dependency structures since existing |
| 3049 |
dependency tests based on ``MockTrilinos`` are expensive to change by their |
| 3050 |
very nature. |
| 3051 |
|
| 3052 |
The primary reason that the ``MockTrilinos`` test project is mentioned in this |
| 3053 |
developers guide is because it contains a variety of packages, subpackages, |
| 3054 |
and TPLs with a variety of different types of dependencies. This variety is |
| 3055 |
is needed to more fully test the TriBITS system but this project and the tests |
| 3056 |
also serve as examples and extra documentation for the behavior of the TriBITS |
| 3057 |
system. Several of the dependency-related examples referenced in this |
| 3058 |
document come from ``MockTrilinos``. |
| 3059 |
|
| 3060 |
Most of the dependency tests involving ``MockTrilinos`` are specified in:: |
| 3061 |
|
| 3062 |
TriBITS/test/core/DependencyUnitTests/CMakeLists.txt |
| 3063 |
|
| 3064 |
A great deal about the current behavior of TriBITS `Package Dependencies and |
| 3065 |
Enable/Disable Logic`_ can be learned from inspecting these tests. There are |
| 3066 |
also some faster-running unit tests involving ``MockTrilinos`` defined in the |
| 3067 |
file:: |
| 3068 |
|
| 3069 |
TriBITS/test/core/TribitsAdjustPackageEnables_UnitTests.cmake |
| 3070 |
|
| 3071 |
|
| 3072 |
ReducedMockTrilinos |
| 3073 |
------------------- |
| 3074 |
|
| 3075 |
The TriBITS project ``ReducedMockTrilinos`` is contained under the directory:: |
| 3076 |
|
| 3077 |
tribits/examples/ReducedMockTrilinos/ |
| 3078 |
|
| 3079 |
It is a scaled-down version of the `MockTrilinos`_ test project with just a |
| 3080 |
handful of packages and some modified dependencies. Its primary purpose for |
| 3081 |
this example project is to be used for examples in the section `Package |
| 3082 |
Dependencies and Enable/Disable Logic`_ and to test a few features of the |
| 3083 |
TriBITS system not covered in other tests. |
| 3084 |
|
| 3085 |
The list of packages in:: |
| 3086 |
|
| 3087 |
tribits/examples/ReducedMockTrilinos/PackagesList.cmake |
| 3088 |
|
| 3089 |
is: |
| 3090 |
|
| 3091 |
.. include:: ../../examples/ReducedMockTrilinos/PackagesList.cmake |
| 3092 |
:literal: |
| 3093 |
|
| 3094 |
All of the listed packages are standard TriBITS packages except for the mock |
| 3095 |
``Thyra`` package which is broken down into subpackages. More details of this |
| 3096 |
example project are described in `Package Dependencies and Enable/Disable |
| 3097 |
Logic`_. |
| 3098 |
|
| 3099 |
Trilinos |
| 3100 |
-------- |
| 3101 |
|
| 3102 |
The real Trilinos project and repository itself is an advanced example for the |
| 3103 |
usage of TriBITS. Almost every single-repository use case for TriBITS is |
| 3104 |
demonstrated somewhere in Trilinos. While some of the usage of TriBITS in |
| 3105 |
Trilinos may not be not exemplary (e.g., because it represents old usage, or |
| 3106 |
was written by CMake/TriBITS beginners) it does represent real working usage. |
| 3107 |
Given that Trilinos is a widely available software repository, anyone should be |
| 3108 |
able to access a newer version of Trilinos and mine it for CMake and TriBITS |
| 3109 |
examples. |
| 3110 |
|
| 3111 |
|
| 3112 |
The TriBITS Test Package |
| 3113 |
------------------------ |
| 3114 |
|
| 3115 |
The last TriBITS example mentioned here is the TriBITS test package named |
| 3116 |
(appropriately) ``TriBITS`` itself defined in the ``TriBITS`` repository. The |
| 3117 |
directory for the ``TriBITS`` test package is the base TriBITS source |
| 3118 |
directory ``tribits``. This allows any TriBITS project to add testing for the |
| 3119 |
TriBITS system by just listing the TriBITS repository in its |
| 3120 |
`<projectDir>/cmake/ExtraRepositoriesList.cmake`_ file. Trilinos lists the |
| 3121 |
TriBITS repository in its ``ExtraRepositoriesList.cmake`` file as: |
| 3122 |
|
| 3123 |
tribits_project_define_extra_repositories( |
| 3124 |
TriBITS "" GIT https://github.com/TriBITSPub/TriBITS "" Continuous |
| 3125 |
... |
| 3126 |
) |
| 3127 |
|
| 3128 |
No `downstream`_ TriBITS packages list a dependency on ``TriBITS`` in their |
| 3129 |
`<packageDir>/cmake/Dependencies.cmake`_ files. Defining the ``TriBITS`` |
| 3130 |
TriBITS package in only done for running the TriBITS tests. |
| 3131 |
|
| 3132 |
Once the ``TriBITS`` test package is added to the list of project/repository |
| 3133 |
packages, it can be enabled just like any other package by adding the |
| 3134 |
following to the ``cmake`` command-line options:: |
| 3135 |
|
| 3136 |
-D <Project>_ENABLE_TriBITS=ON \ |
| 3137 |
-D <Project>_ENABLE_TESTS=ON |
| 3138 |
|
| 3139 |
One can then inspect the added tests prefixed by ``"TriBITS_"`` to see what |
| 3140 |
tests are defined and how they are run. There is a wealth of information |
| 3141 |
about the TriBITS system embedded in these tests and where documentation and |
| 3142 |
these tests disagreed, believe the tests! |
| 3143 |
|
| 3144 |
|
| 3145 |
Package Dependencies and Enable/Disable Logic |
| 3146 |
============================================= |
| 3147 |
|
| 3148 |
Arguably, the more important feature/aspect of the TriBITS system is the |
| 3149 |
partitioning of a large software project into packages and managing the |
| 3150 |
dependencies between these packages to support building, testing, and |
| 3151 |
deploying different pieces as needed (see discussion of `Software Engineering |
| 3152 |
Packaging Principles`_). This is especially useful in incremental CI testing |
| 3153 |
of large projects. However, maintaining such dependencies is also a critical |
| 3154 |
component in creating and maintaining Self-Sustaining Software (see the |
| 3155 |
`TriBITS Lifecycle Model`_). The fundamental mechanism for breaking up a |
| 3156 |
large software into manageable pieces is to partition the software into |
| 3157 |
different `TriBITS Packages`_ and then define the dependencies between these |
| 3158 |
packages (which are defined inside of the |
| 3159 |
`<packageDir>/cmake/Dependencies.cmake`_ files for each package). |
| 3160 |
|
| 3161 |
Note that the basic idea of breaking up a large set of software into pieces, |
| 3162 |
defining dependencies between the pieces, and then applying algorithms to |
| 3163 |
manipulate the dependency data-structures is nothing new. If fact, nearly |
| 3164 |
every binary package deployment system provided in various Linux OS |
| 3165 |
distributions have the concept of packages and dependencies and will |
| 3166 |
automatically install all of the necessary `upstream dependencies`_ when a |
| 3167 |
`downstream dependency`_ install is requested. The main difference (and the |
| 3168 |
added complexity) with TriBITS is that it can handle both required and |
| 3169 |
optional dependencies since it can build from source. A binary package |
| 3170 |
installation system, however, typically can't support optional dependencies |
| 3171 |
because only pre-built binary libraries and tools are available to install. |
| 3172 |
|
| 3173 |
This section is organized and broken-down as follows. First, the subsection |
| 3174 |
`Example ReducedMockTrilinos Project Dependency Structure`_ presents the |
| 3175 |
`ReducedMockTrilinos`_ example project, describes its dependency structure, |
| 3176 |
and uses it to begin to describe how TriBITS sets up and manages package |
| 3177 |
dependencies. The packages in this `ReducedMockTrilinos`_ example project are |
| 3178 |
used in the following subsections so one will be constantly referred back to |
| 3179 |
this subsection. The following subsection `TriBITS Dependency Handling |
| 3180 |
Behaviors`_ defines and describes the nitty-gritty details of the TriBITS |
| 3181 |
package dependency structure and the algorithms that manipulate the various |
| 3182 |
package and test enables and disables. Specific examples for the TriBITS |
| 3183 |
dependency handling algorithms are given in the subsection `Example |
| 3184 |
Enable/Disable Use Cases`_. Finally, the subsection |
| 3185 |
`<Project>PackageDependencies.xml`_ describes the standard XML output |
| 3186 |
data-structure that gets created by TriBITS that defines a project's package |
| 3187 |
dependencies. |
| 3188 |
|
| 3189 |
|
| 3190 |
Example ReducedMockTrilinos Project Dependency Structure |
| 3191 |
-------------------------------------------------------- |
| 3192 |
|
| 3193 |
To demonstrate the TriBITS package dependency handling system, the small |
| 3194 |
simple `ReducedMockTrilinos`_ project is used. The list of packages for this |
| 3195 |
project is defined in the file ``ReducedMockTrilinos/PackagesList.cmake`` (see |
| 3196 |
`<repoDir>/PackagesList.cmake`_) which contents: |
| 3197 |
|
| 3198 |
.. include:: ../../examples/ReducedMockTrilinos/PackagesList.cmake |
| 3199 |
:literal: |
| 3200 |
|
| 3201 |
All of the listed packages are standard TriBITS packages except for the mock |
| 3202 |
``Thyra`` package which is broken down into subpackages as shown in |
| 3203 |
``packages/thyra/cmake/Dependnecies.cmake`` (see |
| 3204 |
`<packageDir>/cmake/Dependencies.cmake`_) which is: |
| 3205 |
|
| 3206 |
.. include:: ../../examples/ReducedMockTrilinos/packages/thyra/cmake/Dependencies.cmake |
| 3207 |
:literal: |
| 3208 |
|
| 3209 |
This gives the full list of top-level TriBITS packages:: |
| 3210 |
|
| 3211 |
Teuchos RTOp Epetra Triutils EpetraExt Thyra |
| 3212 |
|
| 3213 |
Adding in the subpackages defined in the top-level ``Thyra`` package, the full |
| 3214 |
set of `TriBITS Packages`_ for this project is:: |
| 3215 |
|
| 3216 |
Teuchos RTOp Epetra Triutils EpetraExt ThyraCoreLibs ThyraGoodStuff \ |
| 3217 |
ThyraCrazyStuff ThyraEpetra ThyraEpetraExt Thyra |
| 3218 |
|
| 3219 |
Note that one can see this full list of top-level packages and packages in |
| 3220 |
the ``cmake`` configure output lines starting with:: |
| 3221 |
|
| 3222 |
Final set of non-enabled top-level packages: |
| 3223 |
Final set of non-enabled packages: |
| 3224 |
|
| 3225 |
respectively, when configuring with no package enables as shown in the example |
| 3226 |
`Default configure with no packages enabled on input`_. |
| 3227 |
|
| 3228 |
The list of `TriBITS External Packages/TPLs`_ for this example project given |
| 3229 |
in the file ``ReducedMockTrilinos/TPLsList.cmake`` (see |
| 3230 |
`<repoDir>/TPLsList.cmake`_) which is: |
| 3231 |
|
| 3232 |
.. include:: ../../examples/ReducedMockTrilinos/TPLsList.cmake |
| 3233 |
:literal: |
| 3234 |
|
| 3235 |
Take note of the `Package Test Group`_ (i.e. `PT`_, `ST`_, or `EX`_) assigned |
| 3236 |
to each package as it plays a significant role in how the TriBITS dependency |
| 3237 |
system handles enables and disables. |
| 3238 |
|
| 3239 |
The dependency structure of this simple TriBITS project is shown below in |
| 3240 |
`ReducedMockTrilinos Dependencies`_. |
| 3241 |
|
| 3242 |
.. _ReducedMockTrilinos Dependencies: |
| 3243 |
|
| 3244 |
**ReducedMockTrilinos Dependencies:** |
| 3245 |
|
| 3246 |
.. include:: ReducedMockTrilinosOutput/ExpectedDependencies.txt |
| 3247 |
:literal: |
| 3248 |
|
| 3249 |
The above dependency structure printout is produced by configuring with |
| 3250 |
``${PROJECT_NAME}_DUMP_PACKAGE_DEPENDENCIES=ON`` (which also results in more |
| 3251 |
dependency information than what is shown above, e.g. like computed forward |
| 3252 |
package dependencies). Note that the top-level package ``Thyra`` is shown |
| 3253 |
to depend on its subpackages (not the other way around). (Many people are |
| 3254 |
confused about the nature of the dependencies between packages and |
| 3255 |
subpackages. See `<packageDir>/<spkgDir>/cmake/Dependencies.cmake`_ for more |
| 3256 |
discussion.) |
| 3257 |
|
| 3258 |
.. ToDo: Show diagram with this dependency structure. |
| 3259 |
|
| 3260 |
A number of user-settable CMake cache variables determine what packages and |
| 3261 |
what tests and examples get enabled. These cache variables are described in |
| 3262 |
`Selecting the list of packages to enable`_ and are described below. Also, |
| 3263 |
the assigned `Package Test Group`_ (i.e. `PT`_, `ST`_, and `EX`_) also affects |
| 3264 |
what packages get enabled or disabled. |
| 3265 |
|
| 3266 |
Any of these packages can be enabled or disabled with |
| 3267 |
``${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>=(ON|OFF)`` (the default enable is |
| 3268 |
typically empty ``""``, see `PT/ST packages given default unset |
| 3269 |
enable/disable state`_). For ``ReducedMockTrilinos``, this gives the |
| 3270 |
enable/disable cache variables (with the initial default values):: |
| 3271 |
|
| 3272 |
Trilinos_ENABLE_Teuchos="" |
| 3273 |
Trilinos_ENABLE_RTOp"" |
| 3274 |
Trilinos_ENABLE_Epetra="" |
| 3275 |
Trilinos_ENABLE_Triutils="" |
| 3276 |
Trilinos_ENABLE_EpetraExt="" |
| 3277 |
Trilinos_ENABLE_ThyraCore="" |
| 3278 |
Trilinos_ENABLE_ThyraGoodStuff="" |
| 3279 |
Trilinos_ENABLE_ThyraCrazyStuff="OFF" # Because it is 'EX' |
| 3280 |
Trilinos_ENABLE_ThyraEpetra="" |
| 3281 |
Trilinos_ENABLE_ThyraEpetraExt="" |
| 3282 |
|
| 3283 |
Every TriBITS package is assumed to have tests and/or examples so TriBITS |
| 3284 |
defines the following cache variables as well (with the initial default |
| 3285 |
values):: |
| 3286 |
|
| 3287 |
Teuchos_ENABLE_TESTS="" |
| 3288 |
RTOp_ENABLE_TESTS="" |
| 3289 |
Epetra_ENABLE_TESTS="" |
| 3290 |
Triutils_ENABLE_TESTS="" |
| 3291 |
EpetraExt_ENABLE_TESTS="" |
| 3292 |
ThyraCoreLibs_ENABLE_TESTS="" |
| 3293 |
ThyraGoodStuff_ENABLE_TESTS="" |
| 3294 |
ThyraEpetra_ENABLE_TESTS="" |
| 3295 |
ThyraEpetraExt_ENABLE_TESTS="" |
| 3296 |
Thyra_ENABLE_TESTS="" |
| 3297 |
|
| 3298 |
NOTE: TriBITS only sets the variables ``<TRIBITS_PACKAGE>_ENABLE_TESTS`` into |
| 3299 |
the cache if the package ``<TRIBITS_PACKAGE>`` becomes enabled at some |
| 3300 |
point. This cuts down the clutter in the CMake cache for large projects with |
| 3301 |
lots of packages where the user only enables a subset of the packages. |
| 3302 |
|
| 3303 |
NOTE: TriBITS also defines the cache variables |
| 3304 |
``<TRIBITS_PACKAGE>_ENABLE_EXAMPLES`` for each enabled TriBITS package which |
| 3305 |
is handled the same way as the ``<TRIBITS_PACKAGE>_ENABLE_TEST`` variables. |
| 3306 |
|
| 3307 |
Also, every defined external package/TPL is given its own |
| 3308 |
``TPL_ENABLE_<TRIBITS_TPL>`` enable/disable cache variable. For the TPLs in |
| 3309 |
``ReducedMockTrilinos``, this gives the enable/disable cache variables (with |
| 3310 |
default values):: |
| 3311 |
|
| 3312 |
TPL_ENABLE_MPI="" |
| 3313 |
TPL_ENABLE_BLAS="" |
| 3314 |
TPL_ENABLE_LAPACK="" |
| 3315 |
TPL_ENABLE_Boost="" |
| 3316 |
TPL_ENABLE_UMFPACK="" |
| 3317 |
TPL_ENABLE_AMD="" |
| 3318 |
TPL_ENABLE_PETSC="" |
| 3319 |
|
| 3320 |
In addition, for every optional package dependency, TriBITS defines a cache |
| 3321 |
variable ``<TRIBITS_PACKAGE>_ENABLE_<OPTIONAL_DEP>``. For the optional |
| 3322 |
dependencies shown in `ReducedMockTrilinos Dependencies`_, that gives the |
| 3323 |
additional cache variables (with default values):: |
| 3324 |
|
| 3325 |
Teuchos_ENABLE_Boost="" |
| 3326 |
Teuchos_ENABLE_MPI="" |
| 3327 |
Teuchos_ENABLE_Boost="" |
| 3328 |
Epetra_ENABLE_MPI="" |
| 3329 |
EpetraExt_ENABLE_Triutils="" |
| 3330 |
EpetraExt_ENABLE_UMFPACK="" |
| 3331 |
EpetraExt_ENABLE_AMD="" |
| 3332 |
EpetraExt_ENABLE_PETSC="" |
| 3333 |
Thyra_ENABLE_ThyraGoodStuff="" |
| 3334 |
Thyra_ENABLE_ThyraCrazyStuff="" |
| 3335 |
Thyra_ENABLE_ThyraEpetra="" |
| 3336 |
Thyra_ENABLE_ThyraEpetraExt="" |
| 3337 |
|
| 3338 |
The above optional package-specific cache variables allow one to control |
| 3339 |
whether or not support for `upstream`_ dependency X is turned on in package Y |
| 3340 |
independent of whether or not X and Y are themselves both enabled. For |
| 3341 |
example, if the packages ``Triutils`` and ``EpetraExt`` are both enabled, one |
| 3342 |
can explicitly disable support for the optional dependency ``Triutils`` in |
| 3343 |
``EpetraExt`` by setting ``EpetraExt_ENABLE_Triutils=OFF``. One may want to |
| 3344 |
do this for several reasons but the bottom line is that this gives the user |
| 3345 |
more detailed control over package dependencies. See the `TriBITS Dependency |
| 3346 |
Handling Behaviors`_ and `Explicit disable of an optional package |
| 3347 |
dependency`_ for more discussion and examples. |
| 3348 |
|
| 3349 |
Before getting into specific `Example Enable/Disable Use Cases`_, some of the |
| 3350 |
`TriBITS Dependency Handling Behaviors`_ are first defined below. |
| 3351 |
|
| 3352 |
|
| 3353 |
TriBITS Dependency Handling Behaviors |
| 3354 |
------------------------------------- |
| 3355 |
|
| 3356 |
Below, some of the rules and behaviors of the TriBITS dependency management |
| 3357 |
system are described. Examples refer to the `Example ReducedMockTrilinos |
| 3358 |
Project Dependency Structure`_. More detailed examples of these behaviors are |
| 3359 |
given in the section `Example Enable/Disable Use Cases`_. |
| 3360 |
|
| 3361 |
In brief, the rules/behaviors of the TriBITS package dependency management |
| 3362 |
system are: |
| 3363 |
|
| 3364 |
0) `No circular dependencies of any kind are allowed`_ |
| 3365 |
1) `PT/ST packages given default unset enable/disable state`_ |
| 3366 |
2) `EX packages disabled by default`_ |
| 3367 |
3) `Package enable triggers auto-enables of upstream dependencies`_ |
| 3368 |
4) `Package disable triggers auto-disables of downstream dependencies`_ |
| 3369 |
5) `PT/ST TPLs given default unset enable/disable state`_ |
| 3370 |
6) `EX TPLs given default unset enable/disable state`_ |
| 3371 |
7) `Required TPLs are auto-enabled for enabled packages`_ |
| 3372 |
8) `Optional TPLs only enabled explicitly by the user`_ |
| 3373 |
9) `Disables trump enables where there is a conflict`_ |
| 3374 |
10) `Enable/disable of parent package is enable/disable for subpackages`_ |
| 3375 |
11) `Enable/disable of parent package tests/examples is enable/disable for subpackages tests/examples`_ |
| 3376 |
12) `Subpackage enable does not auto-enable the parent package`_ |
| 3377 |
13) `Support for optional package/TPL is enabled by default`_ |
| 3378 |
14) `Support for optional package can be explicitly disabled`_ |
| 3379 |
15) `Explicit enable of optional package/TPL support auto-enables package/TPL`_ |
| 3380 |
16) `ST packages only auto-enabled if ST code is enabled`_ |
| 3381 |
17) `<Project>_ENABLE_ALL_FORWARD_DEP_PACKAGES enables downstream packages/tests`_ |
| 3382 |
18) `<Project>_ENABLE_ALL_PACKAGES enables all PT (cond. ST) packages`_ |
| 3383 |
19) `<Project>_ENABLE_TESTS only enables explicitly enabled package tests`_ |
| 3384 |
20) `If no packages are enabled, nothing will get built`_ |
| 3385 |
21) `TriBITS prints all enables and disables to stdout`_ |
| 3386 |
22) `TriBITS auto-enables/disables done using non-cache local variables`_ |
| 3387 |
|
| 3388 |
In more detail, these rules/behaviors are: |
| 3389 |
|
| 3390 |
.. _No circular dependencies of any kind are allowed: |
| 3391 |
|
| 3392 |
0) **No circular dependencies of any kind are allowed**: The zeroth rule of |
| 3393 |
the TriBITS dependency management system is that no TriBITS package (or |
| 3394 |
its tests) can declare a dependency on a `downstream`_ package, period! |
| 3395 |
This is one of the most basic `Software Engineering Packaging Principles`_ |
| 3396 |
stated as the *ADP (Acyclic Dependencies Principle)*, i.e. "Allow no cycles |
| 3397 |
in the package dependency graph". By default, the TriBITS system will |
| 3398 |
check for circular dependencies and will fail the configure if any are |
| 3399 |
found. For a more detailed discussion of circular package dependencies, see |
| 3400 |
`Design Considerations for TriBITS`_. |
| 3401 |
|
| 3402 |
.. _PT/ST packages given default unset enable/disable state: |
| 3403 |
|
| 3404 |
1) **PT/ST packages given default unset enable/disable state**: A package |
| 3405 |
``<TRIBITS_PACKAGE>`` with testing group ``PT`` or ``ST`` is given an |
| 3406 |
**unset enable/disable state by default** |
| 3407 |
(i.e. ``${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>=""``). For example, the |
| 3408 |
``PT`` package ``Teuchos`` is not enabled or disabled by default and is |
| 3409 |
given the initial value ``Trilinos_ENABLE_Teuchos=""``. For an example, |
| 3410 |
see `Default configure with no packages enabled on input`_. This allows |
| 3411 |
``PT``, and ``ST`` packages to be enabled or disabled using other logic |
| 3412 |
defined by TriBITS which is described below. TriBITS defines persistent |
| 3413 |
cache variables for these with the default value of empty "". Therefore, |
| 3414 |
if the user or other CMake code does not hard enable or disable one of |
| 3415 |
these variables, then on future configures it will be defined but have the |
| 3416 |
value of empty "". |
| 3417 |
|
| 3418 |
.. _EX packages disabled by default: |
| 3419 |
|
| 3420 |
2) **EX packages disabled by default**: An package ``<TRIBITS_PACKAGE>`` |
| 3421 |
with testing group ``EX`` is **disabled by default** |
| 3422 |
(i.e. ``${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>=OFF``). For an example, |
| 3423 |
see `Default configure with no packages enabled on input`_. This results |
| 3424 |
in all required `downstream`_ packages to be disabled by default. |
| 3425 |
However, the user can explicitly set |
| 3426 |
``${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>=ON`` for an ``EX`` package and |
| 3427 |
it will be enabled (unless one of its required dependencies are not enabled |
| 3428 |
for some reason). In this case, the cache variable is given the cache |
| 3429 |
value of ``OFF``. |
| 3430 |
|
| 3431 |
.. _Package enable triggers auto-enables of upstream dependencies: |
| 3432 |
|
| 3433 |
3) **Package enable triggers auto-enables of upstream dependencies**: Any |
| 3434 |
package ``<TRIBITS_PACKAGE>`` can be explicitly **enabled** by the user |
| 3435 |
by setting the cache variable |
| 3436 |
``${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>=ON`` |
| 3437 |
(e.g. ``Trilinos_ENABLE_EpetraExt=ON``). When an package is enabled in |
| 3438 |
this way, the TriBITS system will try to enable all of the required |
| 3439 |
upstream packages and TPLs defined by the package (specified in its |
| 3440 |
``Dependencies.cmake`` file). If an enabled package can't be enabled |
| 3441 |
and has to be disabled, either a warning is printed or processing will stop |
| 3442 |
with an error (depending on the value of |
| 3443 |
``${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES``, see `Disables |
| 3444 |
trump enables where there is a conflict`_). In addition, if |
| 3445 |
``${PROJECT_NAME}_ENABLE_ALL_OPTIONAL_PACKAGES=ON``, then TriBITS will try |
| 3446 |
to enable all of the specified optional `PT`_ packages as well (and also |
| 3447 |
optional upstream `ST`_ packages if |
| 3448 |
``${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON``). For an example, see |
| 3449 |
`Explicit enable of a package, its tests, an optional TPL, with ST |
| 3450 |
enabled`_. |
| 3451 |
|
| 3452 |
.. _Package disable triggers auto-disables of downstream dependencies: |
| 3453 |
|
| 3454 |
4) **Package disable triggers auto-disables of downstream dependencies**: Any |
| 3455 |
package ``<TRIBITS_PACKAGE>`` can be explicitly **disabled** by the user by |
| 3456 |
setting the cache variable ``${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>=OFF`` |
| 3457 |
(or ``TPL_ENABLE_<TRIBITS_PACKAGE>=OFF`` for an external package/TPL) |
| 3458 |
(e.g. ``Trilinos_ENABLE_Teuchos=OFF``, ``TPL_ENABLE_BLAS=OFF``). When an |
| 3459 |
package is explicitly disabled, it will result in the disable of all |
| 3460 |
dependent `downstream`_ external packages/TPLs and internal packages that |
| 3461 |
have required dependency on it. It will also disable optional support for |
| 3462 |
the disabled packages in downstream packages that list it as an optional |
| 3463 |
dependency. For an example, see `Explicit disable of a package`_. |
| 3464 |
|
| 3465 |
.. _PT/ST TPLs given default unset enable/disable state: |
| 3466 |
|
| 3467 |
5) **PT/ST TPLs given default unset enable/disable state**: A TriBITS TPL |
| 3468 |
``<TRIBITS_TPL>`` with testing group ``PT`` or ``ST`` is given an **unset |
| 3469 |
enable/disable state by default** (i.e. ``TPL_ENABLE_<TRIBITS_TPL>=""``). |
| 3470 |
For example, the ``PT`` TPL ``BLAS`` is not enabled or disabled by default |
| 3471 |
(i.e. ``TPL_ENABLE_BLAS=""``). For an example, see `Default configure with |
| 3472 |
no packages enabled on input`_. This allows ``PT``, and ``ST`` TPLs to be |
| 3473 |
enabled or disabled using other logic. |
| 3474 |
|
| 3475 |
.. _EX TPLs given default unset enable/disable state: |
| 3476 |
|
| 3477 |
6) **EX TPLs given default unset enable/disable state**: A TriBITS TPL |
| 3478 |
``<TRIBITS_TPL>`` with testing group ``EX``, is given an **unset |
| 3479 |
enable/disable state by default** (i.e. ``TPL_ENABLE_<TRIBITS_TPL>=""``, |
| 3480 |
same as for ``PT`` and ``EX`` TPLs). For an example, see `Default |
| 3481 |
configure with no packages enabled on input`_. This is different behavior |
| 3482 |
than for ``EX`` packages described above which provides an initial hard |
| 3483 |
disable. However, since TriBITS will never automatically enable an |
| 3484 |
optional TPL (see `Optional TPLs only enabled explicitly by the user`_) and |
| 3485 |
since only `downstream`_ ``EX`` packages are allowed to have a required |
| 3486 |
dependencies on an ``EX`` TPL, there is no need to set the default enable |
| 3487 |
for an ``EX`` TPL to ``OFF``. If an ``EX`` package has a required |
| 3488 |
dependency on an ``EX`` TPL, just enabling the ``EX`` package should |
| 3489 |
automatically enable the ``EX`` TPL as described in `Required TPLs are |
| 3490 |
auto-enabled for enabled packages`_. |
| 3491 |
|
| 3492 |
.. _Required TPLs are auto-enabled for enabled packages: |
| 3493 |
|
| 3494 |
7) **Required TPLs are auto-enabled for enabled packages**: All TPLs listed |
| 3495 |
as required TPL dependencies for the final set of enabled packages are |
| 3496 |
**set to enabled** (i.e. ``TPL_ENABLE_<TRIBITS_TPL>=ON``), unless the |
| 3497 |
listed TPLs are already explicit disabled (in which case the package |
| 3498 |
would be disabled or an error would occur, see `Disables trump enables |
| 3499 |
where there is a conflict`_). For example, if the ``Teuchos`` package is |
| 3500 |
enabled, then that will trigger the enable of its required TPLs ``BLAS`` |
| 3501 |
and ``LAPACK``. For an example, see `Explicit enable of a package and its |
| 3502 |
tests`_. |
| 3503 |
|
| 3504 |
.. _Optional TPLs only enabled explicitly by the user: |
| 3505 |
|
| 3506 |
8) **Optional TPLs only enabled explicitly by the user**: Optional TPLs, |
| 3507 |
regardless of their testing group ``PT``, ``ST`` or ``EX``, will only be |
| 3508 |
enabled if they are explicitly enabled by the user. For example, just |
| 3509 |
because the package ``Teuchos`` is enabled, the optional TPLs ``Boost`` and |
| 3510 |
``MPI`` will **not** be enabled by default, even if |
| 3511 |
``${PROJECT_NAME}_ENABLE_ALL_OPTIONAL_PACKAGES=ON``. To enable the |
| 3512 |
optional TPL ``Boost``, for example, and enable support for ``Boost`` in |
| 3513 |
the ``Teuchos`` package, the user must explicitly set |
| 3514 |
``TPL_ENABLE_Boost=ON``. For an example, see `Explicit enable of a |
| 3515 |
package, its tests, an optional TPL, with ST enabled`_. |
| 3516 |
|
| 3517 |
.. _Disables trump enables where there is a conflict: |
| 3518 |
|
| 3519 |
9) **Disables trump enables where there is a conflict** and TriBITS will |
| 3520 |
never override a disable in order to satisfy some dependency. For |
| 3521 |
example, if the user sets ``Trilinos_ENABLE_Teuchos=OFF`` and |
| 3522 |
``Trilinos_ENABLE_RTOp=ON``, then TriBITS will **not** override the |
| 3523 |
disable of ``Teuchos`` in order to satisfy the required dependency of |
| 3524 |
``RTOp``. In cases such as this, the behavior of the TriBITS dependency |
| 3525 |
adjustment system will depend on the setting of the top-level user cache |
| 3526 |
variable `${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES`_: |
| 3527 |
|
| 3528 |
.. _${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=ON: |
| 3529 |
|
| 3530 |
* If ``${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=ON``, then |
| 3531 |
TriBITS will disable the explicit enable and continue on. In the above |
| 3532 |
example, TriBITS will override ``Trilinos_ENABLE_RTOp=ON`` and set |
| 3533 |
``Trilinos_ENABLE_RTOp=OFF`` and print a verbose warning to the |
| 3534 |
``cmake`` stdout. |
| 3535 |
|
| 3536 |
.. _${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=OFF: |
| 3537 |
|
| 3538 |
* If ``${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=OFF``, then |
| 3539 |
TriBITS will generate a detailed error message printed to ``cmake`` |
| 3540 |
stdout and then abort processing. In the above example, TriBITS will |
| 3541 |
report that ``RTOp`` is enabled but the required package ``Teuchos`` |
| 3542 |
is disabled and therefore ``RTOp`` can't be enabled and processing must |
| 3543 |
stop. |
| 3544 |
|
| 3545 |
For an example of both behaviors, see `Conflicting explicit enable and |
| 3546 |
disable`_. |
| 3547 |
|
| 3548 |
.. _Enable/disable of parent package is enable/disable for subpackages: |
| 3549 |
|
| 3550 |
10) **Enable/disable of parent package is enable/disable for subpackages**: An |
| 3551 |
explicit enable/disable of a top-level parent package with subpackages |
| 3552 |
with ``${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>=(ON|OFF)`` is equivalent |
| 3553 |
to the explicit enable/disable of all of the parent package's subpackages. |
| 3554 |
For example, explicitly setting ``Trilinos_ENABLE_Thyra=ON`` is equivalent |
| 3555 |
to explicitly setting:: |
| 3556 |
|
| 3557 |
Trilinos_ENABLE_ThyraCoreLibs=ON |
| 3558 |
Trilinos_ENABLE_ThyraGoodStuff=ON # Only if enabling ST code! |
| 3559 |
Trilinos_ENABLE_ThyraEpetra=ON |
| 3560 |
Trilinos_ENABLE_ThyraEpetraExt=ON # Only if enabling ST code! |
| 3561 |
|
| 3562 |
(Note that ``Trilinos_ENABLE_ThyraCrazyStuff`` is **not** set to ``ON`` |
| 3563 |
because it is already set to ``OFF`` by default, see `EX packages |
| 3564 |
disabled by default`_.) Likewise, explicitly setting |
| 3565 |
``Trilinos_ENABLE_Thyra=OFF`` is equivalent to explicitly setting all of |
| 3566 |
the ``Thyra`` subpackages to ``OFF`` at the outset. For a ``PT`` example, |
| 3567 |
see `Explicit enable of a package and its tests`_. For a ``ST`` example, |
| 3568 |
see `Explicit enable of a package, its tests, an optional TPL, with ST |
| 3569 |
enabled`_. |
| 3570 |
|
| 3571 |
.. _Enable/disable of parent package tests/examples is enable/disable for subpackages tests/examples: |
| 3572 |
|
| 3573 |
11) **Enable/disable of parent package tests/examples is enable/disable for |
| 3574 |
subpackages tests/examples**: Setting |
| 3575 |
``<TRIBITS_PACKAGE>_ENABLE_TESTS=[ON|OFF]`` is equivalent to setting the |
| 3576 |
default for ``<TRIBITS_PACKAGE><SP>_ENABLE_TESTS=[ON|OFF]`` for each |
| 3577 |
subpackage ``<SP>`` of the parent package ``<TRIBITS_PACKAGE>`` (if |
| 3578 |
``<TRIBITS_PACKAGE>`` has subpackages). Same is true for |
| 3579 |
``<TRIBITS_PACKAGE>_ENABLE_EXAMPLES=[ON|OFF]`` setting the default for |
| 3580 |
``<TRIBITS_PACKAGE><SP>_ENABLE_EXAMPLES=[ON|OFF]``. In addition, setting |
| 3581 |
``<TRIBITS_PACKAGE>_ENABLE_TESTS=[ON|OFF]`` will set |
| 3582 |
``<TRIBITS_PACKAGE>_ENABLE_EXAMPLES=[ON|OFF]`` by default as well (but not |
| 3583 |
vice versa). |
| 3584 |
|
| 3585 |
.. _Subpackage enable does not auto-enable the parent package: |
| 3586 |
|
| 3587 |
12) **Subpackage enable does not auto-enable the parent package**: Enabling an |
| 3588 |
package that is a subpackage does **not** automatically enable the |
| 3589 |
parent package (except for at the very end, mostly just for show). For |
| 3590 |
example, enabling the package ``ThyraEpetra`` does not result in enable |
| 3591 |
of the parent ``Thyra`` package, (except when |
| 3592 |
``${PROJECT_NAME}_ENABLE_ALL_FORWARD_DEP_PACKAGES=ON`` for course). For |
| 3593 |
an example, see `Explicit enable of a subpackage`_. This means that if a |
| 3594 |
`downstream`_ package declares a dependency on the package |
| 3595 |
``ThyraEpetra``, but not the parent package ``Thyra``, then the ``Thyra`` |
| 3596 |
package (and its other subpackages and their dependencies) will not get |
| 3597 |
auto-enabled. Also note that enabling a subset of the required |
| 3598 |
subpackages for a parent package does not require the enable of the other |
| 3599 |
required subpackages. For example, if both ``ThyraEpetra`` and |
| 3600 |
``ThyraCore`` were both required subpackages of the parent package |
| 3601 |
``Thyra``, then enabling ``ThyraCore`` does not require the enabling of |
| 3602 |
``ThyraEpetra``. In these cases where one of a parent package's |
| 3603 |
subpackages is enabled for some reason, the final value of |
| 3604 |
``${PROJECT_NAME}_ENABLE_${PACKAGE_NAME}`` will be set to ``ON`` (but this |
| 3605 |
does imply that all of the required subpackages will be enabled, only that |
| 3606 |
the parent package will be processed.) |
| 3607 |
|
| 3608 |
.. _Support for optional package/TPL is enabled by default: |
| 3609 |
|
| 3610 |
13) **Support for optional package/TPL is enabled by default**: For an package |
| 3611 |
``<TRIBITS_PACKAGE>`` with an optional dependency on an `upstream`_ |
| 3612 |
package or TPL ``<TRIBITS_DEP_PACKAGE_OR_TPL>``, TriBITS will |
| 3613 |
automatically set the intra-enable variable |
| 3614 |
``<TRIBITS_PACKAGE>_ENABLE_<TRIBITS_DEP_PACKAGE_OR_TPL>=ON`` if |
| 3615 |
``<TRIBITS_PACKAGE>`` and ``<TRIBITS_DEP_PACKAGE_OR_TPL>`` are **both |
| 3616 |
enabled**. For example, if the packages ``Triutils`` and ``EpetraExt`` |
| 3617 |
are both enabled, then TriBITS will automatically set |
| 3618 |
``EpetraExt_ENABLE_Triutils=ON`` by default and enables support for |
| 3619 |
``Triutils`` in ``EpetraExt``. Likewise, if ``Teuchos`` and the optional |
| 3620 |
TPL ``Boost`` are both enabled, then TriBITS will automatically set |
| 3621 |
``Teuchos_ENABLE_Boost=ON`` by default. This is obviously the logical |
| 3622 |
behavior. See the full context for these examples in `Explicit enable of |
| 3623 |
a package, its tests, an optional TPL, with ST enabled`_. |
| 3624 |
|
| 3625 |
.. _Support for optional package can be explicitly disabled: |
| 3626 |
|
| 3627 |
14) **Support for optional package can be explicitly disabled:** Even |
| 3628 |
though TriBITS will automatically set |
| 3629 |
``<TRIBITS_PACKAGE>_ENABLE_<TRIBITS_DEP_PACKAGE_OR_TPL>=ON`` by default if |
| 3630 |
``<TRIBITS_PACKAGE>`` and ``<TRIBITS_DEP_PACKAGE_OR_TPL>`` are both |
| 3631 |
enabled at the project level (as described above), the user can explicitly |
| 3632 |
set ``<TRIBITS_PACKAGE>_ENABLE_<TRIBITS_DEP_PACKAGE_OR_TPL>=OFF`` which |
| 3633 |
will turn off optional package-level support for the upstream package |
| 3634 |
or TPL ``<TRIBITS_DEP_PACKAGE_OR_TPL>`` in the downstream package |
| 3635 |
``<TRIBITS_PACKAGE>``. For example, the user can enable ``EpetraExt`` and |
| 3636 |
``Triutils`` at the project level, but set |
| 3637 |
``EpetraExt_ENABLE_Triutils=OFF`` which will turn off package-level |
| 3638 |
support for ``Triutils`` in the ``EpetraExt`` package. Likewise, the user |
| 3639 |
can enable ``Teuchos``, ``Epetra`` and the optional TPL ``Boost``, but set |
| 3640 |
``Epetra_ENABLE_Boost=ON``. This would provide support for ``Boost`` in |
| 3641 |
``Teuchos`` but not in ``Epetra``. For examples, see `Explicit disable of |
| 3642 |
an optional package dependency`_ and `Explicit disable of an optional TPL |
| 3643 |
dependency`_. |
| 3644 |
|
| 3645 |
.. _Explicit enable of optional package/TPL support auto-enables package/TPL: |
| 3646 |
|
| 3647 |
15) **Explicit enable of optional package/TPL support auto-enables |
| 3648 |
package/TPL**: If the user explicitly enables the TriBITS package |
| 3649 |
``<TRIBITS_PACKAGE>`` and explicitly sets |
| 3650 |
``<TRIBITS_PACKAGE>_ENABLE_<TRIBITS_DEP_PACKAGE_OR_TPL>=ON`` on input, |
| 3651 |
then that will automatically enable the package or TPL |
| 3652 |
``<TRIBITS_DEP_PACKAGE_OR_TPL>`` (and all of its upstream dependencies |
| 3653 |
accordingly). For example, if the user sets |
| 3654 |
``Trilinos_ENABLE_EpetraExt=ON`` and ``EpetraExt_ENABLE_Triutils=ON``, |
| 3655 |
then that will result in the auto-enable of |
| 3656 |
``Trilinos_ENABLE_Triutils=ON`` regardless of the value of |
| 3657 |
``${PROJECT_NAME}_ENABLE_ALL_OPTIONAL_PACKAGES`` or |
| 3658 |
``${PROJECT_NAME}_SECONDARY_TESTED_CODE``. This true even if the optional |
| 3659 |
package or TPL is ``EX``. For example, setting |
| 3660 |
``Thyra_ENABLE_ThyraCrazyStuff=ON`` will result in the enabling of the |
| 3661 |
``EX`` package ``ThyraCrazyStuff``. However, always remember that |
| 3662 |
`Disables trump enables where there is a conflict`_. For example, see |
| 3663 |
`Explicit enable of an optional package dependency`_. |
| 3664 |
|
| 3665 |
.. _ST packages only auto-enabled if ST code is enabled: |
| 3666 |
|
| 3667 |
16) **ST packages only auto-enabled if ST code is enabled**: TriBITS will |
| 3668 |
only enable an optional ``ST`` package when |
| 3669 |
``${PROJECT_NAME}_ENABLE_ALL_OPTIONAL_PACKAGES=ON`` if |
| 3670 |
``${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON`` is also set. Otherwise, when |
| 3671 |
an optional ``ST`` `upstream`_ dependent package is not enabled due to |
| 3672 |
``${PROJECT_NAME}_SECONDARY_TESTED_CODE=OFF``, then a one-line warning is |
| 3673 |
printed to stdout. For example, if the ``EpetraExt`` package is enabled |
| 3674 |
but ``ST`` code is not enabled, then the optional package ``Triutils`` |
| 3675 |
will not be enabled (for an example, see `Explicit enable of a package and |
| 3676 |
its tests`_). However, when ``${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON`` |
| 3677 |
and ``EpetraExt`` is enabled, then ``Triutils`` will be enabled too (for an |
| 3678 |
example, see `Explicit enable of a package, its tests, an optional TPL, |
| 3679 |
with ST enabled`_). The TriBITS default is |
| 3680 |
``${PROJECT_NAME}_ENABLE_ALL_OPTIONAL_PACKAGES=ON``. This helps to avoid |
| 3681 |
problems when users try to set a permutation of enables/disables which is |
| 3682 |
not regularly tested. |
| 3683 |
|
| 3684 |
.. _<Project>_ENABLE_ALL_FORWARD_DEP_PACKAGES enables downstream packages/tests: |
| 3685 |
|
| 3686 |
17) **<Project>_ENABLE_ALL_FORWARD_DEP_PACKAGES enables downstream |
| 3687 |
packages/tests**: Setting the user cache-variable |
| 3688 |
``${PROJECT_NAME}_ENABLE_ALL_FORWARD_PACKAGES=ON`` will result in the |
| 3689 |
`downstream`_ ``PT`` internal packages and tests to be enabled (and all |
| 3690 |
``PT`` and ``ST`` packages and tests when |
| 3691 |
``${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON``) for all explicitly enabled |
| 3692 |
internal packages. For example, in the mock Trilinos project, configuring |
| 3693 |
with ``Trilinos_ENABLE_Epetra=ON``, ``Trilinos_ENABLE_TESTS=ON``, and |
| 3694 |
``Trilinos_ENABLE_ALL_FORWARD_PACKAGES=ON`` will result the package |
| 3695 |
enables (and test and example enables) for the downstream packages |
| 3696 |
``Triutils``, ``EpetraExt``, ``ThyraCoreLibs``, ``ThyraEpetra`` and |
| 3697 |
``Thyra``. For an example, see `Explicit enable of a package and |
| 3698 |
downstream packages and tests`_. Note that when setting this option, the |
| 3699 |
enable of an external package/TPL will **not** result in the auto-enable |
| 3700 |
of downstream internal packages. For example, setting |
| 3701 |
``Trilinos_ENABLE_BLAS=ON`` will not result in the auto-enable of any |
| 3702 |
internal packages that depend on ``BLAS`` like ``Teuchos`` (in the mock |
| 3703 |
Trilinos project). |
| 3704 |
|
| 3705 |
.. _${PROJECT_NAME}_ENABLE_ALL_PACKAGES: |
| 3706 |
|
| 3707 |
.. _<Project>_ENABLE_ALL_PACKAGES enables all PT (cond. ST) packages: |
| 3708 |
|
| 3709 |
18) **<Project>_ENABLE_ALL_PACKAGES enables all PT (cond. ST) packages**: |
| 3710 |
Setting the user cache-variable ``${PROJECT_NAME}_ENABLE_ALL_PACKAGES=ON`` |
| 3711 |
will result in the enable of all ``PT`` packages when |
| 3712 |
``${PROJECT_NAME}_SECONDARY_TESTED_CODE=OFF`` and all ``PT`` and ``ST`` |
| 3713 |
packages when ``${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON``. For an |
| 3714 |
example, see `Enable all packages`_. When the project is a meta-project, |
| 3715 |
this will only enable the project's primary meta-project packages (PMPP). |
| 3716 |
That is, a package will only be enabled due to |
| 3717 |
``${PROJECT_NAME}_ENABLE_ALL_PACKAGES=ON`` when its parent repository does |
| 3718 |
not have `${REPOSITORY_NAME}_NO_PRIMARY_META_PROJECT_PACKAGES`_ set to |
| 3719 |
``TRUE``. However, if:: |
| 3720 |
|
| 3721 |
${REPOSITORY_NAME}_NO_PRIMARY_META_PROJECT_PACKAGES=TRUE |
| 3722 |
|
| 3723 |
then the package may be enabled if it (or its parent package) is listed |
| 3724 |
in `${REPOSITORY_NAME}_NO_PRIMARY_META_PROJECT_PACKAGES_EXCEPT`_. |
| 3725 |
|
| 3726 |
.. _<Project>_ENABLE_TESTS only enables explicitly enabled package tests: |
| 3727 |
|
| 3728 |
19) **<Project>_ENABLE_TESTS only enables explicitly enabled package |
| 3729 |
tests**: Setting ``${PROJECT_NAME}_ENABLE_TESTS=ON`` will **only enable |
| 3730 |
tests for explicitly enabled packages** on input. For example, |
| 3731 |
configuring with ``Trilinos_ENABLE_RTOp=ON`` and |
| 3732 |
``Trilinos_ENABLE_TESTS=ON`` will only result in the enable of tests for |
| 3733 |
``RTOp``, not ``Teuchos`` (even through TriBITS will enable ``Teuchos`` |
| 3734 |
because it is a required dependency of ``RTOp``). See an example, see |
| 3735 |
`Explicit enable of a package and its tests`_. When the project is a |
| 3736 |
meta-project, this will only enable tests for the project's primary |
| 3737 |
meta-project packages (PMPP). The uses the same logic as for |
| 3738 |
``${PROJECT_NAME}_ENABLE_ALL_PACKAGES``, see |
| 3739 |
`<Project>_ENABLE_ALL_PACKAGES enables all PT (cond. ST) packages`_. |
| 3740 |
|
| 3741 |
.. _If no packages are enabled, nothing will get built: |
| 3742 |
|
| 3743 |
20) **If no packages are enabled, nothing will get built**: Most TriBITS |
| 3744 |
projects are set up such that if the user does not explicitly enable at |
| 3745 |
least one package in some way, then nothing will be enabled or built. |
| 3746 |
In this case, when ``${PROJECT_NAME}_ALLOW_NO_PACKAGES=TRUE`` a warning |
| 3747 |
will be printed and configuration will complete. However, if |
| 3748 |
``${PROJECT_NAME}_ALLOW_NO_PACKAGES=FALSE``, then the configure will die |
| 3749 |
with an error message. For example, the ``checkin-test.py`` tool sets |
| 3750 |
``${PROJECT_NAME}_ALLOW_NO_PACKAGES=OFF`` to make sure that something gets |
| 3751 |
enabled and tested in order to accept the results of the test and allow a |
| 3752 |
push. For an example, see `Default configure with no packages enabled on |
| 3753 |
input`_. |
| 3754 |
|
| 3755 |
.. _TriBITS prints all enables and disables to stdout: |
| 3756 |
|
| 3757 |
21) **TriBITS prints all enables and disables to stdout**: TriBITS prints out |
| 3758 |
(to ``cmake`` stdout) the initial set of enables/disables on input, prints |
| 3759 |
a line whenever it sets (or overrides) an enable or disable, and prints |
| 3760 |
out the final set of enables/disables. Therefore, the user just needs to |
| 3761 |
grep the ``cmake`` stdout to find out why any particular package or TPL |
| 3762 |
is enabled or disabled in the end. In addition, will print out when |
| 3763 |
tests/examples for a given package gets enabled and when support for |
| 3764 |
optional packages and TPLs is enabled or not. Examples of this output |
| 3765 |
is given in all of the below examples but a detailed description of this |
| 3766 |
output is given in `Explicit enable of a package and its tests`_. |
| 3767 |
|
| 3768 |
.. _TriBITS auto-enables/disables done using non-cache local variables: |
| 3769 |
|
| 3770 |
22) **TriBITS auto-enables/disables done using non-cache local variables**: |
| 3771 |
TriBITS setting (or overrides) of enable/disable cache variables are done |
| 3772 |
by setting local non-cache variables at the top project-level scope |
| 3773 |
(i.e. the ``<projectDir>/CMakeLists.txt`` file scope) and does **not** |
| 3774 |
touch the value of the cache variables that may be set by the user on |
| 3775 |
input or the cache variables with documentation set by TriBITS. This is |
| 3776 |
done so they don't get set in the cache and so that the same dependency |
| 3777 |
enable/disable logic is redone, from scratch, with each re-configure. |
| 3778 |
This results in the same enable/disable logic output as for the initial |
| 3779 |
configure. This is to avoid confusion by the user about why some packages |
| 3780 |
and TPLs are enabled and some are not on subsequent reconfigures. This is |
| 3781 |
also desirable behavior as it preserves the user's input values for these |
| 3782 |
variables to document what was set by the user. However, this |
| 3783 |
implementation choice (and the tricky relationship between cache and |
| 3784 |
non-cache CMake variables) must be clearly understood when one wants to go |
| 3785 |
about tweaking these TriBITS enable/disable variables as described in `How |
| 3786 |
to check for and tweak TriBITS "ENABLE" cache variables`_ and `How to |
| 3787 |
tweak downstream TriBITS "ENABLE" variables during package |
| 3788 |
configuration`_. |
| 3789 |
|
| 3790 |
TriBITS prints out a lot of information about the enable/disable logic as it |
| 3791 |
applies the above rules/behaviors. For a large TriBITS project with lots of |
| 3792 |
packages, this can produce a lot of output to stdout. One just needs to |
| 3793 |
understand what TriBITS is printing out and where to look in the output for |
| 3794 |
different information. The examples in the section `Example Enable/Disable |
| 3795 |
Use Cases`_ show what this output looks like for the various enable/disable |
| 3796 |
scenarios and tries to explain in more detail the reasons for why the given |
| 3797 |
behavior is implemented the way that it is. Given this output, the rule |
| 3798 |
definitions given above, and the detailed `Example Enable/Disable Use Cases`_, |
| 3799 |
one should always be able to figure out exactly why the final set of |
| 3800 |
enables/disables is the way it is, even in the largest and most complex of |
| 3801 |
TriBITS projects. (NOTE: The same can *not* be said for many other large |
| 3802 |
software configuration and deployment systems where basic decisions about what |
| 3803 |
to enable and disable are hidden from the user and can be very difficult to |
| 3804 |
track down and debug.) |
| 3805 |
|
| 3806 |
The above behaviors address the majority of the functionality of the TriBITS |
| 3807 |
dependency management system. However, when dealing with TriBITS projects |
| 3808 |
with multiple repositories, some other behaviors are supported through the |
| 3809 |
definition of a few more variables. The following TriBITS repository-related |
| 3810 |
variables alter what packages in a given TriBITS repository get enabled |
| 3811 |
implicitly or not by TriBITS: |
| 3812 |
|
| 3813 |
``${REPOSITORY_NAME}_NO_IMPLICIT_PACKAGE_ENABLE`` |
| 3814 |
|
| 3815 |
If set to ``ON``, then the packages in Repository ``${REPOSITORY_NAME}`` |
| 3816 |
will not be implicitly enabled in any of the package adjustment logic. |
| 3817 |
|
| 3818 |
``${REPOSITORY_NAME}_NO_IMPLICIT_PACKAGE_ENABLE_EXCEPT`` |
| 3819 |
|
| 3820 |
List of packages in the Repository ``${REPOSITORY_NAME}`` that will be |
| 3821 |
allowed to be implicitly enabled. Only checked if |
| 3822 |
``${REPOSITORY_NAME}_NO_IMPLICIT_PACKAGE_ENABLE`` is true. |
| 3823 |
|
| 3824 |
The above variables typically are defined in the outer TriBITS Project's CTest |
| 3825 |
driver scripts or even in top-level project files in order to adjust how |
| 3826 |
packages in its listed repositories are handled. What these variable do is to |
| 3827 |
allow a large project to turn off the auto-enable of optional packages in a |
| 3828 |
given TriBITS repository to provide more detailed control of what gets used |
| 3829 |
from a given TriBITS repository. This, for example, is used in the CASL VERA |
| 3830 |
project to manage some of its extra repositories and packages to further reduce |
| 3831 |
the number of packages that get auto-enabled. |
| 3832 |
|
| 3833 |
.. ToDo: We should likely change this so that only tests and example don't get |
| 3834 |
.. enabled by default, not the packages itself. This is mainly used to avoid |
| 3835 |
.. certain packages from getting enabled in the CI server when updates are |
| 3836 |
.. received. This needs to be explained in the section on the CI server. |
| 3837 |
|
| 3838 |
|
| 3839 |
Example Enable/Disable Use Cases |
| 3840 |
--------------------------------- |
| 3841 |
|
| 3842 |
Below, a few of the standard enable/disable use cases for a TriBITS project |
| 3843 |
are given using the `Example ReducedMockTrilinos Project Dependency |
| 3844 |
Structure`_ that demonstrate the `TriBITS Dependency Handling Behaviors`_. |
| 3845 |
|
| 3846 |
The use cases covered are: |
| 3847 |
|
| 3848 |
* `Default configure with no packages enabled on input`_ |
| 3849 |
* `Explicit enable of a package and its tests`_ |
| 3850 |
* `Explicit enable of a package, its tests, an optional TPL, with ST enabled`_ |
| 3851 |
* `Explicit disable of a package`_ |
| 3852 |
* `Conflicting explicit enable and disable`_ |
| 3853 |
* `Explicit enable of an optional TPL`_ |
| 3854 |
* `Explicit disable of an optional TPL`_ |
| 3855 |
* `Explicit disable of a required TPL`_ |
| 3856 |
* `Explicit enable of a subpackage`_ |
| 3857 |
* `Explicit enable of an optional package dependency`_ |
| 3858 |
* `Explicit disable of an optional package dependency`_ |
| 3859 |
* `Explicit enable of an optional TPL dependency`_ |
| 3860 |
* `Explicit disable of an optional TPL dependency`_ |
| 3861 |
* `Explicit enable of a package and downstream packages and tests`_ |
| 3862 |
* `Enable all packages`_ |
| 3863 |
|
| 3864 |
All of these use cases and more can be easily run from the command-line by |
| 3865 |
first setting: |
| 3866 |
|
| 3867 |
$ export REDUCED_MOCK_TRILINOS=<base-dir>/tribits/examples/ReducedMockTrilinos |
| 3868 |
|
| 3869 |
and then copy and pasting the ``cmake`` commands shown below. Just make sure |
| 3870 |
to run these in a temp directory because this actually configures a CMake |
| 3871 |
project in the local directory. Just make sure and run:: |
| 3872 |
|
| 3873 |
$ rm -r CMake* |
| 3874 |
|
| 3875 |
before each run to clear the CMake cache. |
| 3876 |
|
| 3877 |
These use cases are now described in detail below. |
| 3878 |
|
| 3879 |
.. _Default configure with no packages enabled on input: |
| 3880 |
|
| 3881 |
**Default configure with no packages enabled on input** |
| 3882 |
|
| 3883 |
The first use-case to consider is the configure of a TriBITS project without |
| 3884 |
enabling any packages. For the ``ReducedMockTrilinos`` project, this is done |
| 3885 |
with:: |
| 3886 |
|
| 3887 |
$ cmake ${REDUCED_MOCK_TRILINOS} |
| 3888 |
|
| 3889 |
which produces the relevant dependency-related output: |
| 3890 |
|
| 3891 |
.. include:: ReducedMockTrilinosOutput/NoEnables.txt |
| 3892 |
:literal: |
| 3893 |
|
| 3894 |
The above example demonstrates the following behaviors of the TriBITS |
| 3895 |
dependency handling system: |
| 3896 |
|
| 3897 |
* `PT/ST packages given default unset enable/disable state`_ |
| 3898 |
* `EX packages disabled by default`_ (i.e., the ``EX`` package |
| 3899 |
``ThyraCrazyStuff`` is set to ``OFF`` by default at the very beginning). |
| 3900 |
* `PT/ST TPLs given default unset enable/disable state`_ |
| 3901 |
* `EX TPLs given default unset enable/disable state`_ |
| 3902 |
* `If no packages are enabled, nothing will get built`_ |
| 3903 |
|
| 3904 |
.. _Explicit enable of a package and its tests: |
| 3905 |
|
| 3906 |
**Explicit enable of a package and its tests** |
| 3907 |
|
| 3908 |
One of the most typical use cases is for the user to explicitly enable one or |
| 3909 |
more top-level TriBITS package and enable its tests. This configuration would |
| 3910 |
be used to drive local development on a specific set of packages (i.e. tests |
| 3911 |
do not need to be enabled for packages not being changed). |
| 3912 |
|
| 3913 |
Consider the configure of the ``ReducedMockTrilinos`` project enabling the |
| 3914 |
top-level ``Thyra`` package and its tests with:: |
| 3915 |
|
| 3916 |
$ cmake -DTrilinos_ENABLE_Thyra:BOOL=ON \ |
| 3917 |
-DTrilinos_ENABLE_TESTS:BOOL=ON \ |
| 3918 |
${REDUCED_MOCK_TRILINOS} |
| 3919 |
|
| 3920 |
which produces the relevant dependency-related output: |
| 3921 |
|
| 3922 |
.. include:: ReducedMockTrilinosOutput/EnableThyra_EnableTests.txt |
| 3923 |
:literal: |
| 3924 |
|
| 3925 |
This is a configuration that a developer would use to develop on the ``Thyra`` |
| 3926 |
package and its subpackages for example. There is no need to be enabling the |
| 3927 |
tests and examples for upstream packages unless those packages are going to be |
| 3928 |
changed as well. |
| 3929 |
|
| 3930 |
This case demonstrates a number of TriBITS dependency handling behaviors that |
| 3931 |
are worth some discussion. |
| 3932 |
|
| 3933 |
First, note that enabling the parent package ``Thyra`` with |
| 3934 |
``Trilinos_ENABLE_Thyra=ON`` right away results in the auto-enable of its |
| 3935 |
``PT`` subpackages ``ThyraCoreLibs`` and ``ThyraEpetra`` which demonstrates |
| 3936 |
the behavior `Enable/disable of parent package is enable/disable for |
| 3937 |
subpackages`_. Note that the ``ST`` subpackages ``ThyraGoodStuff`` and |
| 3938 |
``ThyraEpetraExt`` where *not* enabled because |
| 3939 |
``${PROJECT_NAME}_SECONDARY_TESTED_CODE=OFF`` (which is off by default) which |
| 3940 |
demonstrates the behavior `ST packages only auto-enabled if ST code is |
| 3941 |
enabled`_. |
| 3942 |
|
| 3943 |
Second, note the auto-enable of required upstream packages ``Epetra``, |
| 3944 |
``RTOp`` and ``Teuchos`` shown in lines like:: |
| 3945 |
|
| 3946 |
-- Setting Trilinos_ENABLE_Teuchos=ON because ThyraCoreLibs has a required dependence on Teuchos |
| 3947 |
|
| 3948 |
Lastly, note that the final set of enabled packages, packages, tests/examples |
| 3949 |
and external packages/TPLs can be clearly seen when processing the external |
| 3950 |
packages/TPLs and top-level packages in the lines:: |
| 3951 |
|
| 3952 |
Getting information for all enabled external packages/TPLs ... |
| 3953 |
|
| 3954 |
-- Processing enabled external package/TPL: BLAS |
| 3955 |
-- Processing enabled external package/TPL: LAPACK |
| 3956 |
|
| 3957 |
Configuring individual enabled Trilinos packages ... |
| 3958 |
|
| 3959 |
Processing enabled top-level package: Teuchos (Libs) |
| 3960 |
Processing enabled top-level package: RTOp (Libs) |
| 3961 |
Processing enabled top-level package: Epetra (Libs) |
| 3962 |
Processing enabled top-level package: Thyra (CoreLibs, Epetra, Tests, Examples) |
| 3963 |
|
| 3964 |
Note that subpackage enables are listed with their parent packages along with |
| 3965 |
if the tests and/or examples are enabled. Top-level packages that don't have |
| 3966 |
subpackages just show ``Libs`` and ``Tests`` and ``Examples`` if they have |
| 3967 |
been enabled as well. |
| 3968 |
|
| 3969 |
.. _Explicit enable of a package, its tests, an optional TPL, with ST enabled: |
| 3970 |
|
| 3971 |
**Explicit enable of a package, its tests, an optional TPL, with ST enabled** |
| 3972 |
|
| 3973 |
An extended use case shown here is for the explicit enable of a package and its |
| 3974 |
tests along with the enable of an optional TPL and with ``ST`` code enabled. |
| 3975 |
This is a configuration that would be used to support the local development of |
| 3976 |
a TriBITS package that involves modifying ``ST`` software. |
| 3977 |
|
| 3978 |
Consider the configure of the ``ReducedMockTrilinos`` project with:: |
| 3979 |
|
| 3980 |
$ cmake -DTPL_ENABLE_Boost:BOOL=ON \ |
| 3981 |
-DTrilinos_ENABLE_Thyra:BOOL=ON \ |
| 3982 |
-DTrilinos_ENABLE_TESTS:BOOL=ON \ |
| 3983 |
-DTrilinos_ENABLE_SECONDARY_TESTED_CODE:BOOL=ON \ |
| 3984 |
-DTrilinos_ENABLE_TESTS:BOOL=ON \ |
| 3985 |
${REDUCED_MOCK_TRILINOS} |
| 3986 |
|
| 3987 |
which produces the relevant dependency-related output: |
| 3988 |
|
| 3989 |
.. include:: ReducedMockTrilinosOutput/EnableThyra_EnableTests_EnableBoost_ST.txt |
| 3990 |
:literal: |
| 3991 |
|
| 3992 |
A few more behaviors of the TriBITS system that this particular configuration |
| 3993 |
use-case shows are described below. |
| 3994 |
|
| 3995 |
First, note the enable of the ``ST`` ``Thyra`` subpackages in lines like:: |
| 3996 |
|
| 3997 |
-- Setting subpackage enable Trilinos_ENABLE_ThyraGoodStuff=ON because parent package \ |
| 3998 |
Trilinos_ENABLE_Thyra=ON |
| 3999 |
|
| 4000 |
Second, note the auto-enable of support for optional packages in lines |
| 4001 |
like:: |
| 4002 |
|
| 4003 |
-- Setting EpetraExt_ENABLE_Triutils=ON since Trilinos_ENABLE_EpetraExt=ON \ |
| 4004 |
AND Trilinos_ENABLE_Triutils=ON |
| 4005 |
|
| 4006 |
Third, note the auto-enable of support for the optional TPL ``Boost`` in the |
| 4007 |
line:: |
| 4008 |
|
| 4009 |
-- Setting Teuchos_ENABLE_Boost=ON since TPL_ENABLE_Boost=ON |
| 4010 |
|
| 4011 |
.. _Explicit disable of a package: |
| 4012 |
|
| 4013 |
**Explicit disable of a package** |
| 4014 |
|
| 4015 |
Another common use case is to enable a package but to disable an optional |
| 4016 |
upstream package. This type of configuration would be used as part of a |
| 4017 |
"black list" approach to enabling only a subset of packages and optional |
| 4018 |
support. The "black list" approach is to enable a package with |
| 4019 |
``${PROJECT_NAME}_ENABLE_ALL_OPTIONAL_PACKAGES=ON`` (the TriBITS default) but |
| 4020 |
then to turn off a specific set of packages that you don't want. This is |
| 4021 |
contrasted with a "white list" approach where you would configure with |
| 4022 |
``${PROJECT_NAME}_ENABLE_ALL_OPTIONAL_PACKAGES=OFF`` and then have to manually |
| 4023 |
enable all of the optional packages you want. Experience with projects like |
| 4024 |
Trilinos show that the "black list" approach is generally to be preferred for |
| 4025 |
a few reasons. |
| 4026 |
|
| 4027 |
Consider the configure of the ``ReducedMockTrilinos`` project enabling ``Thyra`` but disabling ``Epetra`` with:: |
| 4028 |
|
| 4029 |
$ cmake -DTrilinos_ENABLE_Thyra:BOOL=ON \ |
| 4030 |
-DTrilinos_ENABLE_Epetra:BOOL=OFF \ |
| 4031 |
-DTrilinos_ENABLE_TESTS:BOOL=ON \ |
| 4032 |
${REDUCED_MOCK_TRILINOS} |
| 4033 |
|
| 4034 |
which produces the relevant dependency-related output: |
| 4035 |
|
| 4036 |
.. include:: ReducedMockTrilinosOutput/EnableThyra_DisableEpetra_EnableTests.txt |
| 4037 |
:literal: |
| 4038 |
|
| 4039 |
Note how the disable of ``Epetra`` wipes out all of the required and optional |
| 4040 |
packages and intra-package dependencies that depend on ``Epetra``. What is |
| 4041 |
left is only the ``ThyraCoreLibs`` and its upstream dependencies that don't |
| 4042 |
depend on ``Epetra`` (which is only ``RTOp`` and ``Teuchos``). |
| 4043 |
|
| 4044 |
.. _Conflicting explicit enable and disable: |
| 4045 |
|
| 4046 |
**Conflicting explicit enable and disable** |
| 4047 |
|
| 4048 |
One use case that occasionally comes up is when a set of inconsistent enables |
| 4049 |
and disables are set. While this seems illogical that anyone would ever do |
| 4050 |
this, when it comes to larger more complex projects with lots of packages and |
| 4051 |
lots of dependencies, this can happen very easily. In some cases, someone is |
| 4052 |
enabling a set of packages they want and is trying to weed out as many of the |
| 4053 |
(what they think) are optional dependencies that they don't need and |
| 4054 |
accidentally disables a package that is an indirect required dependency of one |
| 4055 |
of the packages they want (in which case the configure should likely fail and |
| 4056 |
provide a good error message). The other use case where conflicting |
| 4057 |
enables/disables can occur is in CTest drivers using `tribits_ctest_driver()`_ |
| 4058 |
where an upstream package has failed and is explicitly disabled (in which case |
| 4059 |
it should gracefully disable downstream dependent packages). TriBITS can |
| 4060 |
either be set up to have the disable override the explicit enable or stop the |
| 4061 |
configure in error depending on the value of the cache variable |
| 4062 |
`${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES`_ (see `Disables trump |
| 4063 |
enables where there is a conflict`_). |
| 4064 |
|
| 4065 |
For example, consider what happens with the ``ReducedMockTrilinos`` project if |
| 4066 |
someone tries to enable the ``RTOp`` package and disable the ``Teuchos`` |
| 4067 |
package. This is not consistent because ``RTOp`` has a required dependency on |
| 4068 |
``Teuchos``. The default behavior of TriBITS is this case is shown in the |
| 4069 |
below configure:: |
| 4070 |
|
| 4071 |
$ cmake -DTrilinos_ENABLE_Epetra:BOOL=ON \ |
| 4072 |
-DTrilinos_ENABLE_RTOp:BOOL=ON \ |
| 4073 |
-DTrilinos_ENABLE_Teuchos:BOOL=OFF \ |
| 4074 |
${REDUCED_MOCK_TRILINOS} |
| 4075 |
|
| 4076 |
which produces the relevant dependency-related output: |
| 4077 |
|
| 4078 |
.. include:: ReducedMockTrilinosOutput/EnableEpetra_EnableRTOp_DisableTeuchos.txt |
| 4079 |
:literal: |
| 4080 |
|
| 4081 |
As shown above, the TriBITS default (which is |
| 4082 |
``${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=OFF``) results in a |
| 4083 |
configure-time error with a good error message. |
| 4084 |
|
| 4085 |
However, if one sets |
| 4086 |
``${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=ON`` and configures |
| 4087 |
with:: |
| 4088 |
|
| 4089 |
$ cmake -DTrilinos_ENABLE_Epetra:BOOL=ON \ |
| 4090 |
-DTrilinos_ENABLE_RTOp:BOOL=ON \ |
| 4091 |
-DTrilinos_ENABLE_Teuchos:BOOL=OFF \ |
| 4092 |
-DTrilinos_DISABLE_ENABLED_FORWARD_DEP_PACKAGES:BOOL=ON \ |
| 4093 |
${REDUCED_MOCK_TRILINOS} |
| 4094 |
|
| 4095 |
then the disable trumps the enable and results in a successful configure as |
| 4096 |
shown in the following relevant dependency-related output: |
| 4097 |
|
| 4098 |
.. include:: ReducedMockTrilinosOutput/EnableEpetra_EnableRTOp_DisableTeuchos_DisableEnabledFwdDepPackages.txt |
| 4099 |
:literal: |
| 4100 |
|
| 4101 |
As shown above, what you end up with is just the enabled package ``Epetra`` |
| 4102 |
which does not have a required dependency on the disabled package ``Teuchos``. |
| 4103 |
Developers of large complex TriBITS projects would be wise to set the default |
| 4104 |
for `${PROJECT_NAME}_DISABLE_ENABLED_FORWARD_DEP_PACKAGES`_ to ``ON``, |
| 4105 |
especially in automated builds and testing. |
| 4106 |
|
| 4107 |
.. _Explicit enable of an optional TPL: |
| 4108 |
|
| 4109 |
**Explicit enable of an optional TPL**: |
| 4110 |
|
| 4111 |
ToDo: Set Trilinos_ENABLE_Thyra=ON and TPL_ENABLE_MPI=ON |
| 4112 |
|
| 4113 |
.. _Explicit disable of an optional TPL: |
| 4114 |
|
| 4115 |
**Explicit disable of an optional TPL**: |
| 4116 |
|
| 4117 |
ToDo: Set Trilinos_ENABLE_Thyra=ON and TPL_ENABLE_MPI=OFF |
| 4118 |
|
| 4119 |
.. _Explicit disable of a required TPL: |
| 4120 |
|
| 4121 |
**Explicit disable of a required TPL** |
| 4122 |
|
| 4123 |
ToDo: Set Trilinos_ENABLE_Epetra=ON and Trilinos_ENABLE_BLAS=OFF |
| 4124 |
|
| 4125 |
.. _Explicit enable of a subpackage: |
| 4126 |
|
| 4127 |
**Explicit enable of a subpackage** |
| 4128 |
|
| 4129 |
ToDo: Enable ThyraEpetra and show how it enables other packages and at the |
| 4130 |
end, enables the Thyra package (just for show). |
| 4131 |
|
| 4132 |
.. _Explicit enable of an optional package dependency: |
| 4133 |
|
| 4134 |
**Explicit enable of an optional package dependency** |
| 4135 |
|
| 4136 |
ToDo: Set Trilinos_ENABLE_EpetraExt=ON and EpetraExt_ENABLE_Triutils=ON and |
| 4137 |
shows how it enables Trilinos_ENABLE_Triutils=ON even through ST code is not |
| 4138 |
enabled. |
| 4139 |
|
| 4140 |
.. _Explicit disable of an optional package dependency: |
| 4141 |
|
| 4142 |
**Explicit disable of an optional package dependency** |
| 4143 |
|
| 4144 |
ToDo: Set Trilinos_ENABLE_EpetraExt=ON, Trilinos_ENABLE_Triutils=ON, and |
| 4145 |
EpetraExt_ENABLE_Triutils=OFF. Discuss how EpetraExt's and ThyraEpetraExt's |
| 4146 |
CMakeLists.txt files might turn off some features if they detects that |
| 4147 |
EpetraExt/Triutils support is turned off. |
| 4148 |
|
| 4149 |
.. _Explicit enable of an optional TPL dependency: |
| 4150 |
|
| 4151 |
**Explicit enable of an optional TPL dependency** |
| 4152 |
|
| 4153 |
ToDo: The current ReducedMockTrilinos is not set up to give a good example of |
| 4154 |
this. We should add an optional Boost dependency to say, Epetra. Then we |
| 4155 |
could show the enable of Teuchos and Epetra and Epetra_ENABLE_Boost=ON. That |
| 4156 |
would enable Boost and enable support for Boost in Epetra but would not |
| 4157 |
provide support for Boost in Teuchos. |
| 4158 |
|
| 4159 |
.. _Explicit disable of an optional TPL dependency: |
| 4160 |
|
| 4161 |
**Explicit disable of an optional TPL dependency** |
| 4162 |
|
| 4163 |
ToDo: The current ReducedMockTrilinos is not set up to give a good example of |
| 4164 |
this. We should add an optional Boost dependency to say, Epetra. Then we |
| 4165 |
could show the enable of Teuchos and Epetra and TPL_ENABLE_Boost=ON but set |
| 4166 |
Epetra_ENABLE_Boost=OFF. That would provide support for Boost in Teuchos but |
| 4167 |
not in Epetra. |
| 4168 |
|
| 4169 |
.. _Explicit enable of a package and downstream packages and tests: |
| 4170 |
|
| 4171 |
**Explicit enable of a package and downstream packages and tests** |
| 4172 |
|
| 4173 |
ToDo: Set Trilinos_ENABLE_RTOp=ON, |
| 4174 |
Trilinos_ENABLE_ALL_FORWARD_DEP_PACKAGES=ON, and Trilinos_ENABLE_TESTS=ON and |
| 4175 |
show what packages and tests/examples get enabled. This is the use case for |
| 4176 |
the checkin-test.py tool for PT enabled code. |
| 4177 |
|
| 4178 |
.. _Enable all packages: |
| 4179 |
|
| 4180 |
**Enable all packages** |
| 4181 |
|
| 4182 |
The last use case to consider is enabling all defined packages. This |
| 4183 |
configuration would be used for either doing a full test of all of the |
| 4184 |
packages defined or to create a distribution of the project. |
| 4185 |
|
| 4186 |
Enabling all ``PT`` packages the with:: |
| 4187 |
|
| 4188 |
$ cmake -DTrilinos_ENABLE_ALL_PACKAGES:BOOL=ON \ |
| 4189 |
-DTrilinos_DUMP_PACKAGE_DEPENDENCIES:BOOL=ON \ |
| 4190 |
${REDUCED_MOCK_TRILINOS} |
| 4191 |
|
| 4192 |
produces the relevant dependency-related output: |
| 4193 |
|
| 4194 |
.. include:: ReducedMockTrilinosOutput/EnableAllPackages.txt |
| 4195 |
:literal: |
| 4196 |
|
| 4197 |
As shown above, only the ``PT`` packages get enabled. To also enable the |
| 4198 |
``ST`` packages as well, one additionally set |
| 4199 |
``${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON`` at configure time. |
| 4200 |
|
| 4201 |
.. _<Project>PackageDependencies.xml: |
| 4202 |
|
| 4203 |
|
| 4204 |
TriBITS Project Dependencies XML file and tools |
| 4205 |
----------------------------------------------- |
| 4206 |
|
| 4207 |
The TriBITS CMake configure system can write out the project's package |
| 4208 |
dependencies into a file ``<Project>Dependencies.xml`` (or any name one wants |
| 4209 |
to give it). This file is used by a number of the TriBITS SE-related tools. |
| 4210 |
The structure of this XML file, showing one of the more interesting mock |
| 4211 |
packages from the `MockTrilinos`_ project is shown below:: |
| 4212 |
|
| 4213 |
<PackageDependencies project="Trilinos"> |
| 4214 |
... |
| 4215 |
<Package name="Amesos" dir="packages/amesos" type="PT"> |
| 4216 |
<LIB_REQUIRED_DEP_PACKAGES value="Teuchos,Epetra"/> |
| 4217 |
<LIB_OPTIONAL_DEP_PACKAGES value="EpetraExt"/> |
| 4218 |
<TEST_REQUIRED_DEP_PACKAGES/> |
| 4219 |
<TEST_OPTIONAL_DEP_PACKAGES value="Triutils,Galeri"/> |
| 4220 |
<LIB_REQUIRED_DEP_TPLS/> |
| 4221 |
<LIB_OPTIONAL_DEP_TPLS value="SuperLUDist,ParMETIS,UMFPACK,SuperLU,MUMPS"/> |
| 4222 |
<TEST_REQUIRED_DEP_TPLS/> |
| 4223 |
<TEST_OPTIONAL_DEP_TPLS/> |
| 4224 |
<EmailAddresses> |
| 4225 |
<Regression address="[email protected]"/> |
| 4226 |
</EmailAddresses> |
| 4227 |
<ParentPackage value=""/> |
| 4228 |
</Package> |
| 4229 |
... |
| 4230 |
</PackageDependencies> |
| 4231 |
|
| 4232 |
This XML file contains the names, directories, `Test Test Category`_ |
| 4233 |
(i.e. ``type``), CDash email address, and all of the package and TPL |
| 4234 |
dependencies for every package in the TriBITS project (including add-on |
| 4235 |
repositories if specified). There are several python tools under |
| 4236 |
``tribits/ci_support/`` that read in this file and use the created |
| 4237 |
data-structure for various tasks. This file and these tools are used by |
| 4238 |
`checkin-test.py`_ and `tribits_ctest_driver()`_. But these tools can also be |
| 4239 |
used to construct other workflows and tools. These tools require a Python3 |
| 4240 |
installation and for the `python3` executable to be installed. |
| 4241 |
|
| 4242 |
.. _TribitsDumpDepsXmlScript.cmake: |
| 4243 |
|
| 4244 |
A TriBITS project configure can create this file as a byproduct of |
| 4245 |
configuration by setting the configure option (see `Outputting package |
| 4246 |
dependency information`_), or the CMake -P script |
| 4247 |
**TribitsDumpDepsXmlScript.cmake** can be used to create this file on the fly |
| 4248 |
without having to configure a TriBITS project. To create this file outside of |
| 4249 |
configuration, one can run:: |
| 4250 |
|
| 4251 |
cmake \ |
| 4252 |
[-D PROJECT_SOURCE_DIR=<projectSourceDir>] \ |
| 4253 |
[-D <Project>_PRE_REPOSITORIES=<prepo0>,<prepo1>,...] \ |
| 4254 |
[-D <Project>_EXTRA_REPOSITORIES=<erepo0>,<erepo1>,...] \ |
| 4255 |
-D <Project>_DEPS_XML_OUTPUT_FILE=<projectDepsFileOut> \ |
| 4256 |
-P <tribitsDir>/ci_support/TribitsDumpDepsXmlScript.cmake |
| 4257 |
|
| 4258 |
If TriBITS is snashotted into the project in the standard location |
| 4259 |
``<projectDir>/cmake/tribits`` or the entire TriBITS repo is cloned under |
| 4260 |
``<projectDir>/TriBITS`` (so that the ``tribits`` dir is |
| 4261 |
``<projectDir>/TriBITS/tribits``) then one can leave off |
| 4262 |
``-DPROJECT_SOURCE_DIR=<projectSourceDir>`` and (if not wanting to include |
| 4263 |
extra repos) just run:: |
| 4264 |
|
| 4265 |
cmake \ |
| 4266 |
-D <Project>_DEPS_XML_OUTPUT_FILE=<projectDepsFileOut> \ |
| 4267 |
-P <projectSourceDir>/cmake/tribits/ci_support/TribitsDumpDepsXmlScript.cmake |
| 4268 |
|
| 4269 |
Once the XML file ``<projectDepsFileOut>`` is created, it can be used in |
| 4270 |
various types of analysis and used with different tools and commands. |
| 4271 |
|
| 4272 |
The tool `get-tribits-packages-from-files-list.py`_ can be used to determine |
| 4273 |
the list of TriBITS packages that need to be tested given a list of changed |
| 4274 |
files (e.g. as returned from ``git diff --name-only <from>..<to> > |
| 4275 |
changed-files.txt``). This is used in the `checkin-test.py`_ tool and the |
| 4276 |
`tribits_ctest_driver()`_ function to determine what TriBITS packages need to |
| 4277 |
be tested based on what files have been changed. |
| 4278 |
|
| 4279 |
The tool `get-tribits-packages-from-last-tests-failed.py`_ can be used to |
| 4280 |
extract the list of TriBITS packages that correspond to the failings tests |
| 4281 |
listed in the CTest-generated |
| 4282 |
``<build-dir>/Testing/Temporary/LastTestsFailed*.log`` file. This tool is |
| 4283 |
used in the `tribits_ctest_driver()`_ function in CI-testing mode to determine |
| 4284 |
what packages must be re-tested if they failed in the last CI iteration. |
| 4285 |
|
| 4286 |
The tool `filter-packages-list.py`_ takes in a list of TriBITS package |
| 4287 |
names and then filters the list according the `Test Test Category`_ of the |
| 4288 |
packages. This is used in testing workflows that only test a subset of |
| 4289 |
packages according to the Test Test Category at different stages in the |
| 4290 |
workflow. For example, the `checkin-test.py`_ tool and the |
| 4291 |
`tribits_ctest_driver()`_ function use this filtering to only test Primary |
| 4292 |
Tested (PT) or Secondary Tested (ST) packages for a given set of changed files |
| 4293 |
in a continuous integration workflow (see `Nested Layers of TriBITS Project |
| 4294 |
Testing`_). |
| 4295 |
|
| 4296 |
|
| 4297 |
TriBITS Automated Testing |
| 4298 |
========================= |
| 4299 |
|
| 4300 |
Much of the value provided by the TriBITS system is support for testing of |
| 4301 |
complex projects. Many different types of testing are required in a |
| 4302 |
complex project and development effort. A large project with lots of |
| 4303 |
repositories and packages provides a number of testing and development |
| 4304 |
challenges but also provides a number of opportunities to do testing in an |
| 4305 |
efficient way; especially pre-push and post-push continuous integration (CI) |
| 4306 |
testing. In addition, a number of post-push automated nightly test cases must |
| 4307 |
be managed. TriBITS takes full advantage of the features of raw CMake, CTest, |
| 4308 |
and CDash in support of testing and where gaps exist, TriBITS provides tools |
| 4309 |
and customizations. |
| 4310 |
|
| 4311 |
The following subsections describe several aspects to the TriBITS support for |
| 4312 |
testing. The subsection `Test Classifications for Repositories, Packages, and |
| 4313 |
Tests`_ defines the different types of test-related classifications that are |
| 4314 |
defined by TriBITS. These different test classifications are then used to |
| 4315 |
define a number of different standard `Nested Layers of TriBITS Project |
| 4316 |
Testing`_ which include different types of CI testing as well as nightly and |
| 4317 |
other tests. One of the most important types of CI testing, pre-push testing, |
| 4318 |
is then described in more detail in the subsection `Pre-push Testing using |
| 4319 |
checkin-test.py`_. The subsection `TriBITS CTest/CDash Driver`_ describes the usage |
| 4320 |
of the advanced `tribits_ctest_driver()`_ function to do incremental project |
| 4321 |
testing of a projects using advanced ``ctest -S`` scripts. The final |
| 4322 |
subsection `TriBITS CDash Customizations`_ describes how projects can use a |
| 4323 |
CDash server to more effectively display test results and provide |
| 4324 |
notifications for failures that are compartmentalized for a large project. |
| 4325 |
|
| 4326 |
|
| 4327 |
Test Classifications for Repositories, Packages, and Tests |
| 4328 |
---------------------------------------------------------- |
| 4329 |
|
| 4330 |
TriBITS defines a few different testing-related classifications for a TriBITS |
| 4331 |
project. These different classifications are used to select subsets of the |
| 4332 |
project's repositories, packages (and code within these packages), and tests |
| 4333 |
to be included in a given project build and test definition. These different |
| 4334 |
classification are: |
| 4335 |
|
| 4336 |
* `Repository Test Classification`_ |
| 4337 |
* `Package Test Group`_ |
| 4338 |
* `Test Test Category`_ |
| 4339 |
|
| 4340 |
These different test-related classifications are used to defined several |
| 4341 |
different `Nested Layers of TriBITS Project Testing`_. First, the `Repository |
| 4342 |
Test Classification`_ determines what repositories are even processed in order |
| 4343 |
for their packages to even consider being enabled. Second, if a repository |
| 4344 |
is selected, then the `Package Test Group`_ determines what packages |
| 4345 |
(and optional code in those packages) are even enabled such that their |
| 4346 |
`<packageDir>/CMakeLists.txt`_ files are even processed (i.e. according to |
| 4347 |
`TriBITS Dependency Handling Behaviors`_). Lastly, if an package gets |
| 4348 |
enabled, then the `Test Test Category`_ determines what test executables and |
| 4349 |
test cases get defined using the functions `tribits_add_executable()`_, |
| 4350 |
`tribits_add_test()`_ and `tribits_add_advanced_test()`_. |
| 4351 |
|
| 4352 |
More detailed descriptions of `Repository Test Classifications`_, `Package |
| 4353 |
Test Groups`_, and `Test Test Categories`_ are given in the following |
| 4354 |
subsections. |
| 4355 |
|
| 4356 |
.. _Repository Test Classification: |
| 4357 |
.. _Repository Test Classifications: |
| 4358 |
|
| 4359 |
**Repository Test Classification** |
| 4360 |
|
| 4361 |
The first type of test-related classification is for extra repositories |
| 4362 |
defined in the file `<projectDir>/cmake/ExtraRepositoriesList.cmake`_ (pulled |
| 4363 |
in through the `${PROJECT_NAME}_EXTRAREPOS_FILE`_ cache variable) using the |
| 4364 |
``REPO_CLASSIFICATION`` field in the macro call |
| 4365 |
`tribits_project_define_extra_repositories()`_. These classifications map to |
| 4366 |
the standard CTest dashboard types ``Continuous``, ``Nightly``, and |
| 4367 |
``Experimental`` (see `CTest documentation`_ and `TriBITS CTest/CDash Driver`_ |
| 4368 |
for details). |
| 4369 |
|
| 4370 |
.. _Repository Test Continuous: |
| 4371 |
|
| 4372 |
* Repositories marked **Continuous** match the standard CTest dashboard type |
| 4373 |
``Continuous``. These repositories are pulled in when |
| 4374 |
``${PROJECT_NAME}_ENABLE_KNOWN_EXTERNAL_REPOS_TYPE=Continuous``, or |
| 4375 |
``Nightly``. Repositories marked as ``Continuous`` are cloned, updated, and |
| 4376 |
processed by default in all project automated testing described in `Nested |
| 4377 |
Layers of TriBITS Project Testing`_. NOTE: One should **not** confuse this |
| 4378 |
with the `Test Test Category CONTINUOUS`_. |
| 4379 |
|
| 4380 |
.. _Repository Test Nightly: |
| 4381 |
|
| 4382 |
* Repositories marked **Nightly** match the standard CTest dashboard type |
| 4383 |
``Nightly``. These repositories are pulled in by default when |
| 4384 |
``${PROJECT_NAME}_ENABLE_KNOWN_EXTERNAL_REPOS_TYPE=Nightly``. Repositories |
| 4385 |
marked as ``Nightly`` are not processed by default as part of either |
| 4386 |
`Pre-Push CI Testing`_ or `Post-Push CI Testing`_. One would mark a |
| 4387 |
repository as ``Nightly`` for few reasons. First, an extra repo may be |
| 4388 |
marked as ``Nightly`` if it may not be available to all developers to clone |
| 4389 |
and only the nightly testing processes and machines may have access. Also, |
| 4390 |
an extra repo may also be marked as ``Nightly`` if it does not contain any |
| 4391 |
packages that the project wants to pay the cost to include in even |
| 4392 |
`Post-Push CI Testing`_. NOTE: One should **not** confuse this with the |
| 4393 |
`Test Test Category NIGHTLY`_. |
| 4394 |
|
| 4395 |
.. _Repository Test Experimental: |
| 4396 |
|
| 4397 |
* Repositories marked **Experimental** match the standard CTest dashboard type |
| 4398 |
``Experimental``. Repositories marked as ``Experimental`` are not processed |
| 4399 |
by default as part of any automated testing described in the subsection |
| 4400 |
`Nested Layers of TriBITS Project Testing`_ (except for perhaps some |
| 4401 |
experimental builds). The main reason that an extra repo may be marked as |
| 4402 |
``Experimental`` is that it may only contain ``EX`` packages and |
| 4403 |
therefore none of these packages would be enabled by default anyway. Also, |
| 4404 |
a repo may be marked as ``Experimental`` if it is developed in a very sloppy |
| 4405 |
way such that one cannot even assume that the repository's |
| 4406 |
`<repoDir>/PackagesList.cmake`_, `<repoDir>/TPlsList.cmake`_, and |
| 4407 |
`<packageDir>/cmake/Dependencies.cmake`_ files are not without errors. |
| 4408 |
Since these files are always processed if the repository is included, then |
| 4409 |
any errors in these files will cause the entire configure to fail! |
| 4410 |
|
| 4411 |
.. ToDo: Determine, add unit tests for, and document exactly how |
| 4412 |
.. 'Experimental' repos are handled |
| 4413 |
.. w.r.t. ${PROJECT_NAME}_ENABLE_KNOWN_EXTERNAL_REPOS_TYPE. Currently it |
| 4414 |
.. looks like there are not processed if |
| 4415 |
.. ${PROJECT_NAME}_ENABLE_KNOWN_EXTERNAL_REPOS_TYPE=Experimental. I am not |
| 4416 |
.. sure why we bother with Experimental repos other than to just ignore them. |
| 4417 |
.. However, it does seem they get cloned no matter what their classification |
| 4418 |
.. in the TribitsCTestDriverCore.cmake script. |
| 4419 |
|
| 4420 |
.. _Package Test Group: |
| 4421 |
.. _Package Test Groups: |
| 4422 |
|
| 4423 |
**Package Test Group** |
| 4424 |
|
| 4425 |
Once a set of TriBITS repositories are selected in accordance with their |
| 4426 |
`Repository Test Classification`_, that determines the set of packages defined |
| 4427 |
for the TriBITS project. Given the set of defined packages, the set of |
| 4428 |
packages that get enabled is determined by the **Package Test Group** which is |
| 4429 |
defined and described here. |
| 4430 |
|
| 4431 |
Every `TriBITS Package`_ is assigned a test group. These test groups are for |
| 4432 |
*Primary Tested* (`PT`_) code, *Secondary Tested* (`ST`_) code, and |
| 4433 |
*Experimental* (`EX`_) code. The test group defines *what* package get |
| 4434 |
selected (or are excluded from being selected) to include in a given build for |
| 4435 |
testing-related purposes. packages may also conditionally build in additional |
| 4436 |
code based on the testing group. The detailed rules for when an package is |
| 4437 |
selected or excluded from the build based on the test group is given in |
| 4438 |
`TriBITS Dependency Handling Behaviors`_. We only summarize those rules here. |
| 4439 |
|
| 4440 |
More detailed descriptions of the test groups are given below. |
| 4441 |
|
| 4442 |
.. _PT: |
| 4443 |
|
| 4444 |
* **Primary Tested (PT)** Code is of the highest priority to keep working for |
| 4445 |
the current development effort. Packages may be selected to be ``PT`` for a |
| 4446 |
number of reasons. First, if the capability provided by the code is mature |
| 4447 |
and if a regression would cause major harm to a customer, the code should |
| 4448 |
likely be marked as ``PT``. Also, if the build and correct functioning of |
| 4449 |
the code is needed by other development team members to support their |
| 4450 |
day-to-day development activities, then the code should be marked as ``PT`` |
| 4451 |
as well. An external package/TPL, on the other hand, is marked as ``PT`` if |
| 4452 |
it is required by a downstream ``PT`` package. Every project developer is |
| 4453 |
expected to have every ``PT`` TPL installed on every machine where they do |
| 4454 |
development on and from which they push to the global repo (see |
| 4455 |
`checkin-test.py`_ tool). ``PT`` packages are the foundation for `Pre-Push |
| 4456 |
CI Testing`_. |
| 4457 |
|
| 4458 |
.. _ST: |
| 4459 |
|
| 4460 |
* **Secondary Tested (ST)** Code is still very important code for the project |
| 4461 |
and represents important capability to maintain but is excluded from the |
| 4462 |
``PT`` set of code for one of a few different reasons. First, code may be |
| 4463 |
marked as ``ST`` if is not critical to drive most day-to-day development |
| 4464 |
activities. If ``ST`` code breaks, it usually will not cause immediate and |
| 4465 |
major harm to most developers. Also, code may be marked as ``ST`` if it has |
| 4466 |
required dependencies on ``ST`` external packages/TPLs which are either hard |
| 4467 |
to install or may not be available on all platforms where developers do |
| 4468 |
their development and from where they push changes to the global repo. In |
| 4469 |
addition, code may be marked as ``ST`` if the project is just too big and |
| 4470 |
developers can't be expected to build and test all of this code with every |
| 4471 |
push (so a decision is made to only make some code as ``PT`` so that pushes |
| 4472 |
don't take too long). ``ST`` code can be included in the TriBITS |
| 4473 |
auto-enable algorithms by setting the variable |
| 4474 |
``${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON`` (see `TriBITS Dependency |
| 4475 |
Handling Behaviors`_). Otherwise, ``ST`` code **is not** enabled by |
| 4476 |
auto-enable algorithms. Typically, ``ST`` code is excluded from the default |
| 4477 |
builds in `Pre-Push CI Testing`_ but ``ST`` code **is** typically tested in |
| 4478 |
`Post-Push CI Testing`_, `Nightly Testing`_ as well as in other testing |
| 4479 |
cases. |
| 4480 |
|
| 4481 |
.. _EX: |
| 4482 |
|
| 4483 |
* **Experimental (EX)** Code is usually too unstable, buggy, or non-portable |
| 4484 |
to be maintained as part of the project's automated testing processes. Or |
| 4485 |
the code may not be important enough to the project to bother paying the |
| 4486 |
cost to test it in the project's automated testing processes. The ability |
| 4487 |
to mark some code as ``EX`` allows the developers of that code to include |
| 4488 |
their code in the project's VC repos along with the rest of the project's |
| 4489 |
code and be able to take advantage of all of the development tools provided |
| 4490 |
by the project (using TriBITS) but at the same time not having to "sign up" |
| 4491 |
for all of the responsibilities of maintaining working software that every |
| 4492 |
developer has to take a part in helping to keep working. |
| 4493 |
|
| 4494 |
The test group for each type of entity is assigned in the following places: |
| 4495 |
|
| 4496 |
* The top-level `TriBITS Package`_'s test group is assigned using the |
| 4497 |
``CLASSIFICATION`` field in the macro call |
| 4498 |
`tribits_repository_define_packages()`_ in its parent repository's |
| 4499 |
`<repoDir>/PackagesList.cmake`_ file. |
| 4500 |
|
| 4501 |
* A `TriBITS Subpackage`_'s test group is assigned using the |
| 4502 |
``CLASSIFICATIONS`` field of the `SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS`_ |
| 4503 |
argument in the macro call `tribits_package_define_dependencies()`_ in its |
| 4504 |
parent package's `<packageDir>/cmake/Dependencies.cmake`_ file. |
| 4505 |
|
| 4506 |
* A `TriBITS External Package/TPL`_'s test group is assigned using the |
| 4507 |
``CLASSIFICATION`` field in the macro call |
| 4508 |
`tribits_repository_define_tpls()`_ in its parent repository's |
| 4509 |
`<repoDir>/TPLsList.cmake`_ file. |
| 4510 |
|
| 4511 |
After these files are processed, the variable `${PACKAGE_NAME}_TESTGROUP`_ |
| 4512 |
gives the test group for each defined Package while the variable |
| 4513 |
``${TPL_NAME}_TESTGROUP`` gives the test group for each defined TPL. |
| 4514 |
|
| 4515 |
Note that the test group classification ``PT``/``ST``/``EX`` is *not* to be |
| 4516 |
confused with the *maturity level* of the package as discussed in the |
| 4517 |
`TriBITS Lifecycle Model`_. The test group classification in no way implies |
| 4518 |
the maturity of the given TriBITS Package or piece of code. Instead, the |
| 4519 |
test group is just used to sub-select packages (and pieces of code within |
| 4520 |
those packages) that are the most important to sustain for the various current |
| 4521 |
development group's activities. While more-mature code would typically never |
| 4522 |
be classified as ``EX`` (Experimental), there are cases were immature packages |
| 4523 |
may be classified as ``ST`` or even ``PT``. For example, a very important |
| 4524 |
research project may be driving the development of a very new algorithm with |
| 4525 |
the low *maturity level* of *Research Stable* (``RS``) or even *Exploratory* |
| 4526 |
(``EP``) because keeping that code working may be critical to keeping the |
| 4527 |
research project on track. |
| 4528 |
|
| 4529 |
In addition to just selecting ``PT`` and ``ST`` packages as a whole, a |
| 4530 |
TriBITS ``PT`` package can also contain conditional code and test |
| 4531 |
directories that get enabled when ``${PROJECT_NAME}_SECONDARY_TESTED_CODE=ON`` |
| 4532 |
and therefore represents more ``ST`` code. The package's |
| 4533 |
`<packageDir>/CMakeLists.txt`_ files can contain simple if statements and can |
| 4534 |
use the `tribits_set_st_for_dev_mode()`_ function to automatically select |
| 4535 |
extra code to enable when ``ST`` is enabled or when the project is in release |
| 4536 |
mode. |
| 4537 |
|
| 4538 |
.. ToDo: Provide examples in TribitsExampleProject of ST and EX packages |
| 4539 |
.. and of conditional ST and EX code in an PT package. This is important |
| 4540 |
.. to round out the examples. |
| 4541 |
|
| 4542 |
.. _Test Test Category: |
| 4543 |
.. _Test Test Categories: |
| 4544 |
|
| 4545 |
**Test Test Category** |
| 4546 |
|
| 4547 |
Once a package is even defined (due to its parent repository's selection |
| 4548 |
consistent with its `Repository Test Classification`_) and is the package is |
| 4549 |
enabled (consistent with its `Package Test Group`_) then the set of |
| 4550 |
individual test executables and test cases that are included or not in that |
| 4551 |
package depends on the ``CATEGORIES`` argument in the functions |
| 4552 |
`tribits_add_executable()`_, `tribits_add_test()`_ and |
| 4553 |
`tribits_add_advanced_test()`_, and the `${PROJECT_NAME}_TEST_CATEGORIES`_ |
| 4554 |
variable. This **Test Test Category** defines the last "knob" that the |
| 4555 |
development team has in controlling what tests get run in a particular test |
| 4556 |
scenario as described in the section `Nested Layers of TriBITS Project |
| 4557 |
Testing`_. |
| 4558 |
|
| 4559 |
The currently allowed values for the *Test Test Category* are ``BASIC``, |
| 4560 |
``CONTINUOUS``, ``NIGHTLY``, ``HEAVY``, and ``PERFORMANCE``. Tests are |
| 4561 |
enabled based on their assigned test test category matching the categories set |
| 4562 |
in the CMake cache variable `${PROJECT_NAME}_TEST_CATEGORIES`_. The test test |
| 4563 |
categories ``BASIC``, ``CONTINUOUS``, ``NIGHTLY``, and ``HEAVY`` are subsets |
| 4564 |
of each other. That is, a ``BASIC`` test is automatically included in the set |
| 4565 |
of ``CONTINUOUS``, ``NIGHTLY``, and ``HEAVY`` tests (as set using |
| 4566 |
``${PROJECT_NAME}_TEST_CATEGORIES``). |
| 4567 |
|
| 4568 |
The different test test categories are described below in more detail: |
| 4569 |
|
| 4570 |
.. _Test Test Category BASIC: |
| 4571 |
|
| 4572 |
* Tests marked **BASIC** represent key functionality that is needed by nearly |
| 4573 |
every developer that works on the project and so must be protected at all |
| 4574 |
times and are therefore included in `Pre-Push CI Testing`_. Tests marked as |
| 4575 |
``BASIC`` are enabled for the values of ``${PROJECT_NAME}_TEST_CATEGORIES`` |
| 4576 |
of ``BASIC``, ``CONTINUOUS``, ``NIGHT``, and ``HEAVY``. The category |
| 4577 |
``BASIC`` is the default test test category given to all test executables |
| 4578 |
and tests that don't specify the ``CATEGORIES`` argument. |
| 4579 |
|
| 4580 |
.. _Test Test Category CONTINUOUS: |
| 4581 |
|
| 4582 |
* Tests marked **CONTINUOUS** also represent importantly functionality but are |
| 4583 |
typically not run in `Pre-Push CI Testing`_ testing but instead are run in |
| 4584 |
`Post-Push CI Testing`_, `Nightly Testing`_, and other types of testing. |
| 4585 |
Tests marked as ``CONTINUOUS`` are enabled for the values of |
| 4586 |
``${PROJECT_NAME}_TEST_CATEGORIES`` equal to ``CONTINUOUS``, ``NIGHT``, and |
| 4587 |
``HEAVY``. A test may be marked ``CONTINUOUS`` and not ``BASIC`` for a few |
| 4588 |
different reasons. For example, the code needed to run the test may take |
| 4589 |
too long to build or the test itself may take too long to run in order to |
| 4590 |
afford including it in `Pre-Push CI Testing`_. |
| 4591 |
|
| 4592 |
.. _Test Test Category NIGHTLY: |
| 4593 |
|
| 4594 |
* Tests marked **NIGHTLY** usually take even longer to build and/or run than |
| 4595 |
``CONTINUOUS`` tests and therefore are too expensive to include in |
| 4596 |
`Post-Push CI Testing`_. Tests may also be marked as ``NIGHTLY`` even if |
| 4597 |
they might run relatively fast if there is a desire to not cause the CI |
| 4598 |
server to fail if these tests fail. In this case, the decision is to take |
| 4599 |
the testing and maintenance of these tests and the capabilities they |
| 4600 |
represent "offline" so that they don't influence the daily development cycle |
| 4601 |
for the project but instead are addressed in a "secondary feedback loop". |
| 4602 |
Tests marked as ``NIGHTLY`` are enabled for the values of |
| 4603 |
``${PROJECT_NAME}_TEST_CATEGORIES`` equal to ``NIGHT``, and ``HEAVY``. |
| 4604 |
|
| 4605 |
.. _Test Test Category HEAVY: |
| 4606 |
|
| 4607 |
* Tests marked **HEAVY** are usually reserved for very expensive tests that |
| 4608 |
are too expensive to run nightly. ``HEAVY`` tests require more testing |
| 4609 |
resources and therefore may only be run on a fully optimized build and/or |
| 4610 |
run less frequently. Tests marked as ``HEAVY`` are enabled only for the |
| 4611 |
value of ``${PROJECT_NAME}_TEST_CATEGORIES`` equal to ``HEAVY``. |
| 4612 |
|
| 4613 |
.. _Test Test Category PERFORMANCE: |
| 4614 |
|
| 4615 |
* Tests marked **PERFORMANCE** are a special category of tests that are |
| 4616 |
specially designed to measure the serial (non-parallel) run-time performance |
| 4617 |
of parts of the software (see `Performance Testing`_). Tests marked as |
| 4618 |
``PERFORMANCE`` are enabled only for the value of |
| 4619 |
``${PROJECT_NAME}_TEST_CATEGORIES`` equal to ``PERFORMANCE``. |
| 4620 |
|
| 4621 |
Every TriBITS project has a default setting for |
| 4622 |
``${PROJECT_NAME}_TEST_CATEGORIES`` that is set for a basic ``cmake`` |
| 4623 |
configure of the project (see `${PROJECT_NAME}_TEST_CATEGORIES_DEFAULT`_ for |
| 4624 |
more details). In addition, the different testing processes described in the |
| 4625 |
section `Nested Layers of TriBITS Project Testing`_ set this to different |
| 4626 |
values. |
| 4627 |
|
| 4628 |
|
| 4629 |
Nested Layers of TriBITS Project Testing |
| 4630 |
---------------------------------------- |
| 4631 |
|
| 4632 |
Now that the different types of `Test Classifications for Repositories, |
| 4633 |
Packages, and Tests`_ have been defined, this section describes how these |
| 4634 |
different test-related classifications are used to select repositories, |
| 4635 |
packages (and code) and tests to run in the standard project testing |
| 4636 |
processes. More than any other section in this document, this section will |
| 4637 |
describe and assume a certain class of software development processes (namely |
| 4638 |
agile processes) where testing and *continuous integration* (CI) are critical |
| 4639 |
components. However, detailed descriptions of these processes are deferred to |
| 4640 |
the later sections `Pre-push Testing using checkin-test.py`_ and `TriBITS |
| 4641 |
CTest/CDash Driver`_. |
| 4642 |
|
| 4643 |
The standard TriBITS-supported project testing processes are: |
| 4644 |
|
| 4645 |
* `Pre-Push CI Testing`_ |
| 4646 |
* `Post-Push CI Testing`_ |
| 4647 |
* `Nightly Testing`_ |
| 4648 |
* `Heavy Testing`_ |
| 4649 |
* `Performance Testing`_ |
| 4650 |
|
| 4651 |
.. ToDo: Discuss why we would want to create standardized test cases? The |
| 4652 |
.. answer is that it standardized how testing is done across multiple TriBITS |
| 4653 |
.. projects so that larger meta-projects and run tests in a consistent way. |
| 4654 |
|
| 4655 |
.. ToDo: Show a diagram for these different testing processes and show how |
| 4656 |
.. they nest inside of each other for the most part. |
| 4657 |
|
| 4658 |
These standard testing processes are outlined in more detail below and show |
| 4659 |
how the different test-related categories are used to define each of these. |
| 4660 |
|
| 4661 |
.. _Pre-Push CI Testing: |
| 4662 |
|
| 4663 |
**Pre-Push CI Testing** |
| 4664 |
|
| 4665 |
The first level of testing is *Pre-Push CI Testing* that is performed before |
| 4666 |
changes to the project are pushed to the master branch(es) in the global |
| 4667 |
repository(s). With TriBITS, this type of testing and the following push is |
| 4668 |
typically done using the `checkin-test.py`_ tool. This category of testing is |
| 4669 |
described in much more detail in `Pre-push Testing using checkin-test.py`_. |
| 4670 |
All of the "default builds" used with the ``checkin-test.py`` tool select |
| 4671 |
repositories, packages and code, and individual tests using the following |
| 4672 |
test-related classifications: |
| 4673 |
|
| 4674 |
========================= ================== ==================================== |
| 4675 |
Classification Type Classification (See Reference) |
| 4676 |
========================= ================== ==================================== |
| 4677 |
Repository Test Classif. ``Continuous`` (`Repository Test Continuous`_) |
| 4678 |
Package Test Group ``PT`` (`PT`_) |
| 4679 |
Test Test Category ``BASIC`` (`Test Test Category BASIC`_) |
| 4680 |
========================= ================== ==================================== |
| 4681 |
|
| 4682 |
Typically a TriBITS project will define a "standard development environment" |
| 4683 |
which is comprised of a standard compiler (e.g. GCC 8.3.0), external |
| 4684 |
package/TPL versions (e.g. OpenMPI 4.0.5, Boost 4.9, etc.), and other tools |
| 4685 |
(e.g. cmake 3.23.0, git 2.10.1, etc.). This standard development environment |
| 4686 |
is expected to be used to test changes to the project's code before any push. |
| 4687 |
By using a standard development environment, if the code builds and all the |
| 4688 |
tests pass for the "default" pre-push builds for one developer, then that |
| 4689 |
maximizes the probability that the code will also build and all tests will |
| 4690 |
pass for every other developer using the same development environment. This |
| 4691 |
is critical to keep the development team maximally productive. Portability is |
| 4692 |
also important for most projects but portability testing is best done in a |
| 4693 |
secondary feedback look using `Nightly Testing`_ builds. TriBITS has some |
| 4694 |
support for helping to set up a standard software development environment as |
| 4695 |
described in section `TriBITS Development Toolset`_. |
| 4696 |
|
| 4697 |
The basic assumption of all CI processes (including the one described here) is |
| 4698 |
that if anyone pulls the project's development sources at any time, then all |
| 4699 |
of the code will build and all of the tests will pass for the "default" build |
| 4700 |
cases. For a TriBITS project, this means that the project's |
| 4701 |
``--default-builds`` (see above) will all pass for every ``PT`` package. All |
| 4702 |
of these software development processes make this basic assumption and agile |
| 4703 |
software development methods fall apart if this is not true. |
| 4704 |
|
| 4705 |
.. _Post-Push CI Testing: |
| 4706 |
|
| 4707 |
**Post-Push CI Testing** |
| 4708 |
|
| 4709 |
After changes are pushed to the master branch(es) in the global repository(s), |
| 4710 |
*Post-Push CI Testing* is performed where a CI server detects the changes and |
| 4711 |
immediately fires off a CI build using CTest to test the changes and the |
| 4712 |
results are posted to a CDash server (in the "Continuous" section on the |
| 4713 |
project's dashboard page). This process is driven by CTest driver code that |
| 4714 |
calls `tribits_ctest_driver()`_ as described in the section `TriBITS |
| 4715 |
CTest/CDash Driver`_. Various types of specific CI builds can be constructed |
| 4716 |
and run (see `CTest/CDash CI Server`_) but these post-push CI builds typically |
| 4717 |
select repositories, packages and code, and individual tests using the |
| 4718 |
following test-related classifications: |
| 4719 |
|
| 4720 |
========================= ================== ==================================== |
| 4721 |
Classification Type Classification (See Reference) |
| 4722 |
========================= ================== ==================================== |
| 4723 |
Repository Test Classif. ``Continuous`` (`Repository Test Continuous`_) |
| 4724 |
Package Test Group ``PT`` & ``ST`` (`PT`_ and `ST`_) |
| 4725 |
Test Test Category ``CONTINUOUS`` (`Test Test Category CONTINUOUS`_) |
| 4726 |
========================= ================== ==================================== |
| 4727 |
|
| 4728 |
Post-push CI testing would assume to use the same standard development |
| 4729 |
environment as used for `Pre-Push CI Testing`_. Also, the project may also |
| 4730 |
choose to run additional automated post-push CI builds that exactly match the |
| 4731 |
pre-push CI default builds to help check on the health of these builds |
| 4732 |
continuously and not just rely on the development team to always perform the |
| 4733 |
pre-push CI builds correctly before pushing. |
| 4734 |
|
| 4735 |
.. _Nightly Testing: |
| 4736 |
|
| 4737 |
**Nightly Testing** |
| 4738 |
|
| 4739 |
In addition to pre-push and post-push CI testing, a typical TriBITS project |
| 4740 |
will set up multiple *Nightly Testing* builds (or once-a-day builds, they |
| 4741 |
don't need to only run at night). These builds are also driven by CTest |
| 4742 |
driver scripts described in the section `TriBITS CTest/CDash Driver`_ and post |
| 4743 |
results to the project's CDash server (in the "Nightly" section on the |
| 4744 |
project's dashboard page). Nightly builds don't run in a continuous loop but |
| 4745 |
instead are run once a day (e.g. driven by a cron job) and there tends to be |
| 4746 |
many different nightly build cases that test the project using different |
| 4747 |
compilers (e.g. GCC, Intel, Microsoft, etc., and different versions of each), |
| 4748 |
different external package/TPL versions (e.g. different OpenMPI versions, |
| 4749 |
different MPICH versions, etc.), different platforms (e.g. Linux, Windows, |
| 4750 |
etc.), and varying many other options and settings on these different |
| 4751 |
platforms. What all nightly builds have in common is that they tend to select |
| 4752 |
repositories, packages and code, and individual tests using the following |
| 4753 |
test-related classifications: |
| 4754 |
|
| 4755 |
========================= ================== ==================================== |
| 4756 |
Classification Type Classification (See Reference) |
| 4757 |
========================= ================== ==================================== |
| 4758 |
Repository Test Classif. ``Nightly`` (`Repository Test Nightly`_) |
| 4759 |
Package Test Group ``PT`` & ``ST`` (`PT`_ and `ST`_) |
| 4760 |
Test Test Category ``NIGHTLY`` (`Test Test Category NIGHTLY`_) |
| 4761 |
========================= ================== ==================================== |
| 4762 |
|
| 4763 |
The nightly builds comprise the basic "heart beat" for the project. |
| 4764 |
|
| 4765 |
.. _Heavy Testing: |
| 4766 |
|
| 4767 |
**Heavy Testing** |
| 4768 |
|
| 4769 |
*Heavy Testing* builds are just an extension to the `Nightly Testing`_ builds |
| 4770 |
that add on more expensive tests marked using the `Test Test Category HEAVY`_. |
| 4771 |
For projects that define heavy tests and heavy builds, individual test cases |
| 4772 |
may be allowed to take 24 hours or longer to run so they can't even be run |
| 4773 |
every day in nightly testing. What standard heavy builds have in common is |
| 4774 |
that they tend to select repositories, packages and code, and individual |
| 4775 |
tests using the following test-related classifications: |
| 4776 |
|
| 4777 |
========================= ================== ==================================== |
| 4778 |
Classification Type Classification (See Reference) |
| 4779 |
========================= ================== ==================================== |
| 4780 |
Repository Test Classif. ``Nightly`` (`Repository Test Nightly`_) |
| 4781 |
Package Test Group ``PT`` & ``ST`` (`PT`_ and `ST`_) |
| 4782 |
Test Test Category ``HEAVY`` (`Test Test Category HEAVY`_) |
| 4783 |
========================= ================== ==================================== |
| 4784 |
|
| 4785 |
Project developer teams should strive to limit the number of test cases that |
| 4786 |
are marked as ``HEAVY`` since these tests will typically *not* get run in very |
| 4787 |
may builds or may not be run every day and developers will tend to never |
| 4788 |
enable them when doing more extensive testing using ``--st-extra-builds`` with |
| 4789 |
the `checkin-test.py`_ tool in extended pre-push testing. |
| 4790 |
|
| 4791 |
.. _Performance Testing: |
| 4792 |
|
| 4793 |
**Performance Testing** |
| 4794 |
|
| 4795 |
*Performance Testing* builds are a special class of builds that have tests |
| 4796 |
that are specifically designed to test the run-time performance of a particular |
| 4797 |
piece of code or algorithm. These tests tend to be sensitive to loads on the |
| 4798 |
machine and therefore typically need to be run on an unloaded machine for |
| 4799 |
reliable results. Details on how to write good performance tests with hard |
| 4800 |
pass/fail time limits is beyond the scope of this document. All TriBITS does |
| 4801 |
is to define the special `Test Test Category PERFORMANCE`_ to allow TriBITS |
| 4802 |
packages to declare these tests in a consistent way so that they can be run |
| 4803 |
along with performance tests defined in other TriBITS packages. From a |
| 4804 |
TriBITS standpoint, all performance testing builds would tend to select |
| 4805 |
repositories, packages and code, and individual tests using the following |
| 4806 |
test-related classifications: |
| 4807 |
|
| 4808 |
========================= ================== ==================================== |
| 4809 |
Classification Type Classification (See Reference) |
| 4810 |
========================= ================== ==================================== |
| 4811 |
Repository Test Classif. ``Nightly`` (`Repository Test Nightly`_) |
| 4812 |
Package Test Group ``PT`` & ``ST`` (`PT`_ and `ST`_) |
| 4813 |
Test Test Category ``PERFORMANCE`` (`Test Test Category PERFORMANCE`_) |
| 4814 |
========================= ================== ==================================== |
| 4815 |
|
| 4816 |
.. ToDo: I need to set up automated testing for TriBITS to use as the example |
| 4817 |
.. for all of these types of testing. There is no better example that one |
| 4818 |
.. that actually works. It would also be nice to have a snapshot repo of |
| 4819 |
.. TribitsExampleProject that also had this testing enabled for it but I am |
| 4820 |
.. not sure that really makes sense. |
| 4821 |
|
| 4822 |
|
| 4823 |
.. _checkin-test.py: |
| 4824 |
|
| 4825 |
Pre-push Testing using checkin-test.py |
| 4826 |
-------------------------------------- |
| 4827 |
|
| 4828 |
CMake provides the integrated tool CTest (executable ``ctest``) which is used |
| 4829 |
to define and run different tests. However, a lot more needs to be done to |
| 4830 |
effectively test changes for a large project before pushing to the master |
| 4831 |
branch(es) in the main repository(s). Things get especially complicated and |
| 4832 |
tricky when multiple version-control (VC) repositories are involved. The |
| 4833 |
TriBITS system provides the tool ``checkin-test.py`` for automating the |
| 4834 |
process of: |
| 4835 |
|
| 4836 |
1) Determining if the local VC repo(s) are ready to integrate with the remote |
| 4837 |
master branch(es), |
| 4838 |
|
| 4839 |
2) Pulling and integrating the most current changes from the remote VC repo(s), |
| 4840 |
|
| 4841 |
3) Figuring out what TriBITS packages need to be enabled and testing (by |
| 4842 |
examining VC file diffs), |
| 4843 |
|
| 4844 |
4) Configuring only the necessary TriBITS packages and tests (and their |
| 4845 |
downstream dependencies by default) and building, running tests, and |
| 4846 |
reporting results (via email), and |
| 4847 |
|
| 4848 |
5) Only if all specified builds and tests pass, amending the last commit |
| 4849 |
message with the test results, then pushing local commits to the remove VC |
| 4850 |
repo(s) and sending out summary emails. |
| 4851 |
|
| 4852 |
There are several advantages to using a project's ``checkin-test.py`` tool for |
| 4853 |
pushing changes to the main development branch which include: |
| 4854 |
|
| 4855 |
a) provides a consistent definition for "okay to push" for all developers |
| 4856 |
|
| 4857 |
b) protects other developers from pulling badly broken code |
| 4858 |
|
| 4859 |
c) reduces the number of packages that need to be tested by automatically |
| 4860 |
determining the set based on changed files and package dependencies |
| 4861 |
|
| 4862 |
d) avoids developer mistakes in performing repetitive tasks and forgetting |
| 4863 |
important steps in the process |
| 4864 |
|
| 4865 |
e) marks a set of working commits that are safe to search with ``git bisect`` |
| 4866 |
to find problems (see `Using Git Bisect with checkin-test.py workflows`_) |
| 4867 |
|
| 4868 |
When using the ``checkin-test.py`` tool, every TriBITS project defines one or |
| 4869 |
more "default builds" (specified through the ``--default-builds`` argument) |
| 4870 |
for pre-push CI testing that form the criteria for if it is okay to push code |
| 4871 |
changes or not. The "default builds" select repositories, packages and |
| 4872 |
code, and individual tests as described in `Pre-Push CI Testing`_. A TriBITS |
| 4873 |
project defines its default pre-push builds using the file |
| 4874 |
`<projectDir>/project-checkin-test-config.py`_. For an example, the file |
| 4875 |
`TribitsExampleProject`_/``project-checkin-test-config.py`` is shown below: |
| 4876 |
|
| 4877 |
.. include:: ../../examples/TribitsExampleProject/project-checkin-test-config.py |
| 4878 |
:literal: |
| 4879 |
|
| 4880 |
This gives ``--default-builds=MPI_DEBUG,SERIAL_RELEASE``. As shown, typically |
| 4881 |
two default builds are defined so that various options can be toggled between |
| 4882 |
the two builds. Typical options to toggle include enabling/disabling MPI and |
| 4883 |
enabling/disabling run-time debug mode checking (i.e. toggle |
| 4884 |
``${PROJECT_NAME}_ENABLE_DEBUG``). Typically, other important options will |
| 4885 |
also be toggled between these two builds. |
| 4886 |
|
| 4887 |
Note that both of the default builds shown above, including the ``MPI_DEBUG`` |
| 4888 |
build, actually set optimized compiler flags with |
| 4889 |
``-DCMAKE_BUILD_TYPE:STRING=RELEASE``. What makes the ``MPI_DEBUG`` build a |
| 4890 |
"debug" build is turning on optional run-time debug-mode checking, not |
| 4891 |
disabling optimized code. This is important so that the defined tests run |
| 4892 |
fast. For most projects, the default pre-push builds should **not** be used |
| 4893 |
to debug-enabled code which is suitable to run through a debugger |
| 4894 |
(e.g. ``gdb``). Instead, these "debug" builds are designed to test changes to |
| 4895 |
the project's code efficiently before pushing changes. Typically, a |
| 4896 |
development team should not have to test the chosen compiler's ability to |
| 4897 |
generate non-optimized debug code and suffer slower test times before pushing. |
| 4898 |
|
| 4899 |
Note that turning on ``-DTribitsExProj_ENABLE_CHECKED_STL=ON`` as shown above |
| 4900 |
can only be used when the external packages/TPLs have no C++ code using the |
| 4901 |
C++ STL or if that particular build points to C++ TPLs also compiled with |
| 4902 |
checked STL enabled. The `TribitsExampleProject`_ default builds do not |
| 4903 |
depend on any C++ TPLs that might use the C++ STL so enabling this option adds |
| 4904 |
additional positive debug-mode checking for C++ code. |
| 4905 |
|
| 4906 |
The ``checkin-test.py`` tool is a fairly sophisticated piece of software that |
| 4907 |
is well tested and very robust. The level of testing of this tool is likely |
| 4908 |
greater than any of the software that it will be used to test (unless the |
| 4909 |
project is a real-time flight control system or nuclear reactor control system |
| 4910 |
or something). This is needed so as to provide confidence in the developers |
| 4911 |
that the tool will only push their changes if everything checks out as it |
| 4912 |
should. There are a lot of details and boundary cases that one has to |
| 4913 |
consider and a number of use cases that need to be supported by such a tool. |
| 4914 |
For more detailed documentation, see `checkin-test.py --help`_. |
| 4915 |
|
| 4916 |
Note that the ``checkin-test.py`` tool can also be used to implement |
| 4917 |
"poor-man's" post-push testing processes as described in `Post-Push CI and |
| 4918 |
Nightly Testing using checkin-test.py`_. However, most software projects will |
| 4919 |
want to go with the more elaborate and more feature-full CTest/CDash system |
| 4920 |
described in `TriBITS CTest/CDash Driver`_. |
| 4921 |
|
| 4922 |
.. ToDo: Describe the standard workflow for using the checkin-test.py tool. |
| 4923 |
|
| 4924 |
.. ToDo: Describe why the --default-builds must only include PT code and not |
| 4925 |
.. ST due to changing the behavior of the PT software. As an example, discuss |
| 4926 |
.. Boost usage in Teuchos and how that changes behavior of a few things. |
| 4927 |
.. Another example is BinUtils. |
| 4928 |
|
| 4929 |
.. ToDo: Describe the system for mapping changed files to changed packages. |
| 4930 |
.. Discuss how important the directory structure is. |
| 4931 |
|
| 4932 |
.. ToDo: Describe the local test, remote push process by pulling from the |
| 4933 |
.. Trilinos developers webpage. |
| 4934 |
|
| 4935 |
.. ToDo: Describe the role that checkin-test.py plays in multi-repo ACI |
| 4936 |
|
| 4937 |
|
| 4938 |
TriBITS CTest/CDash Driver |
| 4939 |
-------------------------- |
| 4940 |
|
| 4941 |
The TriBITS system uses a sophisticated and highly customized CTest -S driver |
| 4942 |
script to test TriBITS projects and submit results to a CDash server. The |
| 4943 |
primary code for driving this is contained in the CTest function |
| 4944 |
`tribits_ctest_driver()`_ contained in the file |
| 4945 |
``TribitsCTestDriverCore.cmake``. This script loops through all of the |
| 4946 |
specified TriBITS packages for a given TriBITS project and does a configure, |
| 4947 |
built, and test and then submits results to the specified CDash server |
| 4948 |
incrementally. If the configure or library build of any upstream TriBITS |
| 4949 |
package fails, then that TriBITS package is disabled in all downstream TriBITS |
| 4950 |
package builds so as not to propagate already known failures. Each TriBITS |
| 4951 |
top-level package is assigned its own CDash regression email address (see |
| 4952 |
`CDash regression email addresses`_) and each package configure/build/test is |
| 4953 |
given its own row for the package build in the CDash server. A CTest script |
| 4954 |
using `tribits_ctest_driver()`_ is run in one of three different modes. |
| 4955 |
First, it can run standard once-a-day, from-scratch builds as described in |
| 4956 |
`CTest/CDash Nightly Testing`_. Second, it can run as a CI server as |
| 4957 |
described in `CTest/CDash CI Server`_. Third, it can run in experimental mode |
| 4958 |
testing a local repository using the TriBITS-defined `make dashboard`_ target. |
| 4959 |
|
| 4960 |
.. ToDo: Discuss the behavior of Continuous, Nightly, and Experimental |
| 4961 |
.. CTest/CDash tracks and what that means in terms of sending out CDash error |
| 4962 |
.. emails and allowing for multiple builds of the same name in a single test |
| 4963 |
.. day. |
| 4964 |
|
| 4965 |
.. ToDo: Document CTEST_TEST_TIMEOUT and DART_TESTING_TIMEOUT and how these |
| 4966 |
.. interact. |
| 4967 |
|
| 4968 |
.. ToDo: Import and format the contents of tribits/ctest/README related to |
| 4969 |
.. this topic. |
| 4970 |
|
| 4971 |
|
| 4972 |
CTest/CDash Nightly Testing |
| 4973 |
+++++++++++++++++++++++++++ |
| 4974 |
|
| 4975 |
When a TriBITS CTest script using `tribits_ctest_driver()`_ is run in |
| 4976 |
"Nightly" testing mode, it builds the project from scratch package-by-package |
| 4977 |
and submits results to the TriBITS project's CDash project on the designated |
| 4978 |
CDash server. |
| 4979 |
|
| 4980 |
.. ToDo: Give example of the package-by-package build. |
| 4981 |
|
| 4982 |
CTest/CDash CI Server |
| 4983 |
+++++++++++++++++++++ |
| 4984 |
|
| 4985 |
When a TriBITS ctest driver script is used in continuous integration (CI) |
| 4986 |
mode, it starts every day with a clean from-scratch build and then performs |
| 4987 |
incremental rebuilds as new commits are pulled from the master branch in the |
| 4988 |
main repository(s). In this mode, a continuous loop is performed after the |
| 4989 |
initial baseline build constantly pulling commits from the master git |
| 4990 |
repository(s). If any package changes are detected (looking at git file |
| 4991 |
diffs), then the tests and examples for those packages and all downstream |
| 4992 |
packages are enabled and run using a reconfigure/rebuild. Since all of the |
| 4993 |
upstream package libraries are already built, this rebuild and retest can take |
| 4994 |
place in a fraction of the time of a complete from-scratch build and test of |
| 4995 |
the project. |
| 4996 |
|
| 4997 |
.. ToDo Give some examples of the behavior of this process for the |
| 4998 |
.. ReducedMockTrilins project. We could show example output from the script |
| 4999 |
.. that shows how it behaves. |
| 5000 |
|
| 5001 |
.. ToDo: Discuss the usage of ${REPOSITORY_NAME}_NO_IMPLICIT_PACKAGE_ENABLE |
| 5002 |
.. and ${REPOSITORY_NAME}_NO_IMPLICIT_PACKAGE_ENABLE_EXCEPT to allow the CI |
| 5003 |
.. server to directly processing and running tests for upstream packages which |
| 5004 |
.. are not primary packages in the TriBITS meta-project. |
| 5005 |
|
| 5006 |
.. ToDo: Mention that failing packages are always explicitly processed even if |
| 5007 |
.. the current CI iteration does not pull any changes. |
| 5008 |
|
| 5009 |
.. ToDo: Give some examples from ReducedMockTrilinos of how file changes |
| 5010 |
.. result in package enables and rebuilds. |
| 5011 |
|
| 5012 |
|
| 5013 |
TriBITS CDash Customizations |
| 5014 |
---------------------------- |
| 5015 |
|
| 5016 |
CDash is not currently designed to accommodate multi-package, multi-repository |
| 5017 |
VC projects in the way they are supported by TriBITS. However, CDash provides |
| 5018 |
some ability to customize a CDash project and submits to address missing |
| 5019 |
features. Each TriBITS package is given a CTest/CDash "Label" with the name |
| 5020 |
of the TriBITS package. CDash will then aggregate the different package |
| 5021 |
configure/build/test runs for each package into aggregated "builds". The |
| 5022 |
commits pulled for each of the extra VC repos listed in the |
| 5023 |
`<projectDir>/cmake/ExtraRepositoriesList.cmake`_ file are shown in an |
| 5024 |
uploaded CDash "Notes" file for each TriBITS package configure/build/test |
| 5025 |
submit. This uploaded "Notes" file also contains a cleaned up version of the |
| 5026 |
CMakeCache.txt file as well as a copy of the ``ctest -s`` script that ran the |
| 5027 |
case. |
| 5028 |
|
| 5029 |
CDash offers numerous features such as the ability to construct a number of |
| 5030 |
different types of queries and is extremely helpful in using past test data. |
| 5031 |
|
| 5032 |
.. ToDo: Describe more on what CDash provides and how TriBITS uses it. This |
| 5033 |
.. could be a *long* section with a lot of screen shots. |
| 5034 |
|
| 5035 |
|
| 5036 |
CDash regression email addresses |
| 5037 |
++++++++++++++++++++++++++++++++ |
| 5038 |
|
| 5039 |
Every TriBITS Package has a regression email address associated with it that |
| 5040 |
gets uploaded to a CDash project on a CDash server that is used to determine |
| 5041 |
what email address to use when a package has configure, build, or test |
| 5042 |
failures. Because of the complex organizational nature of different projects |
| 5043 |
and different integration models, a single static email address for a given |
| 5044 |
package in every project build is not practical. |
| 5045 |
|
| 5046 |
The TriBITS system allows for a package's regression email address to be |
| 5047 |
specified in the following order of precedence: |
| 5048 |
|
| 5049 |
.. _${REPOSITORY_NAME}_REPOSITORY_OVERRIDE_PACKAGE_EMAIL_LIST: |
| 5050 |
|
| 5051 |
1) **${REPOSITORY_NAME}_REPOSITORY_OVERRIDE_PACKAGE_EMAIL_LIST** (typically |
| 5052 |
defined in `<projectDir>/cmake/ProjectDependenciesSetup.cmake`_): Defines a |
| 5053 |
single email address for all packages for the repository |
| 5054 |
``${REPOSITORY_NAME}`` and overrides all other package email regression |
| 5055 |
specification variables. This is typically used by a meta-project to redefine |
| 5056 |
the regression email addresses for all of the packages in an externally |
| 5057 |
developed repository. |
| 5058 |
|
| 5059 |
2) **REGRESSION_EMAIL_LIST** (defined in |
| 5060 |
`<packageDir>/cmake/Dependencies.cmake`_): Package-specific email address |
| 5061 |
specified in the package's ``Dependencies.cmake`` file using |
| 5062 |
`tribits_package_define_dependencies()`_. |
| 5063 |
|
| 5064 |
.. _${REPOSITORY_NAME}_REPOSITORY_EMAIL_URL_ADDRESS_BASE: |
| 5065 |
|
| 5066 |
3) **${REPOSITORY_NAME}_REPOSITORY_EMAIL_URL_ADDRESS_BASE** (set in |
| 5067 |
`<repoDir>/cmake/RepositoryDependenciesSetup.cmake`_): A base email address |
| 5068 |
specified at the Repository level creating package-specific email addresses |
| 5069 |
(e.g. ``<lower-case-package-name>[email protected]``, where |
| 5070 |
``${REPOSITORY_NAME}_REPOSITORY_EMAIL_URL_ADDRESS_BASE=some.repo.gov``). |
| 5071 |
This variable is used, for example, by the Trilinos project to provide |
| 5072 |
automatic regression email addresses for packages. |
| 5073 |
|
| 5074 |
.. _${REPOSITORY_NAME}_REPOSITORY_MASTER_EMAIL_ADDRESS: |
| 5075 |
|
| 5076 |
4) **${REPOSITORY_NAME}_REPOSITORY_MASTER_EMAIL_ADDRESS** (set in |
| 5077 |
`<repoDir>/cmake/RepositoryDependenciesSetup.cmake`_): A single email address |
| 5078 |
for all packages specified at the Repository level |
| 5079 |
(e.g. ``[email protected]``). This variable is used for |
| 5080 |
smaller repositories with smaller development groups who just want all |
| 5081 |
regression emails for the repository's packages going to a single email |
| 5082 |
address. This reduces the overhead of managing a bunch of individual package |
| 5083 |
email addresses but at the expense of spamming too many people with CDash |
| 5084 |
failure emails. |
| 5085 |
|
| 5086 |
.. _${PROJECT_NAME}_PROJECT_EMAIL_URL_ADDRESS_BASE: |
| 5087 |
|
| 5088 |
5) **${PROJECT_NAME}_PROJECT_EMAIL_URL_ADDRESS_BASE** (set in |
| 5089 |
`<projectDir>/cmake/ProjectDependenciesSetup.cmake`_): A base email address |
| 5090 |
specified at the Project level creating package-specific email addresses |
| 5091 |
(e.g. ``<lower-case-package-name>[email protected]``, where |
| 5092 |
``${PROJECT_NAME}_PROJECT_EMAIL_URL_ADDRESS_BASE=some.project.gov``). If not |
| 5093 |
already set, this variable will be set to |
| 5094 |
``${REPOSITORY_NAME}_REPOSITORY_EMAIL_URL_ADDRESS_BASE`` for the first |
| 5095 |
repository processed that has this set. This behavior is used, for example by |
| 5096 |
the Trilinos project to automatically assign email addresses for add-on |
| 5097 |
packages and was added to maintain backward compatibility. |
| 5098 |
|
| 5099 |
.. _${PROJECT_NAME}_PROJECT_MASTER_EMAIL_ADDRESS: |
| 5100 |
|
| 5101 |
6) **${PROJECT_NAME}_PROJECT_MASTER_EMAIL_ADDRESS** (set in |
| 5102 |
`<projectDir>/cmake/ProjectDependenciesSetup.cmake`_): A single default email |
| 5103 |
address for all packages specified at the Project level |
| 5104 |
(e.g. ``[email protected]``). If not already set, this |
| 5105 |
variable will be set to |
| 5106 |
``${REPOSITORY_NAME}_REPOSITORY_MASTER_EMAIL_ADDRESS`` for the first |
| 5107 |
repository processed that has this set. Every meta-project should set this |
| 5108 |
variable so that it will be the default email address for any new package |
| 5109 |
added. |
| 5110 |
|
| 5111 |
WARNING: If any of the email lists or URL string variables listed above are |
| 5112 |
set to ``"OFF"`` or ``"FALSE"`` (or some other value that CMake interprets as |
| 5113 |
false, see `CMake Language Overview and Gotchas`_) then the variables are |
| 5114 |
treated as empty and not set. |
| 5115 |
|
| 5116 |
If a TriBITS project does not use CDash, then no email address needs to be |
| 5117 |
assigned to packages at all (and therefore none of the above variables need be |
| 5118 |
set). |
| 5119 |
|
| 5120 |
As a general rule, repository-level settings override project-level settings |
| 5121 |
and package-level settings override both. Also, a project can redefine a |
| 5122 |
repository's regression email list settings by resetting the variables in the |
| 5123 |
project's `<projectDir>/cmake/ProjectDependenciesSetup.cmake`_ file. |
| 5124 |
|
| 5125 |
All of the email dependency management logic must be accessible by just running |
| 5126 |
the macro:: |
| 5127 |
|
| 5128 |
tribits_read_all_project_deps_files_create_deps_graph() |
| 5129 |
|
| 5130 |
The above email address configuration variables are read from the Repository |
| 5131 |
and Project files `<repoDir>/cmake/RepositoryDependenciesSetup.cmake`_ and |
| 5132 |
`<projectDir>/cmake/ProjectDependenciesSetup.cmake`_, respectively. The |
| 5133 |
``RepositoryDependenciesSetup.cmake`` files are read first in the specified |
| 5134 |
repository order followed up by reading the ``ProjectDependenciesSetup.cmake`` |
| 5135 |
file. In this way, the project can override any of the repository settings. |
| 5136 |
|
| 5137 |
In review, the precedence order for how regression email addresses are |
| 5138 |
selected for a given package is: |
| 5139 |
|
| 5140 |
1) Package-specific email list is selected if defined (unless an override is |
| 5141 |
in place). |
| 5142 |
|
| 5143 |
2) Repository-level option is selected over a project-level option. |
| 5144 |
|
| 5145 |
3) Default email form with repository or project address base is selected over |
| 5146 |
single repository or project email address. |
| 5147 |
|
| 5148 |
4) If none of the above are selected, then no email address is assigned for a |
| 5149 |
given package. |
| 5150 |
|
| 5151 |
What the above setup does is it results in the TriBITS system (in the |
| 5152 |
`tribits_ctest_driver()`_ function called in a ``ctest -S`` script) creating a |
| 5153 |
file called ``CDashSubprojectDependencies.xml`` (which contains the list of |
| 5154 |
TriBITS packages, which CDash calls "subprojects", and email address for each |
| 5155 |
package to send regression emails to) and that file gets sent to the CDash |
| 5156 |
server. CDash then takes this file and creates, or updates, a set of CDash |
| 5157 |
users (same name and password as the email list address) and sets up a mapping |
| 5158 |
of Labels (which are used for TriBITS package names) to CDash user emails |
| 5159 |
addresses. CDash is automatically set up to process this XML file and create |
| 5160 |
and update CDash users. There are several consequences of this implementation |
| 5161 |
of which project maintainers need to be aware. |
| 5162 |
|
| 5163 |
First, **one should not list the email address for a CDash user account |
| 5164 |
already on CDash**. This is because labels will be added for the TriBITS |
| 5165 |
packages that this email address is associated with and CDash emails will not |
| 5166 |
be sent out for any other TriBITS packages, no matter what setting that CDash |
| 5167 |
user account has. Therefore, one should only list email addresses as CDash |
| 5168 |
regression email lists that are not already CDash user accounts and wish to be |
| 5169 |
maintained separately. For example, if there is an email list that one wants |
| 5170 |
to have CDash emails sent to but is already a CDash user account, then one can |
| 5171 |
create another email list (e.g. using Mailman) which can then be registered |
| 5172 |
with the TriBITS packages in the ``CDashSubprojectDependencies.xml`` file and |
| 5173 |
then that new email list forward email to the target email list. |
| 5174 |
|
| 5175 |
Second, the CDash implementation currently is not set up to remove labels from |
| 5176 |
existing users when an email address is disassociated with a TriBITS package |
| 5177 |
in the ``CDashSubprojectDependencies.xml`` file. Therefore, **if one changes |
| 5178 |
a TriBITS package's CDash regression email address then one needs to manually |
| 5179 |
remove the associated labels from the old email address**. CDash will not |
| 5180 |
remove them automatically. Otherwise, email will continue to be sent to that |
| 5181 |
email address for that package. |
| 5182 |
|
| 5183 |
Therefore, **to change the mapping of CDash regression email addresses to |
| 5184 |
TriBITS packages, one must perform the following actions**: |
| 5185 |
|
| 5186 |
1) Change the TriBITS CMake files as described above that will result in the |
| 5187 |
desired email addresses in the ``CDashSubprojectDependencies.xml`` |
| 5188 |
file. One can debug this by generating the file |
| 5189 |
`<Project>PackageDependencies.xml`_ by using the cmake -P script |
| 5190 |
`TribitsDumpDepsXmlScript.cmake`_. |
| 5191 |
|
| 5192 |
2) Log onto the CDash server using an administrator account and then remove |
| 5193 |
the auto-generated account for the CDash user email address for which |
| 5194 |
labels are being removed (i.e. no longer associated with a TriBITS |
| 5195 |
package). This is needed since CDash seems to be unable to remove labels |
| 5196 |
from an existing CDash user (however this might be fixed in a current |
| 5197 |
version of CDash). |
| 5198 |
|
| 5199 |
3) The next time a CDash submit is performed by a CTest driver script calling |
| 5200 |
`tribits_ctest_driver()`_, the CDash user associated with the mail list |
| 5201 |
with labels being removed will get automatically recreated with the right |
| 5202 |
list of labels (according to the current |
| 5203 |
``CDashSubprojectDependencies.xml`` file). Also, any new CDash users for |
| 5204 |
new email addresses will be created. |
| 5205 |
|
| 5206 |
Hopefully that should be enough information to manage the mapping of CDash |
| 5207 |
regression email lists to TriBITS packages for single and multi-repository |
| 5208 |
TriBITS projects. |
| 5209 |
|
| 5210 |
|
| 5211 |
Multi-Repository Support |
| 5212 |
======================== |
| 5213 |
|
| 5214 |
TriBITS has built-in support for projects involving multiple `TriBITS |
| 5215 |
Repositories`_ which contain multiple `TriBITS Packages`_ (see `How to set up |
| 5216 |
multi-repository support`_). The basic configuration, build, and test of such |
| 5217 |
projects requires only raw CMake/CTest, just like any other CMake project (see |
| 5218 |
`TriBITS System Project Dependencies`_). Every TriBITS project automatically |
| 5219 |
supports tacking on add-on TriBITS packages and external packages/TPLs through |
| 5220 |
the `${PROJECT_NAME}_EXTRA_REPOSITORIES`_ cmake cache variable as described in |
| 5221 |
`Enabling extra repositories with add-on packages`_. In addition, a TriBITS |
| 5222 |
project can be set up to pull in other TriBITS Repositories using the |
| 5223 |
`<projectDir>/cmake/ExtraRepositoriesList.cmake`_ file. A special form of |
| 5224 |
this type of project is a `TriBITS Meta-Project`_ that contains no native |
| 5225 |
packages or TPLs of its own. The ability to create meta-projects out of |
| 5226 |
individual TriBITS repositories allows TriBITS to be used to provide |
| 5227 |
coordinated builds (or meta-builds) of large aggregations of software. |
| 5228 |
|
| 5229 |
To help set up a full-featured development environment (i.e. not just the |
| 5230 |
basic configure, build, test, and install) for TriBITS projects with multiple |
| 5231 |
repositories, TriBITS provides some extra development tools implemented using |
| 5232 |
Python which are provided in the "extended" parts of TriBITS (see |
| 5233 |
`TriBITS/tribits/ Directory Contents`_). The primary tools supporting |
| 5234 |
multi-repository projects are the Python tools `clone_extra_repos.py`_, |
| 5235 |
`gitdist`_, and `checkin-test.py`_. |
| 5236 |
|
| 5237 |
To demonstrate, consider the TriBITS meta-project with the following |
| 5238 |
``ExtraRepositoriesList.cmake`` file:: |
| 5239 |
|
| 5240 |
tribits_project_define_extra_repositories( |
| 5241 |
ExtraRepo1 "" GIT [email protected]:ExtraRepo1 "" Continuous |
| 5242 |
ExtraRepo2 "ExtraRepo1/ExtraRepos2" GIT [email protected]:ExtraRepo2 |
| 5243 |
NOPACKAGES Continuous |
| 5244 |
ExtraRepo3 "" GIT [email protected]:ExtraRepo3 "" Nightly |
| 5245 |
) |
| 5246 |
|
| 5247 |
Once cloned, the directories would be laid out as:: |
| 5248 |
|
| 5249 |
MetaProject/ |
| 5250 |
.git/ |
| 5251 |
.gitignore |
| 5252 |
ExtraRepo1/ |
| 5253 |
.git/ |
| 5254 |
ExrraRepo2/ |
| 5255 |
.git/ |
| 5256 |
ExtraRepo3/ |
| 5257 |
.git/ |
| 5258 |
|
| 5259 |
.. _clone_extra_repos.py: |
| 5260 |
|
| 5261 |
The tool **clone_extra_repos.py** is used to clone the extra repositories for |
| 5262 |
a multi-repositories TriBITS project. It reads the repository URLs and |
| 5263 |
destination directories from the file |
| 5264 |
`<projectDir>/cmake/ExtraRepositoriesList.cmake`_ and does the clones. For |
| 5265 |
example, to clone all the repos for the ``MetaProject`` project, one would use |
| 5266 |
the commands:: |
| 5267 |
|
| 5268 |
$ git clone [email protected]:MetaProject |
| 5269 |
$ cd MetaProject/ |
| 5270 |
$ ./cmake/tribits/ci_support/clone_extra_repos.py |
| 5271 |
|
| 5272 |
which produces the output like:: |
| 5273 |
|
| 5274 |
... |
| 5275 |
|
| 5276 |
*** |
| 5277 |
*** Clone the selected extra repos: |
| 5278 |
*** |
| 5279 |
|
| 5280 |
Cloning repo ExtraRepo1 ... |
| 5281 |
|
| 5282 |
Running: git clone [email protected]:ExtraRepo1 ExtraRepo1 |
| 5283 |
|
| 5284 |
Cloning repo ExtraRepo2 ... |
| 5285 |
|
| 5286 |
Running: git clone [email protected]:ExtraRepo2 ExtraRepo1/ExtraRepo2 |
| 5287 |
|
| 5288 |
Cloning repo ExtraRepo3 ... |
| 5289 |
|
| 5290 |
Running: git clone [email protected]:ExtraRepo3 ExtraRepo3 |
| 5291 |
|
| 5292 |
See `clone_extra_repos.py --help`_ for more details. |
| 5293 |
|
| 5294 |
.. _gitdist: |
| 5295 |
|
| 5296 |
Once cloned, one needs to work with the multiple repositories to perform basic |
| 5297 |
VC operations. For this, TriBITS provides the tool **gitdist** which is a |
| 5298 |
simple stand-alone Python script that distributes a git command across a set |
| 5299 |
of git repos. This tool is not specific to TriBITS but it is very useful for |
| 5300 |
dealing with TriBITS projects with multiple repositories. It only requires a |
| 5301 |
local base git repo and a set of zero or more git repos cloned under it. |
| 5302 |
|
| 5303 |
To use ``gitdist`` with this aggregate meta-project, one would first set up |
| 5304 |
the file ``MetaProject/.gitdist`` (or a version-controlled |
| 5305 |
``MetaProject.gitdist.default`` file) which would contain the lines:: |
| 5306 |
|
| 5307 |
ExtraRepo1 |
| 5308 |
ExtraRepo1/ExtraRepo2 |
| 5309 |
ExtraRepo3 |
| 5310 |
|
| 5311 |
and one would set up the tracked ignore file ``MetaProject/.gitignore`` which |
| 5312 |
contains the lines:: |
| 5313 |
|
| 5314 |
/ExtraRepo1/ |
| 5315 |
/ExtraRepo1/ExtraRepo2/ |
| 5316 |
/ExtraRepo3/ |
| 5317 |
|
| 5318 |
To use ``gitdist``, one would put ``gitdist`` into their path and also set up |
| 5319 |
the command-line shell aliases ``gitdist-status`` and ``gitdist-mod`` (see |
| 5320 |
`gitdist --dist-help=aliases`_). |
| 5321 |
|
| 5322 |
Some of the aggregate commands that one would typically run under the base |
| 5323 |
``MetaProject/`` directory are:: |
| 5324 |
|
| 5325 |
# See status of all repos at once |
| 5326 |
gitdist-status |
| 5327 |
|
| 5328 |
# Pull updates to all |
| 5329 |
gitdist pull |
| 5330 |
|
| 5331 |
# Push local commits to tracking branches |
| 5332 |
gitdist push |
| 5333 |
|
| 5334 |
The tool ``gitdist`` is provided under TriBITS directory:: |
| 5335 |
|
| 5336 |
cmake/tribits/python_utils/gitidst |
| 5337 |
|
| 5338 |
and can be installed by the `install_devtools.py`_ tool (see `TriBITS |
| 5339 |
Development Toolset`_). See `gitdist documentation`_ for more details. |
| 5340 |
|
| 5341 |
For projects with a standard set of extra repositories defined in the |
| 5342 |
`<projectDir>/cmake/ExtraRepositoriesList.cmake`_ file, the |
| 5343 |
``checkin-test.py`` tool only requires passing in the option |
| 5344 |
``--extra-repos-file=project`` and ``--extra-repos-type=Continuous`` (or |
| 5345 |
``Nightly``, see `Repository Test Classification`_) and it will automatically |
| 5346 |
perform all of the various actions for all of the selected repositories. See |
| 5347 |
`checkin-test.py`_ and `checkin-test.py --help`_ for more details. |
| 5348 |
|
| 5349 |
To keep track of compatible versions of the git repos, TriBITS provides |
| 5350 |
support for a ``<Project>RepoVersion.txt`` file. Any TriBITS project can |
| 5351 |
generate this file automatically by setting the option |
| 5352 |
`${PROJECT_NAME}_GENERATE_REPO_VERSION_FILE`_. For the above example |
| 5353 |
``MetaProject``, this file looks like:: |
| 5354 |
|
| 5355 |
*** Base Git Repo: MetaProject |
| 5356 |
e102e27 [Mon Sep 23 11:34:59 2013 -0400] <[email protected]> |
| 5357 |
First summary message |
| 5358 |
*** Git Repo: ExtraRepo1 |
| 5359 |
b894b9c [Fri Aug 30 09:55:07 2013 -0400] <[email protected]> |
| 5360 |
Second summary message |
| 5361 |
*** Git Repo: ExtraRepo1/ExtraRepo2 |
| 5362 |
97cf1ac [Thu Dec 1 23:34:06 2012 -0500] <author2someurl.com> |
| 5363 |
Third summary message |
| 5364 |
*** Git Repo: ExtraRepo3 |
| 5365 |
cd4a3af [Mon Mar 9 19:39:06 2013 -0400] <author3someurl.com> |
| 5366 |
Fourth summary message |
| 5367 |
|
| 5368 |
This file gets created in the build directory, gets echoed in the configure |
| 5369 |
output, gets installed into the install directory, and get added to the source |
| 5370 |
distributions tarball. It also gets pushed up to CDash for all automated |
| 5371 |
builds. The tool `gitdist`_ can then use this file to checkout and tag |
| 5372 |
compatible versions, difference two versions of the meta-project, etc. (see |
| 5373 |
`gitdist documentation`_ for more details on git operations). |
| 5374 |
|
| 5375 |
The TriBITS approach to managing multiple VC repos described above works well |
| 5376 |
for around 20 or 30 VC repos but is likely not a good solution for many more |
| 5377 |
git repos. For larger numbers of VC repos, one should consider nested |
| 5378 |
integration creating snapshot git repos (e.g. using the tool |
| 5379 |
`snapshot-dir.py`_) that aggregate several related repositories into a single |
| 5380 |
git repo. Another approach might be to use git submodules. (However, note |
| 5381 |
that the TriBITS tools and processes described here are **not** currently set |
| 5382 |
up to support aggregate VC repos that use git submodules.) The design |
| 5383 |
decision with TriBITS was to explicitly handle the different git VC repos by |
| 5384 |
listing them in the `<projectDir>/cmake/ExtraRepositoriesList.cmake`_ file and |
| 5385 |
then using the simple, easy to understand, tools `clone_extra_repos.py`_ and |
| 5386 |
`gitdist`_. There are advantages and disadvantages to explicitly handling the |
| 5387 |
different git repos as is currently employed by the TriBITS software |
| 5388 |
development tools verses using git submodules. It is possible that TriBITS |
| 5389 |
will add support for aggregate git repos using git submodules in the future |
| 5390 |
but only if there are important projects that choose to use them. The |
| 5391 |
discussion of these various approaches and strategies to dealing with |
| 5392 |
aggregate repos is beyond the scope of this document. |
| 5393 |
|
| 5394 |
|
| 5395 |
Development Workflows |
| 5396 |
====================== |
| 5397 |
|
| 5398 |
In this section, the typical development workflows for a TriBITS project are |
| 5399 |
described. First, the `Basic Development Workflow`_ for a single-repository |
| 5400 |
TriBITS project is described. This is followed up with a slightly more |
| 5401 |
complex `Multi-Repository Development Workflow`_. |
| 5402 |
|
| 5403 |
|
| 5404 |
Basic Development Workflow |
| 5405 |
-------------------------- |
| 5406 |
|
| 5407 |
The basic development workflow of a TriBITS project is not much different than |
| 5408 |
with any other CMake project that uses CTest to define and run tests. One |
| 5409 |
pulls updates from the master VC repo then configures with ``cmake``, and |
| 5410 |
iteratively builds, runs tests, adds files, changes files, does a final test, |
| 5411 |
then pushes updates. The major difference is that a well constructed |
| 5412 |
development process will use the `checkin-test.py`_ tool to test and push all |
| 5413 |
changes that affect the build or the tests. The basic steps in configuring, |
| 5414 |
building, running tests, etc., are given in the project's |
| 5415 |
`<Project>BuildReference`_. file (see `Project-Specific Build Reference`_). |
| 5416 |
|
| 5417 |
Multi-Repository Development Workflow |
| 5418 |
------------------------------------- |
| 5419 |
|
| 5420 |
The development workflow for a project with multiple VC repos is very similar |
| 5421 |
to a project with just a single VC repo if the project provides a standard |
| 5422 |
`<projectDir>/cmake/ExtraRepositoriesList.cmake`_ file. The major difference |
| 5423 |
is in making changes, creating commits, etc. The `gitdist`_ tool makes these |
| 5424 |
steps easier and has been shown to work fairly well for up to 20 extra VC |
| 5425 |
repos (as used in the CASL VERA project). The `checkin-test.py`_ tool |
| 5426 |
automatically handles all of the details of pulling, diffing, pushing etc. to |
| 5427 |
all the VC repos. |
| 5428 |
|
| 5429 |
|
| 5430 |
|
| 5431 |
.. ToDo: Discuss usage of 'gitdist' and the repo clone script. |
| 5432 |
|
| 5433 |
|
| 5434 |
Howtos |
| 5435 |
====== |
| 5436 |
|
| 5437 |
This section provides short, succinct lists of the steps to accomplish a few |
| 5438 |
common tasks. Extra details are referenced. |
| 5439 |
|
| 5440 |
|
| 5441 |
How to add a new TriBITS Package |
| 5442 |
-------------------------------- |
| 5443 |
|
| 5444 |
To add a new TriBITS package, it is recommended to take the template from one |
| 5445 |
of the `TribitsExampleProject`_ packages that most closely fits the needs of |
| 5446 |
the new package and modify it for the new package. For example, the files for |
| 5447 |
the ``SimpleCxx`` package can be copied one at a time and modified for the new |
| 5448 |
package. |
| 5449 |
|
| 5450 |
To add a new TriBITS package (with no subpackages), do the following: |
| 5451 |
|
| 5452 |
1) Chose a name ``<packageName>`` for the new package and which TriBITS |
| 5453 |
repository (``<repoDir>``) to put the package into. **WARNING!** The |
| 5454 |
chosen name ``<packageName>`` must be unique across all TriBITS |
| 5455 |
repositories (see `Globally unique TriBITS package names`_). |
| 5456 |
|
| 5457 |
2) Create the directory ``<repoDir>/<packageDir>/`` for the new package and |
| 5458 |
put in skeleton files for `<packageDir>/cmake/Dependencies.cmake`_ and |
| 5459 |
`<packageDir>/CMakeLists.txt`_. Set the desired upstream TPL and package |
| 5460 |
dependencies in the new ``Dependencies.cmake`` file but initially comment |
| 5461 |
out everything in the ``CMakeLists.txt`` file except for the |
| 5462 |
`tribits_package()`_ and `tribits_package_postprocess()`_ commands. |
| 5463 |
|
| 5464 |
3) Add a row for the new package to the `<repoDir>/PackagesList.cmake`_ file |
| 5465 |
after all of its upstream dependent packages. If a mistake is made and it |
| 5466 |
is listed before one of its upstream dependent packages, the TriBITS CMake |
| 5467 |
code will catch this and issue an error. |
| 5468 |
|
| 5469 |
4) Configure the TriBITS project enabling the new empty package |
| 5470 |
``<packageName>``. This will enable the listed dependencies. |
| 5471 |
|
| 5472 |
5) Incrementally fill in the package's ``CMakeLists.txt`` files defining |
| 5473 |
libraries, executables, tests and examples. The project should be built |
| 5474 |
and tests run as new pieces are added. |
| 5475 |
|
| 5476 |
Once the new package is defined, downstream packages can define |
| 5477 |
dependencies on this new package. |
| 5478 |
|
| 5479 |
.. ToDo: Expand on the above bullets a lot! |
| 5480 |
|
| 5481 |
|
| 5482 |
How to add a new TriBITS Package with Subpackages |
| 5483 |
------------------------------------------------- |
| 5484 |
|
| 5485 |
Adding a new package with subpackages is similar to adding a new regular |
| 5486 |
package described in `How to add a new TriBITS Package`_. Again, it is |
| 5487 |
recommended that one copies an example package from `TribitsExampleProject`_. |
| 5488 |
For example, one could copy files and directories from the example package |
| 5489 |
``WithSubpackages``. |
| 5490 |
|
| 5491 |
To add a new TriBITS package with packages, do the following: |
| 5492 |
|
| 5493 |
1) Chose a name ``<packageName>`` for the new package and which TriBITS |
| 5494 |
repository (``<repoDir>``) to put the package into. **WARNING!** The |
| 5495 |
chosen name ``<packageName>`` must be unique across all TriBITS |
| 5496 |
repositories (see `Globally unique TriBITS package names`_). |
| 5497 |
|
| 5498 |
2) Create the directory ``<repoDir>/<packageDir>/`` for the new package and |
| 5499 |
put in skeleton files for `<packageDir>/cmake/Dependencies.cmake`_ and |
| 5500 |
`<packageDir>/CMakeLists.txt`_. Initially don't define any subpackages and |
| 5501 |
comment out everything in the ``CMakeLists.txt`` file except for the |
| 5502 |
`tribits_package()`_ and `tribits_package_postprocess()`_ commands. |
| 5503 |
|
| 5504 |
3) Add a new row for the new package to the `<repoDir>/PackagesList.cmake`_ |
| 5505 |
file after all of the upstream dependencies of its to-be-defined |
| 5506 |
subpackages. |
| 5507 |
|
| 5508 |
4) Configure the TriBITS project enabling the new empty package |
| 5509 |
``<packageName>``. |
| 5510 |
|
| 5511 |
5) Incrementally add the subpackages as described in `How to add a new TriBITS |
| 5512 |
Subpackage`_, filling out the various ``CMakeLists.txt`` files defining |
| 5513 |
libraries, executables, tests and examples. |
| 5514 |
|
| 5515 |
Once the new packages are defined, downstream packages can define |
| 5516 |
dependencies on these. |
| 5517 |
|
| 5518 |
|
| 5519 |
How to add a new TriBITS Subpackage |
| 5520 |
----------------------------------- |
| 5521 |
|
| 5522 |
Given an existing top-level TriBITS package that is already broken down into |
| 5523 |
subpackages (see `How to add a new TriBITS Package with Subpackages`_), adding |
| 5524 |
a new subpackage does not require changing any project-level or |
| 5525 |
repository-level files. One only needs to add the declaration for the new |
| 5526 |
subpackages in its parent's `<packageDir>/cmake/Dependencies.cmake`_ file then |
| 5527 |
fill out the pieces of the new subpackage defined in the section `TriBITS |
| 5528 |
Subpackage Core Files`_. It is recommended to copy files from one of the |
| 5529 |
`TribitsExampleProject`_ subpackages in the ``WithSubpackages`` |
| 5530 |
package. |
| 5531 |
|
| 5532 |
To add a new TriBITS subpackage to a top-level package that already has |
| 5533 |
subpackages, do the following: |
| 5534 |
|
| 5535 |
1) Chose a name ``<spkgName>`` for the new subpackage which only has to be |
| 5536 |
different than the other subpackages in the parent package. This name gets |
| 5537 |
appended to the parent package's name ``<packageName>`` to form the package |
| 5538 |
name ``<packageName><spkgName>``. |
| 5539 |
|
| 5540 |
2) Create the directory ``<packageDir><spkgDir>/`` for the new package and put |
| 5541 |
in skeleton files for `<packageDir>/<spkgDir>/cmake/Dependencies.cmake`_ |
| 5542 |
and `<packageDir>/<spkgDir>/CMakeLists.txt`_. Set the desired upstream TPL |
| 5543 |
and package dependencies in the new ``Dependencies.cmake`` file but |
| 5544 |
initially comment out everything in the ``CMakeLists.txt`` file except for |
| 5545 |
the `tribits_subpackage()`_ and `tribits_subpackage_postprocess()`_ |
| 5546 |
commands. |
| 5547 |
|
| 5548 |
3) Add a row for the new subpackage to the argument |
| 5549 |
`SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS`_ in the macro call |
| 5550 |
`tribits_package_define_dependencies()`_ in the parent package's |
| 5551 |
`<packageDir>/cmake/Dependencies.cmake`_ file after all of its upstream |
| 5552 |
dependent subpackages. If a mistake is made and it is listed before one of |
| 5553 |
its upstream dependent subpackages, the TriBITS CMake code will catch this |
| 5554 |
and issue an error. |
| 5555 |
|
| 5556 |
4) Configure the TriBITS project enabling the new empty package |
| 5557 |
``<packageName><spkgName>``. This will enable the listed dependencies. |
| 5558 |
|
| 5559 |
5) Incrementally fill in the subpackage's ``CMakeLists.txt`` files defining |
| 5560 |
libraries, executables, tests and examples. The project should be built |
| 5561 |
and tests run as new pieces are added. |
| 5562 |
|
| 5563 |
|
| 5564 |
.. _How to add a new TriBITS TPL: |
| 5565 |
|
| 5566 |
How to add a new TriBITS external package/TPL |
| 5567 |
--------------------------------------------- |
| 5568 |
|
| 5569 |
In order for a TriBITS package to define a dependency on a new `TriBITS |
| 5570 |
External Package/TPL`_ (i.e. a TPL that has not already been declared in the |
| 5571 |
current repo's or an upstream repo's `<repoDir>/TPLsList.cmake`_ file), one |
| 5572 |
must add and modify a few repository-level files in addition to modifying |
| 5573 |
files within the TriBITS packages that use the new external package/TPL. |
| 5574 |
|
| 5575 |
To add a new TriBITS TPL, do the following: |
| 5576 |
|
| 5577 |
1) **Chose a name <tplName>** for the new TPL (must be globally unique across |
| 5578 |
all TriBITS repos. (See `Globally unique TriBITS TPL names`_.) |
| 5579 |
|
| 5580 |
2) **Choose the subdirectory <tplDefsDir> for the new TPL files**. These |
| 5581 |
files are usually placed under the TriBITS repository directory |
| 5582 |
``<repoDir>/`` (e.g. ``<repoDir>/cmake/tpls/``) where the first downstream |
| 5583 |
dependent package is defined or can be under a TriBITS package directory |
| 5584 |
``<packageDir>/`` (e.g. ``<packageDir>/cmake/tpls/``) if only one package |
| 5585 |
is using that TPL. |
| 5586 |
|
| 5587 |
3) **Create the FindTPL<tplName>.cmake file** (or some other name, see |
| 5588 |
`<tplName>_FINDMOD`_) under ``<tplDefsDir>/``. (See `Creating the |
| 5589 |
FindTPL<tplName>.cmake file`_.) However, if the external package/TPL is a |
| 5590 |
`TriBITS-compliant external package`_ this file is not needed and is |
| 5591 |
ignored. |
| 5592 |
|
| 5593 |
4) **[Optional] Create the FindTPL<tplName>Dependencies.cmake file** in the |
| 5594 |
same directory as the ``FindTPL<tplName>.cmake`` file, ``<tplDefsDir>/``. |
| 5595 |
(See `FindTPL<tplName>Dependencies.cmake`_.) NOTE: This file is need for a |
| 5596 |
`TriBITS-compliant external package`_ if it has upstream dependent external |
| 5597 |
packages/TPLs (where the file ``FindTPL<tplName>.cmake`` is not needed). |
| 5598 |
|
| 5599 |
5) **Add a row to the <repoDir>/TPLsList.cmake file** for the new external |
| 5600 |
package/TPL after any upstream TPLs that this new TPL may depend on. NOTE: |
| 5601 |
For a `TriBITS-compliant external package`_, the special value |
| 5602 |
``TRIBITS_PKG`` is used for the TPL (See `<repoDir>/TPLsList.cmake`_.) |
| 5603 |
|
| 5604 |
6) **Configure the TriBITS project enabling the new TPL with |
| 5605 |
TPL_ENABLE_<tplName>=ON** and see that the TPL is found correctly at |
| 5606 |
configure time. |
| 5607 |
|
| 5608 |
7) **Add <tplName> to the package Dependencies.cmake files** of downstream |
| 5609 |
dependent packages that will use this TPL (see |
| 5610 |
`<packageDir>/cmake/Dependencies.cmake`_ or |
| 5611 |
`<packageDir>/<spkgDir>/cmake/Dependencies.cmake`_ for a subpackage). |
| 5612 |
|
| 5613 |
8) **[Optional] Add #cmakedefine for an optional package LIB TPL dependency in |
| 5614 |
the package's <packageName>_config.h.in file** using:: |
| 5615 |
|
| 5616 |
#cmakedefine HAVE_<PACKAGE_NAME_UC>_<TPL_NAME_UC> |
| 5617 |
|
| 5618 |
so that the package's LIB code build knows if the TPL is defined or not |
| 5619 |
(see `<packageName>_config.h.in`_ and `tribits_configure_file()`_). (NOTE: |
| 5620 |
Do **not** add this define for a optional test-only TPL dependency. We |
| 5621 |
don't want all of the libraries in a package to have to be rebuild when we |
| 5622 |
enable or disable tests for the package.) |
| 5623 |
|
| 5624 |
9) **Use the TPL functionality in the packages** that define the dependency on |
| 5625 |
the new TPL, configure, test, etc. |
| 5626 |
|
| 5627 |
|
| 5628 |
.. _Creating the FindTPL<tplName>.cmake file: |
| 5629 |
|
| 5630 |
**Creating the FindTPL<tplName>.cmake file** |
| 5631 |
|
| 5632 |
The main axes of variation in defining ``FindTPL<tplName>.cmake`` modules are: |
| 5633 |
|
| 5634 |
a) Use an inner ``find_package(<externalPkg>)`` call or just use a simple list |
| 5635 |
of header file names and library names? |
| 5636 |
|
| 5637 |
b) The call ``find_package(<externalPkg>)`` defines modern IMPORTED targets or |
| 5638 |
only provides variables for the list of include directories and libraries? |
| 5639 |
|
| 5640 |
c) Maintain backward compatibility with the legacy TriBITS TPL system that |
| 5641 |
allows the user to set cache variables for the include directories and link |
| 5642 |
libraries or force users to always call ``find_package(<externalPkg>)``? |
| 5643 |
|
| 5644 |
See the different variations in the below sections: |
| 5645 |
|
| 5646 |
* `Creating FindTPL<tplName>.cmake using find_package()`_ |
| 5647 |
|
| 5648 |
* `Creating FindTPL<tplName>.cmake using find_package() with IMPORTED targets`_ |
| 5649 |
|
| 5650 |
* `Creating FindTPL<tplName>.cmake using find_package() without IMPORTED targets`_ |
| 5651 |
|
| 5652 |
* `Creating a FindTPL<tplName>.cmake module without find_package()`_ |
| 5653 |
|
| 5654 |
* `Requirements for FindTPL<tplName>.cmake modules`_ |
| 5655 |
|
| 5656 |
|
| 5657 |
Creating FindTPL<tplName>.cmake using find_package() |
| 5658 |
++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| 5659 |
|
| 5660 |
When defining a ``FindTPL<tplName>.cmake`` file, it encouraged to utilize |
| 5661 |
``find_package(<externalPkg> ...)`` to provide the default find operation (and |
| 5662 |
also the definition of the IMPORTED targets needed to use the external package |
| 5663 |
when supported). (Here, typically ``<tplName>`` and ``<externalPkg>`` are the |
| 5664 |
same names, but there are cases where the names may be different or use |
| 5665 |
different capitalization.) However, the state of the current ecosystem of |
| 5666 |
``Find<externalPkg>.cmake`` modules and ``<externalPkg>Config.cmake`` package |
| 5667 |
config files is a bit uneven. While all find modules and package config files |
| 5668 |
should be defining modern CMake IMPORTED targets which contains all the needed |
| 5669 |
usage requirements (such as the target properties |
| 5670 |
``INTERFACE_INCLUDE_DIRECTORIES`` and ``INTERFACE_LINK_LIBRARIES``) and use |
| 5671 |
``find_dependency()`` to get all of their required external upstream package |
| 5672 |
dependencies, many do not. Also, many of these don't provide a complete |
| 5673 |
``<tplName>::all_libs`` target which is required for a TriBITS-compliant |
| 5674 |
external package/TPL. |
| 5675 |
|
| 5676 |
In this case, the ``FindTPL<tplName>.cmake`` file provides a thin "glue" layer |
| 5677 |
to adapt the information and objects provided by the |
| 5678 |
``find_package(<externalPkg> ...)`` call into a complete |
| 5679 |
``<tplName>::all_libs`` target and a wrapper ``<tplName>Config.cmake`` file |
| 5680 |
for consumption by downstream TriBITS-compliant packages. |
| 5681 |
|
| 5682 |
The following subsections will describe how to create these TriBITS-compliant |
| 5683 |
``FindTPL<tplName>.cmake`` modules for all of the various cases using an |
| 5684 |
internal call to ``find_package(<externalPkg> ...)``: |
| 5685 |
|
| 5686 |
* `Creating FindTPL<tplName>.cmake using find_package() with IMPORTED targets`_ |
| 5687 |
* `Creating FindTPL<tplName>.cmake using find_package() without IMPORTED targets`_ |
| 5688 |
|
| 5689 |
|
| 5690 |
Creating FindTPL<tplName>.cmake using find_package() with IMPORTED targets |
| 5691 |
.......................................................................... |
| 5692 |
|
| 5693 |
For cases where ``find_package(<externalPkg>)`` provides complete and proper |
| 5694 |
modern (namespaced) IMPORTED targets (but is missing the |
| 5695 |
``<tplName>::all_libs`` target or the name ``<tplName>`` and ``<externalPkg>`` |
| 5696 |
name are different), these ``FindTPL<tplName>.cmake`` modules can call the |
| 5697 |
function `tribits_extpkg_create_imported_all_libs_target_and_config_file()`_ |
| 5698 |
after calling ``find_package(<externalPkg>)`` to create a very thin find |
| 5699 |
module file ``FindTPL<tplName>.cmake``. In these cases, such a |
| 5700 |
``FindTPL<tplName>.cmake`` module file is nothing more than:: |
| 5701 |
|
| 5702 |
find_package(<externalPkg> REQUIRED) |
| 5703 |
tribits_extpkg_create_imported_all_libs_target_and_config_file( |
| 5704 |
<tplName> |
| 5705 |
INNER_FIND_PACKAGE_NAME <externalPkg> |
| 5706 |
IMPORTED_TARGETS_FOR_ALL_LIBS <importedTarget0> <importedTarget1> ... ) |
| 5707 |
|
| 5708 |
The function |
| 5709 |
`tribits_extpkg_create_imported_all_libs_target_and_config_file()`_ creates |
| 5710 |
the target ``<tplName>::all_libs`` and the wrapper file |
| 5711 |
``<tplName>Config.cmake`` which is installed by TriBITS. The only unique |
| 5712 |
information required to create this glue module is the name of the external |
| 5713 |
package ``<externalPkg>`` and the exact full names of the IMPORTED targets |
| 5714 |
``<importedTarget0> <importedTarget1> ...`` provided by the |
| 5715 |
``find_package(<externalPkg> ...)`` call. The TriBITS function |
| 5716 |
``tribits_extpkg_create_imported_all_libs_target_and_config_file()`` takes |
| 5717 |
care of all of the rest of the details. |
| 5718 |
|
| 5719 |
Such simple ``FindTPL<tplName>.cmake`` modules do not follow the legacy |
| 5720 |
TriBITS TPL convention of allowing users to specify a TPL by setting the cache |
| 5721 |
variables ``<tplName>_INCLUDE_DIRS``, ``<tplName>_LIBRARY_DIRS``, and |
| 5722 |
``<tplName>_LIBRARY_NAMES`` or by setting ``TPL_<tplName>_INCLUDE_DIRS`` and |
| 5723 |
``<tplName>_LIBRARIES``. But as the ecosystem of CMake software transitions |
| 5724 |
to modern CMake along with the proper usage of complete |
| 5725 |
``<Package>Config.cmake`` files, this is the reasonable thing to do. |
| 5726 |
|
| 5727 |
However, to maintain backwards compatibility with the legacy TriBITS TPL |
| 5728 |
system (such as when upgrading a existing ``FindTPL<tplName>.cmake`` file), a |
| 5729 |
``FindTPL<tplName>.cmake`` file can be extended to use the function |
| 5730 |
`tribits_tpl_allow_pre_find_package()`_ in combination with the functions |
| 5731 |
``tribits_extpkg_create_imported_all_libs_target_and_config_file()`` and |
| 5732 |
`tribits_tpl_find_include_dirs_and_libraries()`_ as follows:: |
| 5733 |
|
| 5734 |
set(REQUIRED_HEADERS <header0> <header1> ...) |
| 5735 |
set(REQUIRED_LIBS_NAMES <libname0> <libname1> ...) |
| 5736 |
set(IMPORTED_TARGETS_FOR_ALL_LIBS <importedTarget0> <importedTarget1> ...) |
| 5737 |
|
| 5738 |
tribits_tpl_allow_pre_find_package(<tplName> <tplName>_ALLOW_PREFIND) |
| 5739 |
|
| 5740 |
if (<tplName>_ALLOW_PREFIND) |
| 5741 |
message("-- Using find_package(<externalPkg> ...) ...") |
| 5742 |
find_package(<externalPkg>) |
| 5743 |
if (<externalPkg>_FOUND) |
| 5744 |
message("-- Found <externalPkg>_DIR='${<externalPkg>_DIR}'") |
| 5745 |
message("-- Generating <tplName>::all_libs and <tplName>Config.cmake") |
| 5746 |
tribits_extpkg_create_imported_all_libs_target_and_config_file(<tplName> |
| 5747 |
INNER_FIND_PACKAGE_NAME <externalPkg> |
| 5748 |
IMPORTED_TARGETS_FOR_ALL_LIBS ${IMPORTED_TARGETS_FOR_ALL_LIBS} ) |
| 5749 |
endif() |
| 5750 |
endif() |
| 5751 |
|
| 5752 |
if (NOT TARGET <tplName>::all_libs) |
| 5753 |
tribits_tpl_find_include_dirs_and_libraries( <tplName> |
| 5754 |
REQUIRED_HEADERS ${REQUIRED_HEADERS} |
| 5755 |
REQUIRED_LIBS_NAMES ${REQUIRED_LIBS_NAMES} ) |
| 5756 |
endif() |
| 5757 |
|
| 5758 |
Above, if ``find_package(<externalPkg>)`` is called and returns |
| 5759 |
``<externalPkg>_FOUND=FALSE``, then as fallback it will call |
| 5760 |
``tribits_tpl_find_include_dirs_and_libraries()`` to find the components of |
| 5761 |
the TPL. See the documentation for `tribits_tpl_allow_pre_find_package()`_ |
| 5762 |
for conditions where ``<tplName>_ALLOW_PREFIND`` is set to ``FALSE`` (and |
| 5763 |
therefore ``find_package(<externalPkg>)`` is not called). |
| 5764 |
|
| 5765 |
Note in the above ``FindTPL<tplName>.cmake`` file that |
| 5766 |
``find_package(<externalPkg>)`` will be called even on reconfigures. That is |
| 5767 |
critical since ``find_package(<externalPkg>)`` defines IMPORTED targets that |
| 5768 |
must be available each time configure is called. Also, if |
| 5769 |
``find_package(<externalPkg>)`` is called and ``<externalPkg>_FOUND=TRUE``, |
| 5770 |
then the function |
| 5771 |
`tribits_extpkg_create_imported_all_libs_target_and_config_file()`_ |
| 5772 |
is called which defines the targets ``<tplName>::all_libs`` which means that |
| 5773 |
the function ``tribits_tpl_find_include_dirs_and_libraries()`` will **not** be |
| 5774 |
called. |
| 5775 |
|
| 5776 |
A concrete (and tested) example of the latter case can be found in the file |
| 5777 |
``TribitsExampleProject2/cmake/tpls/FindTPLTpl2.cmake``: |
| 5778 |
|
| 5779 |
.. include:: ../../examples/TribitsExampleProject2/cmake/tpls/FindTPLTpl2.cmake |
| 5780 |
:literal: |
| 5781 |
|
| 5782 |
|
| 5783 |
Creating FindTPL<tplName>.cmake using find_package() without IMPORTED targets |
| 5784 |
............................................................................. |
| 5785 |
|
| 5786 |
There are cases where calling ``find_package(<externalPkg>)`` to find the |
| 5787 |
parts of an external package does not create proper IMPORTED targets that can |
| 5788 |
be directly used. For example, legacy ``Find<externalPkg>.cmake`` modules |
| 5789 |
(even many standard ``Find*.cmake`` modules shipped with CMake as of CMake |
| 5790 |
3.23) do not provide IMPORTED targets and instead only provide variables for |
| 5791 |
the list of include directories ``<externalPkg>_INCLUDE_DIRS``, a list of |
| 5792 |
library files ``<externalPkg>_LIBRARIES``, and other information. In cases |
| 5793 |
such as this, one can implement the ``FindTPL<tplName>.cmake`` module by |
| 5794 |
calling ``find_package(<externalPkg>)``, setting |
| 5795 |
``TPL_<tplName>_INCLUDE_DIRS`` and ``TPL_<tplName>_LIBRARIES``, and then |
| 5796 |
calling `tribits_tpl_find_include_dirs_and_libraries()`_ to create the |
| 5797 |
``<tplName>::all_libs`` target and the ``<tplName>Config.cmake`` wrapper |
| 5798 |
config file. |
| 5799 |
|
| 5800 |
The simplest way to implement ``FindTPL<tplName>.cmake`` is to always call |
| 5801 |
``find_package(<externalPkg>)`` as:: |
| 5802 |
|
| 5803 |
message("-- Using find_package(<externalPkg> ...) ...") |
| 5804 |
find_package(<externalPkg> REQUIRED) |
| 5805 |
|
| 5806 |
# Tell TriBITS that we found <tplName> and there no need to look any further |
| 5807 |
set(TPL_<tplName>_INCLUDE_DIRS ${<externalPkg>_INCLUDE_DIRS} CACHE PATH "...") |
| 5808 |
set(TPL_<tplName>_LIBRARIES ${<externalPkg>_LIBRARIES} CACHE FILEPATH "...") |
| 5809 |
set(TPL_<tplName>_LIBRARY_DIRS ${<externalPkg>_LIBRARY_DIRS} CACHE PATH "...") |
| 5810 |
|
| 5811 |
tribits_tpl_find_include_dirs_and_libraries( <tplName> |
| 5812 |
REQUIRED_HEADERS neverFindThisHeader |
| 5813 |
REQUIRED_LIBS_NAMES neverFindThisLib |
| 5814 |
) |
| 5815 |
|
| 5816 |
The above will always call ``find_package(<externalPkg> REQUIRED)``, and if it |
| 5817 |
can't find the package, it will error out and stop the configure process. |
| 5818 |
|
| 5819 |
While the above ``FindTPL<tplName>.cmake`` file is pretty simple, it does not |
| 5820 |
allow for a fall-back using the legacy TriBITS TPL find system using |
| 5821 |
`tribits_tpl_find_include_dirs_and_libraries()`_. One can allow for a |
| 5822 |
fall-back find by passing a set of header files and library names for |
| 5823 |
``tribits_tpl_find_include_dirs_and_libraries()`` to find. This can be done |
| 5824 |
using the ``FindTPL<tplName>.cmake`` module:: |
| 5825 |
|
| 5826 |
set(REQUIRED_HEADERS <header0> <header1> ...) |
| 5827 |
set(REQUIRED_LIBS_NAMES <libname0> <libname1> ...) |
| 5828 |
|
| 5829 |
message("-- Using find_package(<externalPkg> ...) ...") |
| 5830 |
find_package(<externalPkg>) |
| 5831 |
if (<externalPkg>_FOUND) |
| 5832 |
# Tell TriBITS that we found <tplName> and there no need to look any further |
| 5833 |
set(TPL_<tplName>_INCLUDE_DIRS ${<externalPkg>_INCLUDE_DIRS} CACHE PATH "...") |
| 5834 |
set(TPL_<tplName>_LIBRARIES ${<externalPkg>_LIBRARIES} CACHE FILEPATH "...") |
| 5835 |
set(TPL_<tplName>_LIBRARY_DIRS ${<externalPkg>_LIBRARY_DIRS} CACHE PATH "...") |
| 5836 |
endif() |
| 5837 |
|
| 5838 |
tribits_tpl_find_include_dirs_and_libraries( <tplName> |
| 5839 |
REQUIRED_HEADERS ${REQUIRED_HEADERS} |
| 5840 |
REQUIRED_LIBS_NAMES ${REQUIRED_LIBS_NAMES} |
| 5841 |
MUST_FIND_ALL_LIBS ) |
| 5842 |
|
| 5843 |
Above, if ``find_package(<externalPkg>)`` can't find ``<externalPkg>``, then |
| 5844 |
the function ``tribits_tpl_find_include_dirs_and_libraries()`` will try to |
| 5845 |
find the listed required header files and libraries, using the legacy TriBITS |
| 5846 |
TPL find system. And if ``find_package(<externalPkg>)`` can find |
| 5847 |
``<externalPkg>``, then all the call to |
| 5848 |
`tribits_tpl_find_include_dirs_and_libraries()`_ will not look for anything |
| 5849 |
and will just take the information in the variables |
| 5850 |
``TPL_<tplName>_INCLUDE_DIRS`` and ``TPL_<tplName>_LIBRARIES`` and build |
| 5851 |
IMPORTED targets and the ``<tplName>::all_libs`` target. |
| 5852 |
|
| 5853 |
Finally, if one is upgrading an existing ``FindTPL<tplName>.cmake`` file to |
| 5854 |
use ``find_package(<externalPkg>)`` but needs to maintain backward |
| 5855 |
compatibility for existing configure scripts for the project that might be |
| 5856 |
using the legacy TriBITS TPL system, one can use the function |
| 5857 |
`tribits_tpl_allow_pre_find_package()`_ to determine if |
| 5858 |
``find_package(<externalPkg>)`` can be called or if the legacy TriBITS TPL |
| 5859 |
find must be used (to maintain backward compatibility). This |
| 5860 |
``FindTPL<tplName>.cmake`` looks like:: |
| 5861 |
|
| 5862 |
set(REQUIRED_HEADERS <header0> <header1> ...) |
| 5863 |
set(REQUIRED_LIBS_NAMES <libname0> <libname1> ...) |
| 5864 |
|
| 5865 |
tribits_tpl_allow_pre_find_package(<tplName> <tplName>_ALLOW_PREFIND) |
| 5866 |
|
| 5867 |
if (<tplName>_ALLOW_PREFIND) |
| 5868 |
message("-- Using find_package<externalPkg> ...) ...") |
| 5869 |
find_package(<externalPkg>) |
| 5870 |
if (<externalPkg>_FOUND) |
| 5871 |
# Tell TriBITS that we found <tplName> and there no need to look any further |
| 5872 |
set(TPL_<tplName>_INCLUDE_DIRS ${<externalPkg>_INCLUDE_DIRS} CACHE PATH "...") |
| 5873 |
set(TPL_<tplName>_LIBRARIES ${<externalPkg>_LIBRARIES} CACHE FILEPATH "...") |
| 5874 |
set(TPL_<tplName>_LIBRARY_DIRS ${<externalPkg>_LIBRARY_DIRS} CACHE PATH "...") |
| 5875 |
endif() |
| 5876 |
endif() |
| 5877 |
|
| 5878 |
tribits_tpl_find_include_dirs_and_libraries( <tplName> |
| 5879 |
REQUIRED_HEADERS ${REQUIRED_HEADERS} |
| 5880 |
REQUIRED_LIBS_NAMES ${REQUIRED_LIBS_NAMES} |
| 5881 |
MUST_FIND_ALL_LIBS ) |
| 5882 |
|
| 5883 |
The above will result in skipping the call of ``find_package(<externalPkg>)`` |
| 5884 |
if any of the legacy TPL find variables are set. But if the legacy TPL find |
| 5885 |
variables are not set, then the default find will use |
| 5886 |
``find_package(<externalPkg>)`` and will only fall back on the legacy TriBITS |
| 5887 |
TPL find operation if ``find_package(<externalPkg>)`` sets |
| 5888 |
``<externalPkg>_FOUND=FALSE``. |
| 5889 |
|
| 5890 |
With this last approach, the ``FindTPL<tplName>.cmake`` module preserves all |
| 5891 |
of the user behavior described in `Enabling support for an optional |
| 5892 |
Third-Party Library (TPL)`_ for overriding what TPL components to look for, |
| 5893 |
where to look, and finally to override what is actually used. That is, if the |
| 5894 |
user sets the cache variables ``TPL_<tplName>_INCLUDE_DIRS``, |
| 5895 |
``TPL_<tplName>_LIBRARIES``, or ``TPL_<tplName>_LIBRARY_DIRS``, then they |
| 5896 |
should be used without question (which is why the ``set( ... CACHE ...)`` |
| 5897 |
calls in the above example do not use ``FORCE``). |
| 5898 |
|
| 5899 |
If one wants to skip and ignore the standard TriBITS TPL override variables |
| 5900 |
``<tplName>_INCLUDE_DIRS``, ``<tplName>_LIBRARY_NAMES``, or |
| 5901 |
``<tplName>_LIBRARY_DIRS``, then one can set:: |
| 5902 |
|
| 5903 |
set(<tplName>_FORCE_PRE_FIND_PACKAGE TRUE CACHE BOOL |
| 5904 |
"Always first call find_package(<tplName> ...) unless explicit override") |
| 5905 |
|
| 5906 |
at the top of the file ``FindTPL<tplName>.cmake`` and |
| 5907 |
``tribits_tpl_allow_pre_find_package()`` will ignore these variables and |
| 5908 |
return ``TRUE``. This avoids name classes with the variables |
| 5909 |
``<externalPkg>_INCLUDE_DIRS`` and ``<externalPkg>_LIBRARY_DIRS`` with |
| 5910 |
``<tplName>_INCLUDE_DIRS`` and ``<tplName>_LIBRARY_DIRS`` (when ``<tplName> == |
| 5911 |
<externalPkg>``) which are often used in the concrete legacy CMake |
| 5912 |
``Find<tplName>.cmake`` module files themselves. |
| 5913 |
|
| 5914 |
For a slightly more complex (but real-life) example, see ``FindTPLHDF5.cmake`` |
| 5915 |
which is: |
| 5916 |
|
| 5917 |
.. include:: ../../common_tpls/FindTPLHDF5.cmake |
| 5918 |
:literal: |
| 5919 |
|
| 5920 |
Note that some specialized ``Find<externalPkg>.cmake`` modules do more than |
| 5921 |
just return a list of include directories and libraries. Some, like |
| 5922 |
``FindQt4.cmake`` also return other variables that are used in downstream |
| 5923 |
packages. therefore, in these cases, ``find_package(Qt4 ...)`` must be called |
| 5924 |
on every configure. Such find modules cannot completely adhere to the |
| 5925 |
standard legacy TriBITS behavior described in `Enabling support for an |
| 5926 |
optional Third-Party Library (TPL)`_. |
| 5927 |
|
| 5928 |
|
| 5929 |
Creating a FindTPL<tplName>.cmake module without find_package() |
| 5930 |
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| 5931 |
|
| 5932 |
For external packages that don't have a ``Find<externalPkg>.cmake`` module or |
| 5933 |
``<externalPkg>Config.cmake`` package config file, it may make sense to create |
| 5934 |
a simple ``FindTpl<tplName>.cmake`` module that just calls |
| 5935 |
`tribits_tpl_find_include_dirs_and_libraries()`_ with the set of required |
| 5936 |
header files and libraries that must be found. A simple |
| 5937 |
``FindTPL<tplName>.cmake`` module of this form is:: |
| 5938 |
|
| 5939 |
tribits_tpl_find_include_dirs_and_libraries( <tplName> |
| 5940 |
REQUIRED_HEADERS <header0> <header1> ... |
| 5941 |
REQUIRED_LIBS_NAMES <libname0> <libname1> ... |
| 5942 |
MUST_FIND_ALL_LIBS |
| 5943 |
) |
| 5944 |
|
| 5945 |
Note that a set of alternate names for each library can be specified using |
| 5946 |
quotes around the set of alternative library names using the syntax:: |
| 5947 |
|
| 5948 |
tribits_tpl_find_include_dirs_and_libraries( <tplName> |
| 5949 |
... |
| 5950 |
REQUIRED_LIBS_NAMES "<libname0> <libname0alt0> <libname0alt1> ..." ... |
| 5951 |
... |
| 5952 |
) |
| 5953 |
|
| 5954 |
This is most commonly used for simple single-library TPLs like BLAS that has |
| 5955 |
different potential implementations like:: |
| 5956 |
|
| 5957 |
tribits_tpl_find_include_dirs_and_libraries( BLAS |
| 5958 |
REQUIRED_LIBS_NAMES "blas openblas atlas" |
| 5959 |
... |
| 5960 |
) |
| 5961 |
|
| 5962 |
Requirements for FindTPL<tplName>.cmake modules |
| 5963 |
+++++++++++++++++++++++++++++++++++++++++++++++ |
| 5964 |
|
| 5965 |
It is possible to create a ``FindTPL<tplName>.cmake`` find module without |
| 5966 |
using any TriBITS functions. The only firm requirements for a |
| 5967 |
``FindTPL<tplName>.cmake`` file are: |
| 5968 |
|
| 5969 |
* The target ``<tplName>::all_libs`` must be created and it must contain all |
| 5970 |
of the needed libraries, include directories, and other usage requirements |
| 5971 |
(including for all upstream external packages/TPLs) to implement a |
| 5972 |
`TriBITS-compliant package`_ to be consumed by downstream TriBITS packages. |
| 5973 |
|
| 5974 |
* The file ``<buildDir>/external_packages/<tplName>/<tplName>Config.cmake`` |
| 5975 |
must be created in the build directory, and when included, it must define |
| 5976 |
the equivalent IMPORTED target ``<tplName>::all_libs``, pull all of the |
| 5977 |
``<UpstreamTpl>Config.cmake`` files for upstream external packages/TPLs, and |
| 5978 |
define the needed variables to provide a `TriBITS-compliant external |
| 5979 |
package`_. |
| 5980 |
|
| 5981 |
TriBITS will set the remaining variables to provide a complete |
| 5982 |
`TriBITS-Compliant Package`_ for the current CMake project and will add the |
| 5983 |
install target to install the file |
| 5984 |
``<buildDir>/external_packages/<tplName>/<tplName>Config.cmake`` to create a |
| 5985 |
`TriBITS-compliant external package`_. TriBITS will also automatically create |
| 5986 |
an appropriate package version file ``<tplName>ConfigVersion.cmake``. |
| 5987 |
|
| 5988 |
Some of issues to consider in this case (and the role of the |
| 5989 |
``<tplName>ConfigVersion.cmake`` file) are described in the section `Tricky |
| 5990 |
considerations for TriBITS-generated <tplName>Config.cmake files`_. |
| 5991 |
|
| 5992 |
|
| 5993 |
How to add a new TriBITS Repository |
| 5994 |
----------------------------------- |
| 5995 |
|
| 5996 |
To add a new TriBITS and/ git VC repository to a TriBITS project that already |
| 5997 |
contains other extra repositories, do the following: |
| 5998 |
|
| 5999 |
1) Add a row for the new TriBITS and/or git repo to the file |
| 6000 |
`<projectDir>/cmake/ExtraRepositoriesList.cmake`_. Commit this file. |
| 6001 |
|
| 6002 |
2) Add an ignore for the extra repo name to the base project's git repo's |
| 6003 |
``.gitignore`` file (see `Multi-Repository Support`_). Commit this file. |
| 6004 |
|
| 6005 |
3) Set up the new package dependencies for the new package in the new TriBITS |
| 6006 |
repo or make other adjustments. |
| 6007 |
|
| 6008 |
4) Consider the potential for missing upstream repos and packages by using |
| 6009 |
`tribits_allow_missing_external_packages()`_. |
| 6010 |
|
| 6011 |
See `<projectDir>/cmake/ExtraRepositoriesList.cmake`_ for more details and |
| 6012 |
links. |
| 6013 |
|
| 6014 |
.. ToDo: Provide more detail! |
| 6015 |
|
| 6016 |
|
| 6017 |
How to add a new TriBITS Package dependency |
| 6018 |
------------------------------------------- |
| 6019 |
|
| 6020 |
It is often the case where one will want to add a new dependency for an |
| 6021 |
existing `downstream`_ package to an existing `upstream`_ (internal or |
| 6022 |
external) `TriBITS Package`_. This can either be a required dependency or an |
| 6023 |
optional dependency. Here, we will refer to the downstream package as |
| 6024 |
``<packageName>`` with base directory ``<packageDir>`` and will refer to the |
| 6025 |
upstream (internal or external) package as ``<upstreamPackageName>``. |
| 6026 |
|
| 6027 |
The process for adding a new dependency to an existing upstream package is |
| 6028 |
as follows: |
| 6029 |
|
| 6030 |
1) **Add the name of the upstream package to the downstream package's |
| 6031 |
Dependencies.cmake file:** Add ``<upstreamPackagename>`` to the call of |
| 6032 |
`tribits_package_define_dependencies()`_ in the downstream package's |
| 6033 |
`<packageDir>/cmake/Dependencies.cmake`_ file. If this is to be a required |
| 6034 |
library dependency, then ``<upstreamPackagename>`` is added to the |
| 6035 |
``LIB_REQUIRED_PACKAGES`` argument. Alternatively, if this is to be an |
| 6036 |
optional library dependency, then ``<upstreamPackagename>`` is added to the |
| 6037 |
``LIB_OPTIONAL_PACKAGES`` argument. (For example, see the file |
| 6038 |
``packages/EpetaExt/cmake/Dependencies.cmake`` file in the |
| 6039 |
`ReducedMockTrilinos`_ project.) If only the test and/or example sources, |
| 6040 |
and not the package's core library sources, will have the required or |
| 6041 |
optional dependency, then ``<upstreamPackagename>`` is added to the |
| 6042 |
arguments ``TEST_REQUIRED_PACKAGES`` or ``TEST_OPTIONAL_PACKAGES``, |
| 6043 |
respectively. |
| 6044 |
|
| 6045 |
2) **For an optional dependency, add `HAVE_` preprocessor macro to the |
| 6046 |
package's configured header file:** If this is an optional dependency, |
| 6047 |
typically a C/C++ processor macro will be added to the package's configured |
| 6048 |
`<packageDir>/cmake/<packageName>_config.h.in`_ file using the line:: |
| 6049 |
|
| 6050 |
#cmakedefine HAVE_<PACKAGE_NAME_UC>_<UPSTREAM_PACKAGE_NAME_UC> |
| 6051 |
|
| 6052 |
(see `HAVE_<PACKAGE_NAME_UC>_<UPSTREAM_PACKAGE_NAME_UC>`_.) |
| 6053 |
|
| 6054 |
.. _Warning, do not add optional defines for tests/examples to configured header files: |
| 6055 |
|
| 6056 |
**Warning, do not add optional defines for tests/examples to configured |
| 6057 |
header files:** If this is a test-only and/or example-only dependency then |
| 6058 |
please do **not** add a ``#cmakedefine`` to the package's core |
| 6059 |
`<packageDir>/cmake/<packageName>_config.h.in`_ file. Instead, add the |
| 6060 |
``#cmakedefine`` line to a configured header that is only included by |
| 6061 |
sources for the tests/examples or just add a define on the compiler command |
| 6062 |
line (see the ``DEFINES`` argument to `tribits_add_library()`_ and |
| 6063 |
`tribits_add_executable()`_, but see the warning about problems with |
| 6064 |
``add_definitions()`` in `Miscellaneous Notes (tribits_add_library())`_). |
| 6065 |
We don't want the package's header files to change or libraries to have to |
| 6066 |
be rebuilt if tests/examples get enabled or disabled. Otherwise, the |
| 6067 |
`TriBITS CTest/CDash Driver`_ process will result in unnecessary rebuilds |
| 6068 |
of software over and over again. |
| 6069 |
|
| 6070 |
3) **Use the features of the upstream package in the source files of the |
| 6071 |
downstream package sources and/or tests/examples:** Usage of the features |
| 6072 |
of the upstream package ``<upstreamPackageName>`` in the downstream package |
| 6073 |
``<packageName>`` will typically involve adding ``#include |
| 6074 |
<upstreamPackageName>_<fileName>`` in the package's C/C++ source (or |
| 6075 |
test/example) files (or the equivalent in Fortran). If it is an optional |
| 6076 |
dependency, then these includes will typically be protected using |
| 6077 |
preprocessor ifdefs, for example, as:: |
| 6078 |
|
| 6079 |
#include "<packageName>_config.h" |
| 6080 |
|
| 6081 |
#if HAVE_<PACKAGE_NAME_UC>_<UPSTREAM_PACKAGE_NAME_UC> |
| 6082 |
# include "<upstreamPackageName>_<fileName>" |
| 6083 |
#endif |
| 6084 |
|
| 6085 |
4) **For an optional dependency, use CMake if() statements based on |
| 6086 |
${PACKAGE_NAME}_ENABLE_<upstreamPackageName>:** When a package |
| 6087 |
``PACKAGE_NAME`` has an optional dependency on an upstream package |
| 6088 |
``<upstreamPackageName>`` and needs to put in optional logic in a |
| 6089 |
``CMakeLists.txt`` file, then the ``if()`` statements should use the |
| 6090 |
variable ``${PACKAGE_NAME}_ENABLE_<upstreamPackageName>`` and **not** the |
| 6091 |
variable ``${PROJECT_NAME}_ENABLE_<upstreamPackageName>`` or |
| 6092 |
``TPL_ENABLE_<upstreamPackageName>`` (if ``<upstreamPackageName>`` is an |
| 6093 |
external package/TPL). For example, to optionally enable a test that |
| 6094 |
depends on the enable of the optional upstream dependent package |
| 6095 |
``<upstreamPackageName>``, one would use:: |
| 6096 |
|
| 6097 |
tribits_add_test( ... |
| 6098 |
EXCLUDE_IF_NOT_TRUE ${PACKAGE_NAME}_ENABLE_<upstreamPackageName> |
| 6099 |
) |
| 6100 |
|
| 6101 |
or:: |
| 6102 |
|
| 6103 |
if (${PACKAGE_NAME}_ENABLE_<upstreamPackageName>) |
| 6104 |
tribits_add_test( ... ) |
| 6105 |
endif() |
| 6106 |
|
| 6107 |
.. ToDo: Find an example to point to in TribitsExampleProject. |
| 6108 |
|
| 6109 |
NOTE: TriBITS will automatically add the include directories for the upstream |
| 6110 |
package to the compile lines for the downstream package source builds and will |
| 6111 |
add the libraries for the upstream package to the link lines to the downstream |
| 6112 |
package library and executable links. See documentation in the functions |
| 6113 |
`tribits_add_library()`_ and `tribits_add_executable()`_, and the ``DEPLIBS`` |
| 6114 |
argument to these functions, for more details. |
| 6115 |
|
| 6116 |
|
| 6117 |
How to tentatively enable an external package/TPL |
| 6118 |
------------------------------------------------- |
| 6119 |
|
| 6120 |
A TriBITS package can request the tentative enable of any of its optional |
| 6121 |
external packagse/TPLs (see `How to add a new TriBITS Package dependency`_). |
| 6122 |
This is done by calling `tribits_tpl_tentatively_enable()`_ in the package's |
| 6123 |
`<packageDir>/cmake/Dependencies.cmake`_ file. For example:: |
| 6124 |
|
| 6125 |
tribits_package_define_dependencies( |
| 6126 |
... |
| 6127 |
LIB_OPTIONAL_TPLS SomeTpl |
| 6128 |
... |
| 6129 |
) |
| 6130 |
|
| 6131 |
tribits_tpl_tentatively_enable(SomeTpl) |
| 6132 |
|
| 6133 |
This will result is an attempt to find the components for the TPL ``SomeTpl``. |
| 6134 |
But if that attempt fails, then the TPL will be disabled and |
| 6135 |
``${PACKAGE_NAME}_ENABLE_SomeTpl`` will be set to ``OFF``. |
| 6136 |
|
| 6137 |
|
| 6138 |
How to insert a package into an upstream repo |
| 6139 |
--------------------------------------------- |
| 6140 |
|
| 6141 |
Sometimes it is desired to insert a package from a downstream VC repo into an |
| 6142 |
upstream `TriBITS Repository`_ in order for one or more packages in the |
| 6143 |
upstream repo to define a dependency on that package. The way this is |
| 6144 |
supported in TriBITS is to just list the inserted package into the |
| 6145 |
``PackagesList.cmake`` file of the upstream TriBITS repo after the packages it |
| 6146 |
depends on and before the packages that will use it then call the |
| 6147 |
`tribits_allow_missing_external_packages()`_ function to allow the package to |
| 6148 |
be missing. This is demonstrated in `TribitsExampleProject`_ with the package |
| 6149 |
``InsertedPkg`` which is **not** included in the default |
| 6150 |
``TribitsExampleProject`` source tree. The |
| 6151 |
`TribitsExampleProject`_/``PackagesList.cmake`` file looks like: |
| 6152 |
|
| 6153 |
.. include:: ../../examples/TribitsExampleProject/PackagesList.cmake |
| 6154 |
:literal: |
| 6155 |
|
| 6156 |
In this example, ``InsertedPkg`` has a required dependency on ``SimpleCxx`` |
| 6157 |
and the package ``WithSubpackagesB`` has an optional dependency on |
| 6158 |
``InsertedPkg``. Therefore, the inserted package ``InsertedPkg`` has upstream |
| 6159 |
and downstream dependencies on packages in the ``TribitsExampleProject`` repo. |
| 6160 |
|
| 6161 |
The function ``tribits_allow_missing_external_packages()`` tells TriBITS to |
| 6162 |
treat ``InsertedPkg`` the same as any other package if the directory |
| 6163 |
``TribitsExampleProject/InsertedPkg`` exists or to completely ignore the |
| 6164 |
package ``InsertedPkg`` otherwise. In addition, TriBITS will automatically |
| 6165 |
disable of all downstream package dependencies for the missing package (and |
| 6166 |
print a note about the disables). NOTE: By default TriBITS will silently |
| 6167 |
ignore missing inserted packages and disable optional support for the missing |
| 6168 |
package. To see what packages are missing and being ignored, configure with:: |
| 6169 |
|
| 6170 |
-D <Project>_WARN_ABOUT_MISSING_EXTERNAL_PACKAGES=TRUE |
| 6171 |
|
| 6172 |
The way one would set up ``TribitsExampleProject`` to enable ``InsertedPkg``, |
| 6173 |
if these were in separate VC (e.g. git) repos for example, would be to do:: |
| 6174 |
|
| 6175 |
$ git clone <some-url-base>/TribitsExampleProject |
| 6176 |
$ cd TribitsExampleProject |
| 6177 |
$ git clone <some-other-url-base>/ExteranlPkg |
| 6178 |
$ echo /InsertedPkg/ >> .git/info/excludes |
| 6179 |
|
| 6180 |
Then, when you configure ``TribitsExampleProject``, the package |
| 6181 |
``InsertedPkg`` would automatically appear and could then be enabled or |
| 6182 |
disabled like any other TriBITS package. The TriBITS test |
| 6183 |
``Tribits_TribitsExampleProject_InsertedPkg`` demonstrates this. |
| 6184 |
|
| 6185 |
Assuming that one would put the (new) external package in a separate VC repo, |
| 6186 |
one would perform the following steps: |
| 6187 |
|
| 6188 |
1) Pick a name for the new inserted external package ``<insertedPackageName>`` |
| 6189 |
(see `Globally unique TriBITS package names`_). (NOTE: The external |
| 6190 |
package may already exist in which case the name would already be |
| 6191 |
selected). |
| 6192 |
|
| 6193 |
2) Create a new VC repo containing the new package ``<insertedPackageName>`` |
| 6194 |
or add it to an existing extra VC repo. (Or, add the new package to an |
| 6195 |
existing downstream repo but don't add it to the ``PackagesList.cmake`` |
| 6196 |
file to downstream repo. That would define the package twice!) |
| 6197 |
|
| 6198 |
3) Clone the downstream VC repo under the upstream TriBITS repository. (This |
| 6199 |
is currently needed since TriBITS only allows package dirs to be contained |
| 6200 |
under the repository directory.) |
| 6201 |
|
| 6202 |
4) Insert the package into the `<repoDir>/PackagesList.cmake`_ file as |
| 6203 |
described in `How to add a new TriBITS Package`_ except one must also call |
| 6204 |
``tribits_allow_missing_external_packages(<insertedPackageName>)`` as |
| 6205 |
described above. |
| 6206 |
|
| 6207 |
5) Flesh out the new package and use it in downstream packages just like it |
| 6208 |
was any other package. But note that any downstream package that has a |
| 6209 |
required dependency on ``<insertedPackageName>`` will always be hard |
| 6210 |
disabled when the source for ``<insertedPackageName>`` is missing. |
| 6211 |
|
| 6212 |
6) When configuring and building to get the package working, add |
| 6213 |
``-D<insertedPackageName>_ALLOW_MISSING_EXTERNAL_PACKAGE=FALSE`` so that |
| 6214 |
TriBITS will catch mistakes in specifying the package directory. |
| 6215 |
Otherwise, to see notes about ignoring missing inserted/external packages, |
| 6216 |
set the variable ``-D<Project>_WARN_ABOUT_MISSING_EXTERNAL_PACKAGES=TRUE`` |
| 6217 |
and TriBITS will print warnings about missing external packages. |
| 6218 |
|
| 6219 |
|
| 6220 |
How to put a TriBITS and raw CMake build system side-by-side |
| 6221 |
------------------------------------------------------------ |
| 6222 |
|
| 6223 |
There are cases where it is advantageous to have a raw CMake build system and |
| 6224 |
a TriBITS CMake build system sit side-by-side in a CMake project. There are |
| 6225 |
various ways to accomplish this but a very simple way that has minimal impact |
| 6226 |
on the raw CMake build system is described here. An example of how to |
| 6227 |
accomplish this is shown in the example project ``RawAndTribitsHelloWorld``. |
| 6228 |
This CMake project is a copy of the `TribitsHelloWorld`_ project that puts a |
| 6229 |
primary default raw CMake build system side-by-side with a secondary TriBITS |
| 6230 |
CMake build system. The key aspects of this basic approach shown in this |
| 6231 |
example are: |
| 6232 |
|
| 6233 |
1) An ``if()`` statement must be added to the base project ``CMakeLists.txt`` |
| 6234 |
file to switch between the two build systems. (This is required since the |
| 6235 |
raw CMake commands ``cmake_minimum_required()`` and ``project()`` must |
| 6236 |
exist in the base ``CMakeLists.txt`` file and not in and included |
| 6237 |
``*.cmake`` file.) The switch trigger in the ``if()`` statement can be any |
| 6238 |
logic desired, but a simple way is to look for the |
| 6239 |
``${PROJECT_NAME}_TRIBITS_DIR`` cache variable being set. |
| 6240 |
|
| 6241 |
2) The TriBITS build system in every subdirectory is contained in files of the |
| 6242 |
name ``CMakeLists.tribits.cmake`` beside the ``CMakeLists.txt`` files for |
| 6243 |
the raw CMake build system. (The file ``CMakeLists.tribits.cmake`` ends |
| 6244 |
with a ``*.cmake`` extension so that source editors pick it up as a CMake |
| 6245 |
file.) |
| 6246 |
|
| 6247 |
3) At the top of every raw CMake build system ``CMakeLists.txt`` file is a |
| 6248 |
call to a simple macro ``include_tribits_build()`` which includes the |
| 6249 |
``CMakeLists.tribits.cmake`` file and then returns if doing a TriBITS build |
| 6250 |
and otherwise does nothing for a raw CMake build. |
| 6251 |
|
| 6252 |
The top file ``RawAndTribitsHelloWorld/CMakeLists.txt`` file demonstrates the |
| 6253 |
basic approach: |
| 6254 |
|
| 6255 |
.. include:: ../../examples/RawAndTribitsHelloWorld/CMakeLists.txt |
| 6256 |
:literal: |
| 6257 |
|
| 6258 |
Then every raw ``CMakeLists.txt`` file starts with the command |
| 6259 |
``include_tribits_build()`` at the very top as shown in the example file |
| 6260 |
``RawAndTribitsHelloWorld/hello_world/CMakeLists.txt``: |
| 6261 |
|
| 6262 |
.. include:: ../../examples/RawAndTribitsHelloWorld/hello_world/CMakeLists.txt |
| 6263 |
:literal: |
| 6264 |
|
| 6265 |
To configure the project as a raw CMake project, just configure it as with any |
| 6266 |
raw CMake project as:: |
| 6267 |
|
| 6268 |
cmake [options] <some_base_dir>/RawAndTribitsHelloWorld |
| 6269 |
|
| 6270 |
To configure it as a TriBITS project, just set the cache var |
| 6271 |
``RawAndTribitsHelloWorld_TRIBITS_DIR`` to point to valid TriBITS source tree |
| 6272 |
as:: |
| 6273 |
|
| 6274 |
cmake [options] \ |
| 6275 |
-DRawAndTribitsHelloWorld_TRIBITS_DIR=<tribits_dir> \ |
| 6276 |
<some_base_dir>/RawAndTribitsHelloWorld |
| 6277 |
|
| 6278 |
A twist on this use case is for a package that only builds as a TriBITS |
| 6279 |
package inside of some larger TriBITS project and not as its own TriBITS CMake |
| 6280 |
project. In this case, some slight changes are needed to this example but the |
| 6281 |
basic approach is nearly identical. One still needs an ``if()`` statement at |
| 6282 |
the top of the first ``CMakeLists.txt`` file (this time for the package) and |
| 6283 |
the macro ``include_tribits_build()`` needs to be defined at the top of that |
| 6284 |
file as well. Then every ``CMakeLists.txt`` file in subdirectories just calls |
| 6285 |
``include_tribits_build()``. That is it. |
| 6286 |
|
| 6287 |
|
| 6288 |
How to implement a TriBITS-compliant internal package using raw CMake |
| 6289 |
--------------------------------------------------------------------- |
| 6290 |
|
| 6291 |
As described in `TriBITS-Compliant Internal Packages`_, it is possible to |
| 6292 |
create a raw CMake build system for a CMake package that can build under a |
| 6293 |
parent TriBITS CMake project. The raw CMake code for such a package must |
| 6294 |
provide the ``<Package>::all_libs`` target both in the current CMake build |
| 6295 |
system and also in the generated ``<Package>Config.cmake`` file for the build |
| 6296 |
directory and in the installed ``<Package>Config.cmake`` file. Every such |
| 6297 |
TriBITS-compliant internal package therefore is **also capable of installing a |
| 6298 |
TriBITS-compliant external package** ``<Package>Config.cmake`` file (see `How |
| 6299 |
to implement a TriBITS-compliant external package using raw CMake`_). |
| 6300 |
|
| 6301 |
.. ToDo: Consider listing out the key features of a raw CMake build system |
| 6302 |
that is needed for a TriBITS-compliant internal package. |
| 6303 |
|
| 6304 |
A raw CMake build system for a TriBITS-compliant internal package is |
| 6305 |
demonstrated in the `TribitsExampleProject2`_ project ``Package1`` package. |
| 6306 |
The base ``CMakeLists.txt`` file for building ``Package1`` with a raw CMake |
| 6307 |
build system (called ``package1/CMakeLists.raw.cmake`` in that directory) |
| 6308 |
looks like: |
| 6309 |
|
| 6310 |
.. include:: TribitsExampleProject2_Package1_CMakeLists.raw.internal.cmake |
| 6311 |
:literal: |
| 6312 |
|
| 6313 |
As shown above, this simple CMake package contains the basic features of any |
| 6314 |
CMake project/package including calling the ``cmake_minimum_required()`` and |
| 6315 |
``project()`` commands as well as including ``GNUInstallDirs``. In this |
| 6316 |
example, the project/package being built ``Package1`` has a dependency on an |
| 6317 |
external upstream package ``Tpl1`` pulled in with ``find_package(Tpl1)``. |
| 6318 |
Also in this example, the package has native tests it defines with |
| 6319 |
``include(CTest)`` and ``add_subdirectory()`` (if ``Package1_ENABLE_TESTS`` is |
| 6320 |
set to ``ON``). |
| 6321 |
|
| 6322 |
The file ``package1/src/CMakeLists.raw.cmake`` (which gets included from |
| 6323 |
``package1/src/CMakeLists.txt``) creates a library and executable for the |
| 6324 |
package and has the contents: |
| 6325 |
|
| 6326 |
.. include:: ../../examples/TribitsExampleProject2/packages/package1/src/CMakeLists.raw.cmake |
| 6327 |
:literal: |
| 6328 |
|
| 6329 |
This creates a single installable library target ``Package1_package1`` which |
| 6330 |
is aliased as ``Package1::package1`` in the current CMake project and sets up |
| 6331 |
to create the IMPORTED target ``Package1::package1`` in the generated |
| 6332 |
``Package1ConfigTarget.cmake`` file, which gets included in the installed |
| 6333 |
``Package1Config.cmake`` (``<Package>Config.cmake``) file (as recommenced in |
| 6334 |
the book "Professional CMake", see below). In addition, the above code |
| 6335 |
creates the installable executable ``package1-prg``. |
| 6336 |
|
| 6337 |
The ``Package1::all_libs`` (``<Package>::all_libs``) target is defined and set |
| 6338 |
up inside of the included file |
| 6339 |
``package1/cmake/raw/DefineAllLibsTarget.cmake`` which contains the code: |
| 6340 |
|
| 6341 |
.. include:: ../../examples/TribitsExampleProject2/packages/package1/cmake/raw/DefineAllLibsTarget.cmake |
| 6342 |
:literal: |
| 6343 |
|
| 6344 |
The above code contains the ALIAS library target ``Package1::all_libs`` |
| 6345 |
(``<Package>::all_libs``) for the current CMake project as well as sets up for |
| 6346 |
the IMPORTED target ``Package1::all_libs`` (``<Package>::all_libs``) getting |
| 6347 |
put in the generated ``Package1ConfigTargets.cmake`` file (see below). |
| 6348 |
|
| 6349 |
The ``Package1Config.cmake`` (``<Package>Config.cmake``) file for the build |
| 6350 |
directory is generated inside of the included file |
| 6351 |
``package1/cmake/raw/GeneratePackageConfigFileForBuildDir.cmake`` which has |
| 6352 |
the contents: |
| 6353 |
|
| 6354 |
.. include:: ../../examples/TribitsExampleProject2/packages/package1/cmake/raw/GeneratePackageConfigFileForBuildDir.cmake |
| 6355 |
:literal: |
| 6356 |
|
| 6357 |
The above code uses the ``export(EXPORT ...)`` command to generate the file |
| 6358 |
``Package1ConfigTargets.cmake`` for the build directory which provides the |
| 6359 |
IMPORTED targets ``Package1::package1`` and ``Package1::all_libs``. The |
| 6360 |
command ``configure_file(...)`` generates the ``Package1Config.cmake`` file |
| 6361 |
that includes it for the build directory |
| 6362 |
``<buildDir>/cmake_packages/Package1/``. (NOTE: The above code only runs when |
| 6363 |
the package is being built from inside of a TriBITS project which defines the |
| 6364 |
command ``tribits_package``. So this code gets skipped when building |
| 6365 |
``Package1`` as a stand-alone raw CMake project.) |
| 6366 |
|
| 6367 |
Finally, the code for generating and installing the ``Package1Config.cmake`` |
| 6368 |
file for the install directory ``CMAKE_PREFIX_PATH=<installDir>`` is specified |
| 6369 |
in the included file |
| 6370 |
``package1/cmake/raw/GeneratePackageConfigFileForInstallDir.cmake`` with the |
| 6371 |
contents: |
| 6372 |
|
| 6373 |
.. include:: ../../examples/TribitsExampleProject2/packages/package1/cmake/raw/GeneratePackageConfigFileForInstallDir.cmake |
| 6374 |
:literal: |
| 6375 |
|
| 6376 |
The above uses the command ``install(EXPORT ...)`` to have CMake automatically |
| 6377 |
generate and install the file ``Package1ConfigTargets.cmake`` in the install |
| 6378 |
directory ``<installDir>/libs/cmake/Package1/`` which provides the IMPORTED |
| 6379 |
targets ``Package1::package1`` and ``Package1::all_libs``. The command |
| 6380 |
``configure_file()`` is used to generate the file |
| 6381 |
``Package1Config.install.cmake`` in the build directory from the template file |
| 6382 |
``Package1Config.cmake.in``. Finally, the ``install()`` command is used in |
| 6383 |
the file ``GeneratePackageConfigFileForInstallDir.cmake`` to set up the |
| 6384 |
installation of the ``Package1Config.cmake`` file. |
| 6385 |
|
| 6386 |
Note, the template file ``package1/cmake/raw/Package1Config.cmake.in`` (which |
| 6387 |
is unique to ``Package1``) is: |
| 6388 |
|
| 6389 |
.. include:: ../../examples/TribitsExampleProject2/packages/package1/cmake/raw/Package1Config.cmake.in |
| 6390 |
:literal: |
| 6391 |
|
| 6392 |
As shown in the all of the above code, there is a lot of boilerplate CMake |
| 6393 |
code needed to correctly define the targets such that they get put into the |
| 6394 |
installed ``Package1Config.cmake`` file using the correct namespace |
| 6395 |
``Package1::`` and care must be taken to ensure that a consistent "export set" |
| 6396 |
is used for this purpose. (For more details, see the book "Professional |
| 6397 |
CMake".) |
| 6398 |
|
| 6399 |
**NOTE:** One should compare the above raw CMakeLists files to the more |
| 6400 |
compact TriBITS versions for the base ``package1/CMakeLists.txt`` file (called |
| 6401 |
``package1/CMakeLists.tribits.cmake`` in the base directory ``pacakge1/``): |
| 6402 |
|
| 6403 |
.. include:: ../../examples/TribitsExampleProject2/packages/package1/CMakeLists.tribits.cmake |
| 6404 |
:literal: |
| 6405 |
|
| 6406 |
and the TriBITS ``package1/src/CMakeLists.txt`` file (called |
| 6407 |
``package1/src/CMakeLists.tribits.cmake``): |
| 6408 |
|
| 6409 |
.. include:: ../../examples/TribitsExampleProject2/packages/package1/src/CMakeLists.tribits.cmake |
| 6410 |
:literal: |
| 6411 |
|
| 6412 |
This shows the amount of boilerplate code that TriBITS addresses automatically |
| 6413 |
(which reduces the overhead of finer-grained packages and avoids common |
| 6414 |
mistakes with tedious low-level CMake code). |
| 6415 |
|
| 6416 |
|
| 6417 |
How to implement a TriBITS-compliant external package using raw CMake |
| 6418 |
--------------------------------------------------------------------- |
| 6419 |
|
| 6420 |
As described in `TriBITS-Compliant External Packages`_, it is possible to |
| 6421 |
create a raw CMake build system for a CMake package such that once it is |
| 6422 |
installed, satisfies the requirements for a TriBITS-compliant external |
| 6423 |
package. These installed packages provide a ``<Package>Config.cmake`` file |
| 6424 |
that provides the required targets and behaviors as if it was produced by a |
| 6425 |
TriBITS project. For most existing raw CMake projects that already produce a |
| 6426 |
"Professional CMake" compliant ``<Package>Config.cmake`` file, that usually |
| 6427 |
just means adding the IMPORTED target called ``<Package>::all_libs`` to the |
| 6428 |
installed ``<Package>Config.cmake`` file. |
| 6429 |
|
| 6430 |
A raw CMake build system for a TriBITS-compliant external package is |
| 6431 |
demonstrated in the `TribitsExampleProject2`_ project ``Package1`` package. |
| 6432 |
The base ``package1/CMakeLists.txt`` file for building ``Package1`` with a raw |
| 6433 |
CMake build system (called ``package1/CMakeLists.raw.cmake``) for implementing |
| 6434 |
a TriBITS-compliant internal package looks like: |
| 6435 |
|
| 6436 |
.. include:: TribitsExampleProject2_Package1_CMakeLists.raw.external.cmake |
| 6437 |
:literal: |
| 6438 |
|
| 6439 |
Note that the raw build system this example is identical to the build system |
| 6440 |
for the raw TriBITS-compliant internal package described in `How to implement |
| 6441 |
a TriBITS-compliant internal package using raw CMake`_. The only differences |
| 6442 |
are: |
| 6443 |
|
| 6444 |
1) The ``Package1Config.cmake`` (``<Package>Config.cmake``) file does **not** |
| 6445 |
need to be generated for the build directory and therefore the code in |
| 6446 |
``cmake/raw/GeneratePackageConfigFileForBuildDir.cmake`` does **not** need |
| 6447 |
to be included. |
| 6448 |
|
| 6449 |
2) The ALIAS library target ``Package1::all_libs`` (``<Package>::all_libs``) |
| 6450 |
does **not** need to be generated (but should be to be "Professional CMake" |
| 6451 |
compliant). |
| 6452 |
|
| 6453 |
Other than that, see `How to implement a TriBITS-compliant internal package |
| 6454 |
using raw CMake`_ for how to implement a TriBITS-compliant external package. |
| 6455 |
|
| 6456 |
|
| 6457 |
How to use TriBITS testing support in non-TriBITS project |
| 6458 |
--------------------------------------------------------- |
| 6459 |
|
| 6460 |
The TriBITS test support functions `tribits_add_test()`_ and |
| 6461 |
`tribits_add_advanced_test()`_ can be used from any raw (i.e. non-TriBITS) |
| 6462 |
CMake project. To do so, one just needs to include the TriBITS modules: |
| 6463 |
|
| 6464 |
* ``<tribitsDir>/core/test_support/TribitsAddTest.cmake`` |
| 6465 |
* ``<tribitsDir>/core/test_support/TribitsAddAdvancedTest.cmake`` |
| 6466 |
|
| 6467 |
and set the variable ``${PROJECT_NAME}_ENABLE_TESTS`` to ``ON``. For an |
| 6468 |
MPI-enabled CMake project, the CMake variables ``MPI_EXEC``, |
| 6469 |
``MPI_EXEC_PRE_NUMPROCS_FLAGS``, ``MPI_EXEC_NUMPROCS_FLAG`` and |
| 6470 |
``MPI_EXEC_POST_NUMPROCS_FLAGS`` must also be set which define the MPI runtime |
| 6471 |
program launcher command-line used in the TriBITS testing functions:: |
| 6472 |
|
| 6473 |
${MPI_EXEC} ${MPI_EXEC_PRE_NUMPROCS_FLAGS} |
| 6474 |
${MPI_EXEC_NUMPROCS_FLAG} <NP> |
| 6475 |
${MPI_EXEC_POST_NUMPROCS_FLAGS} |
| 6476 |
<TEST_EXECUTABLE_PATH> <TEST_ARGS> |
| 6477 |
|
| 6478 |
(NOTE: These variables are defined automatically in a TriBITS project when |
| 6479 |
``TPL_ENABLE_MPI`` is set to ``ON``.) |
| 6480 |
|
| 6481 |
This is demonstrated in the `TribitsExampleProject2`_ project ``Package1`` |
| 6482 |
package. The base ``pacakge1/CMakeLists.txt`` file for building ``Package1`` |
| 6483 |
with a raw CMake build system using TriBITS testing functions (called |
| 6484 |
``package1/CMakeLists.raw.cmake``) looks like: |
| 6485 |
|
| 6486 |
.. include:: TribitsExampleProject2_Package1_CMakeLists.raw.tribits_test.cmake |
| 6487 |
:literal: |
| 6488 |
|
| 6489 |
The only difference between this base ``package1/CMakeLists.txt`` file and one |
| 6490 |
for a raw CMake project is the inclusion of the file |
| 6491 |
``package1/cmake/raw/EnableTribitsTestSupport.cmake`` which has the contents: |
| 6492 |
|
| 6493 |
.. include:: ../../examples/TribitsExampleProject2/packages/package1/cmake/raw/EnableTribitsTestSupport.cmake |
| 6494 |
:literal: |
| 6495 |
|
| 6496 |
The key lines are:: |
| 6497 |
|
| 6498 |
include("${Package1_TRIBITS_DIR}/core/test_support/TribitsAddTest.cmake") |
| 6499 |
include("${Package1_TRIBITS_DIR}/core/test_support/TribitsAddAdvancedTest.cmake") |
| 6500 |
|
| 6501 |
This defines the CMake functions `tribits_add_test()`_ and |
| 6502 |
`tribits_add_advanced_test()`_, respectively. |
| 6503 |
|
| 6504 |
The above code demonstrates that ``CMAKE_MODULE_PATH`` does **not** need to be |
| 6505 |
updated to use these TriBITS ``test_support`` modules. However, one is free |
| 6506 |
to update ``CMAKE_MODULE_PATH`` and then include the modules by name only |
| 6507 |
like:: |
| 6508 |
|
| 6509 |
list(PREPEND CMAKE_MODULE_PATH "${Package1_TRIBITS_DIR}/core/test_support") |
| 6510 |
include(TribitsAddTest) |
| 6511 |
include(TribitsAddAdvancedTest) |
| 6512 |
|
| 6513 |
Once these TriBITS modules are included, one can use the TriBITS functions as |
| 6514 |
demonstrated in the file ``package1/test/CMakeLists.tribits.cmake`` (which is |
| 6515 |
included from the file ``package1/test/CMakeLists.txt``) and has the contents: |
| 6516 |
|
| 6517 |
.. include:: ../../examples/TribitsExampleProject2/packages/package1/test/CMakeLists.tribits.cmake |
| 6518 |
:literal: |
| 6519 |
|
| 6520 |
Note that in this example, the executable ``package1-prg`` was already |
| 6521 |
created. If new test libraries and executables need to be created, then the |
| 6522 |
raw CMake commands to create those will need to be added as well. |
| 6523 |
|
| 6524 |
|
| 6525 |
How to check for and tweak TriBITS "ENABLE" cache variables |
| 6526 |
----------------------------------------------------------- |
| 6527 |
|
| 6528 |
TriBITS defines a number of special ``<XXX>_ENABLE_<YYY>`` variables for |
| 6529 |
enabling/disabling various entities that allow for a default "undefined" empty |
| 6530 |
``""`` enable status. Examples of these special variables include: |
| 6531 |
|
| 6532 |
* ``${PROJECT_NAME}_ENABLE_<TRIBITS_PACKAGE>`` (Packages) |
| 6533 |
* ``TPL_ENABLE_<tplName>`` (External Packages/TPLs) |
| 6534 |
* ``<TRIBITS_PACKAGE>_ENABLE_<TRIBITS_DEP_PACKAGE>`` (Support for a |
| 6535 |
package ``<TRIBITS_DEP_PACKAGE>`` in a downstream package |
| 6536 |
``<TRIBITS_PACKAGE>``) |
| 6537 |
* ``<TRIBITS_PACKAGE>_ENABLE_TESTS`` (Package tests) |
| 6538 |
* ``<TRIBITS_PACKAGE>_ENABLE_EXAMPLES`` (Package examples) |
| 6539 |
* ``${PROJECT_NAME}_ENABLE_TESTS`` (Tests for explicitly enabled packages) |
| 6540 |
* ``${PROJECT_NAME}_ENABLE_EXAMPLES`` (Examples for explicitly enabled |
| 6541 |
packages) |
| 6542 |
|
| 6543 |
(see `TriBITS Dependency Handling Behaviors`_). |
| 6544 |
|
| 6545 |
To check for and tweak these special "ENABLE" variables, perform the |
| 6546 |
following: |
| 6547 |
|
| 6548 |
1) To check to see if an ``ENABLE`` variable has been enabled or disabled |
| 6549 |
(either explicitly or through auto enable/disable logic), use:: |
| 6550 |
|
| 6551 |
if ("${<XXX>_ENABLE_<YYY>}" STREQUAL "") |
| 6552 |
# Variable has not been set to 'ON' or 'OFF' yet |
| 6553 |
... |
| 6554 |
endif() |
| 6555 |
|
| 6556 |
This will work correctly independent of if the cache variable has been |
| 6557 |
default defined or not. |
| 6558 |
|
| 6559 |
2) To tweak the enable/disable of one or more of these variables after user |
| 6560 |
input but **before** the step "Adjust package and TPLs enables and |
| 6561 |
disables" in `Full Processing of TriBITS Project Files`_: |
| 6562 |
|
| 6563 |
a) To tweak the enables/disables for a TriBITS Repository (i.e. affecting |
| 6564 |
all TriBITS projects) add enable/disable code to the file |
| 6565 |
`<repoDir>/cmake/RepositoryDependenciesSetup.cmake`_. |
| 6566 |
|
| 6567 |
b) To tweak the enables/disables for a specific TriBITS Project |
| 6568 |
(i.e. affecting only that TriBITS project) add enable/disable code to the |
| 6569 |
file `<projectDir>/cmake/ProjectDependenciesSetup.cmake`_. |
| 6570 |
|
| 6571 |
For example, one might default disable a package if it has not been |
| 6572 |
explicitly enabled (or disabled) in one of these files using logic like:: |
| 6573 |
|
| 6574 |
if (NOT ${PROJECT_NAME}_ENABLE_Fortran) |
| 6575 |
if ("${${PROJECT_NAME}_ENABLE_<SomeFortranPackage>}" STREQUAL "") |
| 6576 |
message("-- " "NOTE: Setting ${PROJECT_NAME}_ENABLE_<SomeFortranPackage>=OFF because" |
| 6577 |
"${PROJECT_NAME}_ENABLE_Fortran = '${${PROJECT_NAME}_ENABLE_Fortran}'") |
| 6578 |
set(${PROJECT_NAME}_ENABLE_<SomeFortranPackage> OFF) |
| 6579 |
endif() |
| 6580 |
endif() |
| 6581 |
|
| 6582 |
In order to understand the above steps for properly querying and tweaking |
| 6583 |
these ``ENABLE`` variables, one must understand how TriBITS CMake code defines |
| 6584 |
and interprets variables of this type. |
| 6585 |
|
| 6586 |
First, note that most of these particular ``ENABLE`` variables are not |
| 6587 |
``BOOL`` cache variables but are actually ``STRING`` variables with the |
| 6588 |
possible values of ``ON``, ``OFF`` and empty ``""`` (see the macro |
| 6589 |
`set_cache_on_off_empty()`_). Therefore, just because the value of a |
| 6590 |
``<XXX>_ENABLE_<YYY>`` variable is defined (e.g. ``if (DEFINED |
| 6591 |
<XXX>_ENABLE_<YYY>) ... endif()``) does not mean that it has been set to |
| 6592 |
``ON`` or ``OFF`` yet (or any non-empty values that evaluates to true or false |
| 6593 |
in CMake). To see if an ``ENABLE`` variable is one of these types, look in |
| 6594 |
the ``CMakeCache.txt``. If the type of the variable ``<XXX>_ENABLE_<YYY>`` is |
| 6595 |
``STRING`` and you see another variable set with the name |
| 6596 |
``<XXX>_ENABLE_<YYY>_-STRINGS``, then it is most likely this special type of |
| 6597 |
``ENABLE`` variable with a typical default value of empty ``""``. However, if |
| 6598 |
the cache variable is of type ``BOOL``, then it is likely a standard bool |
| 6599 |
variable that is not allowed to have a value of empty ``""``. |
| 6600 |
|
| 6601 |
Second, note that the value of empty ``""`` evaluates to ``FALSE`` in CMake |
| 6602 |
``if()`` statements. Therefore, if one just wants to know if one of these |
| 6603 |
variables evaluates to true, then just use ``if (<XXX>_ENABLE_<YYY>) |
| 6604 |
... endif()``. |
| 6605 |
|
| 6606 |
Third, note that TriBITS will not define cache variables for these ``ENABLE`` |
| 6607 |
variables until TriBITS processes the ``Dependencies.cmake`` files on the |
| 6608 |
first configure (see `Full TriBITS Project Configuration`_). On future |
| 6609 |
reconfigures, these variables are all defined (but most will have a default |
| 6610 |
value of empty ``""`` stored in the cache). |
| 6611 |
|
| 6612 |
The reason the files ``RepositoryDependenciesSetup.cmake`` and |
| 6613 |
``ProjectDependenciesSetup.cmake`` are the best places to put in these tweaks |
| 6614 |
is because, as shown in `Full Processing of TriBITS Project Files`_, they get |
| 6615 |
processed after all of the user input has been read (in CMake cache variables |
| 6616 |
set with ``-D<variable>=<value>`` and read in from |
| 6617 |
`${PROJECT_NAME}_CONFIGURE_OPTIONS_FILE`_ files) but before TriBITS adjusts |
| 6618 |
the package enables and disables (see `Package Dependencies and Enable/Disable |
| 6619 |
Logic`_). Also, these files get processed in `Reduced Package Dependency |
| 6620 |
Processing`_ as well so they get processed in all contexts where |
| 6621 |
enable/disable logic is applied. |
| 6622 |
|
| 6623 |
However, if one wants to tweak these variables once packages are starting to |
| 6624 |
be processed (in step "For each ``<packageDir>`` in all enabled top-level |
| 6625 |
packages" in `Full TriBITS Project Configuration`_), there are fewer |
| 6626 |
situations where that can be done correctly as described in the next section. |
| 6627 |
|
| 6628 |
|
| 6629 |
How to tweak downstream TriBITS "ENABLE" variables during package configuration |
| 6630 |
------------------------------------------------------------------------------- |
| 6631 |
|
| 6632 |
There are cases where one may need to enable or disable some feature that |
| 6633 |
TriBITS may have enabled by default (such as in "Adjust package and TPLs |
| 6634 |
enables and disables" in `Full Processing of TriBITS Project Files`_) and that |
| 6635 |
decision can only be made while processing a package's |
| 6636 |
`<packageDir>/CMakeLists.txt`_ file and not before. (And therefore the logic |
| 6637 |
for this disable cannot be performed in the ``ProjectDependenciesSetup.cmake`` |
| 6638 |
or ``RepositoryDependenciesSetup.cmake`` files as described in `How to check |
| 6639 |
for and tweak TriBITS "ENABLE" cache variables`_.) Also, there are cases |
| 6640 |
where it is necessary to make this change visible to downstream packages. The |
| 6641 |
main example is when optional support of an upstream package in a downstream |
| 6642 |
package ``<DownstreamPackage>_ENABLE_<UpstreamPackage>`` must be changed in |
| 6643 |
the package's `<packageDir>/CMakeLists.txt`_ file. But there are other |
| 6644 |
examples such as support for a given data-type that may impact multiple |
| 6645 |
downstream packages. |
| 6646 |
|
| 6647 |
When the internal configuration of a package (i.e. while processing its |
| 6648 |
``<packageDir>/CMakeLists.txt`` file) determines that an optional feature |
| 6649 |
``<Package>_ENABLE_<YYY>`` must change the value previously set (e.g. that was |
| 6650 |
set automatically by TriBITS during the "Adjust package and TPLs enables and |
| 6651 |
disables" stage in `Full Processing of TriBITS Project Files`_), one cannot |
| 6652 |
use a simple ``set()`` statement. Changing the value of a |
| 6653 |
``<Package>_ENABLE_<YYY>`` variable inside a package's |
| 6654 |
``<packageDir>/CMakeLists.txt`` file using a raw ``set(<Package>_ENABLE_<YYY> |
| 6655 |
<newValue>)`` statement only changes the variable's value inside the package's |
| 6656 |
scope, but all other packages will see the old value of |
| 6657 |
``<Package>_ENABLE_<YYY>``. To correctly change the value of one of these |
| 6658 |
variables, instead use `dual_scope_set()`_ from the top-level |
| 6659 |
``<packageDir>/CMakeLists.txt`` file. To perform this disable more robustly |
| 6660 |
than calling ``dual_scope_set()`` directly, use the provided macro |
| 6661 |
`tribits_disable_optional_dependency()`_. For example, to disable optional |
| 6662 |
support for ``<UpstreamPackage>`` in ``<DownstreamPackage>`` in |
| 6663 |
``<DownstreamPackage>`` package's ``<packageDir>/CMakeLists.txt`` file based |
| 6664 |
on some criteria, add the CMake code:: |
| 6665 |
|
| 6666 |
if (<some-condition>) |
| 6667 |
tribits_disable_optional_dependency( <UpstreamPackage> |
| 6668 |
"NOTE: ${PACKAGE_NAME}_ENABLE_<UpstreamPackage> being set to OFF because of <reason>" ) |
| 6669 |
endif() |
| 6670 |
|
| 6671 |
Calling ``dual_scope_set()`` in the package's top-level |
| 6672 |
``<packageDir>/CMakeLists.txt`` file sets the value in both the local scope of |
| 6673 |
``<packageDir>/CMakeLists.txt`` (and therefore propagated to all other |
| 6674 |
``CMakeLists.txt`` files in that package) and in base-level (global) project |
| 6675 |
scope. (But this does **not** change the value of a cache variable |
| 6676 |
``<Package>_ENABLE_<YYY>`` that may have been set by the user or some other |
| 6677 |
means which is the desired behavior; see `TriBITS auto-enables/disables done |
| 6678 |
using non-cache local variables`_.) In this way, any downstream package |
| 6679 |
(configured after processing ``<packageDir>/CMakeLists.txt``) will see the new |
| 6680 |
value for ``<Package>_ENABLE_<YYY>``. |
| 6681 |
|
| 6682 |
It is also strongly recommended that a message be printed to CMake STDOUT |
| 6683 |
using ``message("-- " "NOTE: <message>")`` when changing the value of one of |
| 6684 |
these ``<Package>_ENABLE_<YYY>`` variables. The user may have set it |
| 6685 |
explicitly or TriBITS may have printed automatic logic for setting it by |
| 6686 |
default, and user needs to know why and where the value is being overridden. |
| 6687 |
|
| 6688 |
**NOTE:** However, it is **not** allowed to try to change the value of a |
| 6689 |
global enable of a upstream or downstream package by trying to change the |
| 6690 |
value of ``<Project>_ENABLE_<Package>`` or ``TPL_ENABLE_<Package>`` in a |
| 6691 |
``<packageDir>/CMakeLists.txt`` file. Changing the value of these variables |
| 6692 |
after the "Adjust package and TPLs enables and disables" stage in `Full |
| 6693 |
Processing of TriBITS Project Files`_ will result in undefined behavior. |
| 6694 |
|
| 6695 |
|
| 6696 |
How to set up multi-repository support |
| 6697 |
-------------------------------------- |
| 6698 |
|
| 6699 |
The following steps describe how to set up support for TriBITS project |
| 6700 |
managing multiple version control and TriBITS repositories by default (see |
| 6701 |
`Multi-Repository Support`_). |
| 6702 |
|
| 6703 |
1) Add file `<projectDir>/cmake/ExtraRepositoriesList.cmake`_ and list out |
| 6704 |
extra repos |
| 6705 |
|
| 6706 |
For example, this file would contain something like:: |
| 6707 |
|
| 6708 |
tribits_project_define_extra_repositories( |
| 6709 |
ExtraRepo1 "" GIT [email protected]:ExtraRepo1 "" Continuous |
| 6710 |
ExtraRepo2 "ExtraRepo1/ExtraRepos2" GIT [email protected]:ExtraRepo2 |
| 6711 |
NOPACKAGES Continuous |
| 6712 |
ExtraRepo3 "" GIT [email protected]:ExtraRepo3 "" Nightly |
| 6713 |
) |
| 6714 |
|
| 6715 |
NOTE: If one will not be using the `checkin-test.py`_ tool, or |
| 6716 |
`clone_extra_repos.py`_ tool, or the `TriBITS CTest/CDash Driver`_ system, |
| 6717 |
then one can leave the **REPO_VCTYPE** and **REPO_URL** fields empty (see |
| 6718 |
`tribits_project_define_extra_repositories()`_ for details). (TriBITS Core |
| 6719 |
does not have any dependencies on any specific VC tool. These fields are |
| 6720 |
listed here to avoid duplicating the list of repos in another file when |
| 6721 |
using these additional TriBITS tools.) |
| 6722 |
|
| 6723 |
2) Set default values for `${PROJECT_NAME}_EXTRAREPOS_FILE`_ and |
| 6724 |
``${PROJECT_NAME}_ENABLE_KNOWN_EXTERNAL_REPOS_TYPE`` (and possibly |
| 6725 |
``${PROJECT_NAME}_IGNORE_MISSING_EXTRA_REPOSITORIES``) in the file |
| 6726 |
`<projectDir>/ProjectName.cmake`_ |
| 6727 |
|
| 6728 |
For example, add:: |
| 6729 |
|
| 6730 |
set(${PROJECT_NAME}_EXTRAREPOS_FILE cmake/ExtraRepositoriesList.cmake |
| 6731 |
CACHE FILEPATH "Set in ProjectName.cmake") |
| 6732 |
set(${PROJECT_NAME}_ENABLE_KNOWN_EXTERNAL_REPOS_TYPE Continuous |
| 6733 |
CACHE STRING "Set in ProjectName.cmake") |
| 6734 |
|
| 6735 |
to the `<projectDir>/ProjectName.cmake`_ file. Otherwise, no extra repos |
| 6736 |
will be defined or processed by default when configuring the project. |
| 6737 |
|
| 6738 |
And if the project can operate without all of its extra repos, the project |
| 6739 |
can set the following default in this file as well with:: |
| 6740 |
|
| 6741 |
set(${PROJECT_NAME}_IGNORE_MISSING_EXTRA_REPOSITORIES TRUE |
| 6742 |
CACHE STRING "Set in ProjectName.cmake") |
| 6743 |
|
| 6744 |
Otherwise, all of the extra repos need to be present or the project |
| 6745 |
configure will fail. |
| 6746 |
|
| 6747 |
3) If using git as the VC tool, then set the variable |
| 6748 |
``${PROJECT_NAME}_GENERATE_REPO_VERSION_FILE_DEFAULT`` in the |
| 6749 |
`<projectDir>/ProjectName.cmake`_ file |
| 6750 |
|
| 6751 |
For example:: |
| 6752 |
|
| 6753 |
set(${PROJECT_NAME}_GENERATE_REPO_VERSION_FILE_DEFAULT TRUE) |
| 6754 |
|
| 6755 |
4) If wanting a clone tool with git repos, set up a link to the |
| 6756 |
`clone_extra_repos.py`_ tool in the base ``<projectDir>/`` directory |
| 6757 |
|
| 6758 |
Create a symlink to the script `clone_extra_repos.py`_ in the base project |
| 6759 |
repo, for example with:: |
| 6760 |
|
| 6761 |
cd <projecDir>/ |
| 6762 |
ln -s cmake/tribits/ci_support/clone_extra_repos.py . |
| 6763 |
git add clone_extra_repos.py |
| 6764 |
git commit |
| 6765 |
|
| 6766 |
|
| 6767 |
How to submit testing results to a CDash site |
| 6768 |
--------------------------------------------- |
| 6769 |
|
| 6770 |
The following steps describe how to submit results to a CDash site using the |
| 6771 |
`TriBITS CTest/CDash Driver`_ support. |
| 6772 |
|
| 6773 |
1) Create a CDash project ``<ProjectName>`` on the targeted CDash site. |
| 6774 |
|
| 6775 |
To do this, one must have an account on the CDash site and the permissions |
| 6776 |
to create a new CDash project. The name of the project on CDash should |
| 6777 |
generally match the name of the TriBITS project ``PROJECT_NAME`` but it does |
| 6778 |
not have to. In fact, one can generally submit to any CDash project with |
| 6779 |
any name so creating a new CDash project is actually optional. |
| 6780 |
|
| 6781 |
NOTE: For open-source projects, Kitware provides the free CDash site |
| 6782 |
my.cdash.org that allows a limited number of submits and data per day. But |
| 6783 |
it should be enough to test out submitting to CDash. |
| 6784 |
|
| 6785 |
2) Create ``CTestConfig.cmake`` and ``CTestCustom.cmake.in`` files for the |
| 6786 |
project. |
| 6787 |
|
| 6788 |
* The file `<projectDir>/CTestConfig.cmake`_ can be copied and pasted from |
| 6789 |
TribitsExampleProject/CTestConfig.cmake. To customize for your project, |
| 6790 |
you generally just need to update the variables ``CTEST_DROP_SITE``, |
| 6791 |
``CTEST_PROJECT_NAME``, and ``CTEST_DROP_LOCATION``. |
| 6792 |
|
| 6793 |
* The file `<projectDir>/cmake/ctest/CTestCustom.cmake.in`_ can be copied |
| 6794 |
and pasted from ``TribitsExampleProject/cmake/ctest/CTestCustom.cmake.in`` |
| 6795 |
and them modified as desired. |
| 6796 |
|
| 6797 |
3) Test experimental submits with ``make dashboard``. |
| 6798 |
|
| 6799 |
Once the CDash project and the `<projectDir>/CTestConfig.cmake`_ and |
| 6800 |
`<projectDir>/cmake/ctest/CTestCustom.cmake.in`_ files are created, one |
| 6801 |
perform an experimental submission by just configuring the project as normal |
| 6802 |
(except configuring additionally with ``-DCTEST_BUILD_FLAGS=-j8`` and |
| 6803 |
``-DCTEST_PARALLEL_LEVEL=8`` to use parallelism in the build and testing in |
| 6804 |
the ``ctest -S`` script) and then running the build target:: |
| 6805 |
|
| 6806 |
make dashboard |
| 6807 |
|
| 6808 |
That will configure, build, test, and submit results to the CDash site to |
| 6809 |
the ``Experimental`` track of the target CDash project (see `Dashboard |
| 6810 |
Submissions`_). |
| 6811 |
|
| 6812 |
To work out problems locally without spamming the CDash site, one can run |
| 6813 |
with:: |
| 6814 |
|
| 6815 |
env CTEST_DO_SUBMIT=OFF make dashboard |
| 6816 |
|
| 6817 |
To submit to a different CDash site and project, change the cache vars |
| 6818 |
``CTEST_DROP_SITE``, ``CTEST_PROJECT_NAME``, and ``CTEST_DROP_LOCATION`` at |
| 6819 |
configure time. For example, to submit TribitsExampleProject results to a |
| 6820 |
different CDash site, configure with:: |
| 6821 |
|
| 6822 |
cmake \ |
| 6823 |
-DCTEST_DROP_SITE=testing.sandia.gov/cdash \ |
| 6824 |
-DCTEST_PROJECT_NAME=TribitsExProj \ |
| 6825 |
-DCTEST_DROP_LOCATION="/submit.php?project=TribitsExProj" \ |
| 6826 |
[other cmake options] \ |
| 6827 |
<baseDir>/TribitsExamplProject |
| 6828 |
|
| 6829 |
and then run ``make dashboard``. |
| 6830 |
|
| 6831 |
4) Add custom CTest -S driver scripts. |
| 6832 |
|
| 6833 |
For driving different builds and tests, one needs to set up one or more |
| 6834 |
CTest -S driver scripts. There are various ways to do this but a simple |
| 6835 |
approach that avoids duplication is to first create a file like |
| 6836 |
``TribitsExampleProject/cmake/ctest/TribitsExProjCTestDriver.cmake``: |
| 6837 |
|
| 6838 |
.. include:: ../../examples/TribitsExampleProject/cmake/ctest/TribitsExProjCTestDriver.cmake |
| 6839 |
:literal: |
| 6840 |
|
| 6841 |
and then create a set of CTest -S driver scripts that uses that file. One |
| 6842 |
example is the file |
| 6843 |
``TribitsExampleProject/cmake/ctest/general_gcc/ctest_serial_debug.cmake``: |
| 6844 |
|
| 6845 |
.. include:: ../../examples/TribitsExampleProject/cmake/ctest/general_gcc/ctest_serial_debug.cmake |
| 6846 |
:literal: |
| 6847 |
|
| 6848 |
5) Test CTest -S driver scripts |
| 6849 |
|
| 6850 |
Once a CTest -S driver script (like the ``ctest_serial_debug.cmake`` example |
| 6851 |
shown above) is created, one can test it locally and then test a submit to |
| 6852 |
CDash. To test the exact state of the repository locally, one can create a |
| 6853 |
temporary base directory, symbolically link in the local project source |
| 6854 |
directory, and then run the CTest -S script by setting |
| 6855 |
``CTEST_DO_SUBMIT=OFF``. For example, the TribitsExampleProject CTest -S |
| 6856 |
script can be run and tested locally by doing:: |
| 6857 |
|
| 6858 |
$ mkdir MOCK_TRIBITSEXPROJ_SERIAL_DEBUG |
| 6859 |
$ cd MOCK_TRIBITSEXPROJ_SERIAL_DEBUG/ |
| 6860 |
$ ln -s $TRIBITS_DIR/examples/TribitsExampleProject . |
| 6861 |
$ env CTEST_DASHBOARD_ROOT=$PWD \ |
| 6862 |
CTEST_DROP_SITE=testing.sandia.gov/cdash \ |
| 6863 |
CTEST_PROJECT_NAME=TribitsExProj \ |
| 6864 |
CTEST_DROP_LOCATION="/submit.php?project=TribitsExProj" \ |
| 6865 |
CTEST_TEST_TYPE=Experimental \ |
| 6866 |
CTEST_DO_SUBMIT=OFF \ |
| 6867 |
CTEST_DO_UPDATES=OFF \ |
| 6868 |
CTEST_START_WITH_EMPTY_BINARY_DIRECTORY=TRUE \ |
| 6869 |
ctest -V -S \ |
| 6870 |
$TRIBITS_DIR/examples/TribitsExampleProject/cmake/ctest/general_gcc/ctest_serial_debug.cmake \ |
| 6871 |
&> console.out |
| 6872 |
|
| 6873 |
where ``TRIBITS_DIR`` is an env var that points to the location of the |
| 6874 |
``TriBITS/tribits`` directory on the local machine (and the location of the |
| 6875 |
CDash site and project is changed, since the free my.cdash.org site can only |
| 6876 |
accept as small number of builds each day). |
| 6877 |
|
| 6878 |
Once that CTest -S driver script is working correctly without submitting to |
| 6879 |
CDash, the above ``ctest -S`` command can be run with ``CTEST_DO_SUBMIT=ON`` |
| 6880 |
which submit the CDash and then print the location on CDash for the |
| 6881 |
submitted configure, build, and test results. |
| 6882 |
|
| 6883 |
6) Set up automated runs of CTest -S driver scripts |
| 6884 |
|
| 6885 |
The custom CTest -S driver scripts created above can be run and used to |
| 6886 |
submit to CDash in a variety of ways: |
| 6887 |
|
| 6888 |
* Cron jobs can be set up to run them at the same time every day. |
| 6889 |
|
| 6890 |
* Jenkins jobs can be set up to run them based on various criteria. |
| 6891 |
|
| 6892 |
* GitHub Actions can run them to respond to GitHub pushes or to test pull |
| 6893 |
requests. |
| 6894 |
|
| 6895 |
* GitLab CI can run them to respond to GitLab pushes or the test merge |
| 6896 |
requests. |
| 6897 |
|
| 6898 |
* Use the legacy `TriBITS Dashboard Driver`_ system (not recommended). |
| 6899 |
|
| 6900 |
The setup of Jenkins, GitHub Actions, GitLab CI and other more sophisticated |
| 6901 |
automated testing systems will not be described here. What will be briefly |
| 6902 |
outlined below is the setup using cron jobs on a Linux machine. That is |
| 6903 |
sufficient for most smaller projects and provides tremendous value. |
| 6904 |
|
| 6905 |
To set up an automated build using a cron job, one will typically create a |
| 6906 |
shell driver script that sets the env and then calls the ``ctest -S |
| 6907 |
<script>`` command. Then one just adds a call to that shell driver script |
| 6908 |
using ``crontab -e``. That is about all there is to it. |
| 6909 |
|
| 6910 |
.. ToDo: Add some more details and examples on how to do this, using |
| 6911 |
.. TribitsExampleProject implementations. In particular, you need two |
| 6912 |
.. clones of your base git repo. One to provide the CTest -S script, and |
| 6913 |
.. the other cloned and updated by the CTest driver script. |
| 6914 |
|
| 6915 |
|
| 6916 |
How to submit testing results to a custom CDash Group |
| 6917 |
----------------------------------------------------- |
| 6918 |
|
| 6919 |
Following up on `How to submit testing results to a CDash site`_, to submit |
| 6920 |
build and test results to a custom "Group" on CDash (instead of just |
| 6921 |
"Nightly", "Continuous" or "Experimental"), one just has to create the new |
| 6922 |
group on CDash using the CDash GUI interface and then tell the ctest -S local |
| 6923 |
driver to submit results to that CDash group. The steps for doing this are |
| 6924 |
given below. |
| 6925 |
|
| 6926 |
1. Create the new CDash group ``<special_group>`` for CDash project |
| 6927 |
``<ProjectName>`` on the targeted CDash site. |
| 6928 |
|
| 6929 |
If the CDash group ``<special_group>`` is not already created, then one can |
| 6930 |
create it by first logging into CDash with an account that can modify the |
| 6931 |
CDash project ``<ProjectName>``. Once logged in, go to the project edit |
| 6932 |
page and select "Settings" and "Groups". From there, create the new group |
| 6933 |
``<special_group>``. Set the "Build Type" to either "Daily" or "Latest". |
| 6934 |
|
| 6935 |
2. Set ``${PROJECT_NAME}_TRACK=<special_group>`` with the CTest -S driver |
| 6936 |
script. |
| 6937 |
|
| 6938 |
One can either do that by setting ``set(${PROJECT_NAME}_TRACK |
| 6939 |
<special_group>)`` in the CTest -S ``*.cmake`` driver script itself or can |
| 6940 |
set it in the environment when running the ctest -S driver script. For |
| 6941 |
example:: |
| 6942 |
|
| 6943 |
$ env <Project>_TRACK=<special_group> ... \ |
| 6944 |
ctest -V -S <ctest_driver>.cmake |
| 6945 |
|
| 6946 |
If the "build type" for the CDash group ``<special_group>`` was set to |
| 6947 |
"Daily", then set `CTEST_TEST_TYPE`_ to ``Nightly``. Otherwise, |
| 6948 |
``CTEST_TEST_TYPE`` can be set to ``Continuous`` or ``Experimental``. |
| 6949 |
|
| 6950 |
|
| 6951 |
Miscellaneous Topics |
| 6952 |
==================== |
| 6953 |
|
| 6954 |
In this section, a number of miscellaneous topics and TriBITS features are |
| 6955 |
discussed. These features and topics are either not considered primary |
| 6956 |
features of TriBITS or don't neatly fit into one of the other sections. |
| 6957 |
|
| 6958 |
|
| 6959 |
TriBITS Repository Contents |
| 6960 |
--------------------------- |
| 6961 |
|
| 6962 |
The TriBITS git repository is organized as a `TriBITS Project`_, `TriBITS |
| 6963 |
Repository`_, and `TriBITS Package`_ all in the same base directory. The base |
| 6964 |
contents are described in the file:: |
| 6965 |
|
| 6966 |
TriBITS/README.DIRECTORY_CONTENTS.rst |
| 6967 |
|
| 6968 |
The part of TriBITS that is installed or snapshotted in contained in the |
| 6969 |
subdirectory `TriBITS/tribits/`_ and is described in the following section. |
| 6970 |
|
| 6971 |
.. include:: TriBITS.README.DIRECTORY_CONTENTS.rst |
| 6972 |
|
| 6973 |
.. _TriBITS/tribits/: |
| 6974 |
|
| 6975 |
.. include:: ../../README.DIRECTORY_CONTENTS.rst |
| 6976 |
|
| 6977 |
|
| 6978 |
TriBITS System Project Dependencies |
| 6979 |
----------------------------------- |
| 6980 |
|
| 6981 |
The core TriBITS system itself (see ``tribts/core/`` in `TriBITS/tribits/`_) |
| 6982 |
which is used to configure, built, test, create tarballs, and install software |
| 6983 |
has no dependencies other than a basic installation of CMake (which typically |
| 6984 |
includes the executables ``cmake``, ``ctest``, and ``cpack``). Great effort |
| 6985 |
has been expended to implement all of this core functionality of TriBITS just |
| 6986 |
using raw CMake. That means that anyone who needs to configure, build, and |
| 6987 |
install software that uses TriBITS just needs a compatible CMake |
| 6988 |
implementation. CMake is becoming iniquitous enough that many machines will |
| 6989 |
already have a current-enough version of CMake installed by default on their |
| 6990 |
systems and therefore no one will need to download or install any extra |
| 6991 |
software when building and installing a project that uses TriBITS (assuming |
| 6992 |
the necessary compilers etc. required by the project are also installed). If |
| 6993 |
a current-enough version of CMake is not installed on a given system, it is |
| 6994 |
easy to download the source code and all it needs is a basic C++ compiler to |
| 6995 |
build and install. |
| 6996 |
|
| 6997 |
However, note that a specific TriBITS project is free to use any newer CMake |
| 6998 |
features it wants and therefore these projects will require newer versions of |
| 6999 |
CMake than what is required by TriBITS (see discussion of |
| 7000 |
``cmake_minimum_required()`` in `<projectDir>/CMakeLists.txt`_). But also |
| 7001 |
note that specific TriBITS projects and packages will also require additional |
| 7002 |
tools like compilers, Python (see `Python Support`_), Perl, or many other such |
| 7003 |
dependencies. It is just that TriBITS itself does not require any of these in |
| 7004 |
order to perform the basic configure, build, test, and install of software. |
| 7005 |
The goal of TriBITS is not to make the portability of software that uses it |
| 7006 |
any worse than it already is but instead to make it easier in most cases (that |
| 7007 |
after all is the whole goal of CMake). |
| 7008 |
|
| 7009 |
While the TriBITS Core functionality to configure, build, test, and install |
| 7010 |
software is written using only raw CMake, the more sophisticated development |
| 7011 |
tools needed to implement the full TriBITS development environment require |
| 7012 |
Python 2.7 (or higher including Python 3.x) (see `Python Support`_). Python |
| 7013 |
is needed for tools like `checkin-test.py`_ and `gitdist`_. In addition, |
| 7014 |
these python tools are used in `tribits_ctest_driver()`_ to drive automated |
| 7015 |
testing and submits to CDash. Also note that ``git`` is the chosen version |
| 7016 |
control tool for the TriBITS software development tools and all the VC-related |
| 7017 |
functionality in TriBITS. But none of this is required for doing the most |
| 7018 |
basic building, testing, or installation of a project using TriBITS Core. |
| 7019 |
|
| 7020 |
|
| 7021 |
Python Support |
| 7022 |
-------------- |
| 7023 |
|
| 7024 |
TriBITS Core does not require anything other than raw CMake. However, Python |
| 7025 |
Utils, TriBITS CI Support, and other extended TriBITS components require |
| 7026 |
Python. These extra TriBITS tools only require Python 3.6+. By default, when |
| 7027 |
a TriBITS project starts to configure using CMake, it will try to find Python |
| 7028 |
3.6+ on the system (see `Full Processing of TriBITS Project Files`_). If |
| 7029 |
Python is found, it will set the global cache variable ``Python3_EXECUTABLE``. |
| 7030 |
If it is not found, then it will print a warning and ``Python3_EXECUTABLE`` |
| 7031 |
will be empty. With this default behavior, if Python is found, then the |
| 7032 |
TriBITS project can use it. Otherwise, it can do without it. |
| 7033 |
|
| 7034 |
While the default behavior for finding Python described above is useful for |
| 7035 |
many TriBITS projects, some TriBITS projects need different behavior such as: |
| 7036 |
|
| 7037 |
1. The TriBITS project may not ever use Python so there is no need to look for |
| 7038 |
it at all. In this case, the TriBITS project would set |
| 7039 |
`${PROJECT_NAME}_USES_PYTHON`_ to ``FALSE``. |
| 7040 |
|
| 7041 |
2. Some TriBITS projects require Python and should not even configure if it |
| 7042 |
can't be found. In this case, the TriBITS project would set |
| 7043 |
`${PROJECT_NAME}_REQUIRES_PYTHON`_ to ``TRUE``. |
| 7044 |
|
| 7045 |
3. Some TriBITS projects may require a version of Python more recent than 3.6. |
| 7046 |
In this case, the TriBITS project would set |
| 7047 |
`${PROJECT_NAME}_Python3_FIND_VERSION`_ to some value higher than ``3.6``. |
| 7048 |
For example, may newer systems have higher versions of Python installed by |
| 7049 |
default, and projects developed on such a system typically require this |
| 7050 |
version or higher. |
| 7051 |
|
| 7052 |
|
| 7053 |
Project-Specific Build Reference |
| 7054 |
-------------------------------- |
| 7055 |
|
| 7056 |
If a project that uses TriBITS is going to have a significant user base that |
| 7057 |
will configure, build, and test the project, then having some documentation |
| 7058 |
that explains how to do this would be useful. For this purpose, TriBITS |
| 7059 |
provides a mechanism to quickly create a project-specific build reference |
| 7060 |
document in restructured text (RST) format and with HTML and LaTeX/PDF |
| 7061 |
outputs. This document are generally created in the base project source tree |
| 7062 |
and given then name ``<Project>BuildReference.[rst,html,pdf]``. This document |
| 7063 |
consists of two parts. One part is a generic template document:: |
| 7064 |
|
| 7065 |
tribits/doc/TribitsBuildReferenceBody.rst |
| 7066 |
|
| 7067 |
provided in the TriBITS source tree that uses the place-holder ``<Project>`` |
| 7068 |
for the for the real project name. The second part is a project-specific |
| 7069 |
template file:: |
| 7070 |
|
| 7071 |
<projectDir>/cmake/<Project>BuildReferenceTemplate.rst |
| 7072 |
|
| 7073 |
which provides the outer RST document (with title, authors, abstract, |
| 7074 |
introduction, other introductory sections). From these two files, the |
| 7075 |
script:: |
| 7076 |
|
| 7077 |
tribits/doc/build_ref/create-project-build-quickref.py |
| 7078 |
|
| 7079 |
is used to replace ``<Project>`` in the ``TribitsBuildReferenceBody.rst`` file |
| 7080 |
with the real project name (read from the project's ``ProjectName.cmake`` file |
| 7081 |
by default) and then generates the read-only files:: |
| 7082 |
|
| 7083 |
<projectDir>/ |
| 7084 |
<Project>BuildReference.rst |
| 7085 |
<Project>BuildReference.html |
| 7086 |
<Project>BuildReference.pdf |
| 7087 |
|
| 7088 |
For a simple example of this, see:: |
| 7089 |
|
| 7090 |
tribits/doc/build_ref/create-build-ref.sh |
| 7091 |
|
| 7092 |
A project-independent version of this file is provided in the |
| 7093 |
`TribitsBuildReference`_.[rst,html,pdf] which is referred to many times in this |
| 7094 |
developers guide. |
| 7095 |
|
| 7096 |
|
| 7097 |
Project and Repository Versioning and Release Mode |
| 7098 |
---------------------------------------------------- |
| 7099 |
|
| 7100 |
TriBITS has built-in support for project and repository versioning and |
| 7101 |
release-mode control. When the project contains the file |
| 7102 |
`<projectDir>/Version.cmake`_, it is used to define the project's official |
| 7103 |
version. The idea is that when it is time to branch for a release, the only |
| 7104 |
file that needs to be changed is the file `<projectDir>/Version.cmake`_. |
| 7105 |
|
| 7106 |
Each TriBITS repository can also contain a `<repoDir>/Version.cmake`_ file |
| 7107 |
that sets version-related variables which TriBITS packages in that repository |
| 7108 |
can use to derive development and release version information. If the TriBITS |
| 7109 |
repository also contains a `<repoDir>/Copyright.txt`_ file, then the |
| 7110 |
information in ``<repoDir>/Version.cmake`` and ``<repoDir>/Copyright.txt`` are |
| 7111 |
used to configure a repository version header file:: |
| 7112 |
|
| 7113 |
${${REPOSITORY_NAME}_BINARY_DIR}/${REPOSITORY_NAME}_version.h |
| 7114 |
|
| 7115 |
The configured header file ``${REPOSITORY_NAME}_version.h`` defines C |
| 7116 |
pre-processor macros that give the repository version number in several |
| 7117 |
formats, which allows C/C++ code (or any software that uses the C |
| 7118 |
preprocessor) to write conditional code like:: |
| 7119 |
|
| 7120 |
#if Trilinos_MAJOR_MINOR_VERSION > 100200 |
| 7121 |
/* Contains feature X */ |
| 7122 |
... |
| 7123 |
#else |
| 7124 |
/* Does not contain feature X */ |
| 7125 |
... |
| 7126 |
#endif |
| 7127 |
|
| 7128 |
Of course when the TriBITS project and the TriBITS repository are the same |
| 7129 |
directory, the ``<projectDir>/Version.cmake`` and ``<repoDir>/Version.cmake`` |
| 7130 |
files are the same file, which works just fine. |
| 7131 |
|
| 7132 |
|
| 7133 |
TriBITS Environment Probing and Setup |
| 7134 |
------------------------------------- |
| 7135 |
|
| 7136 |
Part of the TriBITS Framework is to probe the environment, set up the |
| 7137 |
compilers, and get ready to compile code. This was mentioned in `Full |
| 7138 |
Processing of TriBITS Project Files`_. This is executed by the TriBITS macro |
| 7139 |
``tribits_setup_env()``. Some of the things this macro does are: |
| 7140 |
|
| 7141 |
.. _Probe and set up the environment: |
| 7142 |
|
| 7143 |
**Probe and set up the environment:** |
| 7144 |
|
| 7145 |
* Set ``CMAKE_BUILD_TYPE`` default (if not already set) |
| 7146 |
* Set up for MPI (MPI compilers, etc.) |
| 7147 |
* Set up C, C++, and Fortran compilers using ``enable_language(<LANG>)`` |
| 7148 |
* ``include(`` `<projectDir>/cmake/ProjectCompilerPostConfig.cmake`_ ``)`` |
| 7149 |
* Find Perl (sets ``PERL_EXECUTABLE``) |
| 7150 |
* Determine mixed language C/Fortran linking |
| 7151 |
* Set up OpenMP (with ``find_package(OpenMP)``) |
| 7152 |
* Set up optional Windows support |
| 7153 |
* Find Doxygen (sets ``DOXYGEN_EXECUTABLE``) |
| 7154 |
* Perform some other configure-time tests (see ``cmake`` configure output) |
| 7155 |
|
| 7156 |
At the completion of this part of the processing, the TriBITS CMake project is |
| 7157 |
ready to compile code. All of the major variables set as part of this process |
| 7158 |
are printed to the ``cmake`` stdout when the project is configured. |
| 7159 |
|
| 7160 |
|
| 7161 |
Tricky considerations for TriBITS-generated <tplName>Config.cmake files |
| 7162 |
----------------------------------------------------------------------- |
| 7163 |
|
| 7164 |
An issue that comes up with external packages/TPLs like HDF5 that needs to be |
| 7165 |
discussed here is the fact that ``FindTPL<tplName>.cmake`` module files create |
| 7166 |
(See `How to add a new TriBITS TPL`_) and TriBITS installs package config |
| 7167 |
files of the name ``<tplName>Config.cmake``. These TriBITS-generated package |
| 7168 |
config files ``<tplName>Config.cmake`` could potentially be found by calls to |
| 7169 |
``find_package(<externalPkg>)`` (i.e. when ``<tplName> == <externalPkg>`` like |
| 7170 |
with HDF5). These TriBITS-generated ``<tplName>Config.cmake`` files are |
| 7171 |
primarily meant to provide a `TriBITS-compliant external package`_ for |
| 7172 |
downstream TriBITS-compliant ``<Package>Config.cmake`` files. These |
| 7173 |
TriBITS-generated ``<tplName>Config.cmake`` files will usually not behave the |
| 7174 |
same way existing ``Find<tplName>.config`` find modules or native |
| 7175 |
``<tplName>Config.cmake`` package config files would behave as expected by |
| 7176 |
downstream projects when found by ``find_package(<tplName>)`` commands called |
| 7177 |
in some arbitrary downstream raw CMake project. Therefore, to avoid having an |
| 7178 |
installed TriBITS-generated ``HDF5Config.cmake`` file, for example, being |
| 7179 |
found by the inner call to ``find_package(HDF5 ...)`` in the file |
| 7180 |
``FindTPLHDF5.cmake`` (which could be disastrous), TriBITS employs two |
| 7181 |
safeguards. |
| 7182 |
|
| 7183 |
First, TriBITS-generated ``<tplName>Config.cmake`` package config files are |
| 7184 |
placed into the build directory under:: |
| 7185 |
|
| 7186 |
<buildDir>/external_packages/<tplName>/<tplName>Config.cmake |
| 7187 |
|
| 7188 |
and installed into the installation directory under:: |
| 7189 |
|
| 7190 |
<installDir>/lib/external_packages/<tplName>/<tplName>Config.cmake |
| 7191 |
|
| 7192 |
so they will not be found by ``find_package(<tplName>)`` by default when |
| 7193 |
``<buildDir>/cmake_packages`` and/or ``<installDir>``, respectively, are added |
| 7194 |
to ``CMAKE_PREFIX_PATH``. |
| 7195 |
|
| 7196 |
Second, even if the directories ``<installDir>/lib/external_packages`` or |
| 7197 |
``<buildDir>/external_packages`` do get added to the search path somehow |
| 7198 |
(e.g. by appending those to ``CMAKE_INSTALL_PREFIX``), the companion |
| 7199 |
TriBITS-generated ``<tplName>ConfigVersion.cmake`` files will set |
| 7200 |
``PACKAGE_VERSION_COMPATIBLE=OFF`` and result in ``find_package(<tplName>)`` |
| 7201 |
not selecting the TriBITS-generated ``<tplName>Config.cmake`` file. (It turns |
| 7202 |
out that CMake's ``find_package(<Package>)`` command always includes the file |
| 7203 |
``<Package>ConfigVersion.cmake``, even if no version information is passed to |
| 7204 |
the command ``find_package(<Package>)``. This allows special logic to be |
| 7205 |
placed in the file ``<Package>ConfigVersion.cmake`` to determine if |
| 7206 |
``find_package(<Package>)`` will select a given ``<Package>Config.cmake`` file |
| 7207 |
that is in the search path based on a number of different criteria such as in |
| 7208 |
this case.) |
| 7209 |
|
| 7210 |
|
| 7211 |
Installation considerations |
| 7212 |
--------------------------- |
| 7213 |
|
| 7214 |
For the most part, installation is pretty straightforward with a TriBITS-based |
| 7215 |
CMake project. TriBITS automatically puts in appropriate default |
| 7216 |
``install()`` commands to install header files, libraries, executables, and |
| 7217 |
other commonly installed artifacts (such as TriBITS-autogenerated |
| 7218 |
``<Package>Config.cmake`` files). And packages can add their own custom |
| 7219 |
``install()`` commands to install items under ``CMAKE_INSTALL_PREFIX`` (or the |
| 7220 |
subdirs under ``CMAKE_INSTALL_PREFIX`` mentioned in `Setting the install |
| 7221 |
prefix`_). However, there are some special situations that need to be |
| 7222 |
addressed and some tweaks to built-in CMake support that need to be made. |
| 7223 |
|
| 7224 |
One issue that can occur is that there are cases where a Unix/Linux system is |
| 7225 |
set up not to honor the group sticky bit and therefore one cannot control what |
| 7226 |
group owns the created installed files and directories (i.e. the default group |
| 7227 |
will be used). Also, there are cases were one cannot easily control the |
| 7228 |
default file or directory creation permissions using ``umask``. And there are |
| 7229 |
cases where one would like to recursively install a set of directories and |
| 7230 |
files where some of these files may be scripts that need to have the execute |
| 7231 |
permission set on them for them to work. The only to flexible accomplish |
| 7232 |
that with CMake (if one does not know the exist list of those files or |
| 7233 |
extensions of those files) is to pass in the ``SOURCE_PERMISSIONS`` option to |
| 7234 |
the ``install(DIRECTORY ...)`` command. An example of this is shown in: |
| 7235 |
|
| 7236 |
* ``TribitsExampleProject/packages/with_subpackages/b/CMakeLists.txt`` |
| 7237 |
|
| 7238 |
that has:: |
| 7239 |
|
| 7240 |
install( DIRECTORY "${CMAKE_CURRENT_LIST_DIR}/stuff" |
| 7241 |
DESTINATION "${CMAKE_INSTALL_PREFIX}/share/${PACKAGE_NAME}" |
| 7242 |
USE_SOURCE_PERMISSIONS PATTERN "*~" EXCLUDE ) |
| 7243 |
|
| 7244 |
In this case, CMake will preserve the execute permission on any of the scripts |
| 7245 |
contained under the ``stuff/`` subdirectory but ``group`` and ``other`` |
| 7246 |
permissions will not be set based on ``umask`` or the default CMake install |
| 7247 |
permissions. Instead, these permissions are set based on the source directory |
| 7248 |
permissions (which is often set to ``700`` or ``rwx------``). |
| 7249 |
|
| 7250 |
To address cases like this, TriBITS can automatically run ``chgrp`` and |
| 7251 |
``chmod`` on the created files and directories that are created during the |
| 7252 |
``install`` target as described in `Setting install ownership and |
| 7253 |
permissions`_. This is completely automatic and requires nothing for the |
| 7254 |
TriBITS Project developers to do to enable support for this (other than to |
| 7255 |
note the below warning). |
| 7256 |
|
| 7257 |
**WARNING**: Do not add any ``install()`` commands after the |
| 7258 |
`tribits_project()`_ command completes. Otherwise, any extra files or |
| 7259 |
directories will not have their group and permissions fixed by these special |
| 7260 |
TriBITS-added ``chgrp`` and ``chmod`` commands run at install time. Instead, |
| 7261 |
try to put all ``install()`` commands inside of a package's |
| 7262 |
`<packageDir>/CMakeLists.txt`_ file. Currently, there really is no good place |
| 7263 |
to add repo-level or project-level ``install()`` commands. But if one had to |
| 7264 |
sneak them in, they could add various ``install()`` commands to files like |
| 7265 |
`<projectDir>/CMakeLists.txt`_ (before the ``tribits_project_()`` command), |
| 7266 |
`<repoDir>/cmake/CallbackSetupExtraOptions.cmake`_, |
| 7267 |
`<projectDir>/cmake/CallbackDefineProjectPackaging.cmake`_ and/or |
| 7268 |
`<repoDir>/cmake/CallbackDefineRepositoryPackaging.cmake`_. (Note that |
| 7269 |
install commands from the former two files are run before install commands for |
| 7270 |
the enabled packages while install commands from the latter two files are run |
| 7271 |
after.) |
| 7272 |
|
| 7273 |
One can also change what compilers are written into the generated |
| 7274 |
``<Project>Config.cmake`` and ``<Package>Config.cmake`` files for the build |
| 7275 |
and the install trees. By default, the compilers pointed to in these |
| 7276 |
``Config.cmake`` files will be ``CMAKE_<LANG>_COMPILER`` where ``<LANG>`` = |
| 7277 |
``CXX``, ``C``, and ``Fortran``. But one can change this by setting any of |
| 7278 |
the following:: |
| 7279 |
|
| 7280 |
set(CMAKE_CXX_COMPILER_FOR_CONFIG_FILE_BUILD_DIR <path>) |
| 7281 |
set(CMAKE_C_COMPILER_FOR_CONFIG_FILE_BUILD_DIR <path>) |
| 7282 |
set(CMAKE_Fortran_COMPILER_FOR_CONFIG_FILE_BUILD_DIR <path>) |
| 7283 |
set(CMAKE_CXX_COMPILER_FOR_CONFIG_FILE_INSTALL_DIR <path>) |
| 7284 |
set(CMAKE_C_COMPILER_FOR_CONFIG_FILE_INSTALL_DIR <path>) |
| 7285 |
set(CMAKE_Fortran_COMPILER_FOR_CONFIG_FILE_INSTALL_DIR <path>) |
| 7286 |
|
| 7287 |
before the ``Config.cmake`` files are generated. These can also be set in the |
| 7288 |
CMake cache using, for example, |
| 7289 |
``-DCMAKE_CXX_COMPILER_FOR_CONFIG_FILE_INSTALL_DIR:FILEPATH=<path>``. |
| 7290 |
|
| 7291 |
This is used, for example, when compiler wrappers are used for the build tree |
| 7292 |
and are set to ``CMAKE_<LANG>_COMPILER`` but when one wants to point to the |
| 7293 |
original underlying compilers for the installed ``Config.cmake`` files. |
| 7294 |
|
| 7295 |
|
| 7296 |
RPATH Handling |
| 7297 |
-------------- |
| 7298 |
|
| 7299 |
As explained in `Setting install RPATH`_, TriBITS changes the CMake defaults |
| 7300 |
to write in the RPATH for shared libraries and executables so that they run |
| 7301 |
right out of the install directory without needing to set paths in the |
| 7302 |
environment (e.g. ``LD_LIBRARY_PATH``). However, these defaults can be |
| 7303 |
changed by changing setting different project defaults for the variables |
| 7304 |
`${PROJECT_NAME}_SET_INSTALL_RPATH`_ and `CMAKE_INSTALL_RPATH_USE_LINK_PATH`_. |
| 7305 |
But most projects should likely keep these defaults in place since they make |
| 7306 |
it so that doing builds and installations on a single machine work correctly |
| 7307 |
by default out of the box. For other installation/distribution use cases, the |
| 7308 |
user is told how to manipulate CMake variables for those cases in `Setting |
| 7309 |
install RPATH`_. |
| 7310 |
|
| 7311 |
|
| 7312 |
Configure-time System Tests |
| 7313 |
--------------------------- |
| 7314 |
|
| 7315 |
CMake has good support for defining configure-time checks of the system to |
| 7316 |
help in configuring the project. One can check for whether a header file |
| 7317 |
exists, if the compiler supports a given data-type or language feature, or |
| 7318 |
perform almost any other type of check that one can imagine that can be done |
| 7319 |
using the configured compilers, libraries, system tools, etc. An example was |
| 7320 |
given in `TribitsExampleProject`_. Just follow that example, look at some of |
| 7321 |
the built-in CMake configure-time test modules, and consult provided on-line |
| 7322 |
CMake documentation in order to learn how to create a configure-time test for |
| 7323 |
almost anything. |
| 7324 |
|
| 7325 |
.. ToDo: Provide more detail how to integrate a configure-time test into a |
| 7326 |
.. TriBITS project and TriBITS package. |
| 7327 |
|
| 7328 |
|
| 7329 |
Creating Source Distributions |
| 7330 |
----------------------------- |
| 7331 |
|
| 7332 |
The TriBITS system uses CMake's built-in CPack support to create source |
| 7333 |
distributions in a variety of zipped and tarred formats. (Note that the term |
| 7334 |
"source tarball" or just "tarball" may be used below but should be interpreted |
| 7335 |
as "source distribution".) TriBITS will automatically add support for CPack |
| 7336 |
when the variable `${PROJECT_NAME}_ENABLE_CPACK_PACKAGING`_ is set to ``ON``. |
| 7337 |
The commands for creating a source distribution are described in `Creating a |
| 7338 |
tarball of the source tree`_ using the built-in ``package_source`` build |
| 7339 |
target. The value added by TriBITS is that TriBITS will automatically exclude |
| 7340 |
the source for any defined packages that are not enabled and TriBITS provides |
| 7341 |
a framework for systematically excluding files and directories from individual |
| 7342 |
repositories and packages. In addition, the source for non-enabled |
| 7343 |
subpackages can also be excluded depending on the value of |
| 7344 |
`${PROJECT_NAME}_EXCLUDE_DISABLED_SUBPACKAGES_FROM_DISTRIBUTION`_. All of |
| 7345 |
this allows one to create distributions which only includes subsets of a |
| 7346 |
larger project (even a single package in some cases). |
| 7347 |
|
| 7348 |
Unlike other build systems (like autotools), CMake will put **EVERYTHING** |
| 7349 |
into the source distribution (e.g. tarball) that is sitting in the source tree |
| 7350 |
by default. Therefore, setting up for a source distribution usually means |
| 7351 |
deciding what extra files and directories should be excluded. Beyond the |
| 7352 |
directories for non-enabled packages, further files can be selected to be |
| 7353 |
excluded on a package-by-package based and at the repository level (see |
| 7354 |
below). |
| 7355 |
|
| 7356 |
Individual packages can list additional files/directories under the package's |
| 7357 |
source tree to be excluded from the project's source distribution using a call |
| 7358 |
to `tribits_exclude_files()`_ in their `<packageDir>/CMakeLists.txt`_ file. |
| 7359 |
Note that if the package is not enabled, these excludes will never get added! |
| 7360 |
That is no problem if these excludes only apply to the given package since |
| 7361 |
TriBITS will add an exclude for the entire package but is a problem if a |
| 7362 |
package lists excludes for files outside of the package's source tree. |
| 7363 |
|
| 7364 |
Additional files and entire directories can also be excluded at the repository |
| 7365 |
level by listing them in the |
| 7366 |
`<repoDir>/cmake/CallbackDefineRepositoryPackaging.cmake`_ file which |
| 7367 |
typically just appends the built-in CMake variable |
| 7368 |
``CPACK_SOURCE_IGNORE_FILES``. However, if a repository is not processed, |
| 7369 |
this file is never processed and therefore no files will be excluded from a |
| 7370 |
repository sitting in the source tree that is not processed (see below). |
| 7371 |
|
| 7372 |
There are a number of project-level settings that need to be defined and these |
| 7373 |
are specified in the file |
| 7374 |
`<projectDir>/cmake/CallbackDefineProjectPackaging.cmake`_. |
| 7375 |
|
| 7376 |
The `TribitsExampleProject`_ is set up for creating source distributions and |
| 7377 |
this is demonstrated in one of the tests defined in:: |
| 7378 |
|
| 7379 |
TriBITS/test/core/ExamplesUnitTests/CMakeLists.txt |
| 7380 |
|
| 7381 |
There are a few points of caution to note about creating source distributions. |
| 7382 |
|
| 7383 |
**NOTE:** It is worth stressing again that **EVERY** file that is in the |
| 7384 |
source tree will be included in the source distribution (tarball) unless there |
| 7385 |
is an exclude regex matching it appended to the variable |
| 7386 |
``CPACK_SOURCE_IGNORE_FILES``. TriBITS can only add excludes for defined |
| 7387 |
non-enabled packaged. Every other file listed in the source tree will be |
| 7388 |
included in the tarball. |
| 7389 |
|
| 7390 |
**NOTE:** The entries in ``CPACK_SOURCE_IGNORE_FILES`` are interpreted as |
| 7391 |
**REGULAR EXPRESSIONS** not globs so if you add ``"someFile.*"`` as an |
| 7392 |
exclude, it will exclude every file in the entire source tree that has |
| 7393 |
``"someFile"`` in the name! This is because, in regex terminology, the |
| 7394 |
trailing ``".*"`` means "match any character zero or more times" and |
| 7395 |
``"someFile"`` can match anywhere in the file name path. Also, note that if |
| 7396 |
you add in an exclude like ``"*.pyc"`` (i.e. trying to exclude all of the |
| 7397 |
generated Python byte code files) that it will exclude every file that has |
| 7398 |
``"pyc"`` in the name and **not** just those with the file extension |
| 7399 |
``"pyc"``. For example, the exclude ``".pyc"`` would exclude the files |
| 7400 |
``"puppyc"``, ``"lpycso"``, etc. If you want to exclude all files with |
| 7401 |
extension ``"pyc"``, you have to add the exclude regex ``".*[.]pyc$"``! |
| 7402 |
One's lack of understanding of this fact will cost someone hours of lost time |
| 7403 |
debugging what happens when random files are missing when one tries to |
| 7404 |
configure what is left. Sometimes, what is left will actually configure and |
| 7405 |
might almost build! |
| 7406 |
|
| 7407 |
**NOTE:** As warned in `TriBITS Package Core Files`_ and `TriBITS Subpackage |
| 7408 |
Core Files`_, Packages must have directories that are strictly independent |
| 7409 |
of the directories of other packages. If they don't, then the source |
| 7410 |
directory for an enabled package will get excluded from the source |
| 7411 |
distribution if its directory is under the directory of a package that is not |
| 7412 |
enabled. For example, if ``PackageA`` is enabled but its package directory |
| 7413 |
``packageb/packagea/`` is under the package directory ``packageb/`` for the |
| 7414 |
disabled package ``PackageB``, then every file and directory under |
| 7415 |
``packageb/`` will be excluded from the source distribution (tarball), |
| 7416 |
including everything under ``packageb/packagea/``! It would be too expensive |
| 7417 |
to put in an automated check for cases like this so package developers should |
| 7418 |
just take care not to nest the directories of packages inside of each other to |
| 7419 |
avoid problems like this. |
| 7420 |
|
| 7421 |
**NOTE:** Extra repositories that are sitting in the source tree but not |
| 7422 |
processed by TriBITS for some reason (e.g. due to explicitly listing in the |
| 7423 |
variable `${PROJECT_NAME}_EXTRA_REPOSITORIES`_ only a subset of the repositories |
| 7424 |
listed in `<projectDir>/cmake/ExtraRepositoriesList.cmake`_ ) will get added |
| 7425 |
to the source distribution in full by default even though there are no enabled |
| 7426 |
packages from these repos. These non-processed repo dirs are like any other |
| 7427 |
random directory sitting in the source tree, they will get copied over into |
| 7428 |
the source distribution!. |
| 7429 |
|
| 7430 |
**NOTE:** When debugging tarball creation problems, always configure with the |
| 7431 |
variable ``<Project>_DUMP_CPACK_SOURCE_IGNORE_FILES=ON``. If you don't see a |
| 7432 |
regex listed for the file or directory you expect to be excluded, then that |
| 7433 |
file/directory it will be included in the source distribution! |
| 7434 |
|
| 7435 |
|
| 7436 |
Using Git Bisect with checkin-test.py workflows |
| 7437 |
----------------------------------------------- |
| 7438 |
|
| 7439 |
There are cases where a customer will do an update of an upstream project from |
| 7440 |
a git repo and then find out that some feature or behavior is broken with |
| 7441 |
respect to their usage. This can happen even if the upstream project's own |
| 7442 |
test suite passes all of its tests. Depending on the situation, there may be |
| 7443 |
hundreds to thousands of commits between the last known "good" working version |
| 7444 |
of the code and pulled "bad" version. To help customers find the first commit |
| 7445 |
that contains the changes which are causing the breakage, git supplies ``git |
| 7446 |
bisect``. This set of commands does a binary search of the commits in the |
| 7447 |
range ``<good-sha>..<bad_sha>`` and finds the first commit that is "bad" (or a |
| 7448 |
range of commits which contains the first "bad" commit if commits are skipped, |
| 7449 |
as described below). |
| 7450 |
|
| 7451 |
But the ``git bisect`` commands require that all of the commits in the range |
| 7452 |
``<good-sha>..<bad_sha>`` be complete commits that provide full working |
| 7453 |
versions of the upstream project. However, many beginning git developers and |
| 7454 |
even many experienced developers don't always create individual git commits |
| 7455 |
that build and pass all of the upstream project's tests and therefore can |
| 7456 |
create false "bad" commits during the binary search. This can happen when |
| 7457 |
developers create intermediate "check-point" commits during the development |
| 7458 |
process but did not squash the intermediate commits together to create |
| 7459 |
cohesive working commits. This can also happen when experienced developers |
| 7460 |
have a set of what they believe are all working commits but do not actually |
| 7461 |
test all of the commits to know that they pass *all* of the upstream project's |
| 7462 |
tests before pushing these commits to the main development branch. This lack |
| 7463 |
of detailed testing of each and every individual commit can give rise to false |
| 7464 |
"bad" commits which will result in ``git bisect`` reporting the wrong first |
| 7465 |
"bad" commit. |
| 7466 |
|
| 7467 |
Projects that use the `checkin-test.py`_ tool to push sets of commits to the |
| 7468 |
main development branch have an advantage in the usage of ``git bisect``. |
| 7469 |
This is because the default mode of the ``checkin-test.py`` script is to amend |
| 7470 |
the top commit message with a summary of what was tested and therefore marks a |
| 7471 |
set of commits that are known to have more complete testing. For example, the |
| 7472 |
``checkin-test.py`` tool amends the top commit (after the final pull and |
| 7473 |
rebase by default) as shown in the following Trilinos commit:: |
| 7474 |
|
| 7475 |
commit 71ce56bd2d268922fda7b8eca74fad0ffbd7d807 |
| 7476 |
Author: Roscoe A. Bartlett <[email protected]> |
| 7477 |
Date: Thu Feb 19 12:04:11 2015 -0500 |
| 7478 |
|
| 7479 |
Define HAVE_TEUCHOSCORE_CXX11 in TeuchosCore_config.h |
| 7480 |
|
| 7481 |
This makes TeuchosCore a good example for how Trilinos (or any TriBITS) |
| 7482 |
subpackages should put in an optional dependency on C++11. |
| 7483 |
|
| 7484 |
Build/Test Cases Summary |
| 7485 |
Enabled Packages: TeuchosCore |
| 7486 |
Disabled Packages: [...] |
| 7487 |
0) MPI_DEBUG => passed: passed=44,notpassed=0 (2.61 min) |
| 7488 |
1) SERIAL_RELEASE => passed: passed=43,notpassed=0 (1.08 min) |
| 7489 |
|
| 7490 |
Therefore, these special known-tested commits can be flagged by grepping the |
| 7491 |
``git log -1 HEAD`` output for the string ``"Build/Test Cases Summary"``. By |
| 7492 |
bisecting on these commits, one has a lower chance of encountering false "bad" |
| 7493 |
commits and has a higher chance of finding a smaller range of commits where |
| 7494 |
the first true "bad" commit might be found. To aid in performing ``git |
| 7495 |
bisect`` and only checking ``checkin-test.py``-tested commits, the tool |
| 7496 |
`is_checkin_tested_commit.py`_ is provided. |
| 7497 |
|
| 7498 |
To demonstrate how the ``is_checkin_tested_commit.py`` tool can be used with |
| 7499 |
``git bisect``, suppose that someone writes a customized script |
| 7500 |
``build_and_test_customer_code.sh`` that will build the upstream project and |
| 7501 |
the downstream customer's code and then run a set of tests to see if the "bad" |
| 7502 |
behavior seen by the customer code is the current ``HEAD`` version of the |
| 7503 |
upstream project. Assume this script is copied into the upstream project's |
| 7504 |
local git repo using:: |
| 7505 |
|
| 7506 |
$ cd <upstream-repo>/ |
| 7507 |
$ cp ~/build_and_test_customer_code.sh . |
| 7508 |
$ cat /build_and_test_customer_code.sh >> .git/info/exclude |
| 7509 |
|
| 7510 |
Now, one could use ``build_and_test_customer_code.sh`` directly with:: |
| 7511 |
|
| 7512 |
$ git bisect run ./build_and_test_customer_code.sh |
| 7513 |
|
| 7514 |
but that would result in testing *all* the commits which may have a high |
| 7515 |
chance of producing false "bad" commits as described above and fail to |
| 7516 |
correctly bracket the location of the true first "bad" commit. |
| 7517 |
|
| 7518 |
So instead, one can write a filtered testing script |
| 7519 |
``safe_build_and_test_customer_code.sh`` which calls |
| 7520 |
``is_checkin_tested_commit.py`` and ``build_and_test_customer_code.sh`` as |
| 7521 |
follows:: |
| 7522 |
|
| 7523 |
#!/bin/bash |
| 7524 |
# |
| 7525 |
# Script: safe_build_and_test_customer_code.sh |
| 7526 |
|
| 7527 |
$TRIBITS_DIR/ci_support/is_checkin_tested_commit.py |
| 7528 |
IS_CHECKIN_TESTED_COMMIT_RTN=$? |
| 7529 |
if [ "$IS_CHECKIN_TESTED_COMMIT_RTN" != "0" ] ; then |
| 7530 |
exit 125 # Skip the commit because HEAD is not known to be tested! |
| 7531 |
fi |
| 7532 |
|
| 7533 |
./build_and_test_customer_code.sh # Rtn 0 "good", or [1, 124] if "bad" |
| 7534 |
|
| 7535 |
The above test script ``safe_build_and_test_customer_code.sh`` will skip the |
| 7536 |
testing of commits that are not marked by the ``checkin-test.py`` tool. |
| 7537 |
|
| 7538 |
To demonstrate how to use the ``is_checkin_tested_commit.py`` script with |
| 7539 |
``git bisect``, an example from Trilinos is used below. (Note that the |
| 7540 |
current Trilinos public repository may have been filtered so the commit SHA1s |
| 7541 |
shown below may not match what is in the current Trilinos repository. But one |
| 7542 |
can use the commit summary message, author, and author date to find the |
| 7543 |
updated SHA1s and then to update this example for the current repository.) |
| 7544 |
|
| 7545 |
Consider a scenario where a customer application updates Trilinos from the |
| 7546 |
commit:: |
| 7547 |
|
| 7548 |
d44c17d "Merge branch 'master' of software.sandia.gov:/space/git/Trilinos" |
| 7549 |
Author: Roscoe A. Bartlett <[email protected]> |
| 7550 |
Date: Tue May 26 12:43:25 2015 -0400 |
| 7551 |
|
| 7552 |
to the commit:: |
| 7553 |
|
| 7554 |
605b91b "Merge branch 'master' of software.sandia.gov:/git/Trilinos" |
| 7555 |
Author: Vitus Leung <[email protected]> |
| 7556 |
Date: Tue Sep 29 20:18:54 2015 -0600 |
| 7557 |
|
| 7558 |
and it is found that some critical feature broke or is not behaving acceptably |
| 7559 |
for the customer code (but all of the tests for Trilinos pass just fine). |
| 7560 |
This range of commits ``d44c17d..605b91b`` gives 2257 commits to search as |
| 7561 |
shown by:: |
| 7562 |
|
| 7563 |
$ cd Trilinos/ |
| 7564 |
$ git log-oneline d44c17d..605b91b | wc -l |
| 7565 |
2257 |
| 7566 |
|
| 7567 |
However, as described above, it is likely that doing ``git bisect`` on that |
| 7568 |
full set of 2257 commits may result in hitting false "bad" commits and |
| 7569 |
therefore result in a false bracketing of the first "bad" commit. This is |
| 7570 |
where the usage of the ``checkin-test.py`` tool helps which is used by many |
| 7571 |
(but not currently all) Trilinos developers to push changes to the Trilinos |
| 7572 |
'master' branch in the current single-branch workflow. The commits marked |
| 7573 |
with the ``checkin-test.py`` tool are known (with some caveats mentioned |
| 7574 |
below) to be working commits and for this the range of commits |
| 7575 |
``d44c17d..605b91b`` yields 166 commits as shown by:: |
| 7576 |
|
| 7577 |
$ git log-oneline --grep="Build/Test Cases Summary" d44c17d..605b91b | wc -l |
| 7578 |
166 |
| 7579 |
|
| 7580 |
That is an average of 2257/166 = 13.6 commits between commits pushed with the |
| 7581 |
``checkin-test.py`` tool. So bisecting on just the commits marked by |
| 7582 |
``checkin-test.py`` should bound the "bad" commit in a set of 13.6 commits on |
| 7583 |
average. Bisecting on this set of 166 commits should likely give no false |
| 7584 |
“bad” commits, and therefore result in the correct bracketing of the first |
| 7585 |
"bad" commit. |
| 7586 |
|
| 7587 |
Using the ``safe_build_and_test_customer_code.sh`` shown above, one would |
| 7588 |
search for the first bad commit over this range using:: |
| 7589 |
|
| 7590 |
$ git bisect start 605b91b d44c17d |
| 7591 |
$ env DUMMY_TEST_COMMIT_BAD_SHA=83f05e8 \ |
| 7592 |
time git bisect run ./safe_build_and_test_customer_code.sh |
| 7593 |
|
| 7594 |
and this would return the range of commits that contains the first "bad" |
| 7595 |
commit (listed at the end of ``git bisect log`` output, see example below). |
| 7596 |
|
| 7597 |
To provide a concrete example, suppose the commit that first introduced the |
| 7598 |
problem in the range of commits ``d44c17d..605b91b`` was:: |
| 7599 |
|
| 7600 |
83f05e8 "MueLu: stop semi-coarsening if no z-layers are left." |
| 7601 |
Author: Tobias Wiesner <[email protected]> |
| 7602 |
Date: Wed Jul 1 14:54:20 2015 -0600 |
| 7603 |
|
| 7604 |
And instead of using the script ``safe_build_and_test_customer_code.sh``, we |
| 7605 |
use a dummy driver script ``dummy_test_commit.sh`` to simulate this which is |
| 7606 |
provided in the set of TriBITS documentation:: |
| 7607 |
|
| 7608 |
$TRIBITS_DIR/doc/developers_guide/scripts/dummy_test_commit.sh |
| 7609 |
|
| 7610 |
as: |
| 7611 |
|
| 7612 |
.. include:: scripts/dummy_test_commit.sh |
| 7613 |
:literal: |
| 7614 |
|
| 7615 |
This driver script allows one to simulate the usage of ``git bisect`` to |
| 7616 |
understand how it works without having to actually build and test code. It is |
| 7617 |
a useful training and experimentation tool. |
| 7618 |
|
| 7619 |
Using ``git bisect`` (with git version 2.1.0) over the range of commits |
| 7620 |
``d44c17d..605b91b`` searching for the first "bad" commit is done by running |
| 7621 |
the commands:: |
| 7622 |
|
| 7623 |
$ git bisect start 605b91b d44c17d |
| 7624 |
$ env DUMMY_TEST_COMMIT_BAD_SHA=83f05e8 \ |
| 7625 |
time git bisect run \ |
| 7626 |
$TRIBITS_DIR/doc/developers_guide/scripts/dummy_test_commit.sh \ |
| 7627 |
&> ../git_bisect_run.log |
| 7628 |
$ git2 bisect log &> ../git_bisect_log.log |
| 7629 |
$ cat ../git_bisect_log.log | grep "possible first bad commit" | \ |
| 7630 |
sed "s/possible first bad commit://g" | sed "s/[a-z0-9]\{30\}\]/]/g" |
| 7631 |
$ git bisect reset |
| 7632 |
|
| 7633 |
This set of commands yield the output:: |
| 7634 |
|
| 7635 |
Bisecting: 1128 revisions left to test after this (roughly 10 steps) |
| 7636 |
[9634d462dba77704b598e89ba69ba3ffa5a71471] Revert "Trilinos: remove _def.hpp [...]" |
| 7637 |
|
| 7638 |
real 1m22.961s |
| 7639 |
user 0m57.157s |
| 7640 |
sys 3m40.376s |
| 7641 |
|
| 7642 |
# [165067ce53] MueLu: SemiCoarsenPFactory. Use strided maps to properly transfer [...] |
| 7643 |
# [ada21a95a9] MueLu: refurbish LineDetectionFactory |
| 7644 |
# [83f05e8970] MueLu: stop semi-coarsening if no z-layers are left. |
| 7645 |
|
| 7646 |
Previous HEAD position was 83f05e8... MueLu: stop semi-coarsening if no z-layers are left. |
| 7647 |
Switched to branch 'master' |
| 7648 |
|
| 7649 |
This output shows the dummy bad commit 83f05e8 in a set of just 3 commits, |
| 7650 |
bounded in the set of commits ``8b79832..165067c``:: |
| 7651 |
|
| 7652 |
165067c "MueLu: SemiCoarsenPFactory. Use strided maps to properly [...]." |
| 7653 |
Author: Tobias Wiesner <[email protected]> |
| 7654 |
Date: Thu Jul 2 12:11:24 2015 -0600 |
| 7655 |
|
| 7656 |
8b79832 "Ifpack2: RBILUK: adding additional ETI types" |
| 7657 |
Author: Jonathan Hu <[email protected]> |
| 7658 |
Date: Thu Jul 2 14:17:40 2015 -0700 |
| 7659 |
|
| 7660 |
The set of commits that were actually tested by ``git bisect run <script>`` is |
| 7661 |
shown by:: |
| 7662 |
|
| 7663 |
$ cat ../git_bisect_log.log | grep "\(good:\|bad:\)" | sed "s/[a-z0-9]\{30\}\]/]/g" |
| 7664 |
# bad: [605b91b012] Merge branch 'master' of software.sandia.gov:/git/Trilinos |
| 7665 |
# good: [d44c17d5d2] Merge branch 'master' of software.sandia.gov:/space/git/Trilinos |
| 7666 |
# good: [7e13a95774] Ifpack2: If the user does not provide the bandwidth of the banded [...] |
| 7667 |
# bad: [7335d8bc92] MueLu: fix documentation |
| 7668 |
# bad: [9997ecf0ba] Belos::LSQRSolMgr: Fixed bug in setParameters. |
| 7669 |
# bad: [b6e0453224] MueLu: add a nightly test for the combination of semicoarsening [...] |
| 7670 |
# bad: [165067ce53] MueLu: SemiCoarsenPFactory. Use strided maps to properly [...] |
| 7671 |
# good: [3b5453962e] Ifpack2: Nuking the old ETI system |
| 7672 |
# good: [8b79832f1d] Ifpack2: RBILUK: adding additional ETI types |
| 7673 |
|
| 7674 |
This is only 9 commits out of the possible set of 166 ``checkin-test.py`` |
| 7675 |
marked commits which is out of the total set of 2257 possible commits. With |
| 7676 |
just 9 build/test cycles, it bounded the first "bad" commit in a set of 3 |
| 7677 |
commits in this case. And it does not matter how sloppy or broken the |
| 7678 |
intermediate commits are in Trilinos. All that matters is the usage of the |
| 7679 |
``checkin-test.py`` tool (another motivation for the usage of the |
| 7680 |
``checkin-test.py`` tool, see `Pre-push Testing using checkin-test.py`_ for |
| 7681 |
others as well). |
| 7682 |
|
| 7683 |
Note that above, we grep the output from ``git bisect log`` for the set of |
| 7684 |
possible "bad" commits instead of just looking at the output from the ``git |
| 7685 |
bisect run <script>`` command (which also lists the set of possible "bad" |
| 7686 |
commits). This is because the direct output from the ``git bisect run |
| 7687 |
<script>`` command (shown in the log file ``git_bisect_run.log``) shows the |
| 7688 |
set of possible bad commits at the end of the output but they are unsorted and |
| 7689 |
give no other git commit information:: |
| 7690 |
|
| 7691 |
There are only 'skip'ped commits left to test. |
| 7692 |
The first bad commit could be any of: |
| 7693 |
83f05e89706590c4b384dd191f51ef4ab00ce9bb |
| 7694 |
ada21a95a991cd238581e5a6a96800d209a57924 |
| 7695 |
165067ce538af2cd0bd403e2664171726ec86f3f |
| 7696 |
We cannot bisect more! |
| 7697 |
bisect run cannot continue any more |
| 7698 |
|
| 7699 |
The problem with unsorted commits is that it is difficult to use an unsorted |
| 7700 |
set to do further bisection. However, the output of the set of commits from |
| 7701 |
``git bisect log`` is sorted and also shows the commit summary message and |
| 7702 |
therefore is much more useful. (But note that older versions of git don’t |
| 7703 |
show this set of commits at the end of ``git bisect log`` so make sure and use |
| 7704 |
an updated version of git, preferably >= 2.1.0.) |
| 7705 |
|
| 7706 |
Now that one has the range of possible "bad" commits (just 3 in this example) |
| 7707 |
doing a further manual bisection or even manual inspection of these commits |
| 7708 |
may be enough to find the change that is causing the problem for the |
| 7709 |
downstream customer application. |
| 7710 |
|
| 7711 |
Without the usage of the ``checkin-test.py`` tool, one would not have an |
| 7712 |
automated way to ensure that ``git bisect`` avoids false "bad" commits. This |
| 7713 |
allows for less experienced developers to create commits and push to the main |
| 7714 |
development branch but still ensure effective usage of ``git bisect``. (This |
| 7715 |
is another example where automated tools in TriBITS help to overcome lacking |
| 7716 |
developer experience and discipline.) |
| 7717 |
|
| 7718 |
|
| 7719 |
Multi-Repository Almost Continuous Integration |
| 7720 |
---------------------------------------------- |
| 7721 |
|
| 7722 |
The `checkin-test.py`_ tool can be used to the implement staged integration of |
| 7723 |
the various repositories in a multi-repo TriBITS project (see |
| 7724 |
`Multi-Repository Support`_) . This is referred to here as Almost Continuous |
| 7725 |
Integration (ACI). The basic concept of Almost Continuous Integration (ACI) |
| 7726 |
is defined and described in the paper [`Integration Strategies for CSE, |
| 7727 |
2009`_]. |
| 7728 |
|
| 7729 |
This topic is broken down into the following subsections: |
| 7730 |
|
| 7731 |
* `ACI Multi-Git/TriBITS Repo Integration Example`_ |
| 7732 |
* `ACI Local Sync Git Repo Setup`_ |
| 7733 |
* `ACI Integration Build Directory Setup`_ |
| 7734 |
* `ACI Sync Driver Script`_ |
| 7735 |
* `ACI Cron Job Setup`_ |
| 7736 |
* `Addressing ACI Failures and Summary`_ |
| 7737 |
|
| 7738 |
|
| 7739 |
ACI Introduction |
| 7740 |
++++++++++++++++ |
| 7741 |
|
| 7742 |
The TriBITS system allows for setting up composite meta-builds of large |
| 7743 |
collections of software pulled in from many different git/TriBITS code |
| 7744 |
repositories as described in the section `Multi-Repository Support`_. The |
| 7745 |
`checkin-test.py`_ tool is a key tool to enable the testing of a set of |
| 7746 |
packages in different git/TriBITS repos before pushing to remote tracking |
| 7747 |
branches for the set of git repos; all in one robust command invocation. |
| 7748 |
|
| 7749 |
While the ``checkin-test.py`` tool was originally designed and its default |
| 7750 |
behavior is to test a set of local commits created by a developer before |
| 7751 |
pushing changes to one or more (public) git repos, it can also be used to set |
| 7752 |
up an Almost Continuous Integration (ACI) process to keep these various |
| 7753 |
git/TriBITS repos in sync thereby integrating the work of various disconnected |
| 7754 |
development teams and projects. To use the ``checkin-test.py`` tool for ACI |
| 7755 |
requires some setup and changing what the tool does a little by passing in |
| 7756 |
additional options that a regular developer typically never uses. |
| 7757 |
|
| 7758 |
The following subsections describe how to use the `checkin-test.py`_ tool to |
| 7759 |
implement an ACI process for a given set of git/TriBITS repositories and also |
| 7760 |
provides a little background and context behind ACI. |
| 7761 |
|
| 7762 |
|
| 7763 |
ACI Multi-Git/TriBITS Repo Integration Example |
| 7764 |
++++++++++++++++++++++++++++++++++++++++++++++ |
| 7765 |
|
| 7766 |
In order to set up the context for the ACI process, consider the following |
| 7767 |
simple TriBITS project with two extra repositories:: |
| 7768 |
|
| 7769 |
BaseProj/ |
| 7770 |
ExtraRepo1 |
| 7771 |
ExtraRepo2 |
| 7772 |
|
| 7773 |
Here, ``BaseProj`` is the base TriBITS project/repository and ``ExtraRepo1`` |
| 7774 |
and ``ExtraRepo2`` are extra repositories that supply additional TriBITS |
| 7775 |
packages that are appended to the TriBITS packages defined in ``BaseProj`` |
| 7776 |
(see `<projectDir>/cmake/ExtraRepositoriesList.cmake`_). Also, assume that |
| 7777 |
``BaseProj``, ``ExtraRepo1``, and ``ExtraRepo2`` are developed by three |
| 7778 |
different development teams that all have different funding sources and |
| 7779 |
different priorities so they tend not to work closely together or consider the |
| 7780 |
other efforts too much when developing their software. However, in this |
| 7781 |
example, there is great value in combining all of this software into a single |
| 7782 |
integrated TriBITS meta-project. This combined meta-build is driven by a 4th |
| 7783 |
integration/development team. In this case, the core developers for each of |
| 7784 |
these three different git/TriBITS repos do not test compatibility with the |
| 7785 |
other git/TriBITS repos when pushing commits to their own git/TriBITS repos. |
| 7786 |
This gives three different git repos on three different machines: |
| 7787 |
|
| 7788 |
* ``BaseProj`` main repo: Pushed to by the core BaseProj team:: |
| 7789 |
|
| 7790 |
url1.gov:/git/BaseProj |
| 7791 |
|
| 7792 |
* ``ExtraRepo1`` main repo: Pushed to by the core ExtraRepo1 team:: |
| 7793 |
|
| 7794 |
url2.gov:/git/ExtraRepo1 |
| 7795 |
|
| 7796 |
* ``ExtraRepo2`` main repo: Pushed to by the core ExtraRepo2 team:: |
| 7797 |
|
| 7798 |
url3.gov:/git/ExtraRepo2 |
| 7799 |
|
| 7800 |
Because of the independent development processes of these three teams, unless |
| 7801 |
these development teams maintain 100% backward compatibility w.r.t. the |
| 7802 |
interfaces and behavior of the combined software, one cannot at any time pull |
| 7803 |
the code from these three different git repos and expect to be able to |
| 7804 |
successfully build all of the code and have all of the tests pass. Therefore, |
| 7805 |
how does the 4th integration team expect to be able to build, test, and |
| 7806 |
possibly extend the combined software? In this case, the integration team |
| 7807 |
would set up their own clones of all three git/TriBITS repos on their own |
| 7808 |
machine such as: |
| 7809 |
|
| 7810 |
**Integration project mirrored git repos**:: |
| 7811 |
|
| 7812 |
url4.gov:/git/BaseProj |
| 7813 |
url4.gov:/git/ExtraRepo1 |
| 7814 |
url4.gov:/git/ExtraRepo2 |
| 7815 |
|
| 7816 |
Once an initial collaboration effort between the integration team and the |
| 7817 |
three other development teams is able to get a version of all three |
| 7818 |
git/TriBITS repos to work correctly in the combined meta-project, these |
| 7819 |
versions (assume the ``master`` branches) would be pushed to the git repos on |
| 7820 |
the git integration server ``url4.gov``. The state where the TriBITS packages |
| 7821 |
in the three different git/TriBITS repos in the ``master`` branch on |
| 7822 |
``url4.gov`` all work together correctly constitutes the initial condition for |
| 7823 |
the ACI process described below. From that initial condition, the ACI |
| 7824 |
processes ensures that updates the ``master`` branches for the git/TriBITS |
| 7825 |
repos on ``url4.gov`` do not break any builds or tests of the integrated |
| 7826 |
software. |
| 7827 |
|
| 7828 |
In order to describe how to set up an ACI process using the |
| 7829 |
``checkin-test.py`` tool, the following subsections will focus on the update |
| 7830 |
of the git/TriBITS ``ExtraRepo1`` repo keeping the other two git/TriBITS repos |
| 7831 |
``BaseProj`` and ``ExtraRepo2`` constant as the ACI use case. |
| 7832 |
|
| 7833 |
|
| 7834 |
ACI Local Sync Git Repo Setup |
| 7835 |
+++++++++++++++++++++++++++++ |
| 7836 |
|
| 7837 |
In order to set up an ACI process for the multi-git/TriBITS repo example |
| 7838 |
outlined above, first local repos are created by cloning the repos on the |
| 7839 |
integration server url4.gov as follows (all of which become 'origin'):: |
| 7840 |
|
| 7841 |
$ cd $SYNC_BASE_DIR |
| 7842 |
$ git clone url4.gov:/git/BaseProj |
| 7843 |
$ cd BaseProj |
| 7844 |
$ git clone url4.gov:/git/ExtraRepo1 |
| 7845 |
$ git clone url4.gov:/git/ExtraRepo2 |
| 7846 |
|
| 7847 |
where, ``SYNC_BASE_DIR=~/sync_base_dir`` for example, which must already be |
| 7848 |
created. |
| 7849 |
|
| 7850 |
Next, one defines a remote to pull changes for the ``ExtraRepo1`` from the |
| 7851 |
main development repo: |
| 7852 |
|
| 7853 |
$ cd $SYNC_BASE_DIR/BaseProj/ExtraRepo1 |
| 7854 |
$ git remote add public url2.gov:/git/ExtraRepo1 |
| 7855 |
|
| 7856 |
Here, one should pick a name for the remote repo for ``ExtraRepo1`` that is |
| 7857 |
most descriptive for that particular situation. In this case, the name |
| 7858 |
``public`` is chosen to signify the main public development repo. |
| 7859 |
|
| 7860 |
This gives the remotes:: |
| 7861 |
|
| 7862 |
$ cd $SYNC_BASE_DIR/BaseProj |
| 7863 |
$ gitdist remote -v | grep -v push | grep -v "^$" |
| 7864 |
*** Base Git Repo: BaseProj |
| 7865 |
origin url4.gov:/git/BaseProj (fetch) |
| 7866 |
*** Git Repo: ExtraRepo1 |
| 7867 |
origin url4.gov:/git/ExtraRepo1 (fetch) |
| 7868 |
public url2.gov:/git/ExtraRepo1 (fetch) |
| 7869 |
*** Git Repo: ExtraRepo2 |
| 7870 |
origin url4.gov:/git/ExtraRepo2 (fetch) |
| 7871 |
|
| 7872 |
The remote ``public`` is used by the ``checkin-test.py`` wrapper script (see |
| 7873 |
`ACI Sync Driver Script`_ below) to pull and merge in additional changes that |
| 7874 |
will be tested and pushed to the 'origin' repos on ``url4.gov``. In this |
| 7875 |
case, the ``ExtraRepo1`` remote ``public`` will result in updates being pulled |
| 7876 |
from the main development repo on ``url2.gov``, thereby facilitating the |
| 7877 |
update of ``ExtraRepo1`` in the integrated meta-project. |
| 7878 |
|
| 7879 |
|
| 7880 |
ACI Integration Build Directory Setup |
| 7881 |
+++++++++++++++++++++++++++++++++++++ |
| 7882 |
|
| 7883 |
After the git repos are cloned and the remotes are set up as described above, |
| 7884 |
a build base directory is set up as:: |
| 7885 |
|
| 7886 |
$ cd $SYNC_BASE_DIR |
| 7887 |
$ mkdir BUILDS |
| 7888 |
$ mkdir BUILDS/CHECKIN |
| 7889 |
|
| 7890 |
An ACI wrapper script for ``checkin-test.py`` is created to drive the syncing |
| 7891 |
process. It is assumed that this script would be called only once a day and |
| 7892 |
not continuously in a loop (but that is possible as well but is not documented |
| 7893 |
here). |
| 7894 |
|
| 7895 |
NOTE: Other build directory structures are possible, it all depends how one |
| 7896 |
writes the ``checkin-test.py`` wrapper scripts but the above directory |
| 7897 |
structure is fairly standard in the usage of the ``checkin-test.py`` script. |
| 7898 |
|
| 7899 |
|
| 7900 |
ACI Sync Driver Script |
| 7901 |
++++++++++++++++++++++ |
| 7902 |
|
| 7903 |
The sync driver script for this example should be called something like |
| 7904 |
``sync_ExtraRepo1.sh``, placed under version control, and would look something |
| 7905 |
like:: |
| 7906 |
|
| 7907 |
#!/bin/bash -e |
| 7908 |
|
| 7909 |
# Set up the environment (i.e. PATH; needed for cron jobs) |
| 7910 |
... |
| 7911 |
|
| 7912 |
SYNC_BASE_DIR=~/sync_base_dir |
| 7913 |
CHECKIN_TEST_WRAPPER=$SYNC_BASE_DIR/BaseProj/sampleScripts/checkin-test-foo.sh |
| 7914 |
|
| 7915 |
cd $SYNC_BASE_DIR/BUILDS/CHECKIN |
| 7916 |
|
| 7917 |
$CHECKIN_TEST_WRAPPER \ |
| 7918 |
--extra-pull-from=ExtraRepo1:public:master \ |
| 7919 |
--abort-gracefully-if-no-changes-to-push \ |
| 7920 |
--enable-extra-packages=Package1A \ |
| 7921 |
--send-build-case-email=only-on-failure \ |
| 7922 |
[email protected] \ |
| 7923 |
[email protected] \ |
| 7924 |
--no-append-test-results --no-rebase \ |
| 7925 |
--do-all --push \ |
| 7926 |
-j16 \ |
| 7927 |
--wipe-clean \ |
| 7928 |
"$@" |
| 7929 |
|
| 7930 |
NOTE, in the above example ``sync_ExtraRepo1.sh`` script, the variable |
| 7931 |
``CHECKIN_TEST_WRAPPER`` is set to a wrapper script:: |
| 7932 |
|
| 7933 |
BaseProj/sampleScripts/checkin-test-foo.sh |
| 7934 |
|
| 7935 |
which would be set up to call the project's `checkin-test.py`_ tool with |
| 7936 |
configure options for the specific machine. The location and the nature of |
| 7937 |
the wrapper script will vary from project to project and machine to machine. |
| 7938 |
In some simple cases, ``CHECKIN_TEST_WRAPPER`` might just be set to be the raw |
| 7939 |
machine-independent ``checkin-test.py`` tool for the project. |
| 7940 |
|
| 7941 |
A description of each option passed into this invocation of the |
| 7942 |
`checkin-test.py`_ tool is given below (see `checkin-test.py --help`_ for more |
| 7943 |
details): |
| 7944 |
|
| 7945 |
``--extra-pull-from=ExtraRepo1:public:master`` |
| 7946 |
|
| 7947 |
This option instructs the ``checkin-test.py`` tool to pull and merge in |
| 7948 |
commits that define the integration. One could do the pull(s) manually of |
| 7949 |
doing so has the disadvantage that if they fail for some reason, they will |
| 7950 |
not be seen by the ``checkin-test.py`` tool and no notification email |
| 7951 |
would go out. |
| 7952 |
|
| 7953 |
``--abort-gracefully-if-no-changes-to-push`` |
| 7954 |
|
| 7955 |
The option ``--abort-gracefully-if-no-changes-to-push`` makes the |
| 7956 |
``checkin-test.py`` tool gracefully terminate without sending out any |
| 7957 |
emails if after all the pulls, there are no local changes to push to the |
| 7958 |
'origin' repos. This can happen, for example, if no commits were pushed |
| 7959 |
to the main development git repo for ``ExtraRepo1`` at |
| 7960 |
``url2.gov:/git/ExtraRepo1`` since the last time this sync process was |
| 7961 |
run. This avoids getting confusing and annoying emails like ``"PUSH |
| 7962 |
FAILED"``. The reason this option is not generally needed for local |
| 7963 |
developer usage of the ``checkin-test.py`` tool is that in general a |
| 7964 |
developer will not run the ``checkin-test.py`` tool with ``--push`` unless |
| 7965 |
they have made local changes; it just does not make any sense at all to do |
| 7966 |
that and if they do by accident, they should get an error email. However, |
| 7967 |
for an automated ACI sync process, there is no easy way to know a-priori |
| 7968 |
if changes need to be synced so the script supports this option to deal |
| 7969 |
with that case gracefully. |
| 7970 |
|
| 7971 |
``--enable-extra-packages=Package1A`` |
| 7972 |
|
| 7973 |
This option should be set if one wants to ensure that all commits get |
| 7974 |
synced, even when these changes don't impact the build or the tests of the |
| 7975 |
project. If not setting ``--enable-extra-packages=<some-package>`` , then |
| 7976 |
the ``checkin-test.py`` tool will only decide on its own what packages to |
| 7977 |
test just based on what packages have changed files in the ``ExtraRepo1`` |
| 7978 |
repo and if no modified files map to a package, then no packages will be |
| 7979 |
auto-enabled and therefore no packages will be enabled at all. For |
| 7980 |
example, if a top-level README file in the base ``ExtraRepo1`` repo gets |
| 7981 |
modified that does not sit under a package directory, then the automatic |
| 7982 |
logic in the checkin-test.py tool will not trigger a package enable. In |
| 7983 |
that case, no configure, build, testing, or push will take place (must run |
| 7984 |
at least some tests in order to assume it is safe to push) and therefore |
| 7985 |
the sync will not occur. Therefore, if one wants to ensure that every |
| 7986 |
commit gets safely synced over on every invocation, then the safest way to |
| 7987 |
that is to always enable at least one or more packages by specify |
| 7988 |
``--enable-extra-packages=<pkg0>,<pkg1>``. **WARNING:** it is not |
| 7989 |
advisable to manually set ``--enable-packages=<package-list>`` since it |
| 7990 |
turns off the auto-enable logic for changed files. This is because if |
| 7991 |
there are changes to other packages, then these packages will not get |
| 7992 |
enabled and not get tested, which could break the global build and tests. |
| 7993 |
Also, this is fragile if new packages are added to ``ExtraRepo1`` later |
| 7994 |
that are not listed in ``--enable-packages=<pkg0>,<pkg1>,...`` as they |
| 7995 |
will not be included in the testing. Also, if someone makes local commits |
| 7996 |
in other local git repos before running the sync script again, then these |
| 7997 |
packages will not get enabled and tested. Therefore, in general, don't |
| 7998 |
set ``--enable-packages=<pkg0>,<pkg1>,...`` in a sync script, only set |
| 7999 |
``--enable-extra-packages=<pkg0>,<pkg1>,...`` to be robust and safe. |
| 8000 |
|
| 8001 |
``--send-build-case-email=only-on-failure`` |
| 8002 |
|
| 8003 |
This makes the checkin-test.py tool skip sending email about a build case |
| 8004 |
(e.g. ``MPI_DEBUG``) unless it fails. That way, if everything passes, |
| 8005 |
then only a final ``DID PUSH`` email will go out. But if a build case |
| 8006 |
does fail (i.e. configure, build, or tests fail), then an "early warning" |
| 8007 |
email will still go out. However, if one wants to never get the early |
| 8008 |
build-case emails, one can turn this off by setting |
| 8009 |
``--send-build-case-email=never``. |
| 8010 |
|
| 8011 |
``[email protected]`` |
| 8012 |
|
| 8013 |
The results of the builds will be sent this email address. If you only |
| 8014 |
want an email sent when a push actually happens, you can set |
| 8015 |
``--send-email-to=''`` and rely on ``--send-email-to-on-push``. |
| 8016 |
|
| 8017 |
``[email protected]`` |
| 8018 |
|
| 8019 |
A confirmation and summary email will be sent to this address if the push |
| 8020 |
happens. This can be a different email address than set by the |
| 8021 |
``--send-email-to`` option. It is highly recommended that a mail list be |
| 8022 |
used for this email address since this will be the only more permanent |
| 8023 |
logging of the ACI process. |
| 8024 |
|
| 8025 |
``--no-append-test-results --no-rebase`` |
| 8026 |
|
| 8027 |
These options are needed to stop the ``checkin-test.py`` tool from |
| 8028 |
modifying the commits being tested and pushed from one public git repo to |
| 8029 |
another. The option ``--no-append-test-results`` is needed to instruct |
| 8030 |
the ``checkin-test.py`` tool to **NOT** amend the last commit with the |
| 8031 |
test results. The option ``--no-rebase`` is needed to avoid rebasing the |
| 8032 |
new commits pulled. While the default behavior of the ``checkin-test.py`` |
| 8033 |
tool is to amend the last commit message and rebase the local commits |
| 8034 |
(which is considered best practice when testing local commits), this is a |
| 8035 |
very bad thing to do when a ACI sync server is only testing and moving |
| 8036 |
commits between public repos. Amending the last commit would change the |
| 8037 |
SHA1 of the commit (just as a rebase would) and would fork the history and |
| 8038 |
mess up a number of workflows that otherwise should work smoothly. Since |
| 8039 |
an email logging what was tested will go out if a push happens due to the |
| 8040 |
``--send-email-to-on-push`` argument, there is no value in appending the |
| 8041 |
test results to the last commit pulled and merged (which will generally |
| 8042 |
not be a merge commit but just a fast-forward). There are cases, however, |
| 8043 |
where appending the test results in an ACI process might be acceptable but |
| 8044 |
they are not discussed here. |
| 8045 |
|
| 8046 |
``--do-all --push -j16`` |
| 8047 |
|
| 8048 |
These are standard options that always get used when invoking the |
| 8049 |
``checkin-test.py`` tool and need no further explanation. |
| 8050 |
|
| 8051 |
``--wipe-clean`` |
| 8052 |
|
| 8053 |
This option is added if you want to make the sync server more robust to |
| 8054 |
changes that might require a clean configure from script. If you care |
| 8055 |
more about using less computer resources and testing that rebuilds work |
| 8056 |
smoothly, remove this option. |
| 8057 |
|
| 8058 |
The sync script can be created and tested locally to ensure that it works |
| 8059 |
correctly first, before setting it as a cron job as described next. Also, the |
| 8060 |
sync script should be version controlled in one of the project's git repos. |
| 8061 |
This ensures that changes to script pushed to the repos will get invoked |
| 8062 |
automatically when they are pulled (but only on the second invocation of the |
| 8063 |
script). If a change to script is critical in order to do the pull, then one |
| 8064 |
must manually pull the updated commit to the local sync repo. |
| 8065 |
|
| 8066 |
Note, if using this in a continuous sync server that runs many times in a day |
| 8067 |
in a loop, you also want to set the option |
| 8068 |
``--abort-gracefully-if-no-changes-pulled`` in addition to the option |
| 8069 |
``--abort-gracefully-if-no-changes-to-push``. That is because if the updated |
| 8070 |
repos are in a broken state such that there are always local changes at every |
| 8071 |
CI iteration (because they have not been pushed to origin), you don't want to |
| 8072 |
do a new CI build unless something has changed that would otherwise perhaps |
| 8073 |
make the error go away. That allows the CI server to sit ready to try out any |
| 8074 |
change that gets pulled that might allow the integrated build to work and then |
| 8075 |
push the updates. |
| 8076 |
|
| 8077 |
|
| 8078 |
ACI Cron Job Setup |
| 8079 |
++++++++++++++++++ |
| 8080 |
|
| 8081 |
Once the sync script ``sync_ExtraRepo1.sh`` has been locally tested, then it |
| 8082 |
should be committed to a version control git repo and then run automatically |
| 8083 |
as a cron job. For example, the cron script shown below would fire off the |
| 8084 |
daily ACI process at 8pm local time every night:: |
| 8085 |
|
| 8086 |
# ----------------- minute (0 - 59) |
| 8087 |
# | -------------- hour (0 - 23) |
| 8088 |
# | | ----------- day of month (1 - 31) |
| 8089 |
# | | | -------- month (1 - 12) |
| 8090 |
# | | | | ----- day of week (0 - 7) (Sunday=0 or 7) |
| 8091 |
# | | | | | |
| 8092 |
# * * * * * command to be executed |
| 8093 |
00 20 * * * ~/sync_base_dir/sync_ExtraRepo1.sh &> ~/sync_base_dir/sync_ExtraRepo1.out |
| 8094 |
|
| 8095 |
In the above crontab file (set with ``'crontab -e'`` or ``'crontab |
| 8096 |
my-crontab-file.txt'``), the script:: |
| 8097 |
|
| 8098 |
~/sync_base_dir/sync_ExtraRepo1.sh |
| 8099 |
|
| 8100 |
is assumed to be a soft symbolic link to some version controlled copy of the |
| 8101 |
ACI sync script. For example, it might make sense for this script to be |
| 8102 |
version controlled in the ``BaseProj`` repo and therefore the symbolic link |
| 8103 |
would be created with something like:: |
| 8104 |
|
| 8105 |
$ cd ~/sync_base_dir/ |
| 8106 |
$ ln -s BaseProj/sampleScripts/sync_ExtraRepo1.sh . |
| 8107 |
|
| 8108 |
Such a setup would ensure that sync scripts would always be up-to-date due to |
| 8109 |
the git pulls part of the ACI process. |
| 8110 |
|
| 8111 |
|
| 8112 |
Addressing ACI Failures and Summary |
| 8113 |
+++++++++++++++++++++++++++++++++++ |
| 8114 |
|
| 8115 |
After the above cron job starts running (setup described above), the |
| 8116 |
``checkin-test.py`` tool will send out emails to the email addresses passed |
| 8117 |
into the underlying ``checkin-test.py`` tool. If the emails report an update, |
| 8118 |
configure, build, or test failure, then someone will need to log onto the |
| 8119 |
machine where the ACI sync server is running and investigate what went wrong, |
| 8120 |
just like they would if they were running the ``checkin-test.py`` tool for |
| 8121 |
testing locally modified changes before pushing. |
| 8122 |
|
| 8123 |
In the above example, only a single git/TriBITS repo is integrated in this ACI |
| 8124 |
sync scripts. For a complete system, other ACI sync scripts would be written |
| 8125 |
to sync the two other git/TriBITS repos in order to maintain some |
| 8126 |
independence. Or, a single ACI sync script that tries to update all three |
| 8127 |
git/TriBITS repos at could would be written and used. The pattern of |
| 8128 |
integrations chosen will depend many different factors and these patterns can |
| 8129 |
change over time according to current needs and circumstances. |
| 8130 |
|
| 8131 |
In summary, the `checkin-test.py`_ tool can be used to set up robust and |
| 8132 |
effective Almost Continuous Integration (ACI) sync servers that can be used to |
| 8133 |
integrate large collections of software in logical configurations at any |
| 8134 |
logical frequency. Such an approach, together with the practice of `Regulated |
| 8135 |
Backward Compatibility and Deprecated Code`_, can allow very large collections |
| 8136 |
of software to be kept integrated in short time intervals using ACI. |
| 8137 |
|
| 8138 |
Post-Push CI and Nightly Testing using checkin-test.py |
| 8139 |
------------------------------------------------------ |
| 8140 |
|
| 8141 |
While the post-push CI and Nightly testing processes using ``ctest -S`` |
| 8142 |
scripts using `tribits_ctest_driver()`_ (see `TriBITS CTest/CDash Driver`_) |
| 8143 |
which posts results to a CDash server (see `TriBITS CDash Customizations`_) is |
| 8144 |
a very attractive testing system with many advantages, setting up a CDash |
| 8145 |
server can be a bit difficult and a CDash server can require non-trivial |
| 8146 |
storage and CPU resources (due to the MySQL DB of test results) and requires |
| 8147 |
some amount of extra maintenance. As an intermediate approach, one can |
| 8148 |
consider just using the project's `checkin-test.py`_ tool to implement basic |
| 8149 |
post-push CI and/or Nightly testing servers using simple cron jobs and some |
| 8150 |
other helper scripts. The ``checkin-test.py`` tool will robustly pull new |
| 8151 |
commits, configure the project, build, run tests, and send out emails with |
| 8152 |
results and pass/fail. A bunch of builds can be run at once using multiple |
| 8153 |
builds specified in the ``--default-builds``, ``--st-extra-builds``, and |
| 8154 |
``--extra-build`` arguments, or different invocations of the |
| 8155 |
``checkin-test.py`` tool can be run in parallel for better machine |
| 8156 |
utilization. |
| 8157 |
|
| 8158 |
What one gives up with this approach over the full-blow CTest/CDash |
| 8159 |
implementation is: |
| 8160 |
|
| 8161 |
1) Test results over multiple builds/days are not archived in a database for |
| 8162 |
later retrieval and query. Only the latest build's detailed results are |
| 8163 |
accessible. |
| 8164 |
|
| 8165 |
2) Detailed test results (pass or fail) cannot be easily looked up on a |
| 8166 |
website. Only detailed results from the last build can be found and one |
| 8167 |
has to go to the machine where the build was run from to see the results. |
| 8168 |
|
| 8169 |
The process for setting up a basic nightly build using ``checkin-test.py`` as |
| 8170 |
a cron job is a subset of the steps needed to set up a driver and cron job for |
| 8171 |
the more complex ACI process described in `ACI Sync Driver Script`_ and `ACI |
| 8172 |
Cron Job Setup`_. Setting up a CI server using ``checkin-test.py`` is a |
| 8173 |
little more involved than setting up a nightly build (and less desirable |
| 8174 |
because it blows away old CI iteration results pretty fast) but can be done |
| 8175 |
using existing tools. |
| 8176 |
|
| 8177 |
.. ToDo: Show how to set up a cron job using checkin-test.py. |
| 8178 |
|
| 8179 |
For smaller, private, less-critical projects with few developers, setting up |
| 8180 |
CI and Nightly testing using ``checkin-test.py`` may be quite adequate. In |
| 8181 |
fact, post-push testing processes implemented with ``checkin-test.py`` are |
| 8182 |
much more solid and feature-full than have been employed in many software |
| 8183 |
projects that we have seen over the years that were larger, more public, had |
| 8184 |
many developers, and were quite important to many users and development |
| 8185 |
teams. |
| 8186 |
|
| 8187 |
|
| 8188 |
TriBITS Dashboard Driver |
| 8189 |
------------------------ |
| 8190 |
|
| 8191 |
TriBITS also includes a system based on CMake/CTest/CDash to drive the builds |
| 8192 |
of a TriBITS project and post results to another CDash project. This system |
| 8193 |
is contained under the directory:: |
| 8194 |
|
| 8195 |
tribits/ctest/tdd/ |
| 8196 |
|
| 8197 |
If the TriBITS project name is ``<projectName>``, the TDD driver CDash project |
| 8198 |
is typically called ``<projectName>Driver``. Using CTest to drive the Nightly |
| 8199 |
builds of a TriBITS project makes sense because CTest can run different builds |
| 8200 |
in parallel, can time-out builds that are taking too long, and will report |
| 8201 |
results to a dashboard and submit notification emails when things fail. |
| 8202 |
However, this is the most confusing and immature part of the TriBITS system. |
| 8203 |
The `TriBITS CTest/CDash Driver`_ system using the `tribits_ctest_driver()`_ |
| 8204 |
can be used without this TriBITS Dashboard Driver (TDD) system. |
| 8205 |
|
| 8206 |
However, this TriBITS subsystem is not well tested with automated tests, is |
| 8207 |
difficult to extend and test manually, and has other problems. Therefore, it |
| 8208 |
is not recommended that projects adopt the usage of this subsystem. A simple |
| 8209 |
set of cron jobs or a tool like Jenkins is likely a better option (if done |
| 8210 |
carefully). |
| 8211 |
|
| 8212 |
.. ToDo: Import and format the contents of tribits/ctest/README related to |
| 8213 |
.. this topic into this section. |
| 8214 |
|
| 8215 |
|
| 8216 |
Regulated Backward Compatibility and Deprecated Code |
| 8217 |
---------------------------------------------------- |
| 8218 |
|
| 8219 |
The motivation and ideas behind `Regulated Backward Compatibility` and |
| 8220 |
deprecated code are described in the `TriBITS Lifecycle Model`_ document. |
| 8221 |
Here, the details of the implementation in TriBITS are given and how |
| 8222 |
transitions between non-backward compatible major versions is accomplished. |
| 8223 |
|
| 8224 |
This section describes the process for deprecating code within a major |
| 8225 |
backward-compatible version sequence and finally removing deprecated code when |
| 8226 |
transitioning to a new major version X to X+1 for the semantic versioning |
| 8227 |
numbering scheme X.Y.Z. This information is given as a process with different |
| 8228 |
phases. |
| 8229 |
|
| 8230 |
The processing for managing deprecated code is as follows and more details are |
| 8231 |
given below: |
| 8232 |
|
| 8233 |
* `Setting up support for deprecated code handling`_ |
| 8234 |
|
| 8235 |
* `Deprecating code but maintaining backward compatibility`_ |
| 8236 |
|
| 8237 |
* `Deprecating C/C++ classes, structs, functions, typdefs, etc.`_ |
| 8238 |
* `Deprecating preprocessor macros`_ |
| 8239 |
* `Deprecating an entire header and/or source file`_ |
| 8240 |
|
| 8241 |
* `Hiding deprecated code to certify and facilitate later removal`_ |
| 8242 |
|
| 8243 |
* `Hiding C/C++ entities`_ |
| 8244 |
* `Hiding entire deprecated header and source files`_ |
| 8245 |
|
| 8246 |
* `Physically removing deprecated code`_ |
| 8247 |
|
| 8248 |
* `Removing entire deprecated header and source files`_ |
| 8249 |
* `Removing deprecated code from remaining files`_ |
| 8250 |
|
| 8251 |
|
| 8252 |
Setting up support for deprecated code handling |
| 8253 |
+++++++++++++++++++++++++++++++++++++++++++++++ |
| 8254 |
|
| 8255 |
Setting up support for managing deprecated code in a TriBITS package requires |
| 8256 |
just two simple changes to the TriBITS-related files in a package. First, the |
| 8257 |
top-level package `<packageDir>/CMakeLists.txt`_ file needs to have a call |
| 8258 |
to:: |
| 8259 |
|
| 8260 |
tribits_add_show_deprecated_warnings_option() |
| 8261 |
|
| 8262 |
Second, the package's configure header:: |
| 8263 |
|
| 8264 |
<packageDir>/cmake/<PACKAGE_UCNAME>_config.h.in |
| 8265 |
|
| 8266 |
file needs to have:: |
| 8267 |
|
| 8268 |
@<PACKAGE_UCNAME>_DEPRECATED_DECLARATIONS@ |
| 8269 |
|
| 8270 |
where ``<PACKAGE_UCNAME>`` is the upper-case name of the TriBITS package. |
| 8271 |
|
| 8272 |
That is all that is needed to provide support for TriBITS deprecated code. |
| 8273 |
|
| 8274 |
When a TriBITS project is configured with:: |
| 8275 |
|
| 8276 |
-D<Project>_SHOW_DEPRECATED_WARNINGS=ON |
| 8277 |
|
| 8278 |
by default, all packages will show deprecated warnings. These deprecated |
| 8279 |
warnings can be turned on and off on a package-by-package basis by setting:: |
| 8280 |
|
| 8281 |
-D<PackageName>_SHOW_DEPRECATED_WARNINGS=[ON|OFF] |
| 8282 |
|
| 8283 |
This gives developers and users a little extra control over what deprecated |
| 8284 |
warnings are shown. |
| 8285 |
|
| 8286 |
In addition, deprecated code can be hidden from the build to help certify that |
| 8287 |
downstream code is clean by configuring with:: |
| 8288 |
|
| 8289 |
-D<Project>_HIDE_DEPRECATED_CODE=ON |
| 8290 |
|
| 8291 |
This will remove the deprecated code from the build and install (see details |
| 8292 |
below) so that other software in the TriBITS project can be shown to build |
| 8293 |
clean without deprecated code and so that outside client code can be shown to |
| 8294 |
be clean of deprecated code. |
| 8295 |
|
| 8296 |
As with deprecated warnings, showing or hiding deprecated code can be |
| 8297 |
controlled on a package-by-package basis by setting:: |
| 8298 |
|
| 8299 |
-D<PackageName>_HIDE_DEPRECATED_CODE=[ON|OFF] |
| 8300 |
|
| 8301 |
In this case, hiding deprecated code on a package-by-package basis may not |
| 8302 |
work because deprecated code in a downstream package might rely on deprecated |
| 8303 |
code in an upstream package (which might have its deprecated code hidden). |
| 8304 |
|
| 8305 |
|
| 8306 |
Deprecating code but maintaining backward compatibility |
| 8307 |
+++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| 8308 |
|
| 8309 |
One of the most important aspects of the `TriBITS Lifecycle Model`_ for |
| 8310 |
later-stage Production Growth and Production Maintenance code is to provide |
| 8311 |
backward compatibility between a continuous stream of versions of the software |
| 8312 |
within a major version number X in the version numbering scheme X.Y.Z. In all |
| 8313 |
cases, if a piece of client code builds and runs correctly against version |
| 8314 |
X0.Y0.Z0, it should also build, without modification, against versions |
| 8315 |
X0.Y1,Z1 for all Y1 >= Y0 and all Z1 and up to (but not including) X0+1.0.0. |
| 8316 |
There are many types of constructs that one will want to deprecate and |
| 8317 |
therefore later remove. When deprecating code, one wants to give users |
| 8318 |
compile-time warnings of the usage of deprecated features so that they know |
| 8319 |
what they need to remove. One also wants to allow them to certify that their |
| 8320 |
code is free of deprecated warnings by hiding deprecated code. Below, the |
| 8321 |
different types of entities that one wants to deprecate and how to support |
| 8322 |
hiding code (which also facilitates removing it later) are described. |
| 8323 |
|
| 8324 |
|
| 8325 |
Deprecating C/C++ classes, structs, functions, typdefs, etc. |
| 8326 |
............................................................ |
| 8327 |
|
| 8328 |
To deprecate standard C/C++ constructs, one can just use the standard TriBITS |
| 8329 |
compile-time macro ``<PACKAGE_UCNAME>_DEPRECATED`` which is properly ifdefed |
| 8330 |
by the TriBITS system to add a GCC/Intel deprecated attribute or not. For |
| 8331 |
example, one would deprecate standard C/C++ constructs for the package |
| 8332 |
``SomePackage`` with:: |
| 8333 |
|
| 8334 |
// Deprecate a class (or struct) |
| 8335 |
class SOMEPACKAGE_DEPRECATED SomeClass { ... }; |
| 8336 |
|
| 8337 |
// Deprecate a function |
| 8338 |
SOMEPACKAGE_DEPRECATED int somefunc(...); |
| 8339 |
|
| 8340 |
// Deprecate a typedef |
| 8341 |
SOMEPACKAGE_DEPRECATED typedef someTypeDef int; |
| 8342 |
|
| 8343 |
The GCC (version 3.1 and newer) and Intel C and C++ compilers both support |
| 8344 |
adding extra attributes including the ``__deprecated__`` attribute. When this |
| 8345 |
attribute is applied to a given C/C++ entity, it produces a compiler warning |
| 8346 |
that can be searched for in the compiler output and elevated to an error (when |
| 8347 |
``-Werror`` is also passed to the compiler). |
| 8348 |
|
| 8349 |
In addition to the basic deprecated warning, one can also add an optional |
| 8350 |
deprecated warning message using the macro |
| 8351 |
``<PACKAGE_UCNAME>_deprecated_msg()``. For example, if a new function is |
| 8352 |
replacing an old function, one might use:: |
| 8353 |
|
| 8354 |
// Deprecated old unsafe function taking raw pointers |
| 8355 |
somepackage_deprecated_msg( |
| 8356 |
"Please use the safe somefunc(const Ptr<const std::string>&) instead!") |
| 8357 |
void somefunc(const std::string *str); |
| 8358 |
|
| 8359 |
// New version take does not take raw pointers (and is therefore safer) |
| 8360 |
void somefunc(const Teuchos::Ptr<const std::string> &str); |
| 8361 |
|
| 8362 |
Then, if user code calls the version ``somefunc(const std::string*)`` they |
| 8363 |
will get the string:: |
| 8364 |
|
| 8365 |
"Please use the safe somefunc(const Ptr<const std::string>&) instead!" |
| 8366 |
|
| 8367 |
printed as well by the compiler. Note that the custom message will only be |
| 8368 |
printed for GCC versions 4.5 and newer. If an older version of GCC is used, |
| 8369 |
then the message string is ignored. |
| 8370 |
|
| 8371 |
|
| 8372 |
Deprecating preprocessor macros |
| 8373 |
............................... |
| 8374 |
|
| 8375 |
A C/C++ preprocessor macro is not an entity seen by the C/C++ compiler and |
| 8376 |
therefore cannot directly take advantage of a feature such as the |
| 8377 |
``__deprecated__`` attribute of the GCC/Intel compilers. However, in some |
| 8378 |
cases, for function-like macros can such as:: |
| 8379 |
|
| 8380 |
// The definition of the macro |
| 8381 |
#define some_old_func_macro(ARG1, ARG2, ...) ... |
| 8382 |
... |
| 8383 |
// A use of the macro |
| 8384 |
some_old_func_macro(a, b, ...) |
| 8385 |
|
| 8386 |
there is a strategy where one can define the macro to call a dummy deprecated |
| 8387 |
function such as with:: |
| 8388 |
|
| 8389 |
SOMEPACKAGE_DEPRECATED |
| 8390 |
inline void some_old_func_macro_is_deprecated() |
| 8391 |
{} |
| 8392 |
|
| 8393 |
// The definition of the macro |
| 8394 |
#define some_old_func_macro(ARG1, ARG2, ...) \ |
| 8395 |
{ \ |
| 8396 |
some_old_func_macro_is_deprecated(); \ |
| 8397 |
... \ |
| 8398 |
} |
| 8399 |
|
| 8400 |
... |
| 8401 |
|
| 8402 |
// A use of the macro |
| 8403 |
some_old_func_macro(a, b, ...) |
| 8404 |
|
| 8405 |
In the above example, client code calling ``some_old_func_macro()`` will now |
| 8406 |
result in a deprecated compiler warning which should make it clear what is |
| 8407 |
being deprecated. |
| 8408 |
|
| 8409 |
Note that this approach will not work in cases were a macro is only providing |
| 8410 |
a single value such as a constant (but one should not use macros for providing |
| 8411 |
constants anyway). |
| 8412 |
|
| 8413 |
|
| 8414 |
Deprecating an entire header and/or source file |
| 8415 |
............................................... |
| 8416 |
|
| 8417 |
There are times when one wants to deprecate an entire set of files and all of |
| 8418 |
the contents in those files. In addition to deprecating the contents of the |
| 8419 |
files one will want to deprecate the entire file as well. There are a few |
| 8420 |
steps to this. First, one wants to put a warning in the file such as:: |
| 8421 |
|
| 8422 |
#ifdef __GNUC__ |
| 8423 |
# warning "This header <THIS_HEADER> is deprecated! Use <NEW_HEADER> instead!" |
| 8424 |
#endif |
| 8425 |
|
| 8426 |
The above ``ifdef`` on ``__GNUC__`` is needed in order avoid the non-standard |
| 8427 |
``#warning`` preprocessor directive on non-compliant compilers (but should |
| 8428 |
work on all later version GCC and Intel compilers). |
| 8429 |
|
| 8430 |
|
| 8431 |
Hiding deprecated code to certify and facilitate later removal |
| 8432 |
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| 8433 |
|
| 8434 |
In addition to adding deprecation warnings at preprocessing or compile-time, |
| 8435 |
it is also highly desirable to allow the deprecated code to be removed from |
| 8436 |
the build to help certify that client code indeed no longer needs the |
| 8437 |
deprecated code. The following subsections describe how to hide deprecated |
| 8438 |
code from existing files and how to hide deprecated files entirely. |
| 8439 |
|
| 8440 |
|
| 8441 |
Hiding C/C++ entities |
| 8442 |
..................... |
| 8443 |
|
| 8444 |
In the case when various C/C++ entities will be removed from an existing file, |
| 8445 |
but the file will remain, then the deprecated code can be ifdefed out, for the |
| 8446 |
package ``SomePackage`` for example, using:: |
| 8447 |
|
| 8448 |
#ifndef SOMEPACKAGE_HIDE_DEPRECATED_CODE |
| 8449 |
// Deprecate a class (or struct) |
| 8450 |
class SOMEPACKAGE_DEPRECATED SomeClass { ... }; |
| 8451 |
... |
| 8452 |
#endif /* SOMEPACKAGE_HIDE_DEPRECATED_CODE */ |
| 8453 |
|
| 8454 |
In this way, when the CMake variable ``SomePackae_HIDE_DEPRECATED_CODE=ON``, |
| 8455 |
then the deprecated code will be completely removed resulting in compile |
| 8456 |
errors for any downstream code still using them. |
| 8457 |
|
| 8458 |
|
| 8459 |
Hiding entire deprecated header and source files |
| 8460 |
................................................ |
| 8461 |
|
| 8462 |
In order to hide entire deprecated header and source files when the CMake |
| 8463 |
variable ``<PackageName>_HIDE_DEPRECATED_CODE=ON`` is set, one needs to move |
| 8464 |
the headers and sources to another directory and provide for conditional |
| 8465 |
inclusion in the TriBITS build of the library. For example, suppose one wants |
| 8466 |
to remove support for the deprecated files ``SomeOldStuff.hpp`` and |
| 8467 |
``SomeOldStuff.cpp``. In this case, one would move the files onto a new |
| 8468 |
``deprecated/`` sub-directory and then write the ``CMakeLists.txt`` file |
| 8469 |
like:: |
| 8470 |
|
| 8471 |
set(HEADERS "") |
| 8472 |
set(SOURCES "") |
| 8473 |
|
| 8474 |
set_and_inc_dirs(DIR ${CMAKE_CURRENT_SOURCE_DIR}) |
| 8475 |
append_glob(HEADERS ${DIR}/*.hpp) |
| 8476 |
append_glob(SOURCES ${DIR}/*.cpp) |
| 8477 |
|
| 8478 |
if (NOT ${PACKAGE_NAME}_HIDE_DEPRECATED_CODE) |
| 8479 |
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/deprecated) |
| 8480 |
append_set(HEADERS |
| 8481 |
SomeOldStuff.hpp |
| 8482 |
) |
| 8483 |
append_set(SOURCES |
| 8484 |
SomeOldStuff.cpp |
| 8485 |
) |
| 8486 |
endif() |
| 8487 |
|
| 8488 |
... |
| 8489 |
|
| 8490 |
tribits_add_library( |
| 8491 |
<LIBRARY_NAME> |
| 8492 |
HEADERS ${HEADERS} |
| 8493 |
SOURCES ${SOURCES} |
| 8494 |
) |
| 8495 |
|
| 8496 |
In this way, when ``${PACKAGE_NAME}_HIDE_DEPRECATED_CODE=TRUE``, then the |
| 8497 |
directory for the deprecated headers will not be in the include path (so |
| 8498 |
downstream clients will not even be able to see them) and they will not be |
| 8499 |
installed so external clients will not be able to see them either. However, |
| 8500 |
when ``${PACKAGE_NAME}_HIDE_DEPRECATED_CODE=FALSE``, then these files will be |
| 8501 |
included in the build and include path and downstream clients can use them. |
| 8502 |
|
| 8503 |
Once these files need to be permanently removed, one just then needs to remove |
| 8504 |
them from the version control repository (i.e. ``git rm <files_to_remove>``) |
| 8505 |
and then remove them from the above ``CMakeLists.txt`` code. |
| 8506 |
|
| 8507 |
|
| 8508 |
Physically removing deprecated code |
| 8509 |
+++++++++++++++++++++++++++++++++++ |
| 8510 |
|
| 8511 |
The final step in the code deprecation cycle is to actually remove the |
| 8512 |
deprecated code. This is necessary to clean house, remove clutter and finally |
| 8513 |
get the payoff in the reduction of technical debt that occurs when removing |
| 8514 |
what is no longer needed or wanted. |
| 8515 |
|
| 8516 |
It is recommended to remove deprecated files first, then remove deprecated |
| 8517 |
file fragments from remaining files. Also, it is recommended to create git |
| 8518 |
commits after each step. |
| 8519 |
|
| 8520 |
It is also recommended that some time before deprecated code is actually |
| 8521 |
removed, that a TriBITS repo change the default of |
| 8522 |
``<Project>_HIDE_DEPRECATED_CODE`` from ``OFF`` to ``ON`` so that downstream |
| 8523 |
clients will see the effects of hiding the deprecated code before the code is |
| 8524 |
actually removed. In fact, this default should be changed several days to a |
| 8525 |
week or more before the code is actually removed. This way, downstream code |
| 8526 |
developers will get a "shock" about removal of the deprecated code but can |
| 8527 |
manually configure with ``-D<Project>_HIDE_DEPRECATED_CODE=OFF`` to keep |
| 8528 |
building in the short-term until they can remove their usage of deprecated |
| 8529 |
code. |
| 8530 |
|
| 8531 |
|
| 8532 |
Removing entire deprecated header and source files |
| 8533 |
.................................................. |
| 8534 |
|
| 8535 |
To remove entire deprecated header and source files one just needs to first |
| 8536 |
remove them from the version control repository and local directories |
| 8537 |
(e.g. ``git rm deprecated/*``) and then remove any traces of them from the |
| 8538 |
``CMakeLists.txt`` file. For the example in `Hiding entire deprecated header |
| 8539 |
and source files`_, one would just remove the files ``SomeOldStuff.hpp`` and |
| 8540 |
``SomeOldStuff.cpp`` from the ``CMakeLists.txt`` file leaving:: |
| 8541 |
|
| 8542 |
if (NOT ${PACKAGE_NAME}_HIDE_DEPRECATED_CODE) |
| 8543 |
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/deprecated) |
| 8544 |
append_set(HEADERS |
| 8545 |
) |
| 8546 |
append_set(SOURCES |
| 8547 |
) |
| 8548 |
endif() |
| 8549 |
|
| 8550 |
Since more files may be deprecated later, it may be a good idea to leave the |
| 8551 |
machinery for conditionally including deprecated files by leaving the above |
| 8552 |
empty CMake code or just commenting it out. |
| 8553 |
|
| 8554 |
To find which ``CMakeLists.txt`` files need to be modified, do a search like:: |
| 8555 |
|
| 8556 |
$ find . -name CMakeLists.txt -exec grep -nH HIDE_DEPRECATED_CODE {} \; |
| 8557 |
|
| 8558 |
After removing the files, create a local commit of the removed files and the |
| 8559 |
updated ``CMakeLists.txt`` files before removing deprecated fragments from the |
| 8560 |
source files. In other words, do:: |
| 8561 |
|
| 8562 |
$ emacs -nw CMakeLists.txt # Remove the references to the deprecated files |
| 8563 |
$ git rm SomeOldStuff.hpp SomeOldStuff.cpp |
| 8564 |
$ git commit -m "Removing deprecated files" |
| 8565 |
|
| 8566 |
|
| 8567 |
Removing deprecated code from remaining files |
| 8568 |
............................................. |
| 8569 |
|
| 8570 |
The deprecated ifdefed blocks described in `Hiding C/C++ entities`_ can be |
| 8571 |
removed manually but it is generally preferred to use a tool. One simple tool |
| 8572 |
that can do this is called ``unifdef``, that can be downloaded and it is |
| 8573 |
documented at: |
| 8574 |
|
| 8575 |
http://dotat.at/prog/unifdef/ |
| 8576 |
|
| 8577 |
Just download, build, and install the program ``unifdef`` (see |
| 8578 |
``unifdef/INSTALL`` in untarred source) and then run it as described below. |
| 8579 |
In the example below, assume the program is installed in the user's home |
| 8580 |
directory under:: |
| 8581 |
|
| 8582 |
~/install/bin/unifdef |
| 8583 |
|
| 8584 |
For a given TriBITS package, the program is then run as:: |
| 8585 |
|
| 8586 |
$ find . -name "*pp" -exec ~/install/bin/unifdef \ |
| 8587 |
-DSomePackage_HIDE_DEPRECATED_CODE {} -o {} \; |
| 8588 |
|
| 8589 |
After the above command runs, look at the diffs to make sure the ifdef |
| 8590 |
deprecated code blocks were removed correctly. For example, run:: |
| 8591 |
|
| 8592 |
$ git diff -- . |
| 8593 |
|
| 8594 |
If the diffs look correct, commit the changes:: |
| 8595 |
|
| 8596 |
$ git commit -m "Removing deprecated code blocks" -- . |
| 8597 |
|
| 8598 |
Then test everything and push using the `checkin-test.py`_ tool. |
| 8599 |
|
| 8600 |
After that, all deprecated code is removed and the next period of incremental |
| 8601 |
change and deprecation begins. |
| 8602 |
|
| 8603 |
|
| 8604 |
Installation and Backward Compatibility Testing |
| 8605 |
----------------------------------------------- |
| 8606 |
|
| 8607 |
TriBITS has some built-in support for installation testing and backward |
| 8608 |
compatibility testing. The way it works is that one can install the headers, |
| 8609 |
libraries, and executables for a TriBITS project and then configure the tests |
| 8610 |
and examples in the TriBITS project against the installed |
| 8611 |
headers/libraries/executables. In this mode, the TriBITS project's libraries |
| 8612 |
and executables are **not** build and the header file locations to local |
| 8613 |
source are not included. |
| 8614 |
|
| 8615 |
When the same version of the project sources are used to build the |
| 8616 |
tests/examples against the installed headers/libraries/executables, then this |
| 8617 |
constitutes *installation testing*. When an older version of the project is |
| 8618 |
used to build and run tests and examples against the |
| 8619 |
headers/libraries/executables for a version of the project, then this |
| 8620 |
constitutes *backward compatibility testing* which also includes installation |
| 8621 |
testing of course. |
| 8622 |
|
| 8623 |
.. ToDo: Describe how the installation testing and backward compatibility |
| 8624 |
.. testing process works with some examples. |
| 8625 |
|
| 8626 |
.. ToDo: Discuss how this fits into the TriBITS lifecycle model. |
| 8627 |
|
| 8628 |
|
| 8629 |
Wrapping Externally Configured/Built Software |
| 8630 |
--------------------------------------------- |
| 8631 |
|
| 8632 |
It is possible to take an external piece of software that uses any arbitrary |
| 8633 |
build system and wrap it as a TriBITS package and have it integrate in with |
| 8634 |
the package dependency infrastructure. The `TribitsExampleProject`_ package |
| 8635 |
``WrapExternal`` shows how this can be done. Support for this in TriBITS is |
| 8636 |
slowly evolving but some key TriBITS features that have been added to support |
| 8637 |
the arbitrary case include: |
| 8638 |
|
| 8639 |
* `tribits_determine_if_current_package_needs_rebuilt()`_: Uses brute-force |
| 8640 |
searches for recently modified files in upstream packages to determine if |
| 8641 |
the external piece of software needs to be rebuilt. |
| 8642 |
|
| 8643 |
* `tribits_write_flexible_package_client_export_files()`_: Write an export |
| 8644 |
makefile or a ``XXXConfig.cmake`` file for usage by the wrapped externally |
| 8645 |
configured and built software. |
| 8646 |
|
| 8647 |
.. ToDo: Show how this is done in detail once TriBITS has nice support for |
| 8648 |
.. external software and the WrapExternal example looks cleaner. |
| 8649 |
|
| 8650 |
While it is possible to wrap nearly any externally configured and built piece |
| 8651 |
of software as a TriBITS package, in most cases, it is usually better to just |
| 8652 |
create a TriBITS build system for the software. For projects that use a raw |
| 8653 |
CMake build system, a TriBITS build system can be created side-by-side with |
| 8654 |
the existing raw CMake build using a number of approaches. The common approach |
| 8655 |
that is not too invasive is to create a ``CMakeLists.tribits.txt`` file along |
| 8656 |
side every native ``CMakeLists.txt`` file in the external software project and |
| 8657 |
have the native ``CMakeLists.txt`` file defined like:: |
| 8658 |
|
| 8659 |
if (DOING_A_TRIBITS_BUILD) |
| 8660 |
include("${CMAKE_CURRENT_SOURCE_DIR}/CMakeLists.tribits.txt") |
| 8661 |
return() |
| 8662 |
endif() |
| 8663 |
|
| 8664 |
# Rest of native CMakeLists.txt file ... |
| 8665 |
|
| 8666 |
Experience from the CASL VERA project has shown that, overall, there is less |
| 8667 |
hassle, less work, and better portability when creating a native TriBITS |
| 8668 |
build, even if it is a secondary build system for a given piece of software. |
| 8669 |
|
| 8670 |
The decision whether to just wrap the build system for the existing software |
| 8671 |
or to create a (secondary) TriBITS build system for it depends on a number of |
| 8672 |
factors. |
| 8673 |
|
| 8674 |
Note that while it is generally recommended to create a TriBITS build for an |
| 8675 |
existing piece of software, it is generally not recommended with switch over |
| 8676 |
all of the tests to use CTest (unless the existing software is going to ditch |
| 8677 |
their current test driver and reporting system). Instead, the external |
| 8678 |
software's native test system can just be called by the wrapped TriBITS |
| 8679 |
package in one or more CTest tests. |
| 8680 |
|
| 8681 |
|
| 8682 |
TriBITS directory snapshotting |
| 8683 |
------------------------------ |
| 8684 |
|
| 8685 |
Some TriBITS projects choose to snapshot the `TriBITS/tribits/`_ directory |
| 8686 |
source tree into their project's source tree, typically under |
| 8687 |
`<projectDir>/cmake/tribits/`_. The independent ``TriBITS/tribts/`` source |
| 8688 |
tree contains the tool ``snapshot_tribits.py`` (calls `snapshot-dir.py`_) that |
| 8689 |
allows one to update the snapshot of the TriBITS source tree as simply as:: |
| 8690 |
|
| 8691 |
$ cd <projectDir>/cmake/tribits/ |
| 8692 |
$ <some-base-dir>/TriBITS/tribits/snapshot_tribits.py |
| 8693 |
|
| 8694 |
This will create a git commit in the local ``<projectDir>/`` git repo that |
| 8695 |
looks like:: |
| 8696 |
|
| 8697 |
Automatic snapshot commit from tribits at f8c1682 |
| 8698 |
|
| 8699 |
Origin repo remote tracking branch: 'casl-dev-collab/tribits_reorg_26' |
| 8700 |
Origin repo remote repo URL: 'casl-dev-collab = git@casl-dev:collaboration/TriBITS' |
| 8701 |
|
| 8702 |
At commit: |
| 8703 |
|
| 8704 |
f8c1682 Assert TriBITS min CMake version in TriBITS itself |
| 8705 |
Author: Roscoe A. Bartlett <[email protected]> |
| 8706 |
Date: Fri Dec 5 05:40:49 2014 -0500 |
| 8707 |
|
| 8708 |
This, of course, assumes that ``<projectDir>/`` is a local git repo (or is in |
| 8709 |
local git repo). If that is not the case, then one cannot use the script |
| 8710 |
``snapshot_tribits.py`` or must use it with the ``--skip-commit`` option. |
| 8711 |
|
| 8712 |
See `snapshot-dir.py --help`_ for more details. Note the guidance on using a |
| 8713 |
different branch for the snapshot sync followed by a merge. This allows for |
| 8714 |
one to maintain local changes to TriBITS and use git to manage the merges. |
| 8715 |
However, this will increase the changes of merge conflicts so one should |
| 8716 |
consider just directly snapshotting into the master branch to avoid merge |
| 8717 |
conflicts. |
| 8718 |
|
| 8719 |
|
| 8720 |
TriBITS Development Toolset |
| 8721 |
--------------------------- |
| 8722 |
|
| 8723 |
Most TriBITS projects need git, a compiler (e.g. GCC), MPI, and a number of |
| 8724 |
other standard TPLs and other tools in order to develop on and test the |
| 8725 |
project code. To this end, TriBITS contains some helper scripts for |
| 8726 |
downloading, configuring, building, and installing packages like git, cmake, |
| 8727 |
GCC, MPICH, and others needed to set up a development environment for a |
| 8728 |
typical computational science software project. These tools are used to set |
| 8729 |
up development environments on new machines for projects like Trilinos and |
| 8730 |
CASL VERA. Scripts with names like ``install-gcc.py`` are defined which pull |
| 8731 |
sources from public git repos then configure, build, and install into |
| 8732 |
specified installation directories. |
| 8733 |
|
| 8734 |
.. _install_devtools.py: |
| 8735 |
|
| 8736 |
The script **install_devtools.py** is provided in the directory:: |
| 8737 |
|
| 8738 |
tribits/devtools_install/ |
| 8739 |
|
| 8740 |
To use this script, one just needs to create some scratch directory like:: |
| 8741 |
|
| 8742 |
$ mkdir scratch |
| 8743 |
$ cd scratch/ |
| 8744 |
|
| 8745 |
then install the tools using, for example:: |
| 8746 |
|
| 8747 |
$ install_devtools.py --install-dir=~/install/tribits_devtools \ |
| 8748 |
--parallel=16 --do-all |
| 8749 |
|
| 8750 |
Then to access installed development environment, one just needs to source the |
| 8751 |
script:: |
| 8752 |
|
| 8753 |
~/install/tribits_devtools/load_dev_env.sh |
| 8754 |
|
| 8755 |
and then the installed versions of GCC, MPICH, CMake, and `gitdist`_ are |
| 8756 |
placed in one's path. |
| 8757 |
|
| 8758 |
See `install_devtools.py --help`_ for more details. |
| 8759 |
|
| 8760 |
|
| 8761 |
.. Words specific to this documentation collection: |
| 8762 |
|
| 8763 |
.. LocalWords: projectDir packageDir |
| 8764 |
|
| 8765 |
.. TriBITS words: |
| 8766 |
|
| 8767 |
.. LocalWords: TRIBITS TriBITS tribits TPL TPLs |
| 8768 |
.. LocalWords: Subpackages subpackages Subpackage subpackage |
| 8769 |
.. LocalWords: PackagesList TPLsList |
| 8770 |
.. LocalWords: NativeRepositoriesList ExtraRepositoriesList |
| 8771 |
.. LocalWords: TribitsCTestDriverCore |
| 8772 |
.. LocalWords: TribitsExampleProject TribitsExProj DTribitsExProj SimpleCXX MixedLang |
| 8773 |
.. LocalWords: MockTrilinos |
| 8774 |
.. LocalWords: WithSubpackages WithSubpackagesA WithSubpackagesB WithSubpackagesC |
| 8775 |
|
| 8776 |
.. General CMake words: |
| 8777 |
|
| 8778 |
.. LocalWords: CMAKE CMake cmake CTEST CTest ctest CDash |
| 8779 |
.. LocalWords: CMakeLists CMakeCache CTestConfig |
| 8780 |
.. LocalWords: Kitware |
| 8781 |
.. LocalWords: endif foreach endforeach endmacro subdirectory |
| 8782 |
.. LocalWords: Fortran |
| 8783 |
|
| 8784 |
|
| 8785 |
.. Other general words: |
| 8786 |
|
| 8787 |
.. LocalWords: Trilinos executables Versioning versioning |
| 8788 |
.. LocalWords: Namespaced namespaced symlinks initializations |