| 1 |
.. _user-guide: |
| 2 |
|
| 3 |
Lesser known features |
| 4 |
===================== |
| 5 |
|
| 6 |
Environment Modules provides many capabilities beyond the commonly used |
| 7 |
:command:`module` :subcmd:`load` command. Some features are lesser known, despite being very |
| 8 |
useful for users, modulefile developers, and site administrators. |
| 9 |
|
| 10 |
This document gathers and explains a selection of these features through |
| 11 |
practical examples and common use cases. |
| 12 |
|
| 13 |
It originates from the presentation *Beyond "module load": Exploring the |
| 14 |
Capabilities of Environment Modules* given at `HPSFCon 2026 |
| 15 |
<https://hpsf.io/event/hpsf-conference-2026/>`_. Original slides and video |
| 16 |
demonstrations can be found on the `event session page |
| 17 |
<https://hpsf2026.sched.com/event/2ESia/beyond-module-load-exploring-the-capabilities-of-environment-modules-part-1-adrien-cotte-as+>`_. |
| 18 |
|
| 19 |
Short command names |
| 20 |
------------------- |
| 21 |
|
| 22 |
Most Modules sub-commands can be abbreviated. The :command:`ml` command also |
| 23 |
provides a shorter alternative interface for common operations. |
| 24 |
|
| 25 |
.. list-table:: |
| 26 |
:header-rows: 1 |
| 27 |
:widths: 50 50 |
| 28 |
|
| 29 |
* - Standard command |
| 30 |
- Short form |
| 31 |
|
| 32 |
* - ``module <cmd>`` |
| 33 |
- ``ml <cmd>`` |
| 34 |
|
| 35 |
* - ``module avail`` |
| 36 |
- ``module av`` |
| 37 |
|
| 38 |
* - ``module purge`` |
| 39 |
- ``module pu`` |
| 40 |
|
| 41 |
* - ``module list`` |
| 42 |
- ``ml`` |
| 43 |
|
| 44 |
* - ``module load <module>`` |
| 45 |
- ``ml <module>`` |
| 46 |
|
| 47 |
* - ``module unload <module>`` |
| 48 |
- ``ml -<module>`` |
| 49 |
|
| 50 |
Display/Parsing |
| 51 |
--------------- |
| 52 |
|
| 53 |
``--output=LIST`` |
| 54 |
^^^^^^^^^^^^^^^^^ |
| 55 |
|
| 56 |
The :option:`--output`, or :option:`-o`, option customizes the information |
| 57 |
displayed by Modules commands. |
| 58 |
|
| 59 |
It is supported by :command:`module` :subcmd:`avail`, :subcmd:`list`, and |
| 60 |
:subcmd:`spider` sub-commands. |
| 61 |
|
| 62 |
By default, these commands display additional formatting elements such as |
| 63 |
modulepaths, tags, default markers, or variant information. With |
| 64 |
:option:`--output`, it is possible to restrict the displayed information to |
| 65 |
only the needed fields. |
| 66 |
|
| 67 |
This option is especially useful when parsing command output in shell scripts, |
| 68 |
Python programs, or monitoring tools. |
| 69 |
|
| 70 |
For example, with the following available modulefiles: |
| 71 |
|
| 72 |
.. parsed-literal:: |
| 73 |
|
| 74 |
:ps:`$` module avail |
| 75 |
------------------- :sgrmp:`/usr/share/modulefiles` ------------------ |
| 76 |
gcc/8.3 :sgrde:`gcc/11.1` gcc/14.0 openmpi/5.0.8 |
| 77 |
|
| 78 |
When *LIST* is set to an empty value, only module names are displayed: |
| 79 |
|
| 80 |
.. parsed-literal:: |
| 81 |
|
| 82 |
:ps:`$` module avail -o '' |
| 83 |
gcc/8.3 gcc/11.1 gcc/14.0 openmpi/5.0.8 |
| 84 |
|
| 85 |
The output format is defined as a colon-separated list of elements. Supported |
| 86 |
elements are documented in the corresponding sub-command manual pages. |
| 87 |
|
| 88 |
*LIST* may also be prefixed by ``+`` or ``-`` to respectively append elements |
| 89 |
to or subtract them from the currently configured value. For example, report |
| 90 |
the variants declared by each modulefile in addition to the default output: |
| 91 |
|
| 92 |
.. parsed-literal:: |
| 93 |
|
| 94 |
:ps:`$` module avail -o +variant |
| 95 |
------------------- :sgrmp:`/usr/share/modulefiles` ------------------ |
| 96 |
gcc/8.3 :sgrde:`gcc/11.1` gcc/14.0 openmpi/5.0.8\ :sgrse:`{`:sgrva:`cuda=on`:sgrse:`,`\ :sgrva:`off`:sgrse:`}` |
| 97 |
|
| 98 |
This feature helps avoid fragile text parsing pipelines. |
| 99 |
|
| 100 |
Instead of parsing formatted output: |
| 101 |
|
| 102 |
.. code-block:: bash |
| 103 |
|
| 104 |
modules_list=$(module avail | grep -v '^---' \ |
| 105 |
| sed 's/(default)//g' | head -n -2 | xargs) |
| 106 |
|
| 107 |
A simpler and more robust approach is: |
| 108 |
|
| 109 |
.. code-block:: bash |
| 110 |
|
| 111 |
modules_list=$(module avail -o '') |
| 112 |
|
| 113 |
``spider`` |
| 114 |
^^^^^^^^^^ |
| 115 |
|
| 116 |
The :command:`module` :subcmd:`spider` sub-command lists all available |
| 117 |
modulefiles found in enabled modulepaths, including modulepaths recursively |
| 118 |
added by modulefiles. |
| 119 |
|
| 120 |
Unlike :subcmd:`avail`, :subcmd:`spider` explores the full dependency tree |
| 121 |
created by :mfcmd:`module use<module>`, :mfcmd:`append-path`, or |
| 122 |
:mfcmd:`prepend-path` instructions. |
| 123 |
|
| 124 |
This command is especially useful with hierarchical module layouts, where some |
| 125 |
modulefiles only become visible after loading compiler or MPI dependencies. |
| 126 |
|
| 127 |
The :subcmd:`spider` sub-command accepts the same display and formatting |
| 128 |
options as :subcmd:`avail`, including :option:`--output`, |
| 129 |
:option:`--latest`, :option:`--default`, :option:`--json`, |
| 130 |
:option:`--indepth`, and :option:`--no-indepth`. |
| 131 |
|
| 132 |
For example, in a hierarchical environment: |
| 133 |
|
| 134 |
.. parsed-literal:: |
| 135 |
|
| 136 |
:ps:`$` module avail |
| 137 |
------------------- :sgrmp:`/usr/share/modulefiles/common` ------------------ |
| 138 |
gcc/14.0 intel/25.0 |
| 139 |
|
| 140 |
Only compiler modules are initially visible. |
| 141 |
|
| 142 |
The :subcmd:`spider` sub-command reveals additional modules together with |
| 143 |
their dependency path: |
| 144 |
|
| 145 |
.. parsed-literal:: |
| 146 |
|
| 147 |
:ps:`$` module spider |
| 148 |
------------------- :sgrmp:`/usr/share/modulefiles/common` ------------------ |
| 149 |
gcc/14.0 intel/25.0 |
| 150 |
|
| 151 |
|
| 152 |
---------- :sgrmp:`/usr/share/modulefiles/intel` (via intel/25.0) ----------- |
| 153 |
intelmpi/25.0 |
| 154 |
|
| 155 |
------------ :sgrmp:`/usr/share/modulefiles/gcc` (via gcc/14.0) ------------- |
| 156 |
openmpi/5.0.8 |
| 157 |
|
| 158 |
|
| 159 |
This output indicates which modules must be loaded before these MPI stacks |
| 160 |
become available. |
| 161 |
|
| 162 |
Modules caches are automatically used by :subcmd:`spider` when configured, |
| 163 |
which significantly improves search performance on large installations. |
| 164 |
|
| 165 |
``--latest`` and ``--default`` |
| 166 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 167 |
|
| 168 |
The :option:`--latest`, or :option:`-L`, and :option:`--default`, or |
| 169 |
:option:`-d`, options restrict the output of the :command:`module` |
| 170 |
:subcmd:`avail` and :subcmd:`spider` sub-commands. |
| 171 |
|
| 172 |
For example, with the following available modulefiles: |
| 173 |
|
| 174 |
.. parsed-literal:: |
| 175 |
|
| 176 |
:ps:`$` module avail |
| 177 |
------------------- :sgrmp:`/usr/share/modulefiles` ------------------ |
| 178 |
gcc/8.3 :sgrde:`gcc/11.1` gcc/14.0 intel/24.0 intel/25.0 |
| 179 |
|
| 180 |
The :option:`--latest` option displays only the highest numerically sorted |
| 181 |
version of each module name. |
| 182 |
|
| 183 |
.. parsed-literal:: |
| 184 |
|
| 185 |
:ps:`$` module avail --latest |
| 186 |
------------------- :sgrmp:`/usr/share/modulefiles` ------------------ |
| 187 |
gcc/14.0 intel/25.0 |
| 188 |
|
| 189 |
The :option:`--default` option displays only the default version of each |
| 190 |
module name. This version is either explicitly defined (as here for ``gcc``) |
| 191 |
or implicitly the highest one (as for ``intel``). |
| 192 |
|
| 193 |
.. parsed-literal:: |
| 194 |
|
| 195 |
:ps:`$` module avail --default |
| 196 |
------------------- :sgrmp:`/usr/share/modulefiles` ------------------ |
| 197 |
:sgrde:`gcc/11.1` intel/25.0 |
| 198 |
|
| 199 |
These options may also be combined with other display options such as |
| 200 |
:option:`--output` or :option:`--json`. |
| 201 |
|
| 202 |
``--indepth`` and ``--no-indepth`` |
| 203 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 204 |
|
| 205 |
The :option:`--indepth` and :option:`--no-indepth` options control how the |
| 206 |
:command:`module` :subcmd:`avail` and :subcmd:`spider` sub-commands search |
| 207 |
for matching modulefiles. |
| 208 |
|
| 209 |
By default, Modules searches recursively and returns all matching |
| 210 |
modulefiles. This behavior is equivalent to :option:`--indepth` and |
| 211 |
can be configured with :command:`module` :subcmd:`config` :mconfig:`avail_indepth` |
| 212 |
and :mconfig:`spider_indepth`. |
| 213 |
|
| 214 |
.. parsed-literal:: |
| 215 |
|
| 216 |
:ps:`$` module avail |
| 217 |
------------------- :sgrmp:`/usr/share/modulefiles` ------------------ |
| 218 |
intel/24.0 intel/25.0 gcc/8.3 gcc/11.1 gcc/14.0 |
| 219 |
|
| 220 |
The :option:`--no-indepth` option limits results to the depth level |
| 221 |
expressed by the search query. Modulefiles contained in matching directories |
| 222 |
are not displayed. |
| 223 |
|
| 224 |
.. parsed-literal:: |
| 225 |
|
| 226 |
:ps:`$` module avail --no-indepth |
| 227 |
------------------- :sgrmp:`/usr/share/modulefiles` ------------------ |
| 228 |
:sgrdi:`intel`/ :sgrdi:`gcc`/ |
| 229 |
|
| 230 |
``--json`` |
| 231 |
^^^^^^^^^^ |
| 232 |
|
| 233 |
The :option:`--json`, or :option:`-j`, option displays command results in |
| 234 |
JSON format. |
| 235 |
|
| 236 |
It is supported by the :command:`module` :subcmd:`avail`, :subcmd:`list`, |
| 237 |
:subcmd:`savelist`, :subcmd:`search`, :subcmd:`spider`, |
| 238 |
:subcmd:`stashlist`, and :subcmd:`whatis` sub-commands. |
| 239 |
|
| 240 |
JSON output is intended for machine consumption and can be processed directly |
| 241 |
by tools such as ``jq`` or Python scripts. |
| 242 |
|
| 243 |
.. parsed-literal:: |
| 244 |
|
| 245 |
:ps:`$` module avail -j | jq . |
| 246 |
{ |
| 247 |
"/usr/share/modulefiles": { |
| 248 |
"gcc/14.0": { |
| 249 |
"name": "gcc/14.0", |
| 250 |
"type": "modulefile", |
| 251 |
"symbols": [], |
| 252 |
"tags": [], |
| 253 |
"pathname": "/usr/share/modulefiles/gcc/14.0", |
| 254 |
"via": "" |
| 255 |
}, |
| 256 |
... |
| 257 |
} |
| 258 |
|
| 259 |
The JSON format is also useful for monitoring, reporting, and integration |
| 260 |
with external tools. |
| 261 |
|
| 262 |
Manipulating the environment for users |
| 263 |
-------------------------------------- |
| 264 |
|
| 265 |
Collections |
| 266 |
^^^^^^^^^^^ |
| 267 |
|
| 268 |
Collections save and restore sets of loaded modules. |
| 269 |
|
| 270 |
They provide a convenient way to switch between software environments without |
| 271 |
having to manually load and unload each module. |
| 272 |
|
| 273 |
Collections are managed through several :command:`module` sub-commands: |
| 274 |
|
| 275 |
.. list-table:: |
| 276 |
:header-rows: 1 |
| 277 |
:widths: 30 70 |
| 278 |
|
| 279 |
* - Sub-command |
| 280 |
- Description |
| 281 |
|
| 282 |
* - :subcmd:`save` [collection] |
| 283 |
- Save the current module environment to a collection. |
| 284 |
|
| 285 |
* - :subcmd:`restore` [collection] |
| 286 |
- Restore a collection or collection file. |
| 287 |
|
| 288 |
* - :subcmd:`savelist` |
| 289 |
- List saved collections. |
| 290 |
|
| 291 |
* - :subcmd:`saverm` [collection] |
| 292 |
- Remove a saved collection. |
| 293 |
|
| 294 |
* - :subcmd:`saveshow` [collection] |
| 295 |
- Display information about a collection. |
| 296 |
|
| 297 |
* - :subcmd:`is-saved` [collection] |
| 298 |
- Test whether one or more collections exist. |
| 299 |
|
| 300 |
Collections are stored under the user collection directory, usually |
| 301 |
:file:`$HOME/.module`. |
| 302 |
|
| 303 |
The :envvar:`MODULES_COLLECTION_TARGET` environment variable may be defined |
| 304 |
to append a suffix to collection names. This feature is particularly useful |
| 305 |
on systems sharing a common home directory across multiple clusters. |
| 306 |
|
| 307 |
.. parsed-literal:: |
| 308 |
|
| 309 |
:ps:`$` module save |
| 310 |
:ps:`$` module purge |
| 311 |
:ps:`$` module restore |
| 312 |
|
| 313 |
``stash`` commands |
| 314 |
^^^^^^^^^^^^^^^^^^ |
| 315 |
|
| 316 |
Stash collections temporarily save the current module environment and make it |
| 317 |
easy to switch to another environment. |
| 318 |
|
| 319 |
This feature is similar to the :command:`git stash` workflow: save the current |
| 320 |
state, perform other work, then restore the original environment later. |
| 321 |
|
| 322 |
Stash collections are managed through several :command:`module` |
| 323 |
sub-commands: |
| 324 |
|
| 325 |
.. list-table:: |
| 326 |
:header-rows: 1 |
| 327 |
:widths: 30 70 |
| 328 |
|
| 329 |
* - Sub-command |
| 330 |
- Description |
| 331 |
|
| 332 |
* - :subcmd:`stash` |
| 333 |
- Save the current environment and reset it. |
| 334 |
|
| 335 |
* - :subcmd:`stashpop` |
| 336 |
- Restore and remove a stash collection. |
| 337 |
|
| 338 |
* - :subcmd:`stashrm` |
| 339 |
- Remove a stash collection. |
| 340 |
|
| 341 |
* - :subcmd:`stashshow` |
| 342 |
- Display information about a stash collection. |
| 343 |
|
| 344 |
* - :subcmd:`stashclear` |
| 345 |
- Remove all stash collections. |
| 346 |
|
| 347 |
* - :subcmd:`stashlist` |
| 348 |
- List stash collections. |
| 349 |
|
| 350 |
Save the current environment: |
| 351 |
|
| 352 |
.. parsed-literal:: |
| 353 |
|
| 354 |
:ps:`$` module stash |
| 355 |
|
| 356 |
Load a different software stack: |
| 357 |
|
| 358 |
.. parsed-literal:: |
| 359 |
|
| 360 |
:ps:`$` module purge |
| 361 |
:ps:`$` module load ... |
| 362 |
|
| 363 |
Restore the original environment: |
| 364 |
|
| 365 |
.. parsed-literal:: |
| 366 |
|
| 367 |
:ps:`$` module stashpop |
| 368 |
|
| 369 |
Unlike regular collections, stash collections are intended to be short-lived |
| 370 |
and are automatically removed when restored with |
| 371 |
:subcmd:`stashpop`. |
| 372 |
|
| 373 |
``mogui`` |
| 374 |
^^^^^^^^^ |
| 375 |
|
| 376 |
``mogui`` provides a graphical interface to browse available modules and |
| 377 |
manage collections. |
| 378 |
|
| 379 |
It offers an alternative to the command line and can be useful for new users |
| 380 |
discovering a software stack or for users who prefer an interactive |
| 381 |
interface. |
| 382 |
|
| 383 |
Any environment change made from the GUI, such as loading modules or |
| 384 |
restoring collections, is applied back to the shell session that launched |
| 385 |
the application. |
| 386 |
|
| 387 |
The project is available on GitHub: |
| 388 |
`cea-hpc/mogui <https://github.com/cea-hpc/mogui>`_. It can be installed |
| 389 |
from PyPI with :command:`pip` or from Spack. |
| 390 |
|
| 391 |
.. parsed-literal:: |
| 392 |
|
| 393 |
:ps:`$` pip install modules-gui |
| 394 |
|
| 395 |
or: |
| 396 |
|
| 397 |
.. parsed-literal:: |
| 398 |
|
| 399 |
:ps:`$` spack install py-modules-gui |
| 400 |
|
| 401 |
Then start the graphical interface: |
| 402 |
|
| 403 |
.. parsed-literal:: |
| 404 |
|
| 405 |
:ps:`$` mogui |
| 406 |
|
| 407 |
.. image:: https://raw.githubusercontent.com/cea-hpc/mogui/main/doc/sneak_peek.gif |
| 408 |
:alt: mogui graphical interface |
| 409 |
:align: center |
| 410 |
|
| 411 |
``envml`` |
| 412 |
^^^^^^^^^ |
| 413 |
|
| 414 |
The :ref:`envml(1)` command executes a command in a specific module |
| 415 |
environment. |
| 416 |
|
| 417 |
It applies the requested module actions, executes the command, then restores |
| 418 |
the original environment. As a result, the current shell session remains |
| 419 |
unchanged. |
| 420 |
|
| 421 |
This feature is particularly useful in scripts, tests, and automation |
| 422 |
workflows where commands must be executed in a controlled software |
| 423 |
environment. |
| 424 |
|
| 425 |
Supported module actions include: |
| 426 |
|
| 427 |
.. list-table:: |
| 428 |
:header-rows: 1 |
| 429 |
:widths: 35 65 |
| 430 |
|
| 431 |
* - Action |
| 432 |
- Description |
| 433 |
|
| 434 |
* - ``purge`` |
| 435 |
- Purge all loaded modules. |
| 436 |
|
| 437 |
* - ``restore=<collection>`` |
| 438 |
- Restore a saved collection. |
| 439 |
|
| 440 |
* - ``unload=<module>`` |
| 441 |
- Unload one or more modules. |
| 442 |
|
| 443 |
* - ``switch=<module1>&<module2>`` |
| 444 |
- Switch from one module to another. |
| 445 |
|
| 446 |
* - ``load=<module>`` |
| 447 |
- Load one or more modules. |
| 448 |
|
| 449 |
* - ``<module>`` |
| 450 |
- Shorthand form of ``load=<module>``. |
| 451 |
|
| 452 |
Multiple actions may be chained with the ``:`` separator. When specifying |
| 453 |
multiple modules for a single action, use the ``&`` separator. In some |
| 454 |
shells, such as Bash, the ``&`` character must be escaped or quoted. |
| 455 |
|
| 456 |
For example, execute a command in a clean environment: |
| 457 |
|
| 458 |
.. parsed-literal:: |
| 459 |
|
| 460 |
:ps:`$` envml purge -- gcc hello.c |
| 461 |
|
| 462 |
The :ref:`envml(1)` command can simplify scripts by replacing sequences of |
| 463 |
module operations with a single command. |
| 464 |
|
| 465 |
Instead of: |
| 466 |
|
| 467 |
.. code-block:: bash |
| 468 |
|
| 469 |
#!/bin/bash |
| 470 |
module purge |
| 471 |
module load gcc |
| 472 |
gcc hello.c |
| 473 |
|
| 474 |
One can write: |
| 475 |
|
| 476 |
.. code-block:: bash |
| 477 |
|
| 478 |
#!/bin/bash |
| 479 |
envml purge:gcc -- gcc hello.c |
| 480 |
|
| 481 |
Protected environment variables |
| 482 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 483 |
|
| 484 |
The :mconfig:`protected_envvars` configuration option prevents Modules from |
| 485 |
modifying selected environment variables. Multiple variables may be specified |
| 486 |
using the ``:`` separator. |
| 487 |
|
| 488 |
This feature is useful to protect critical environment settings from |
| 489 |
unintended changes. |
| 490 |
|
| 491 |
When a modulefile attempts to modify a protected variable, Modules ignores |
| 492 |
the modification and emits a warning. |
| 493 |
|
| 494 |
.. parsed-literal:: |
| 495 |
|
| 496 |
:ps:`$` module config protected_envvars LD_PRELOAD |
| 497 |
:ps:`$` module load intel |
| 498 |
Loading :sgrhi:`intel/25.0` |
| 499 |
:sgrwa:`WARNING`: Modification of protected environment variable LD_PRELOAD ignored |
| 500 |
|
| 501 |
Developing and configuring modulefiles |
| 502 |
-------------------------------------- |
| 503 |
|
| 504 |
``module edit`` |
| 505 |
^^^^^^^^^^^^^^^ |
| 506 |
|
| 507 |
The :command:`module` :subcmd:`edit` sub-command opens a modulefile in the |
| 508 |
configured text editor (cf. :subcmd:`config` :mconfig:`editor`). |
| 509 |
|
| 510 |
Instead of manually locating a modulefile: |
| 511 |
|
| 512 |
.. parsed-literal:: |
| 513 |
|
| 514 |
:ps:`$` vi $(module path appA) |
| 515 |
|
| 516 |
One can directly edit it with: |
| 517 |
|
| 518 |
.. parsed-literal:: |
| 519 |
|
| 520 |
:ps:`$` module edit appA |
| 521 |
|
| 522 |
``module lint`` |
| 523 |
^^^^^^^^^^^^^^^ |
| 524 |
|
| 525 |
The :command:`module` :subcmd:`lint` sub-command analyzes modulefiles and |
| 526 |
reports potential issues. |
| 527 |
|
| 528 |
It relies on the Nagelfar Tcl syntax checker to detect syntax errors and |
| 529 |
common mistakes. |
| 530 |
|
| 531 |
Analyze a specific modulefile: |
| 532 |
|
| 533 |
.. parsed-literal:: |
| 534 |
|
| 535 |
:ps:`$` module lint appA |
| 536 |
|
| 537 |
Reported issues may include: |
| 538 |
|
| 539 |
* Unknown variables |
| 540 |
* Unknown commands |
| 541 |
* Invalid command arguments |
| 542 |
* Tcl syntax errors |
| 543 |
|
| 544 |
``source-sh`` |
| 545 |
^^^^^^^^^^^^^ |
| 546 |
|
| 547 |
The :mfcmd:`source-sh` modulefile command evaluates a shell script and tracks |
| 548 |
the environment changes it performs. |
| 549 |
|
| 550 |
The detected changes are translated into modulefile actions, making it |
| 551 |
possible to load and unload shell-based environments through Modules. |
| 552 |
|
| 553 |
It is particularly useful to integrate software distributions that provide |
| 554 |
environment setup scripts such as ``setup-env.sh``, ``mpivars.sh``, or |
| 555 |
similar rc-provided files. |
| 556 |
|
| 557 |
For example: |
| 558 |
|
| 559 |
.. code-block:: tcl |
| 560 |
|
| 561 |
#%Module |
| 562 |
|
| 563 |
source-sh bash /opt/spack/share/spack/setup-env.sh |
| 564 |
|
| 565 |
``sh-to-mod`` |
| 566 |
^^^^^^^^^^^^^ |
| 567 |
|
| 568 |
The :command:`module` :subcmd:`sh-to-mod` sub-command evaluates a shell script |
| 569 |
and reports the resulting environment changes as modulefile commands. |
| 570 |
|
| 571 |
It can be used to convert existing shell initialization scripts into |
| 572 |
modulefiles. |
| 573 |
|
| 574 |
For example: |
| 575 |
|
| 576 |
.. parsed-literal:: |
| 577 |
|
| 578 |
:ps:`$` module sh-to-mod bash setup-env.sh > setup-env.mod |
| 579 |
|
| 580 |
.. tip:: |
| 581 |
|
| 582 |
Use :subcmd:`sh-to-mod` when you want to generate and maintain a regular |
| 583 |
modulefile from an existing shell script. |
| 584 |
|
| 585 |
``mod-to-sh`` |
| 586 |
^^^^^^^^^^^^^ |
| 587 |
|
| 588 |
The :command:`module` :subcmd:`mod-to-sh` sub-command evaluates one or more |
| 589 |
modulefiles and reports the resulting environment changes as shell code. |
| 590 |
|
| 591 |
For example: |
| 592 |
|
| 593 |
.. parsed-literal:: |
| 594 |
|
| 595 |
:ps:`$` module mod-to-sh bash appA > setup-env.sh |
| 596 |
|
| 597 |
.. tip:: |
| 598 |
|
| 599 |
The :subcmd:`mod-to-sh` sub-command can be seen as the reverse operation of |
| 600 |
:subcmd:`sh-to-mod`. |
| 601 |
|
| 602 |
``prereq`` |
| 603 |
^^^^^^^^^^ |
| 604 |
|
| 605 |
The :mfcmd:`prereq` modulefile command declares one or more modules as a |
| 606 |
requirement of the current modulefile. When the automated module handling |
| 607 |
mode is enabled (see :mconfig:`auto_handling`), which is the default, a |
| 608 |
requirement is automatically loaded if not yet present in the user |
| 609 |
environment. |
| 610 |
|
| 611 |
Requirements are sometimes expressed with a condition block: |
| 612 |
|
| 613 |
.. code-block:: tcl |
| 614 |
|
| 615 |
if { ! [ is-loaded appA ] } { |
| 616 |
module load appA |
| 617 |
} |
| 618 |
|
| 619 |
This must be avoided: when ``appA`` is already loaded, the condition is |
| 620 |
false and no requirement rule is defined, so Modules is not aware that the |
| 621 |
current module depends on ``appA``. |
| 622 |
|
| 623 |
Declare the requirement unconditionally instead: |
| 624 |
|
| 625 |
.. code-block:: tcl |
| 626 |
|
| 627 |
prereq appA |
| 628 |
|
| 629 |
Loading a module declaring this requirement (here ``appB``) automatically |
| 630 |
loads the required module when missing: |
| 631 |
|
| 632 |
.. parsed-literal:: |
| 633 |
|
| 634 |
:ps:`$` module load appB |
| 635 |
Loading :sgrhi:`appB` |
| 636 |
:sgrin:`Loading requirement`: appA |
| 637 |
|
| 638 |
When automated module handling is disabled, a missing requirement is not |
| 639 |
automatically loaded: an error is reported and the modulefile load fails. |
| 640 |
|
| 641 |
The :mfcmd:`module load<module>` modulefile command equally declares a |
| 642 |
requirement rule, and it loads the missing required module even when |
| 643 |
automated module handling is disabled. The two commands also differ when |
| 644 |
several modules are specified: :mfcmd:`module load<module>` loads them all |
| 645 |
(logical AND) whereas :mfcmd:`prereq` requires only one of them (logical |
| 646 |
OR). Multiple :mfcmd:`prereq` commands act as a logical AND. |
| 647 |
|
| 648 |
``conflict`` |
| 649 |
^^^^^^^^^^^^ |
| 650 |
|
| 651 |
The :mfcmd:`conflict` modulefile command declares one or more modules that |
| 652 |
cannot be loaded together with the current modulefile. |
| 653 |
|
| 654 |
Instead of manually checking for incompatible modules: |
| 655 |
|
| 656 |
.. code-block:: tcl |
| 657 |
|
| 658 |
if { [ is-loaded appB ] } { |
| 659 |
puts stderr "ERROR: appB conflicts with appA!" |
| 660 |
exit 1 |
| 661 |
} |
| 662 |
|
| 663 |
Declare the conflict with: |
| 664 |
|
| 665 |
.. code-block:: tcl |
| 666 |
|
| 667 |
conflict appB |
| 668 |
|
| 669 |
When the :mconfig:`conflict_unload` configuration option is enabled in |
| 670 |
addition to the automated module handling mode (see :mconfig:`auto_handling`), |
| 671 |
loading a module automatically unloads the conflicting modules found in the |
| 672 |
user environment: |
| 673 |
|
| 674 |
.. parsed-literal:: |
| 675 |
|
| 676 |
:ps:`$` module load appA |
| 677 |
Loading :sgrhi:`appA` |
| 678 |
:sgrin:`Unloading conflict`: appB |
| 679 |
|
| 680 |
When these options are disabled, an error is reported instead and the |
| 681 |
modulefile load fails. |
| 682 |
|
| 683 |
A modulefile declaring a conflict on its own module name defines a |
| 684 |
*reflexive conflict*: only one version of this module can then be loaded at |
| 685 |
a time. |
| 686 |
|
| 687 |
.. code-block:: tcl |
| 688 |
|
| 689 |
conflict appA |
| 690 |
|
| 691 |
.. parsed-literal:: |
| 692 |
|
| 693 |
:ps:`$` module load appA/2.0 |
| 694 |
Loading :sgrhi:`appA/2.0` |
| 695 |
:sgrin:`Unloading conflict`: appA/1.0 |
| 696 |
|
| 697 |
.. tip:: |
| 698 |
|
| 699 |
When only one module should be loaded at a time for any given module |
| 700 |
name, enable the :mconfig:`unique_name_loaded` configuration option |
| 701 |
instead of declaring a reflexive conflict in every modulefile. As this |
| 702 |
option applies to all existing modules, every module name must follow |
| 703 |
this rule. It is generally the case when only *application* modules are |
| 704 |
provided, but may not be if application configurations are also handled |
| 705 |
through modulefiles. |
| 706 |
|
| 707 |
``variant`` |
| 708 |
^^^^^^^^^^^ |
| 709 |
|
| 710 |
The :mfcmd:`variant` modulefile command declares variants and their accepted |
| 711 |
values. |
| 712 |
|
| 713 |
This command is only available inside modulefiles. |
| 714 |
|
| 715 |
Variants allow users to customize the behavior of a modulefile at load time. |
| 716 |
They follow the same syntax as Spack variants. |
| 717 |
|
| 718 |
Boolean variants are enabled with ``+`` and disabled with ``~``: |
| 719 |
|
| 720 |
.. code-block:: tcl |
| 721 |
|
| 722 |
#%Module |
| 723 |
variant --boolean cuda |
| 724 |
|
| 725 |
.. parsed-literal:: |
| 726 |
|
| 727 |
:ps:`$` module load openmpi +cuda |
| 728 |
|
| 729 |
Variants with multiple values are declared by listing the accepted values: |
| 730 |
|
| 731 |
.. code-block:: tcl |
| 732 |
|
| 733 |
#%Module |
| 734 |
variant compiler gcc intel |
| 735 |
|
| 736 |
And selected with the ``=`` syntax: |
| 737 |
|
| 738 |
.. parsed-literal:: |
| 739 |
|
| 740 |
:ps:`$` module load openmpi compiler=intel |
| 741 |
|
| 742 |
The :mfcmd:`getvariant` modulefile command retrieves the value of a declared |
| 743 |
variant, allowing the modulefile to adapt its behavior. |
| 744 |
|
| 745 |
For example: |
| 746 |
|
| 747 |
.. code-block:: tcl |
| 748 |
|
| 749 |
switch -- [getvariant compiler] { |
| 750 |
gcc { |
| 751 |
prereq gcc |
| 752 |
} |
| 753 |
intel { |
| 754 |
prereq intel |
| 755 |
} |
| 756 |
} |
| 757 |
|
| 758 |
Configuration option :mconfig:`variant_shortcut` defines shortcut characters |
| 759 |
for variants. |
| 760 |
|
| 761 |
For example: |
| 762 |
|
| 763 |
.. parsed-literal:: |
| 764 |
|
| 765 |
:ps:`$` module config variant_shortcut compiler=% |
| 766 |
|
| 767 |
This allows users to write: |
| 768 |
|
| 769 |
.. parsed-literal:: |
| 770 |
|
| 771 |
:ps:`$` module load openmpi %gcc |
| 772 |
|
| 773 |
Advanced manipulations |
| 774 |
---------------------- |
| 775 |
|
| 776 |
Advanced module version specifiers |
| 777 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 778 |
|
| 779 |
:ref:`advanced_module_version_specifiers` provide a concise way to select modulefiles based on |
| 780 |
their version or declared variants. They can be used with several |
| 781 |
:command:`module` sub-commands, including :subcmd:`avail`, |
| 782 |
:subcmd:`spider`, :subcmd:`load`, and :subcmd:`switch`. |
| 783 |
|
| 784 |
Version specifications use the ``@`` character: |
| 785 |
|
| 786 |
.. list-table:: |
| 787 |
:header-rows: 1 |
| 788 |
:widths: 40 60 |
| 789 |
|
| 790 |
* - Specification |
| 791 |
- Description |
| 792 |
|
| 793 |
* - ``module avail cuda@default`` |
| 794 |
- Select the default version. |
| 795 |
|
| 796 |
* - ``module avail cuda@latest`` |
| 797 |
- Select the latest version. |
| 798 |
|
| 799 |
* - ``module avail cuda@12:`` |
| 800 |
- Select versions greater than or equal to ``12``. |
| 801 |
|
| 802 |
* - ``module avail cuda@:12`` |
| 803 |
- Select versions lower than or equal to ``12``. |
| 804 |
|
| 805 |
* - ``module avail cuda@12:13`` |
| 806 |
- Select versions between ``12`` and ``13``. |
| 807 |
|
| 808 |
Variant specifications may also be used: |
| 809 |
|
| 810 |
.. list-table:: |
| 811 |
:header-rows: 1 |
| 812 |
:widths: 40 60 |
| 813 |
|
| 814 |
* - Specification |
| 815 |
- Description |
| 816 |
|
| 817 |
* - ``module avail +mpi`` |
| 818 |
- Select modules declaring the ``mpi`` variant. |
| 819 |
|
| 820 |
* - ``module avail cuda=12`` |
| 821 |
- Select modules where the ``cuda`` variant is set to ``12``. |
| 822 |
|
| 823 |
* - ``module avail +mpi +cuda`` |
| 824 |
- Select modules matching both variant requirements. |
| 825 |
|
| 826 |
``--timer`` and ``--debug`` |
| 827 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 828 |
|
| 829 |
The :option:`--timer` and :option:`--debug`, or :option:`-D`, options help |
| 830 |
analyze and troubleshoot Modules commands. |
| 831 |
|
| 832 |
They are supported by most :command:`module` sub-commands. |
| 833 |
|
| 834 |
The :option:`--timer` option reports the total execution time of a command. |
| 835 |
|
| 836 |
For example: |
| 837 |
|
| 838 |
.. parsed-literal:: |
| 839 |
|
| 840 |
:ps:`$` module avail --timer |
| 841 |
------------------- :sgrmp:`/usr/share/modulefiles` ------------------ |
| 842 |
gcc/8.3 :sgrde:`gcc/11.1` gcc/14.0 openmpi/5.0.8 |
| 843 |
|
| 844 |
:sgrtr:`TIMER Total execution took 20.045 ms` |
| 845 |
|
| 846 |
The :option:`--debug`, or :option:`-D`, option displays debugging messages |
| 847 |
describing the internal execution of the command. |
| 848 |
|
| 849 |
For example: |
| 850 |
|
| 851 |
.. parsed-literal:: |
| 852 |
|
| 853 |
:ps:`$` module avail -D |
| 854 |
:sgrtr:`DEBUG setState: cmdline set to 'modulecmd.tcl bash avail -D'` |
| 855 |
:sgrtr:`DEBUG setState: shell set to 'bash'` |
| 856 |
:sgrtr:`DEBUG setState: subcmd set to 'avail'` |
| 857 |
:sgrtr:`DEBUG setConf: verbosity set to 'debug'` |
| 858 |
... |
| 859 |
------------------- :sgrmp:`/usr/share/modulefiles` ------------------ |
| 860 |
gcc/8.3 :sgrde:`gcc/11.1` gcc/14.0 openmpi/5.0.8 |
| 861 |
|
| 862 |
The :option:`--timer` and :option:`--debug` options may be combined. In this |
| 863 |
case, regular debug messages are replaced by execution time reports for each |
| 864 |
internal procedure call. |
| 865 |
|
| 866 |
.. parsed-literal:: |
| 867 |
|
| 868 |
:ps:`$` module avail --timer -D |
| 869 |
:sgrtr:`TIMER parseModuleCommandName avail help (0.151 ms)` |
| 870 |
:sgrtr:`TIMER isIcase (0.088 ms)` |
| 871 |
:sgrtr:`TIMER defineModStartNbProc 1 (0.060 ms)` |
| 872 |
... |
| 873 |
:sgrtr:`TIMER Total execution took 31.722 ms` |
| 874 |
|
| 875 |
For even more detailed debugging information, use :option:`-DD`. |
| 876 |
|
| 877 |
``.modulerc`` files |
| 878 |
^^^^^^^^^^^^^^^^^^^ |
| 879 |
|
| 880 |
A :file:`.modulerc` file contains Tcl code automatically evaluated by |
| 881 |
Modules when encountered during a command execution. |
| 882 |
|
| 883 |
Rc files are used to define module aliases, virtual |
| 884 |
modules, tags, hidden modules, forbidden modules, ... |
| 885 |
|
| 886 |
Several :file:`.modulerc` files may be evaluated during a single |
| 887 |
:command:`module` command. |
| 888 |
|
| 889 |
Depending on the executed sub-command, Modules may evaluate |
| 890 |
:file:`.modulerc` files found in different locations, including: |
| 891 |
|
| 892 |
* :envvar:`MODULERCFILE` |
| 893 |
* :file:`$HOME/.modulerc` |
| 894 |
* Modulepath and modules directories |
| 895 |
|
| 896 |
For example, the following directory hierarchy contains several |
| 897 |
:file:`.modulerc` files: |
| 898 |
|
| 899 |
.. code-block:: text |
| 900 |
|
| 901 |
/home/user/.modulerc |
| 902 |
modulefiles/ |
| 903 |
├── .modulerc |
| 904 |
├── app/ |
| 905 |
│ ├── .modulerc |
| 906 |
│ └── 1.0 |
| 907 |
└── mpi/ |
| 908 |
├── .modulerc |
| 909 |
└── openmpi/5.0 |
| 910 |
|
| 911 |
Depending on the requested operation, one or more of these files are |
| 912 |
evaluated automatically. |
| 913 |
|
| 914 |
See :ref:`Modulecmd startup` for details on the startup sequence, |
| 915 |
:mconfig:`rcfile` and :mconfig:`ignore_user_rc` to configure the evaluation |
| 916 |
of user rc files, and :sitevar:`modulerc_extra_vars` and |
| 917 |
:sitevar:`modulerc_extra_cmds` to extend :file:`.modulerc` files. |
| 918 |
|
| 919 |
``module-tag`` |
| 920 |
^^^^^^^^^^^^^^ |
| 921 |
|
| 922 |
The :mfcmd:`module-tag` modulerc command associates one or more tags with |
| 923 |
modulefiles. |
| 924 |
|
| 925 |
This command is only available inside :file:`.modulerc` files. |
| 926 |
|
| 927 |
Tags may be used to provide additional information about modulefiles or to |
| 928 |
modify their behavior. |
| 929 |
|
| 930 |
For example: |
| 931 |
|
| 932 |
.. code-block:: tcl |
| 933 |
|
| 934 |
module-tag experimental app/2.0 |
| 935 |
|
| 936 |
The tag is then reported along the module name in search results: |
| 937 |
|
| 938 |
.. parsed-literal:: |
| 939 |
|
| 940 |
:ps:`$` module avail |
| 941 |
------------------- :sgrmp:`/usr/share/modulefiles` ------------------ |
| 942 |
app/1.0 app/2.0 :sgrse:`<`\ experimental\ :sgrse:`>` |
| 943 |
|
| 944 |
Several predefined tags affect the behavior of Modules: |
| 945 |
|
| 946 |
.. list-table:: |
| 947 |
:header-rows: 1 |
| 948 |
:widths: 25 75 |
| 949 |
|
| 950 |
* - Tag |
| 951 |
- Description |
| 952 |
|
| 953 |
* - ``keep-loaded`` |
| 954 |
- Prevent automatic unloading when unloading dependent modules. |
| 955 |
|
| 956 |
* - ``sticky`` |
| 957 |
- Prevent unloading unless :option:`--force` is specified. |
| 958 |
|
| 959 |
* - ``super-sticky`` |
| 960 |
- Prevent unloading under any circumstances. |
| 961 |
|
| 962 |
For example: |
| 963 |
|
| 964 |
.. code-block:: tcl |
| 965 |
|
| 966 |
module-tag sticky core |
| 967 |
|
| 968 |
Once loaded, the ``core`` module cannot be unloaded, even with the |
| 969 |
:subcmd:`purge` sub-command. The ``sticky`` tag is rendered with a specific |
| 970 |
background color in the output. |
| 971 |
|
| 972 |
.. parsed-literal:: |
| 973 |
|
| 974 |
:ps:`$` module purge |
| 975 |
Unloading :sgrshi:`core/1.0` |
| 976 |
:sgrer:`ERROR`: Unload of sticky module skipped |
| 977 |
:ps:`$` module list |
| 978 |
Currently Loaded Modulefiles: |
| 979 |
1) :sgrs:`core/1.0` |
| 980 |
|
| 981 |
Sticky modules can help ensure that essential software stacks remain loaded. |
| 982 |
The behavior of :subcmd:`purge` with sticky modules can be configured through |
| 983 |
:mconfig:`sticky_purge`. |
| 984 |
|
| 985 |
``module-hide`` |
| 986 |
^^^^^^^^^^^^^^^ |
| 987 |
|
| 988 |
The :mfcmd:`module-hide` modulerc command hides modulefiles from module |
| 989 |
searches and selection. |
| 990 |
|
| 991 |
This command is only available inside :file:`.modulerc` files. |
| 992 |
|
| 993 |
A hidden module is excluded from regular searches but may still be selected |
| 994 |
when explicitly referred to by its exact name. |
| 995 |
|
| 996 |
For example: |
| 997 |
|
| 998 |
.. code-block:: tcl |
| 999 |
|
| 1000 |
module-hide app/1.0 |
| 1001 |
|
| 1002 |
Visibility may also be restricted to specific users or groups. |
| 1003 |
|
| 1004 |
Time-based restrictions can be defined with ``--after`` and ``--before``. |
| 1005 |
|
| 1006 |
For example: |
| 1007 |
|
| 1008 |
.. code-block:: tcl |
| 1009 |
|
| 1010 |
module-hide --after 2027-01-01 app/1.0 |
| 1011 |
|
| 1012 |
See :mfcmd:`module-hide` for additional visibility options, including **soft** |
| 1013 |
and **hard** hiding, as well as **users** and **groups** restrictions. |
| 1014 |
|
| 1015 |
``module-forbid`` |
| 1016 |
^^^^^^^^^^^^^^^^^ |
| 1017 |
|
| 1018 |
The :mfcmd:`module-forbid` modulerc command prevents designated modulefiles |
| 1019 |
from being loaded. |
| 1020 |
|
| 1021 |
This command is only available inside :file:`.modulerc` files. |
| 1022 |
|
| 1023 |
Unlike a hidden module, a forbidden module remains visible in module searches |
| 1024 |
but cannot be loaded. |
| 1025 |
|
| 1026 |
For example: |
| 1027 |
|
| 1028 |
.. code-block:: tcl |
| 1029 |
|
| 1030 |
module-forbid app/1.0 |
| 1031 |
|
| 1032 |
A custom message may be displayed when a user attempts to load a forbidden |
| 1033 |
module with the ``--message`` option. |
| 1034 |
|
| 1035 |
For example: |
| 1036 |
|
| 1037 |
.. code-block:: tcl |
| 1038 |
|
| 1039 |
module-forbid --message {Please use app/2.0 instead} app/1.0 |
| 1040 |
|
| 1041 |
See :mfcmd:`module-forbid` for additional restriction options, including |
| 1042 |
custom messages, as well as **after**/**before**, **users** and **groups** restrictions. |
| 1043 |
|
| 1044 |
``module-virtual`` |
| 1045 |
^^^^^^^^^^^^^^^^^^ |
| 1046 |
|
| 1047 |
The :mfcmd:`module-virtual` modulerc command associates a virtual module name |
| 1048 |
with an existing modulefile. |
| 1049 |
|
| 1050 |
This command is only available inside :file:`.modulerc` files. |
| 1051 |
|
| 1052 |
Several virtual modules may refer to the same modulefile, reducing the number |
| 1053 |
of files required to define similar modules. |
| 1054 |
|
| 1055 |
For example: |
| 1056 |
|
| 1057 |
.. code-block:: text |
| 1058 |
|
| 1059 |
modulefiles/app/ |
| 1060 |
├── .common |
| 1061 |
└── .modulerc |
| 1062 |
|
| 1063 |
The :file:`.modulerc` file may define several virtual versions: |
| 1064 |
|
| 1065 |
.. code-block:: tcl |
| 1066 |
|
| 1067 |
#%Module |
| 1068 |
module-virtual 1.0 .common |
| 1069 |
module-virtual 1.2 .common |
| 1070 |
module-virtual 1.5 .common |
| 1071 |
module-virtual 2.0 .common |
| 1072 |
|
| 1073 |
.. parsed-literal:: |
| 1074 |
|
| 1075 |
:ps:`$` module avail |
| 1076 |
------------------- :sgrmp:`/usr/share/modulefiles` ------------------ |
| 1077 |
app/1.0 app/1.2 app/1.5 app/2.0 |
| 1078 |
|
| 1079 |
All these virtual modules are evaluated using the same :file:`.common` |
| 1080 |
modulefile. |
| 1081 |
Its behavior can be adapted according to the virtual module |
| 1082 |
name being evaluated. |
| 1083 |
|
| 1084 |
.. code-block:: tcl |
| 1085 |
|
| 1086 |
#%Module |
| 1087 |
|
| 1088 |
set version [lindex [split [module-info name] /] end] |
| 1089 |
|
| 1090 |
setenv APP_VERSION $version |
| 1091 |
|
| 1092 |
Module logger |
| 1093 |
^^^^^^^^^^^^^ |
| 1094 |
|
| 1095 |
Modules can log module activity through the :mconfig:`logger` and |
| 1096 |
:mconfig:`logged_events` configuration options. |
| 1097 |
|
| 1098 |
The :mconfig:`logger` option defines the command used to record log messages. |
| 1099 |
|
| 1100 |
For example: |
| 1101 |
|
| 1102 |
.. parsed-literal:: |
| 1103 |
|
| 1104 |
:ps:`$` module config logger "/usr/bin/logger -t modules" |
| 1105 |
|
| 1106 |
The :mconfig:`logged_events` option defines which module events are recorded. |
| 1107 |
|
| 1108 |
.. list-table:: |
| 1109 |
:header-rows: 1 |
| 1110 |
:widths: 30 70 |
| 1111 |
|
| 1112 |
* - Event |
| 1113 |
- Description |
| 1114 |
|
| 1115 |
* - ``requested_cmd`` |
| 1116 |
- Record module commands requested by users. |
| 1117 |
|
| 1118 |
* - ``requested_eval`` |
| 1119 |
- Record modulefile evaluations requested by users. |
| 1120 |
|
| 1121 |
* - ``auto_eval`` |
| 1122 |
- Record modulefile evaluations automatically triggered by Modules. |
| 1123 |
|
| 1124 |
Multiple events may be specified using the ``:`` separator. |
| 1125 |
|
| 1126 |
For example: |
| 1127 |
|
| 1128 |
.. parsed-literal:: |
| 1129 |
|
| 1130 |
:ps:`$` module config logged_events requested_cmd:requested_eval |
| 1131 |
|
| 1132 |
By default, no events are logged. |
| 1133 |
|
| 1134 |
The logger command can be customized to integrate Modules activity with an |
| 1135 |
existing logging or monitoring infrastructure. |