Debian Codegraph @

doc/source/MIGRATING.rst

raw
1 .. _MIGRATING:
2
3 New features
4 ============
5
6 This document describes the major changes occurring between versions of
7 Modules. It provides an overview of the new features and changed behaviors
8 that will be encountered when upgrading.
9
10
11 v5.7
12 ----
13
14 This new version is backward-compatible with previous version 5 release. It
15 fixes bugs but also introduces new functionalities that are described in this
16 section. See the :ref:`5.7 release notes<5.7 release notes>` for a complete
17 list of the changes between Modules v5.6 and v5.7.
18
19 Improving evaluation performances
20 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
21
22 We observed that Modules could feel slow when evaluating more than one hundred
23 modulefiles within a single command. To address this, several improvements
24 have been implemented:
25
26 * In-memory caching of expensive operations: the results of costly operations,
27 such as module specification comparisons, are now cached in memory.
28
29 * Reworked environment synchronization between Tcl interpreters: environment
30 synchronization during Tcl modulefile evaluation has been redesigned to rely
31 on Tcl’s internal mechanisms rather than a custom reimplementation. As part
32 of this change, calls to the Tcl ``append`` command have been replaced with
33 ``set``, which reliably triggers synchronization when environment variables
34 are modified.
35
36 * More efficient dependency checks: when verifying dependencies against loaded
37 modules, comparisons are now limited to the subset of loaded modules that
38 match the module root name specification, instead of comparing against all
39 loaded modules.
40
41 * Lighter-weight resolution of an exact ``name/version`` specification: sites
42 with a modulepath directory holding hundreds or thousands of sibling
43 modulefiles reported slow ``module load`` commands even when the requested
44 version already exists. Correctly computing the implicit ``default`` and
45 ``latest`` symbolic versions of such a directory used to require validating
46 every sibling modulefile in it. The lookup now walks candidate versions from
47 the dictionary-highest to the dictionary-lowest and stops validating as soon
48 as this can be answered: either a higher valid version is found (the
49 requested one is then known not to be ``default`` or ``latest``), or the
50 requested version itself is reached, in which case
51 nothing higher exists. This bypasses the exhaustive scan without changing
52 the resolved result.
53
54 For this work, the :command:`mb` utility was extended with new benchmark tests
55 that loads 137 modulefiles (``load2``), lists the loaded modules (``list2``),
56 and then purges them. Running this benchmark highlights the performance
57 improvements, as shown in the table below:
58
59 +---------+-------------+--------------------+
60 | | v5.6.1 | v5.7.0 |
61 +=========+=============+====================+
62 | load2 | 1038 ms | 384 ms (-63%) |
63 +---------+-------------+--------------------+
64 | list2 | 222 ms | 29 ms (-87%) |
65 +---------+-------------+--------------------+
66 | purge | 568 ms | 303 ms (-46%) |
67 +---------+-------------+--------------------+
68
69 A ``load3`` benchmark test was also added, loading a low, non-default
70 version of a module out of 500 sibling versions in the same directory. It
71 specifically exercises the lighter-weight resolution described above:
72
73 +---------+-------------+--------------------+
74 | | v5.6.1 | v5.7.0 |
75 +=========+=============+====================+
76 | load3 | 30 ms | 19 ms (-36%) |
77 +---------+-------------+--------------------+
78
79 These measurements were obtained on a system where all modulefiles are stored
80 on local flash storage. If your modulefiles reside on a shared network
81 filesystem, using a `Module cache`_ is strongly recommended.
82
83 A module cache file aggregates the content of all the modulercs and
84 modulefiles found in a modulepath and this whole file is evaluated when a
85 module search occurs, even if the search targets specific modules. On setups
86 with a very large number of modulefiles, resulting in cache files of several
87 megabytes, evaluating the whole cache file may take longer than walking
88 through the modulepath directory to find the few requested modules. The new
89 :mconfig:`ignore_cache_subcmds` configuration option defines a list of
90 module sub-commands on which module cache files are ignored. For instance to
91 keep cache benefit on listing sub-commands but avoid its penalty on targeted
92 ones:
93
94 .. parsed-literal::
95
96 :ps:`$` module config ignore_cache_subcmds load:try-load:load-any
97
98 When the configuration option is changed from its default value using the
99 :subcmd:`config` sub-command, the :envvar:`MODULES_IGNORE_CACHE_SUBCMDS`
100 environment variable is defined accordingly. See
101 :envvar:`MODULES_IGNORE_CACHE_SUBCMDS` for details.
102
103 Reorder existing entries in path-like variables
104 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
105
106 A new :mconfig:`path_entry_reorder` configuration option is introduced to
107 control how path-like environment variables are modified when
108 :mfcmd:`prepend-path`, :mfcmd:`append-path` or :subcmd:`use` target a path
109 entry that is already defined in the environment variable.
110
111 By default, this configuration option is set to ``0``, meaning that when a
112 path entry already exists, the environment variable is left unchanged. When
113 set to ``1``, an existing entry is reordered to the beginning or the end of
114 the variable, depending on the command used, unless duplicates are explicitly
115 allowed.
116
117 .. parsed-literal::
118
119 :ps:`$` module append-path PATHVAR /foo
120 :ps:`$` module append-path PATHVAR /bar
121 :ps:`$` module append-path PATHVAR /foo
122 :ps:`$` echo $PATHVAR
123 /foo:/bar
124 :ps:`$` module config path_entry_reorder 1
125 :ps:`$` module append-path PATHVAR /foo
126 :ps:`$` echo $PATHVAR
127 /bar:/foo
128 :ps:`$` module append-path --duplicates PATHVAR /bar
129 :ps:`$` echo $PATHVAR
130 /bar:/foo:/bar
131
132 The default value of :mconfig:`path_entry_reorder` can be defined at
133 installation time with the :instopt:`--enable-path-entry-reorder` option. When
134 the configuration option is changed from its default value using the
135 :subcmd:`config` sub-command, the :envvar:`MODULES_PATH_ENTRY_REORDER`
136 environment variable is defined accordingly. See
137 :envvar:`MODULES_PATH_ENTRY_REORDER` for details.
138
139 .. _Linked environment variables:
140
141 Linked environment variables
142 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
143
144 A new configuration option, :mconfig:`linked_envvars`, can be used to
145 automatically mirror environment variable changes onto other environment
146 variables.
147
148 When a modulefile modifies an environment variable using commands such as
149 :mfcmd:`setenv`, :mfcmd:`unsetenv`, :mfcmd:`prepend-path`,
150 :mfcmd:`append-path`, :mfcmd:`remove-path` or :mfcmd:`pushenv`, the same
151 operation is also applied to all variables linked through the
152 :mconfig:`linked_envvars` configuration option.
153
154 For example, with the following configuration:
155
156 .. parsed-literal::
157
158 :ps:`$` module config linked_envvars "CPATH&INCLUDE_PATH"
159
160 loading a module that prepends a directory to ``CPATH``:
161
162 .. parsed-literal::
163
164 :sgrcm:`prepend-path` CPATH /opt/pkg/include
165
166 will also prepend the same directory to ``INCLUDE_PATH``.
167
168 Links are directional. In the previous example, changes made to ``CPATH`` are
169 also applied to ``INCLUDE_PATH``, but changes made directly to
170 ``INCLUDE_PATH`` are not applied back to ``CPATH`` unless an explicit reverse
171 link is configured.
172
173 When enabling this feature, note that existing values are not synchronized
174 when the link is established. Only subsequent modifications performed through
175 modulefile environment variable commands are propagated to linked variables.
176
177 The output of ``module display`` also includes the environment variable
178 changes applied to linked environment variables, making these propagated
179 updates visible before a module is loaded.
180
181 .. _Init environment variables:
182
183 Initialize environment variables on first modification
184 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
185
186 A new configuration option, :mconfig:`init_envvars`, has been added to define
187 initial values for environment variables that are modified through path-like
188 modulefile commands.
189
190 This feature is particularly useful for variables such as ``MANPATH``, which
191 require a leading or trailing colon character to preserve access to the system
192 default search paths. Previously, modifying an undefined ``MANPATH`` could
193 prevent system manual pages from being found. See the :ref:`man-path` cookbook
194 for details.
195
196 The :mconfig:`init_envvars` option defines a colon-separated list of
197 ``VARIABLE_NAME=initial_value`` entries. When a path-like operation is applied
198 to an environment variable that is not currently set, the configured initial
199 value is assigned before the modification is performed.
200
201 For example, with the following configuration:
202
203 .. parsed-literal::
204
205 :ps:`$` module config init_envvars MANPATH=
206
207 loading a module that prepends a directory to ``MANPATH``:
208
209 .. parsed-literal::
210
211 :sgrcm:`prepend-path` MANPATH /opt/pkg/man
212
213 will keep a colon character at the end of ``MANPATH``
214
215 .. parsed-literal::
216
217 :ps:`$` echo $MANPATH
218 /opt/pkg/man:
219
220 This mechanism applies to all path-like environment variable management
221 commands, including :mfcmd:`append-path`, :mfcmd:`prepend-path` and
222 :mfcmd:`remove-path`.
223
224 When values are later removed from an environment variable, it is
225 automatically unset if its resulting value matches the configured initial
226 value and no explicit reference counter is associated with it.
227
228 Hook API
229 ^^^^^^^^
230
231 Site code that needs to run right before or after a modulefile or modulerc
232 evaluation could so far only be attached through the ``trace`` Tcl command or
233 by renaming an internal :file:`modulecmd.tcl` procedure, both of which bind
234 to implementation details that may change from one Modules version to the
235 next.
236
237 A new ``add-hook`` siteconfig command is introduced to register a procedure
238 on one of four stable events without relying on such internal details:
239 :mhook:`before-modulefile-eval`, :mhook:`after-modulefile-eval`,
240 :mhook:`before-modulerc-eval` and :mhook:`after-modulerc-eval`. Several
241 procedures can be registered on the same event; they are then called in
242 their registration order.
243
244 .. code-block:: tcl
245
246 proc auditRequestedLoad {modfile modname modnamevr modspec mode requested} {
247 if {$requested && $mode eq {load}} {
248 set fd [open /var/log/modules-audit.log a]
249 puts $fd "[clock format [clock seconds]] $modnamevr"
250 close $fd
251 }
252 }
253 add-hook before-modulefile-eval auditRequestedLoad
254
255 An error raised by a hook procedure is reported but does not abort the
256 running ``module`` command nor prevent other procedures registered on the
257 same event from running, since these hooks fire on nearly every modulefile
258 or modulerc evaluation.
259
260 See the *Hooks* section of :manpage:`module(1)` man page for the full
261 argument contract of each event. The ``trace``/rename techniques remain
262 available for anything not (yet) covered by a hook event.
263
264 These four events are only the first ones introduced through ``add-hook``;
265 additional hook events will be added in the future as real site needs for
266 them come up. See :ref:`add-new-hook-event` for the guide contributors can
267 follow to propose one.
268
269 Ignore files in modulepath
270 ^^^^^^^^^^^^^^^^^^^^^^^^^^
271
272 Modulepath directories sometimes hold files that are not modulefiles, like
273 helper scripts sourced by modulefiles or software installation files when
274 modulefiles are mixed with the software trees they describe. The new
275 :mfcmd:`modulepath-ignore` modulefile command lets modulepath maintainers
276 describe from rc files such content to ignore when searching for
277 modulefiles. Ignored files and directories are handled as if they do not
278 exist: they are not checked, not walked down, not reported and not recorded
279 in modulepath cache file. Cache build time and size are thus kept
280 proportional to the number of actual modulefiles, whatever the amount of
281 other files stored in modulepath.
282
283 Patterns follow the same syntax and behavior than git with
284 :file:`.gitignore` files. For instance to ignore every file with the
285 ``.tcl`` extension and the whole content of the ``soft`` directory, add to
286 the ``.modulerc`` file at the root of the modulepath::
287
288 modulepath-ignore *.tcl soft/
289
290 Negated patterns switch to an *ignore all, accept some* logic. For instance
291 to only take files with the ``.mod`` extension into account::
292
293 modulepath-ignore * !*/ !*.mod
294
295 When defined in a ``.modulerc`` file located within a modulepath, patterns
296 are relative to the directory of this rc file. When defined in a global or
297 user rc file, patterns apply to every modulepath. As ignored elements are
298 excluded from cache files when they are built, pattern definitions should
299 not be conditional.
300
301 The :mconfig:`modulepath_ignore` configuration option, enabled by default,
302 controls whether pattern definitions are applied. Patterns can be skipped
303 for one command execution with the :option:`--no-modulepath-ignore` command
304 line switch, which helps finding out whether a module is missing because it
305 is ignored.
306
307
308 v5.6
309 ----
310
311 This new version is backward-compatible with previous version 5 release. It
312 fixes bugs but also introduces new functionalities that are described in this
313 section. See the :ref:`5.6 release notes<5.6 release notes>` for a complete
314 list of the changes between Modules v5.5 and v5.6.
315
316 spider sub-command
317 ^^^^^^^^^^^^^^^^^^
318
319 Support has been added for the :subcmd:`spider` sub-command introduced by the
320 `Lmod`_ project. This sub-command lists available modules found in enabled
321 modulepaths and recursively found in modulepaths enabled by available modules.
322
323 .. parsed-literal::
324
325 :ps:`$` module avail
326 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
327 foo/1 foo/2
328 :ps:`$` module show foo/1
329 -------------------------------------------------------------------
330 :sgrhi:`/path/to/modulefiles/foo/1`:
331
332 :sgrcm:`conflict` foo
333 :sgrcm:`module` use /path/to/modulefiles.2
334 -------------------------------------------------------------------
335 :ps:`$` module show foo/2
336 -------------------------------------------------------------------
337 :sgrhi:`/path/to/modulefiles/foo/2`:
338
339 :sgrcm:`conflict` foo
340 :sgrcm:`append-path` MODULEPATH /path/to/modulefiles.3
341 -------------------------------------------------------------------
342 :ps:`$` module spider
343 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
344 foo/1 foo/2
345
346 -------------- :sgrdi:`/path/to/modulefiles.2` (via foo/1) --------------
347 bar/1 bar/2 bar/3
348
349 -------------- :sgrdi:`/path/to/modulefiles.3` (via foo/2) --------------
350 bar/3 bar/4
351
352 -------------- :sgrdi:`/path/to/modulefiles.3` (via bar/1) --------------
353 qux/1 qux/2
354
355 The :subcmd:`spider` sub-command relies on the :ref:`Extra match search`
356 mechanism to scan modulefiles and identify the modulepaths they enable. Since
357 all modulefiles are evaluated during this scan, it is recommended to build
358 and use a :ref:`Module cache` to improve search performance.
359
360 The output of :subcmd:`spider` is similar to that of the :subcmd:`avail`
361 sub-command and supports the same set of options and queries.
362
363 .. parsed-literal::
364
365 :ps:`$` module spider -t bar@2:
366 :sgrdi:`/path/to/modulefiles.2`:
367 :sgrhi:`bar/2`
368 bar/3
369
370 :sgrdi:`/path/to/modulefiles.3`:
371 bar/3
372 bar/4
373
374 The :subcmd:`spider` sub-command has its own configuration options to control
375 the content of its output: :mconfig:`spider_indepth`,
376 :mconfig:`spider_output`, and :mconfig:`spider_terse_output`.
377
378 These settings make it possible to define output for :subcmd:`spider` that
379 differs from the :subcmd:`avail` sub-command.
380
381 .. parsed-literal::
382
383 :ps:`$` module config spider_output -modulepath
384 :ps:`$` module config spider_indepth 0
385 :ps:`$` module spider
386 :sgrdi:`bar`/ :sgrdi:`foo`/ :sgrdi:`qux`/
387
388 When a modulepath is enabled by a specific module, that modulepath is
389 considered available *via* that module. When the module is loaded, this *via*
390 information is stored in the :envvar:`__MODULES_LMUSE` environment variable to
391 track which modules enable which modulepaths.
392
393 By default, the *via* information is included in the standard output of the
394 :subcmd:`spider` sub-command. Additionally, the JSON output for both the
395 :subcmd:`avail` and :subcmd:`spider` sub-commands has been updated to include
396 *via* details for each modulefile.
397
398 .. parsed-literal::
399
400 :ps:`$` module spider -j bar@2:
401 {"/path/to/modulefiles.2": {
402 "bar/2": { "name": "bar/2", "type": "modulefile", "symbols": [], "tags": [], "pathname": "/path/to/modulefiles.2/bar/2", "via": "foo/1"}
403 "bar/2": { "name": "bar/3", "type": "modulefile", "symbols": [], "tags": [], "pathname": "/path/to/modulefiles.2/bar/3", "via": "foo/1"}
404 },
405 "/path/to/modulefiles.3": {
406 "bar/3": { "name": "bar/3", "type": "modulefile", "symbols": [], "tags": [], "pathname": "/path/to/modulefiles.3/bar/3", "via": "foo/2"},
407 "bar/4": { "name": "bar/4", "type": "modulefile", "symbols": [], "tags": [], "pathname": "/path/to/modulefiles.3/bar/4", "via": "foo/2"}
408 }}
409
410 .. _Requiring via module:
411
412 Requiring *via* module
413 ^^^^^^^^^^^^^^^^^^^^^^
414
415 A loaded module that enables a modulepath is considered the *via* module of
416 other loaded modules whose modulefiles are stored in this modulepath. The
417 :mconfig:`require_via` configuration option is introduced to consider a *via*
418 module a requirement for the loaded module stored in the modulepath it
419 enables.
420
421 A *module hierarchy* mechanism, as introduced by the Lmod_ project, is now
422 supported with this feature. It allows organizing modulefiles through a
423 primary modulepath, where loading certain modules enables additional
424 modulepaths. :mconfig:`require_via` maintains a link between modulefiles in
425 these additional paths and the *via* module that activated them.
426
427 The :mconfig:`require_via` setting is disabled by default to maintain
428 compatibility with previous Modules 5 releases. However, users are encouraged
429 to enable this feature if they want to use the *module hierarchy* mechanism.
430
431 .. parsed-literal::
432
433 :ps:`$` module config require_via 1
434
435 When unloading a *via* module, all the modules stored in its enabled
436 modulepath are automatically unloaded thanks to the *Dependent Unload*
437 mechanism of the :ref:`Automated module handling mode<MODULES_AUTO_HANDLING>`.
438
439 .. parsed-literal::
440
441 :ps:`$` module load foo/1 bar/2
442 :ps:`$` module avail
443 -------------- :sgrdi:`/path/to/modulefiles.2` (via foo/1) --------------
444 bar/1 :sgrl:`bar/2` bar/3
445
446 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
447 :sgrl:`foo/1` foo/2
448
449 Key:
450 :sgrl:`loaded` :sgrdi:`modulepath`
451 :ps:`$` module unload foo
452 Unloading :sgrhi:`foo/1`
453 :sgrin:`Unloading dependent`: bar/2
454
455 When a *via* module is replaced by another, any modulepaths it enabled are
456 updated accordingly. If the newly enabled modulepath provides alternatives to
457 modules previously loaded from the original path, those modules are reloaded
458 from the new location. If no alternatives are found, the dependent modules are
459 unloaded.
460
461 .. parsed-literal::
462
463 :ps:`$` module config conflict_unload 1
464 :ps:`$` module load foo/1 bar/3
465 :ps:`$` module load foo/2
466 Loading :sgrhi:`foo/2`
467 :sgrin:`Unloading dependent`: bar/3
468 :sgrin:`Unloading conflict`: foo/1
469 :sgrin:`Reloading dependent`: bar/3
470 :ps:`$` module avail
471 -------------- :sgrdi:`/path/to/modulefiles.3` (via foo/2) --------------
472 :sgrl:`bar/3` bar/4
473
474 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
475 foo/1 :sgrl:`foo/2`
476
477 Key:
478 :sgrl:`loaded` :sgrdi:`modulepath`
479
480 Always see hidden modules
481 ^^^^^^^^^^^^^^^^^^^^^^^^^
482
483 The ``hidden`` element is a new allowed value for the :mconfig:`avail_output`,
484 :mconfig:`avail_terse_output`, :mconfig:`list_output`,
485 :mconfig:`list_terse_output`, :mconfig:`spider_output` and
486 :mconfig:`spider_terse_output` configuration options. When set into the value
487 list of these options hidden modules are shown on the output of corresponding
488 commands (:subcmd:`avail`, :subcmd:`list` and :subcmd:`spider`).
489
490 .. parsed-literal::
491
492 --------------- :sgrdi:`/path/to/modulefiles` ---------------
493 bar/1.0
494 :ps:`$` module config avail_output +hidden
495 :ps:`$` ml av
496 --------------- :sgrdi:`/path/to/modulefiles` ---------------
497 bar/1.0 foo/1.0 <H>
498
499 When set, this is equivalent to always adding the :option:`--all` command-line
500 option.
501
502 Issue warning upon module evaluation
503 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
504
505 The mfcmd:`module-warn` modulefile Tcl command is introduced to define warning
506 message to print when loading specified modulefiles.
507
508 .. parsed-literal::
509
510 :ps:`$` cat /path/to/modulefiles/foo/1.0
511 #%Module
512 module-warn --message {Please consider using "bar" module instead} foo
513 :ps:`$` module load foo
514 Loading :sgrhi:`foo/1.0`
515 :sgrwa:`WARNING`: Please consider using "bar" module instead
516
517 The warning message is printed when targeted modulefile is evaluated in
518 *load*, *display*, *help* or *test* mode.
519
520 :mfcmd:`module-warn` supports the same kind of options than
521 :mfcmd:`module-tag` or :mfcmd:`module-hide`: it is possible to specify a list
522 of affected or unaffected users or groups. Warning may also be only effective
523 before or after a particular date.
524
525 The ``warning`` tag is set on modules targeted by a mfcmd:`module-warn`
526 command.
527
528 Simplified way to define module help message
529 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
530
531 Until now module help message was defined through a specific Tcl procedure
532 that outputs text to *stderr* channel:
533
534 .. code-block:: tcl
535
536 #%Module
537 proc ModulesHelp {} {
538 puts stderr {Module Name: foo, Version: 1.0}
539 puts stderr {This modulefile does this and that}
540 }
541
542 A new modulefile Tcl command is introduced, :mfcmd:`module-help`, to simplify
543 a bit the definition of help message:
544
545 .. code-block:: tcl
546
547 #%Module
548 module-help {Module Name: foo, Version: 1.0}
549 module-help {This modulefile does this and that}
550
551 With this modulefile command there is no need to define the ``ModulesHelp``
552 procedure anymore. As :mfcmd:`module-help` is reported on module *display*
553 evaluation, help message will also be visible there.
554
555 Support for newline character in environment variable value
556 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
557
558 It is now possible on *sh*, *bash*, *ksh*, *zsh* and *fish* shells to set
559 value for environment variable that contains newline character:
560
561 .. parsed-literal::
562
563 :ps:`$` cat /path/to/modulefiles/bar/1.0
564 #%Module
565 setenv BAR "multi\nline\nvalue"
566 :ps:`$` module load bar
567 :ps:`$` echo "$BAR"
568 multi
569 line
570 value
571
572 Generated shell code on these shells has been changed to enclose value within
573 single quotes rather escaping each special characters:
574
575 .. parsed-literal::
576
577 :ps:`$` module mod-to-sh bash bar/1.0
578 BAR='multi
579 line
580 value'; export BAR;
581
582 Declare additional elements provided by a module
583 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
584
585 New :mfcmd:`provide` modulefile command is added to declare additional
586 elements included in a modulefile. When the modulefile is loaded, each
587 argument specified in the :mfcmd:`provide` command is registered as an alias
588 for the module.
589
590
591 .. parsed-literal::
592
593 :ps:`$` module show foo
594 -------------------------------------------------------------------
595 :sgrhi:`/path/to/modulefiles/foo/1`:
596
597 :sgrcm:`provide` liba/1.2 libb/2.1
598 :sgrcm:`conflict` foo
599 :sgrcm:`conflict` liba libb
600 -------------------------------------------------------------------
601 :ps:`$` module load foo
602
603 It is advised to define :mfcmd:`conflict` over the name of these additional
604 elements to ensure only one provider is found loaded in user environment.
605
606 .. parsed-literal::
607
608 :ps:`$` module show bar/1
609 -------------------------------------------------------------------
610 :sgrhi:`/path/to/modulefiles/bar/1`:
611
612 :sgrcm:`provide` liba/1.2 libb/2.1
613 :sgrcm:`conflict` bar
614 :sgrcm:`conflict` liba libb
615 -------------------------------------------------------------------
616 :ps:`$` module load bar
617 Loading :sgrhi:`bar/1`
618 :sgrer:`ERROR`: Module cannot be loaded due to a conflict.
619 HINT: Might try "module unload foo/1" first.
620
621 Once a module with a provided alias is loaded, it fulfills any requirements
622 declared by other modules for that name and version, just like any other type
623 of module alias.
624
625 .. parsed-literal::
626
627 :ps:`$` module show qux
628 -------------------------------------------------------------------
629 :sgrhi:`/path/to/modulefiles/qux/1`:
630
631 :sgrcm:`prereq` liba/1.2
632 -------------------------------------------------------------------
633 :ps:`$` module load qux
634 :ps:`$` module list
635 Currently Loaded Modulefiles:
636 1) foo/1 2) qux/1
637
638 Module aliases defined via the :mfcmd:`provide` command are not available
639 during module resolution, as modulefiles are not evaluated in that context.
640 Therefore, it is recommended to define a global module alias in a ``modulerc``
641 file using the :mfcmd:`module-alias` command to specify a default target for
642 the alias.
643
644 .. parsed-literal::
645
646 :ps:`$` cat /path/to/modulefiles/.modulerc
647 #%Module
648 module-alias liba/1.2 bar/1
649 :ps:`$` module purge
650 :ps:`$` module load qux
651 Loading :sgrhi:`qux/1`
652 :sgrin:`Loading requirement`: bar/1
653
654 The :mfcmd:`provide` modulefile command also serves as an alias for the
655 :mfcmd:`extensions` command, ensuring compatibility with modulefiles written
656 for `Lmod`_.
657
658 By default an :subcmd:`avail` command includes module aliases defined in
659 ``modulerc`` files. To also get aliases defined within modulefiles with
660 :mfcmd:`provide` or :mfcmd:`family`, output element ``provided-alias`` has to
661 be included. It is supported on :mconfig:`avail_output`,
662 :mconfig:`avail_terse_output`, :mconfig:`spider_output` and
663 :mconfig:`spider_terse_output` configuration options.
664
665 Having ``provided-alias`` included in output triggers :ref:`Extra match
666 search` mechanism that scans all modulefiles to find these aliases. It is thus
667 advised to have :ref:`Module cache` built and used to improve search speed.
668
669 .. parsed-literal::
670
671 :ps:`$` module avail
672 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
673 bar/1 foo/1 :sgrali:`liba/1.2` qux/1
674
675 Key:
676 :sgrdi:`modulepath` :sgrali:`module-alias`
677 :ps:`$` module avail -o +provided-alias
678 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
679 bar/1 foo/1 :sgrali:`liba/1.2` :sgrali:`libb/2.1` qux/1
680
681 Key:
682 :sgrdi:`modulepath` :sgrali:`module-alias`
683
684 The ``provided-alias`` :ref:`Extra specifier` is also introduced to find
685 modules that define such alias with :mfcmd:`provide` or :mfcmd:`family`
686 modulefile commands.
687
688 .. parsed-literal::
689
690 :ps:`$` module avail provided-alias:liba/1.2
691 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
692 bar/1 foo/1
693
694 Key:
695 :sgrdi:`modulepath`
696
697
698 v5.5
699 ----
700
701 This new version is backward-compatible with previous version 5 release. It
702 fixes bugs but also introduces new functionalities that are described in this
703 section. See the :ref:`5.5 release notes<5.5 release notes>` for a complete
704 list of the changes between Modules v5.4 and v5.5.
705
706 .. _Logging activity:
707
708 Logging activity
709 ^^^^^^^^^^^^^^^^
710
711 The ability to log module command activity is now available out of the box. It
712 could be enabled on previous versions by using a specific
713 :file:`siteconfig.tcl` configuration script as described in the
714 :ref:`log-module-command` cookbook recipe.
715
716 Integrated logging feature relies on two configuration options:
717
718 * :mconfig:`logger`, the command run to transmit messages to the log system
719 * :mconfig:`logged_events`, list of module event to log
720
721 :mconfig:`logger` option relies on the :command:`logger` command by default,
722 which is usually available and already installed on most systems.
723
724 The :mconfig:`logged_events` option is empty by default, so no content is sent
725 to logs by default. It recognizes the following events:
726
727 * ``requested_cmd``: record module commands typed by users
728 * ``requested_eval``: record modulefile evaluations requested by users
729 * ``auto_eval``: record modulefile evaluations automatically triggered
730
731 .. parsed-literal::
732
733 :ps:`$` module config logged_events +requested_cmd:requested_eval
734 :ps:`$` ml av
735 --------------- :sgrdi:`/path/to/modulefiles` ---------------
736 bar/1.0 foo/1.0 qux/1.0
737 :ps:`$` module load bar
738 Loading :sgrhi:`bar/1.0`
739 :sgrin:`Loading requirement`: qux/1.0
740 :ps:`$` module purge
741
742 In the above example the module command and modulefile evaluations directly
743 requested by users are sent to the system log:
744
745 .. parsed-literal::
746
747 :ps:`$` journalctl -q -t modules
748 Apr 29 07:47:42 hostname modules[3777797]: user="username" command="avail" arguments=""
749 Apr 29 07:48:10 hostname modules[3777876]: user="username" command="load" arguments="bar"
750 Apr 29 07:48:10 hostname modules[3777876]: user="username" mode="load" module="bar/1.0" specified="bar" modulefile="/path/to/modulefiles/bar/1.0" requested="1"
751 Apr 29 07:48:17 hostname modules[3777914]: user="username" command="purge" arguments=""
752 Apr 29 07:48:17 hostname modules[3777914]: user="username" mode="unload" module="bar/1.0" specified="bar/1.0" modulefile="/path/to/modulefiles/bar/1.0" requested="1"
753
754 Some messages can also be sent during modulefile evaluation by using the
755 ``log`` channel of the :mfcmd:`puts` command:
756
757 .. parsed-literal::
758
759 :ps:`$` cat /path/to/modulefiles/foo/1.0
760 #%Module
761 puts log {some message sent to log}
762 :ps:`$` ml foo/1.0
763 :ps:`$` journalctl -q -t modules
764 Apr 29 07:47:51 hostname modules[3777835]: user="username" command="ml" arguments="foo/1.0"
765 Apr 29 07:47:51 hostname modules[3777835]: some message sent to log
766 Apr 29 07:47:51 hostname modules[3777835]: user="username" mode="load" module="foo/1.0" specified="foo/1.0" modulefile="/path/to/modulefiles/foo/1.0" requested="1"
767
768 Negating extra match search criteria
769 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
770
771 The ``not:`` prefix string is introduced for extra specifier and variant
772 criteria. When this prefix is used, the search criteria where it is applied is
773 negated.
774
775 .. parsed-literal::
776
777 :ps:`$` module config avail_output +variant
778 :ps:`$` module config variant_shortcut toolchain=%
779 :ps:`$` module avail
780 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
781 bar/1.0{:sgrva:`%x86_64`} foo/2.0{:sgrva:`%x86_64`,\ :sgrva:`arm64`}
782 bar/2.0{:sgrva:`%x86_64`,\ :sgrva:`arm64`} qux/1.0{:sgrva:`%x86_64`}
783 foo/1.0{:sgrva:`%x86_64`} qux/2.0{:sgrva:`%x86_64`,\ :sgrva:`arm64`}
784 :ps:`$` module avail not:envvar:FOO,BAR
785 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
786 qux/1.0{:sgrva:`%x86_64`} qux/2.0{:sgrva:`%x86_64`,\ :sgrva:`arm64`}
787
788 The above search query returns all modules not defining ``FOO`` and ``BAR``
789 environment variables.
790
791 .. parsed-literal::
792
793 :ps:`$` module avail not:envvar:BAR not:%arm64
794 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
795 foo/1.0{:sgrva:`%x86_64`} qux/1.0{:sgrva:`%x86_64`}
796
797 The above example returns all modules not defining ``BAR`` environment
798 variable and not defining ``toolchain`` variant or defining this variant
799 without ``arm64`` among the possible values.
800
801 This prefix is recognized on module search context (i.e., :subcmd:`avail`,
802 :subcmd:`whatis` and :subcmd:`paths` sub-commands).
803
804 PowerShell support
805 ^^^^^^^^^^^^^^^^^^
806
807 Support for PowerShell (``pwsh``) is added. This shell may be used with
808 Modules on either Unix or Windows platforms. All main :ref:`kind of
809 environment changes are supported<Shell support>`.
810
811 A ``envmodule`` shell function is configured on PowerShell, as ``module`` is a
812 reserved keyword on this shell.
813
814 The Windows-specific distribution zipball of Modules has been updated to also
815 contain the required files to setup PowerShell support.
816
817 .. _Conflict unload MIGRATING:
818
819 Conflict unload
820 ^^^^^^^^^^^^^^^
821
822 Automated module handling is enhanced with the addition of a new mechanism:
823 *Conflict Unload*. When loading a module, this new mechanism automatically
824 unloads conflicting modules and their dependent.
825
826 .. parsed-literal::
827
828 :ps:`$` module load foo/1
829 Loading :sgrhi:`foo/1`:sgrse:`{`:sgrvahi:`-debug`:sgrse:`}`
830 :sgrin:`Loading requirement`: qux/1 bar/1
831 :ps:`$` module load foo/2
832 Loading :sgrhi:`foo/2`
833 :sgrer:`ERROR`: Module cannot be loaded due to a conflict.
834 HINT: Might try "module unload foo/1" first.
835 :ps:`$` module config conflict_unload 1
836 :ps:`$` module load foo/2
837 Loading :sgrhi:`foo/2`:sgrse:`{`:sgrvahi:`-debug`:sgrse:`}`
838 :sgrin:`Unloading conflict`: foo/1\ :sgrse:`{`:sgrva:`-debug`:sgrse:`}`
839
840 As seen in the above example, *Conflict Unload* is controlled by the
841 :mconfig:`conflict_unload` configuration option. It is disabled by default
842 as it changes behaviors of the :envvar:`automated module handling
843 mode<MODULES_AUTO_HANDLING>`. But everyone is encouraged to enable this new
844 option to benefit from an highly automated experience. This option can be
845 changed at installation time with :instopt:`--enable-conflict-unload`
846 configure script option.
847
848 *Conflict Unload* handles all kind of conflicts: conflicts defined by already
849 loaded modules, conflict declared by loading module through :mfcmd:`conflict`,
850 :mfcmd:`family` or :mfcmd:`module unload<module>` commands. It also supports
851 the unload of a module that is loaded again but with different variant values.
852
853 .. parsed-literal::
854
855 :ps:`$` module load foo/2 +debug
856 Loading :sgrhi:`foo/2`:sgrse:`{`:sgrvahi:`+debug`:sgrse:`}`
857 :sgrin:`Unloading conflict`: foo/2\ :sgrse:`{`:sgrva:`-debug`:sgrse:`}`
858
859 Along with *Conflict Unload*, the *Useless Requirement Unload* mechanism is
860 added to load evaluation. With it, auto loaded requirements of modules
861 unloaded by *Conflict Unload* mechanism are automatically unloaded.
862
863 .. parsed-literal::
864
865 :ps:`$` module load foo/3
866 Loading :sgrhi:`foo/3`:sgrse:`{`:sgrvahi:`-debug`:sgrse:`}`
867 :sgrin:`Unloading conflict`: foo/2\ :sgrse:`{`:sgrva:`+debug`:sgrse:`}` bar/1
868 :sgrin:`Loading requirement`: bar/2
869 :sgrin:`Unloading useless requirement`: qux/1
870
871 Dependent modules of *Conflict Unload* modules are considered *Dependent
872 Reload* modules. They are unloaded prior unloading the *Conflict Unload*
873 module that depend on them. Reload occurs after loading main module and only
874 if the loaded environment satisfies the dependencies of the module to reload.
875
876 .. parsed-literal::
877
878 :ps:`$` module load bar/3
879 Loading :sgrhi:`bar/3`
880 :sgrin:`Loading requirement`: qux/2
881 :ps:`$` module load qux/1
882 Loading :sgrhi:`qux/1`
883 :sgrin:`Unloading dependent`: bar/3
884 :sgrin:`Unloading conflict`: qux/2
885 :sgrin:`Reloading dependent`: bar/3
886
887 Improved automated module handling mechanisms
888 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
889
890 Along with the introduction of *Conflict Unload* mechanism, some of the
891 existing automated module handling mechanisms have been adapted and enhanced.
892
893 *Useless Requirement Unload* (UReqUn) mechanism is moved during a
894 :subcmd:`switch` sub-command from the end of the unload phase to the end of
895 the load phase. Both *UReqUn* modules coming from unload and load phases are
896 this way treated jointly. If *UReqUn* modules resulting from the switch unload
897 phase are detected as conflict to the switched-on module, it is advised to
898 enable the :mconfig:`conflict_unload` mechanism to automatically handle them.
899
900 .. parsed-literal::
901
902 :ps:`$` module switch foo/3 foo/4
903 Loading :sgrhi:`foo/4`
904 :sgrer:`ERROR`: Module cannot be loaded due to a conflict.
905 HINT: Might try "module unload bar/2" first.
906
907 Switching from :sgrhi:`foo/3`:sgrse:`{`:sgrvahi:`-debug`:sgrse:`}` to :sgrhi:`foo/4`
908 :sgrer:`ERROR`: Load of switched-on foo/4 failed
909 :ps:`$` module config conflict_unload 1
910 :ps:`$` module switch foo/3 foo/4
911 Switching from :sgrhi:`foo/3`:sgrse:`{`:sgrvahi:`-debug`:sgrse:`}` to :sgrhi:`foo/4`
912 :sgrin:`Unloading conflict`: bar/2
913
914 *Dependent Reload* (DepRe) modules that also are *UReqUn* modules are now
915 unloaded during the *DepRe* unload phase instead of during the *UReqUn*
916 process. As a consequence these modules are now unloaded prior main module
917 action, as they are a dependent of this main module. This kind of *UReqUn*
918 modules may not be a dependency of a module unloaded in the current
919 processing.
920
921 Modules from *Dependent Unload* (DepUn) and *Dependent Reload* (DepRe)
922 mechanisms are now mixed together to proceed to their unload. They are this
923 way unloaded in their reverse loading order. So if a module is part of the
924 *DepUn* process and one of its requirement is part of the *DepRe* process, the
925 requirement is now unloaded after its dependent.
926
927 Reload of *Dependent Reload* (DepRe) module is now skipped if this module is
928 not found loadable in environment. Either because its requirements are not
929 loaded or a conflict is spotted. Such module is considered *Dependent Unload*
930 rather *Dependent Reload*. If *DepRe* module is sticky, reload is attempted
931 anyway.
932
933 Reload of all *Dependent Reload* (DepRe) modules now occurs after main module
934 evaluation rather after each sub module evaluation that triggered the
935 *Dependent Reload* unload phase.
936
937 When a loading module requires a *Dependent Unload* (DepUn) module, the unload
938 of this module was forced when it was triggered by a Conflict Unload or a
939 switched-off module. The unload of such *DepUn* module is not forced anymore
940 by default and an error is raised if a loading module requiring the *DepUn*
941 module is detected.
942
943 When unloading *Dependent Reload* modules, do not force by default this unload
944 unless asked with :option:`--force` option. As a result an error is obtained
945 when trying to reload Dependent Reload modules that conflict with other loaded
946 modules. This error may be by-passed with :option:`--force` command-line
947 option.
948
949 .. _Specific modulepath for requirements:
950
951 Specific modulepath for requirements
952 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
953
954 A ``--modulepath`` option is introduced on the :mfcmd:`always-load`,
955 :mfcmd:`depends-on`, :mfcmd:`prereq`, :mfcmd:`prereq-all` and
956 :mfcmd:`prereq-any` modulefile commands. This option indicates that
957 requirement should be specifically searched in the provided list of
958 modulepaths. Such mechanism helps to ensure loaded requirements are those
959 expected and they are not coming from an unexpected module tree.
960
961 In the following example, ``foo`` module requires ``bar``. It expects the
962 ``bar`` module from the same modulepath than ``foo``. User's environment has
963 enabled another modulepath that also provides a ``bar`` module. Use of
964 ``--modulepath`` option guides the requirement load mechanism to the expected
965 module.
966
967 .. parsed-literal::
968
969 :ps:`$` module avail
970 --------------------- :sgrdi:`/path/to/othermfiles` ---------------------
971 bar/1
972
973 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
974 bar/1 foo/1 foo/2
975 :ps:`$` module show foo/1
976 -------------------------------------------------------------------
977 :sgrhi:`/path/to/modulefiles/foo/1`:
978
979 :sgrcm:`prereq` --modulepath .. bar/1
980 -------------------------------------------------------------------
981 :ps:`$` module load foo/1
982 Loading :sgrhi:`foo/1`
983 :sgrin:`Loading requirement`: bar/1
984 :ps:`$` echo $_LMFILES_
985 /path/to/modulefiles/bar/1:/path/to/modulefiles/foo/1
986
987 Specific modulepath set may guide to a directory not necessarily enabled
988 (i.e., not defined in :envvar:`MODULEPATH` environment variable):
989
990 .. parsed-literal::
991
992 :ps:`$` module show foo/2
993 -------------------------------------------------------------------
994 :sgrhi:`/path/to/modulefiles/foo/2`:
995
996 :sgrcm:`prereq` --modulepath /path/to/alternatefiles bar/2
997 -------------------------------------------------------------------
998 :ps:`$` module load foo/2
999 Loading :sgrhi:`foo/2`
1000 :sgrin:`Loading requirement`: bar/2
1001 :ps:`$` module list
1002 Currently Loaded Modulefiles:
1003 1) bar/2 2) foo/2
1004 :ps:`$` echo $_LMFILES_
1005 /path/to/alternatefiles/bar/2:/path/to/modulefiles/foo/2
1006
1007 If a required module is already loaded from a modulepath that does not
1008 correspond to the requirement definition, an error is obtained. However if
1009 the *Conflict Unload* mechanism described above is enabled, the conflicting
1010 module will automatically be unloaded.
1011
1012 .. parsed-literal::
1013
1014 :ps:`$` module purge
1015 :ps:`$` module load bar/1
1016 :ps:`$` echo $_LMFILES_
1017 /path/to/othermfiles/bar/1
1018 :ps:`$` module load foo/1
1019 Loading :sgrhi:`bar/1`
1020 :sgrer:`ERROR`: Module already loaded from a different modulepath
1021
1022 Loading :sgrhi:`foo/1`
1023 :sgrer:`ERROR`: Load of requirement bar/1 (specific path) failed
1024 :ps:`$` module config conflict_unload 1
1025 :ps:`$` module load foo/1
1026 Loading :sgrhi:`foo/1`
1027 :sgrin:`Unloading conflict`: bar/1
1028 :sgrin:`Loading requirement`: bar/1
1029 :ps:`$` echo $_LMFILES_
1030 /path/to/modulefiles/bar/1:/path/to/modulefiles/foo/1
1031
1032
1033 v5.4
1034 ----
1035
1036 This new version is backward-compatible with previous version 5 release. It
1037 fixes bugs but also introduces new functionalities that are described in this
1038 section. See the :ref:`5.4 release notes<5.4 release notes>` for a complete
1039 list of the changes between Modules v5.3 and v5.4.
1040
1041 Enhancing extra specifiers
1042 ^^^^^^^^^^^^^^^^^^^^^^^^^^
1043
1044 Extra specifier ``tag`` is introduced to search modules based on tags applied.
1045 It applies to :subcmd:`avail`, :subcmd:`whatis` and :subcmd:`paths`
1046 sub-commands. Tag abbreviation may also be used as extra specifier name.
1047
1048 .. parsed-literal::
1049
1050 :ps:`$` module avail tag:sticky
1051 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1052 :sgrs:`bar/1`
1053
1054 Multiple names can now be specified on one extra specifier criterion to select
1055 modules matching any of these names.
1056
1057 .. parsed-literal::
1058
1059 :ps:`$` module avail tag:S,F
1060 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1061 :sgrs:`bar/1` :sgrf:`qux/1`
1062
1063 Multiple values can now be specified on one variant criterion to select
1064 modules providing any of these variant values.
1065
1066 .. parsed-literal::
1067
1068 :ps:`$` module config variant_shortcut toolchain=%
1069 :ps:`$` module avail %a,c
1070 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1071 baz/1{:sgrvahi:`%c`} foo/1{:sgrvahi:`%a`,\ :sgrva:`b`,\ :sgrvahi:`c`}
1072 :ps:`$` module avail %a,c %b
1073 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1074 foo/1{:sgrvahi:`%a`,\ :sgrvahi:`b`,\ :sgrvahi:`c`}
1075
1076 Purging sticky modules
1077 ^^^^^^^^^^^^^^^^^^^^^^
1078
1079 The :mconfig:`sticky_purge` configuration option is added to define the
1080 behavior of :subcmd:`purge` sub-command when unloading a sticky or
1081 super-sticky module.
1082
1083 By default an ``error`` is raised. :mconfig:`sticky_purge` can be changed to
1084 emit a ``warning`` message instead or to be ``silent``.
1085
1086 .. parsed-literal::
1087
1088 :ps:`$` module purge
1089 Unloading :sgrshi:`foo/1.0`
1090 :sgrer:`ERROR`: Unload of sticky module skipped
1091 :ps:`$` module config sticky_purge warning
1092 :ps:`$` module purge
1093 Unloading :sgrshi:`foo/1.0`
1094 :sgrwa:`WARNING`: Unload of sticky module skipped
1095 :ps:`$` module config sticky_purge silent
1096 :ps:`$` module purge
1097 :ps:`$` module list
1098 Currently Loaded Modulefiles:
1099 1) :sgrs:`foo/1.0`
1100
1101 Key:
1102 :sgrs:`sticky`
1103
1104
1105 Specific modulepath labels
1106 ^^^^^^^^^^^^^^^^^^^^^^^^^^
1107
1108 :mfcmd:`modulepath-label` command is introduced to define a label to use to
1109 designate modulepath in module :subcmd:`avail` output. This new command should
1110 be used in global or modulepath-specific rc files.
1111
1112 .. parsed-literal::
1113
1114 :ps:`$` cat /path/to/modulefiles/.modulerc
1115 #%Module
1116 modulepath-label . Tools
1117 :ps:`$` module avail foo
1118 ---------------------- :sgrdi:`Tools` ----------------------
1119 foo/1.0 foo/2.0
1120
1121 Unique module name loaded
1122 ^^^^^^^^^^^^^^^^^^^^^^^^^
1123
1124 The configuration option :mconfig:`unique_name_loaded` is introduced to allow
1125 only one module loaded per module name. It is similar to the *One name rule*
1126 feature introduced by the `Lmod`_ project.
1127
1128 When enabled, :mconfig:`unique_name_loaded` produces a conflict definition
1129 at the start of modulefile evaluation for each actual and alternative module
1130 root name. For instance when loading a module named ``foo/sub/1.0`` with a
1131 ``qux/1.0`` alias, it produces a conflict against ``foo`` and ``qux``.
1132
1133 When loading a module that shares a name with an already loaded module, an
1134 error is raised due to the conflict definition. This error aborts the module
1135 load evaluation.
1136
1137 .. parsed-literal::
1138
1139 :ps:`$` cat /path/to/modulefiles/bar/1.0
1140 #%Module
1141 :ps:`$` cat /path/to/modulefiles/bar/2.0
1142 #%Module
1143 :ps:`$` module config unique_name_loaded 1
1144 :ps:`$` module load bar/1.0
1145 :ps:`$` module load bar/2.0
1146 Loading :sgrhi:`bar/2.0`
1147 :sgrer:`ERROR`: Module cannot be loaded due to a conflict.
1148 HINT: Might try "module unload bar" first.
1149
1150 :mconfig:`unique_name_loaded` is disabled by default. It can be changed with
1151 module :subcmd:`config` sub-command or at installation time with
1152 :instopt:`--enable-unique-name-loaded` configure script option.
1153
1154 Cache sourced files
1155 ^^^^^^^^^^^^^^^^^^^
1156
1157 You may have gathered common Tcl code files used by all your modulefiles in
1158 Tcl files that are sourced with :manpage:`source(n)` Tcl command. When loading
1159 multiple modulefiles at once, these sourced files may be read for every loaded
1160 modules, which may induce some I/O load.
1161
1162 The configuration option :mconfig:`source_cache` is introduced to cache files
1163 evaluated through ``source`` Tcl command in order to only read them once
1164 during a :file:`modulecmd.tcl` execution. When this option is enabled, sourced
1165 file read also benefits from optimized I/O calls of Modules Tcl extension
1166 library.
1167
1168 In the following example, 20 modules are loaded, each of them sources the same
1169 Tcl file:
1170
1171 .. parsed-literal::
1172
1173 :ps:`$` syscall_list=close,fcntl,ioctl,newfstatat,openat,read,readlink
1174 :ps:`$` strace -f -e $syscall_list -c $MODULES_CMD bash load foo/20
1175 % time seconds usecs/call calls errors syscall
1176 ------ ----------- ----------- --------- --------- ----------------
1177 31.48 0.000781 1 466 read
1178 28.58 0.000709 5 131 16 openat
1179 14.19 0.000352 2 142 3 newfstatat
1180 9.71 0.000241 1 124 close
1181 8.02 0.000199 0 212 211 readlink
1182 4.55 0.000113 1 67 47 ioctl
1183 3.47 0.000086 1 60 fcntl
1184 ------ ----------- ----------- --------- --------- ----------------
1185 100.00 0.002481 2 1202 277 total
1186
1187 When sourced file is cached, it is only read once which saves a substantial
1188 amount of I/O operations:
1189
1190 .. parsed-literal::
1191
1192 :ps:`$` module config source_cache 1
1193 :ps:`$` strace -f -e $syscall_list -c $MODULES_CMD bash load foo/20
1194 % time seconds usecs/call calls errors syscall
1195 ------ ----------- ----------- --------- --------- ----------------
1196 34.41 0.000523 4 112 16 openat
1197 27.37 0.000416 0 428 read
1198 11.84 0.000180 1 122 3 newfstatat
1199 11.51 0.000175 1 105 close
1200 7.11 0.000108 2 47 27 ioctl
1201 5.99 0.000091 0 92 91 readlink
1202 1.78 0.000027 0 40 fcntl
1203 ------ ----------- ----------- --------- --------- ----------------
1204 100.00 0.001520 1 946 137 total
1205
1206 .. _Abort on error:
1207
1208 Abort on error
1209 ^^^^^^^^^^^^^^
1210
1211 When an error occurs during the evaluation of a modulefile, several module
1212 sub-commands, like :subcmd:`load` or :subcmd:`unload`, continue their
1213 processing. It means these sub-commands do not stop if they have multiple
1214 modulefiles to evaluate. This is named the *continue on error* behavior.
1215
1216 The :mconfig:`abort_on_error` configuration option is introduced to abort, for
1217 the sub-commands listed in its value, the evaluation of multiple modulefiles
1218 if one fails to evaluate.
1219
1220 The *abort on error* behavior is applied by default on :command:`ml` command,
1221 :subcmd:`reload` sub-command and the unload phase of :subcmd:`switch`
1222 sub-command. This behavior may also be turned on for :subcmd:`load`,
1223 :subcmd:`mod-to-sh`, :subcmd:`purge`, :subcmd:`switch`, :subcmd:`try-load` and
1224 :subcmd:`unload` sub-commands. If a command is removed from
1225 :mconfig:`abort_on_error`'s value, it applies the *continue on error*
1226 behavior.
1227
1228 In the following example, *abort on error* behavior is applied to
1229 :subcmd:`load` sub-command to stop evaluation instead of loading the remaining
1230 modulefile in the sequence if first modulefile fails to evaluate.
1231
1232 .. parsed-literal::
1233
1234 :ps:`$` module load foo bar
1235 Loading :sgrhi:`foo/1`
1236 :sgrer:`ERROR`: Module evaluation aborted
1237 :ps:`$` module list
1238 Currently Loaded Modulefiles:
1239 1) bar/1
1240 :ps:`$` module purge
1241 :ps:`$` module config abort_on_error +load
1242 :ps:`$` module load foo bar
1243 Loading :sgrhi:`foo/1`
1244 :sgrer:`ERROR`: Module evaluation aborted
1245 :ps:`$` module list
1246 No Modulefiles Currently Loaded.
1247
1248 When enabled for :subcmd:`switch` sub-command, whole action aborts if
1249 switched-on modulefile fails to load. By default, :subcmd:`switch` only aborts
1250 if switched-off modulefile fails to unload.
1251
1252 .. parsed-literal::
1253
1254 :ps:`$` module load bar
1255 :ps:`$` module switch bar foo
1256 Loading :sgrhi:`foo/1`
1257 :sgrer:`ERROR`: Module evaluation aborted
1258
1259 Switching from :sgrhi:`bar/1` to :sgrhi:`foo/1`
1260 :sgrwa:`WARNING`: Load of switched-on foo/1 failed
1261 :ps:`$` module list
1262 No Modulefiles Currently Loaded.
1263 :ps:`$` module load bar
1264 :ps:`$` module config abort_on_error +switch
1265 :ps:`$` module switch bar foo
1266 Loading :sgrhi:`foo/1`
1267 :sgrer:`ERROR`: Module evaluation aborted
1268
1269 Switching from :sgrhi:`bar/1` to :sgrhi:`foo/1`
1270 :sgrwa:`WARNING`: Load of switched-on foo/1 failed
1271 :ps:`$` module list
1272 Currently Loaded Modulefiles:
1273 1) bar/1
1274
1275 :mconfig:`abort_on_error` configure option has no impact on :mfcmd:`module`
1276 commands defined in modulefiles. When :option:`--force` option is set,
1277 sub-commands apply the *continue on error* behavior.
1278
1279 When :instopt:`--enable-new-features` installation option is set, the
1280 *abort on error* behavior is also enabled on :subcmd:`load` and
1281 :subcmd:`switch` sub-commands.
1282
1283 Improve error reporting
1284 ^^^^^^^^^^^^^^^^^^^^^^^
1285
1286 Module evaluation error reports have been tweaked to appear where these errors
1287 occur rather reporting them on the message block of the main action. Such
1288 change helps to better understand the module evaluation flow.
1289
1290 .. parsed-literal::
1291
1292 :ps:`$` module load baz
1293 Loading :sgrhi:`baz/1`
1294 :sgrin:`Loading requirement`: foo/1
1295 :ps:`$` module switch foo bar
1296 Unloading :sgrhi:`baz/1`
1297 :sgrme:`Module ERROR`: invalid command name "bad_command"
1298 while executing
1299 "bad_command"
1300 (file "/path/to/modulefiles/baz/1" line 4)
1301 Please contact <root@localhost>
1302
1303 Unloading :sgrhi:`foo/1`
1304 :sgrer:`ERROR`: Unload of dependent baz/1 failed
1305
1306 Switching from :sgrhi:`foo/1` to :sgrhi:`bar`
1307 :sgrer:`ERROR`: Unload of switched-off foo/1 failed
1308
1309 :subcmd:`switch` sub-command specific error messages are not reported when
1310 :mfcmd:`module switch<module>` command is run from a modulefile (like *unload
1311 of switched-off module failed*, *load of switched-on module failed* messages
1312 or *Switching* block message). On :subcmd:`switch` sub-command, load failure
1313 of switched-on module is now reported as an error.
1314
1315 .. parsed-literal::
1316
1317 :ps:`$` module load foo
1318 :ps:`$` module switch foo qux
1319 Loading :sgrhi:`qux/1`
1320 :sgrme:`Module ERROR`: invalid command name "bad_command"
1321 while executing
1322 "bad_command"
1323 (file "/path/to/modulefiles/qux/1" line 2)
1324 Please contact <root@localhost>
1325
1326 Switching from :sgrhi:`foo/1` to :sgrhi:`qux/1`
1327 :sgrer:`ERROR`: Load of switched-on qux/1 failed
1328
1329 Conflict error messages now describe the loaded module name and version that
1330 conflicts rather the generic conflict module specification.
1331
1332 .. parsed-literal::
1333
1334 :ps:`$` cat /path/to/modulefiles/bar/1
1335 #%Module
1336 conflict foo
1337 :ps:`$` module load foo
1338 :ps:`$` module load bar
1339 Loading :sgrhi:`bar/1`
1340 :sgrer:`ERROR`: Module cannot be loaded due to a conflict.
1341 HINT: Might try "module unload foo/1" first.
1342
1343 New options for source-sh modulefile command
1344 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1345
1346 Support for new *shell* mode named ``bash-eval`` is added to
1347 :mfcmd:`source-sh` modulefile command. With this new mode, the generated
1348 output of the bash shell script is evaluated to get the environment changes
1349 instead of sourcing this script.
1350
1351 .. parsed-literal::
1352
1353 :ps:`$` cat /path/to/modulefiles/foo/foo-output.sh
1354 #!/bin/bash
1355 cat <<EOF
1356 export FOO=value;
1357 alias foo_alias='echo foo_alias;'
1358 foo_function() {
1359 echo foo_function;
1360 }
1361 EOF
1362 :ps:`$` cat /path/to/modulefiles/foo/1
1363 #%Module
1364 set foo_script [file dirname $ModulesCurrentModulefile]/foo-output.sh
1365 :sgrhi:`source-sh bash-eval` $foo_script
1366
1367 :ps:`$` module show foo/1
1368 -------------------------------------------------------------------
1369 :sgrhi:`/path/to/modulefiles/foo/1`:
1370
1371 :sgrcm:`set-alias` foo_alias {echo foo_alias;}
1372 :sgrcm:`set-function` foo_function {
1373 echo foo_function}
1374 :sgrcm:`setenv` FOO value
1375 -------------------------------------------------------------------
1376 :ps:`$` module load foo/1
1377 :ps:`$` foo_alias
1378 foo_alias
1379
1380 ``bash-eval`` *shell* support is also available on :subcmd:`sh-to-mod`
1381 sub-command.
1382
1383 A new option is added to :mfcmd:`source-sh` modulefile command: ``--ignore``.
1384 It filters shell elements changed by script, not to get these changes applied
1385 when loading modulefile. This option accepts a list, separated by colon
1386 character, of shell elements: ``envvar``, ``alias``, ``function``,
1387 ``complete`` or ``chdir``.
1388
1389 .. parsed-literal::
1390
1391 :ps:`$` module purge
1392 :ps:`$` cat /path/to/modulefiles/foo/2
1393 #%Module
1394 set foo_script [file dirname $ModulesCurrentModulefile]/foo-output.sh
1395 :sgrhi:`source-sh --ignore alias:function bash-eval` $foo_script
1396
1397 :ps:`$` module show foo/2
1398 -------------------------------------------------------------------
1399 :sgrhi:`/path/to/modulefiles/foo/2`:
1400
1401 :sgrcm:`setenv` FOO value
1402 -------------------------------------------------------------------
1403 :ps:`$` module load foo/2
1404 :ps:`$` foo_alias
1405 bash: foo_alias: command not found
1406
1407 This ``--ignore`` option can be used with any *shell* supported by
1408 :mfcmd:`source-sh`.
1409
1410
1411 v5.3
1412 ----
1413
1414 This new version is backward-compatible with previous version 5 release. It
1415 fixes bugs but also introduces new functionalities that are described in this
1416 section. See the :ref:`5.3 release notes<5.3 release notes>` for a complete
1417 list of the changes between Modules v5.2 and v5.3.
1418
1419 Module cache
1420 ^^^^^^^^^^^^
1421
1422 A module cache file can be created under each modulepath directory with new
1423 :subcmd:`cachebuild` sub-command. Cache file is named :file:`.modulecache` and
1424 contains in one file all modulefiles and modulercs found in modulepath
1425 directory.
1426
1427 When cache file is available, module search mechanism uses this file rather
1428 walking through the content of modulepath directory. I/O operations are saved
1429 this way which reduces search processing time.
1430
1431 When searching for available modules without cache, each file contained
1432 in enabled modulepaths is opened to check if it is a modulefile or not. Such
1433 checks lead to a large number of I/O operations on large module setup like in
1434 the below example where a total of 1051 modulefiles are available:
1435
1436 .. parsed-literal::
1437
1438 :ps:`$` module -o "" avail -t | wc -l
1439 1051
1440 :ps:`$` syscall_list=access,close,getdents64,newfstatat,openat,read
1441 :ps:`$` strace -f -e $syscall_list -c $MODULES_CMD bash avail
1442 % time seconds usecs/call calls errors syscall
1443 ------ ----------- ----------- --------- --------- ----------------
1444 31.09 0.003776 2 1424 9 openat
1445 28.56 0.003469 2 1649 3 newfstatat
1446 14.08 0.001710 1 1421 close
1447 11.85 0.001439 3 460 getdents64
1448 10.88 0.001321 0 1505 read
1449 3.54 0.000430 4 107 5 access
1450 ------ ----------- ----------- --------- --------- ----------------
1451 100.00 0.012145 1 6566 17 total
1452
1453 After building cache file for every enabled modulepaths in this example setup,
1454 a lot of I/O operations are saved when searching for available modules:
1455
1456 .. parsed-literal::
1457
1458 :ps:`$` module cachebuild
1459 Creating :sgrhi:`/path/to/modulefiles/.modulecache`
1460 Creating :sgrhi:`/path/to/modulefiles.2/.modulecache`
1461 Creating :sgrhi:`/path/to/modulefiles.3/.modulecache`
1462 :ps:`$` module config cache_buffer_bytes 1000000
1463 :ps:`$` strace -f -e $syscall_list -c $MODULES_CMD bash avail
1464 % time seconds usecs/call calls errors syscall
1465 ------ ----------- ----------- --------- --------- ----------------
1466 70.19 0.000544 2 255 read
1467 13.16 0.000102 2 38 9 openat
1468 8.90 0.000069 1 35 close
1469 6.06 0.000047 1 31 2 newfstatat
1470 1.68 0.000013 1 11 2 access
1471 ------ ----------- ----------- --------- --------- ----------------
1472 100.00 0.000775 2 370 13 total
1473
1474 A significant execution time drop may be noticed, especially if modulepath
1475 directories are stored on heavily loaded network filesystem.
1476
1477 To further optimize I/O operation count, the :mconfig:`cache_buffer_bytes`
1478 configuration option can be set like in the above example to use fewer number
1479 of ``read`` operation to load cache file content.
1480
1481 To build cache file, user should be granted write access on modulepath
1482 directory. Modulefiles or directories that are not accessible for everyone are
1483 not recorded in cache. An indication is saved instead to test these limited
1484 access elements when cache is loaded to determine if they are available to
1485 currently running user.
1486
1487 Cache file can be ignored with :option:`--ignore-cache` command line switch or
1488 more permanently with :mconfig:`ignore_cache` configuration option.
1489
1490 Cache file is valid indefinitely by default but :mconfig:`cache_expiry_secs`
1491 configuration option can be used to define the number of seconds a cache file
1492 is considered valid after being generated. Expired cache file is ignored.
1493
1494 Cache file of enabled modulepaths can be deleted all at once with
1495 :subcmd:`cacheclear` sub-command.
1496
1497 Querying available module variants
1498 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1499
1500 A new mechanism named :ref:`Extra match search` is introduced to evaluate
1501 modulefiles during a module search to find those matching an extra query on a
1502 variant value, a dependency or an environment variable definition.
1503
1504 During this specific evaluation, modulefiles are interpreted in *scan* mode to
1505 collect the different Tcl modulefile commands they use. Special care should be
1506 given when writing modulefiles to ensure they cope with such evaluation mode.
1507
1508 :ref:`Extra match search` mechanism is available on :subcmd:`avail`,
1509 :subcmd:`whatis` and :subcmd:`paths` sub-commands.
1510
1511 With this new mechanism, it is possible to list all available variant defined
1512 in modulefiles with their associated values:
1513
1514 .. parsed-literal::
1515
1516 :ps:`$` module config avail_output modulepath:alias:dirwsym:sym:tag:key:variant
1517 :ps:`$` module config variant_shortcut toolchain=%
1518 :ps:`$` module avail
1519 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1520 bar/1.0{:sgrva:`%a`,\ :sgrva:`b`} foo/1.0{:sgrvade:`%a`} qux/1.0{:sgrva:`%a`,\ :sgrva:`b`}
1521 bar/2.0{:sgrvade:`%b`} foo/2.0{:sgrva:`%a`,\ :sgrva:`b`,\ :sgrva:`c`} qux/2.0{:sgrva:`%b`,\ :sgrva:`c`}
1522
1523 Key:
1524 :sgrdi:`modulepath` {:sgrva:`%value`}={:sgrva:`toolchain=value`}
1525 :sgrde:`default-version` {:sgrva:`variant=value`}
1526
1527 You can also search for modules defining a specific variant value:
1528
1529 .. parsed-literal::
1530
1531 :ps:`$` module avail %a
1532 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1533 bar/1.0{:sgrvahi:`%a`,\ :sgrva:`b`} foo/1.0{:sgrvadehi:`%a`} foo/2.0{:sgrvahi:`%a`,\ :sgrva:`b`,\ :sgrva:`c`} qux/1.0{:sgrvahi:`%a`,\ :sgrva:`b`}
1534
1535 Key:
1536 :sgrdi:`modulepath` {:sgrva:`%value`}={:sgrva:`toolchain=value`}
1537 :sgrde:`default-version` {:sgrva:`variant=value`}
1538
1539 .. note:: As extra match search implies additional modulefile evaluations, it
1540 is advised to build and use `Module cache`_ to improve search speed.
1541
1542 Two new elements, ``variant`` and ``variantifspec``, are added to the allowed
1543 value list of :mconfig:`avail_output` and :mconfig:`avail_terse_output`
1544 configuration options. The latter is set in the default value list of both
1545 options. When ``variant`` is set, variants and their possible values are
1546 reported along module they are associated to. When ``variantifspec`` is set,
1547 available variants are reported only if a variant is specified in search
1548 query.
1549
1550 .. parsed-literal::
1551
1552 :ps:`$` module config --reset avail_output
1553 :ps:`$` module config avail_output
1554 Modules Release 5.3.0 (2023-05-14)
1555
1556 - :sgrhi:`Config. name` ---------.- :sgrhi:`Value (set by if default overridden)` ---------------
1557 avail_output modulepath:alias:dirwsym:sym:tag:variantifspec:key
1558 :ps:`$` module avail foo
1559 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1560 :sgrhi:`foo`/1.0 :sgrhi:`foo`/2.0
1561
1562 Key:
1563 :sgrdi:`modulepath`
1564 :ps:`$` module avail foo %a
1565 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1566 :sgrhi:`foo`/1.0{:sgrvadehi:`%a`} :sgrhi:`foo`/2.0{:sgrvahi:`%a`,\ :sgrva:`b`,\ :sgrva:`c`}
1567
1568 Key:
1569 :sgrdi:`modulepath` {:sgrva:`%value`}={:sgrva:`toolchain=value`}
1570 :sgrde:`default-version` {:sgrva:`variant=value`}
1571
1572 Extra specifiers
1573 ^^^^^^^^^^^^^^^^
1574
1575 Extra specifiers are introduced to query content of modulefiles. They can be
1576 specified with *element:name* syntax as part of module specification on module
1577 search commands (:subcmd:`avail`, :subcmd:`paths` and and :subcmd:`whatis`).
1578
1579 Extra specifiers trigger :ref:`Extra match search` mechanism when found in
1580 module specification. Available modulefiles are evaluated in *scan* mode to
1581 collect the different Tcl modulefile commands they use.
1582
1583 .. parsed-literal::
1584
1585 :ps:`$` module avail variant:toolchain
1586 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1587 bar/1.0 bar/2.0 foo/1.0 foo/2.0 qux/1.0 qux/2.0
1588
1589 In the above example, all modulefiles defining a ``toolchain`` variant are
1590 returned.
1591
1592 Most Tcl modulefile commands can be queried with extra specifiers:
1593 ``variant``, ``setenv``, ``unsetenv``, ``append-path``, ``prepend-path``,
1594 ``remove-path``, ``pushenv``, ``complete``, ``uncomplete``, ``set-alias``,
1595 ``unset-alias``, ``set-function``, ``unset-function``, ``chdir``, ``family``,
1596 ``prereq``, ``prereq-any``, ``prereq-all``, ``depends-on``, ``always-load``,
1597 ``load``, ``load-any``, ``try-load``, ``switch``, ``switch-on``,
1598 ``switch-off``, ``conflict`` and ``unload``. Commands that handle environment
1599 variables may be aliased ``envvar``. Commands that define a module requirement
1600 may be aliased ``require`` and those that define a module incompatibility may
1601 be aliased ``incompat``.
1602
1603 When several extra specifiers are set in query, modules returned are those
1604 matching both conditions. In the following example, all modulefiles
1605 interacting with :envvar:`PATH` environment variable and requiring ``foo``
1606 module are returned.
1607
1608 .. parsed-literal::
1609
1610 :ps:`$` module avail envvar:PATH require:foo
1611 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1612 bar/1.0 bar/2.0
1613
1614 Extra specifiers related to module requirement or incompatibility may leverage
1615 the :ref:`Advanced module version specifiers` syntax. On following example,
1616 modulefiles returned are those defining a requirement on ``foo`` module with
1617 version higher or equal to ``1.2`` and variant ``toolchain=a`` selected.
1618
1619 .. parsed-literal::
1620
1621 :ps:`$` module avail "require:[email protected]: toolchain=a"
1622 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1623 bar/1.0
1624
1625 .. note:: Module aliases or symbolic versions used either in modulefile
1626 definitions or as extra specifier values are not resolved.
1627
1628 Append or subtract elements to current option value
1629 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1630
1631 For command-line switches and configuration options whose value is a
1632 colon-separated list, it is now possible to append or subtract elements to the
1633 current value. With a ``+`` prefix elements are appended and with a ``-``
1634 prefix elements are subtracted.
1635
1636 This new feature applies to :option:`--output`/:option:`-o` command-line
1637 switches and :mconfig:`avail_output`, :mconfig:`avail_terse_output`,
1638 :mconfig:`list_output`, :mconfig:`list_terse_output`, :mconfig:`colors`,
1639 :mconfig:`protected_envvars`, :mconfig:`shells_with_ksh_fpath`,
1640 :mconfig:`tag_abbrev`, :mconfig:`tag_color_name`, :mconfig:`variant_shortcut`
1641 configuration options.
1642
1643 For instance, to output available *foo* modules without modulepath and tag
1644 information:
1645
1646 .. parsed-literal::
1647
1648 :ps:`$` module -o -modulepath:tag avail foo
1649 :sgrhi:`foo`/1.0 :sgrhi:`foo`/2.0
1650
1651 On following example, terse output mode of *list* sub-command is permanently
1652 updated to report variant information:
1653
1654 .. parsed-literal::
1655
1656 :ps:`$` module config list_terse_output
1657 Modules Release 5.3.0 (2023-05-14)
1658
1659 - :sgrhi:`Config. name` ---------.- :sgrhi:`Value (set by if default overridden)` ---------------
1660 list_terse_output header
1661 :ps:`$` module config list_terse_output +variant
1662 :ps:`$` module config list_terse_output
1663 Modules Release 5.3.0 (2023-05-14)
1664
1665 - :sgrhi:`Config. name` ---------.- :sgrhi:`Value (set by if default overridden)` ---------------
1666 list_terse_output header:variant (env-var)
1667
1668
1669 v5.2
1670 ----
1671
1672 This new version is backward-compatible with previous version 5 release. It
1673 fixes bugs but also introduces new functionalities that are described in this
1674 section. See the :ref:`5.2 release notes<5.2 release notes>` for a complete
1675 list of the changes between Modules v5.1 and v5.2.
1676
1677 Optional requirements
1678 ^^^^^^^^^^^^^^^^^^^^^
1679
1680 The ``--optional`` option has been added to the :mfcmd:`prereq`,
1681 :mfcmd:`prereq-all`, :mfcmd:`depends-on` and :mfcmd:`always-load` modulefile
1682 commands to indicate that specified requirement is optional. An automatic
1683 load attempt is also performed for optional requirements. If requirement is
1684 not found or cannot be loaded, the dependency is yet considered satisfied as
1685 it is optional.
1686
1687 .. parsed-literal::
1688
1689 :ps:`$` module show foo
1690 -------------------------------------------------------------------
1691 :sgrhi:`/path/to/modulefiles/foo`:
1692
1693 :sgrcm:`prereq` --optional bar
1694 -------------------------------------------------------------------
1695 :ps:`$` module load foo
1696 Loading :sgrhi:`foo`
1697 :sgrin:`Loading requirement`: bar
1698
1699
1700 If the optional requirement is unloaded or loaded afterward, the dependent
1701 module is automatically reloaded thanks to the :mconfig:`auto_handling`
1702 mechanism.
1703
1704 .. parsed-literal::
1705
1706 :ps:`$` module unload bar
1707 Unloading :sgrhi:`bar`
1708 :sgrin:`Unloading dependent`: foo
1709 :sgrin:`Reloading dependent`: foo
1710 :ps:`$` module list
1711 Currently Loaded Modulefiles:
1712 1) foo
1713 :ps:`$` module load bar
1714 Loading :sgrhi:`bar`
1715 :sgrin:`Unloading dependent`: foo
1716 :sgrin:`Reloading dependent`: foo
1717
1718 Modules loaded by other modules with the :mfcmd:`module try-load<module>`
1719 command are now considered optional requirements. Dependent module can be
1720 loaded without the *try-load* modules, but now it gets automatically reloaded
1721 if *try-load* module is loaded afterward, to take it into account.
1722
1723 Linting modulefiles
1724 ^^^^^^^^^^^^^^^^^^^
1725
1726 Static analysis of modulefile, modulerc and global/user rc is now possible
1727 with :subcmd:`lint` sub-command. It relies on an external program defined with
1728 :mconfig:`tcl_linter` configuration option. Modules or files specified on the
1729 command-line are resolved then passed to the Tcl linter program.
1730
1731 .. parsed-literal::
1732
1733 :ps:`$` cat /path/to/modulefiles/foo/1.0
1734 #%Module
1735 if {"str" eq} {
1736 else {
1737 }
1738 :ps:`$` module lint foo/1.0
1739 Linting :sgrhi:`/path/to/modulefiles/foo/1.0`
1740 :sgrer:`ERROR line 2`: Could not complete statement.
1741 One close brace would complete the first line
1742 One close brace would complete at end of line 4.
1743 One close brace would complete the script body at line 5.
1744 Assuming completeness for further processing.
1745 :sgrer:`ERROR line 2`: Bad expression: missing operand at _@_
1746 in expression ""str" eq_@_"
1747 :sgrwa:`WARNING line 3`: Unknown command "else"
1748 :sgrin:`NOTICE line 4`: Close brace not aligned with line 3 (1 0)
1749
1750 `Nagelfar`_ is the Tcl linter recommended for Modules and set by default. This
1751 default can be changed at installation time with :instopt:`--with-tcl-linter`
1752 and :instopt:`--with-tcl-linter-opts` options. It can also be configured later
1753 on through :mconfig:`tcl_linter` config option.
1754
1755 Specific syntax databases and plugins for `Nagelfar`_ are added by Modules to
1756 precisely lint modulefile commands syntax in addition to regular Tcl syntax.
1757 The installation of these specific files is controlled with
1758 :instopt:`--enable-nagelfar-addons` option (enabled by default). Their
1759 location is controlled by the :instopt:`--nagelfardatadir` option.
1760
1761 .. parsed-literal::
1762
1763 :ps:`$` module lint bar@:1 /path/to/modulefiles/.modulerc
1764 Linting :sgrhi:`/path/to/modulefiles/.modulerc`
1765 :sgrer:`ERROR line 35`: Wrong number of arguments (3) to "module-alias"
1766 :sgrer:`ERROR line 41`: Wrong number of arguments (3) to "module-virtual"
1767
1768 Linting :sgrhi:`/path/to/modulefiles/bar/1.2`
1769 :sgrwa:`WARNING line 19`: Unknown command "unk"
1770 :ps:`$` module lint ~/.modulerc
1771 Linting :sgrhi:`/home/user/.modulerc`
1772 :sgrwa:`WARNING line 2`: Command "setenv" should not be be used in global rc file
1773
1774 :subcmd:`lint` sub-command outputs messages returned by the Tcl linter
1775 program. Nagelfar produces NOTICE, WARNING and ERROR messages. If linter does
1776 not report a thing, :subcmd:`lint` sub-command will be silent, unless if the
1777 :option:`--verbose`/:option:`-v` is set.
1778
1779 .. parsed-literal::
1780
1781 :ps:`$` module lint /path/to/modulefiles/bar/.version bar/1.4
1782 :ps:`$` module lint -v /path/to/modulefiles/bar/.version bar/1.4
1783 Linting :sgrhi:`/path/to/modulefiles/bar/.version`
1784 Linting :sgrhi:`/path/to/modulefiles/bar/1.4`
1785
1786 When no file is specified to :subcmd:`lint` sub-command, all the global/user
1787 rc files and all the modulefiles and modulercs from enabled modulepaths are
1788 analyzed. If the :option:`--all`/:option:`-a` option is set, all hidden
1789 modulefiles are also linted.
1790
1791 .. parsed-literal::
1792
1793 :ps:`$` module lint
1794 Linting :sgrhi:`/home/user/.modulerc`
1795 :sgrwa:`WARNING line 2`: Command "setenv" should not be be used in global rc file
1796
1797 Linting :sgrhi:`/path/to/modulefiles/.modulerc`
1798 :sgrer:`ERROR line 35`: Wrong number of arguments (3) to "module-alias"
1799 :sgrer:`ERROR line 41`: Wrong number of arguments (3) to "module-virtual"
1800
1801 Linting :sgrhi:`/path/to/modulefiles/bar/1.2`
1802 :sgrwa:`WARNING line 19`: Unknown command "unk"
1803 ...
1804
1805 To use `Nagelfar`_ as Tcl linter for Modules, this open source tool has to be
1806 installed on your system. When installing from tarball distribution, make sure
1807 :command:`nagelfar.tcl` command is found through a :envvar:`PATH` lookup or
1808 that :mconfig:`tcl_linter` option is set to its full path location.
1809 `Nagelfar`_ is also made available as a RPM package in `EPEL and Fedora
1810 repositories`_.
1811
1812 .. _Nagelfar: http://nagelfar.sourceforge.net/
1813 .. _EPEL and Fedora repositories: https://src.fedoraproject.org/rpms/nagelfar
1814
1815 mod-to-sh sub-command
1816 ^^^^^^^^^^^^^^^^^^^^^
1817
1818 New sub-command is added to translate modulefile into shell code:
1819 :subcmd:`mod-to-sh`. It evaluates modulefiles passed as argument and produces
1820 code for specified shell.
1821
1822 .. parsed-literal::
1823
1824 :ps:`$` cat /path/to/modulefiles/foo
1825 #%Module
1826 setenv FOO value
1827 set-function foo {echo foo}
1828 :ps:`$` module mod-to-sh bash foo
1829 FOO=value; export FOO;
1830 foo () { echo foo; }; export -f foo;
1831
1832 Designated modulefiles are evaluated as if they were loading. But instead of
1833 producing shell code that is evaluated in current shell session,
1834 :command:`module` command outputs this shell code.
1835
1836 .. parsed-literal::
1837
1838 :ps:`$` cat /path/to/modulefiles/bar
1839 #%Module
1840 setenv BAR othervalue
1841 set-alias bar {echo bar}
1842 :ps:`$` module mod-to-sh fish foo bar
1843 set -xg FOO value;
1844 set -xg BAR othervalue;
1845 alias bar echo\ bar;
1846 function foo; echo foo; end;
1847 :ps:`$` module list
1848 No Modulefiles Currently Loaded.
1849
1850 All shells supported by :file:`modulecmd.tcl` script are supported by
1851 :subcmd:`mod-to-sh`.
1852
1853 .. parsed-literal::
1854
1855 :ps:`$` module mod-to-sh python foo bar
1856 import os
1857 os.environ['FOO'] = 'value'
1858 os.environ['BAR'] = 'othervalue'
1859
1860 .. _Initial environment:
1861
1862 Initial environment
1863 ^^^^^^^^^^^^^^^^^^^
1864
1865 When Modules initializes, it evaluates the :file:`initrc` and
1866 :file:`modulespath` configuration files to enable default modulepaths and load
1867 default modules. Initial environment corresponds to the environment state
1868 after this initialization.
1869
1870 Initial environment is now saved in an environment variable
1871 (:envvar:`__MODULES_LMINIT`) in current shell session to remember what are the
1872 initial modulepaths and initial modules with their tags and variants if any.
1873
1874 :subcmd:`reset` sub-command is introduced, in a similar fashion than on
1875 `Lmod`_, to restore the initial environment. Here, :subcmd:`reset` relies on
1876 the :ref:`collection<collections>` mechanism based and the content of
1877 :envvar:`__MODULES_LMINIT`. Currently enabled modulepaths and loaded modules
1878 are respectively unused and unloaded to use the modulepaths and load the
1879 modules with tags and variants as described by initial environment.
1880
1881 .. parsed-literal::
1882
1883 :ps:`$` module list
1884 Currently Loaded Modulefiles:
1885 1) foo/1.0 2) bar/1.0
1886 :ps:`$` module switch bar/1.0 qux/1.0
1887 :ps:`$` module reset
1888 Unloading :sgrhi:`qux/1.0`
1889 Loading :sgrhi:`bar/1.0`
1890
1891 :subcmd:`restore` sub-command has been adapted to reinitialize the environment
1892 to its initial state when no collection name is provided and no *default*
1893 collection exists or if ``__init__`` virtual collection name is provided.
1894
1895 It is possible to view the content of the initial environment with
1896 :subcmd:`saveshow` sub-command. It is displayed when no argument is provided
1897 and no collection exists or if ``__init__`` name is provided.
1898
1899 .. parsed-literal::
1900
1901 :ps:`$` module saveshow __init__
1902 -------------------------------------------------------------------
1903 :sgrhi:`initial environment`:
1904
1905 :sgrcm:`module` use --append /path/to/modulefiles
1906 :sgrcm:`module` load foo/1.0
1907 :sgrcm:`module` load bar/1.0
1908
1909 -------------------------------------------------------------------
1910
1911 Users have the ability to define what is their initial environment state and
1912 thus adapt the behavior of :subcmd:`reset` sub-command with
1913 :mconfig:`reset_target_state` configuration option. Default value is
1914 ``__init__`` and it corresponds to the behavior described above. When set to
1915 ``__purge__``, a :subcmd:`purge` command is performed when resetting. Any
1916 other value corresponds to the name of a collection to restore.
1917
1918 .. parsed-literal::
1919
1920 :ps:`$` module config reset_target_state __purge__
1921 :ps:`$` module reset
1922 Unloading bar/1.0
1923 Unloading foo/1.0
1924
1925 .. _Stashing environment:
1926
1927 Stashing environment
1928 ^^^^^^^^^^^^^^^^^^^^
1929
1930 The ability to stash current environment is added with the introduction of the
1931 :subcmd:`stash` sub-command. When called current environment is saved in a
1932 *stash* collection then initial environment is restored.
1933
1934 .. parsed-literal::
1935
1936 :ps:`$` module list
1937 Currently Loaded Modulefiles:
1938 1) bar/2.0 2) foo/2.0
1939 :ps:`$` module stash
1940 Unloading :sgrhi:`foo/2.0`
1941 Unloading :sgrhi:`bar/2.0`
1942 :ps:`$` module list
1943 No Modulefiles Currently Loaded.
1944
1945 Sub-commands are added to specifically handle stash collections. Their names
1946 are prefixed with *stash*, like :subcmd:`stashlist` to list existing stash
1947 collections or :subcmd:`stashshow` to display their content.
1948
1949 .. parsed-literal::
1950
1951 :ps:`$` module stashlist
1952 Stash collection list:
1953 0) stash-1665377597432 1) stash-1664946764252
1954 :ps:`$` module stashshow
1955 -------------------------------------------------------------------
1956 :sgrhi:`/home/user/.module/stash-1665377597432:`
1957
1958 :sgrcm:`module` use --append /path/to/modulefiles
1959 :sgrcm:`module` load bar
1960 :sgrcm:`module` load foo
1961
1962 -------------------------------------------------------------------
1963
1964 Stash collections can be designated on sub-commands by their collection name
1965 or stash index. Most recent stash collection has index ``0``, the one after is
1966 designated with index ``1``, and so on. When no stash collection is specified,
1967 most recent one is assumed.
1968
1969 .. parsed-literal::
1970
1971 :ps:`$` module stashshow 1
1972 -------------------------------------------------------------------
1973 :sgrhi:`/home/user/.module/stash-1664946764252:`
1974
1975 :sgrcm:`module` use --append /path/to/modulefiles
1976 :sgrcm:`module` load foobar
1977
1978 -------------------------------------------------------------------
1979
1980 Stash collections are restored with the :subcmd:`stashpop` sub-command. It
1981 changes the user environment to match the stash definition, then it deletes
1982 the stash collection file.
1983
1984 .. parsed-literal::
1985
1986 :ps:`$` module stashpop
1987 Loading :sgrhi:`bar/2.0`
1988 Loading :sgrhi:`foo/2.0`
1989 :ps:`$` module stashlist
1990 Stash collection list:
1991 0) stash-1664946764252
1992
1993 Stash collections can be deleted one by one with :subcmd:`stashrm` sub-command
1994 or all together with :subcmd:`stashclear`.
1995
1996 .. parsed-literal::
1997
1998 :ps:`$` module stashrm
1999 :ps:`$` module stashlist
2000 No stash collection.
2001
2002 Siteconfig hook variables
2003 ^^^^^^^^^^^^^^^^^^^^^^^^^
2004
2005 Several Tcl variables are introduced for :ref:`Site-specific configuration`
2006 script to define specific commands and variables in the evaluation context of
2007 modulefiles and modulercs. These commands and variables setup in
2008 :file:`siteconfig.tcl` can be used in modulefile or modulerc. Sites can easily
2009 extend modulefile and modulerc syntax with specific elements.
2010
2011 :sitevar:`modulefile_extra_cmds` variable defines a list of commands to expose
2012 in the modulefile evaluation context and the associated procedure to run when
2013 this command is called. This variable has to be defined in
2014 :file:`siteconfig.tcl` located for instance at |file etcdir_siteconfig|.
2015
2016 In the following example :sitevar:`modulefile_extra_cmds` is used to define
2017 the ``sys`` command and bound it to the ``sys`` procedure that is also defined
2018 in :file:`siteconfig.tcl`.
2019
2020 .. code-block:: tcl
2021
2022 proc sys {mode} {
2023 switch -- $mode {
2024 name { return myhost-$::tcl_platform(machine) }
2025 default { error "Unknown mode '$mode'" }
2026 }
2027 }
2028 set modulefile_extra_cmds {sys sys}
2029
2030 Once :file:`siteconfig.tcl` is setup, the ``sys`` command can be called by
2031 modulefiles. In the following example it is used to determine the application
2032 path.
2033
2034 .. parsed-literal::
2035
2036 :ps:`$` cat /path/to/modulefiles/foo/1.2
2037 #%Module
2038 append-path PATH /path/to/apps/foo-1.2/[sys name]/bin
2039
2040 :ps:`$` module show foo/1.2
2041 -------------------------------------------------------------------
2042 :sgrhi:`/path/to/modulefiles/foo/1.2`:
2043
2044 :sgrcm:`append-path` PATH /path/to/apps/foo-1.2/myhost-x86_64/bin
2045 -------------------------------------------------------------------
2046
2047 :sitevar:`modulerc_extra_cmds` follows the same approach than
2048 :sitevar:`modulefile_extra_cmds` and makes specific commands available during
2049 modulerc evaluation.
2050
2051 :sitevar:`modulefile_extra_vars` variable defines a list of variables to
2052 expose in the modulefile evaluation context and their associated value. This
2053 variable has to be defined in :file:`siteconfig.tcl`.
2054
2055 In the following example :sitevar:`modulefile_extra_vars` is used to define
2056 the ``APP_ROOT`` variable with ``/path/to/apps`` as value.
2057
2058 .. code-block:: tcl
2059
2060 set modulefile_extra_vars {APP_ROOT /path/to/apps}
2061
2062 Once :file:`siteconfig.tcl` is setup, the ``APP_ROOT`` variable can be used in
2063 modulefiles.
2064
2065 .. parsed-literal::
2066
2067 :ps:`$` cat /path/to/modulefiles/bar/2.1
2068 #%Module
2069 append-path PATH $APP_ROOT/bar-2.1/[sys name]/bin
2070
2071 :ps:`$` module show bar/2.1
2072 -------------------------------------------------------------------
2073 :sgrhi:`/path/to/modulefiles/bar/2.1`:
2074
2075 :sgrcm:`append-path` PATH /path/to/apps/bar-2.1/myhost-x86_64/bin
2076 -------------------------------------------------------------------
2077
2078 :sitevar:`modulerc_extra_vars` follows the same approach than
2079 :sitevar:`modulefile_extra_vars` and makes specific variables available during
2080 modulerc evaluation.
2081
2082
2083 v5.1
2084 ----
2085
2086 This new version is backward-compatible with previous version 5 release. It
2087 fixes bugs but also introduces new functionalities that are described in this
2088 section. See the :ref:`5.1 release notes<5.1 release notes>` for a complete
2089 list of the changes between Modules v5.0 and v5.1.
2090
2091 Control output redirection
2092 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2093
2094 Since version 4.0, the :command:`module` function is initialized differently
2095 on *sh*, *bash*, *ksh*, *zsh* and *fish* shells when their session is found
2096 interactive. In such situation :command:`module` redirects its output from
2097 *stderr* to *stdout*. Once initialized the redirection behavior is inherited
2098 in sub-sessions.
2099
2100 The :mconfig:`redirect_output` configuration option is introduced in version
2101 5.1, to supersede the default behavior set at initialization time.
2102
2103 .. parsed-literal::
2104
2105 :ps:`$` module load unknown >/dev/null
2106 :ps:`$` module config redirect_output 0
2107 :ps:`$` module load unknown >/dev/null
2108 :sgrer:`ERROR`: Unable to locate a modulefile for 'unknown'
2109
2110 The :option:`--redirect` and :option:`--no-redirect` command-line switches are
2111 also added to change the output redirection behavior for a single command:
2112
2113 .. parsed-literal::
2114
2115 :ps:`$` module load unknown --redirect >/dev/null
2116 :ps:`$` module load unknown --no-redirect >/dev/null
2117 :sgrer:`ERROR`: Unable to locate a modulefile for 'unknown'
2118
2119 .. _Change modulefile command behavior:
2120
2121 Change modulefile command behavior
2122 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2123
2124 Depending on the evaluation mode of the modulefile (e.g. *load*, *unload*,
2125 *display*, etc) commands have different behavior. Most common example is the
2126 :mfcmd:`setenv` command that sets an environment variable when modulefile is
2127 loaded and unsets it when it is unloaded. A different behavior may be wished
2128 sometimes for commands. This is why options are introduced for some modulefile
2129 commands to control what happens on particular evaluation mode.
2130
2131 The ``--return-value`` option is added to the :mfcmd:`getenv` and
2132 :mfcmd:`getvariant` modulefile commands to ensure that the value of the
2133 designated environment variable or variant is returned even if modulefile is
2134 evaluated in *display* mode:
2135
2136 .. parsed-literal::
2137
2138 :ps:`$` cat /path/to/modulefiles/foo/1.0
2139 #%Module5.1
2140 if {[getenv --return-value VAR] eq {}} {
2141 setenv VAR value
2142 }
2143 :ps:`$` module display foo/1.0
2144 -------------------------------------------------------------------
2145 :sgrhi:`/path/to/modulefiles/foo/1.0`:
2146
2147 :sgrcm:`setenv` VAR value
2148 -------------------------------------------------------------------
2149
2150 ``--remove-on-unload``, ``--append-on-unload``, ``--prepend-on-unload`` and
2151 ``--noop-on-unload`` options are added to the :mfcmd:`remove-path` and
2152 :mfcmd:`module unuse<module>` modulefile commands to control the behavior
2153 applied when modulefile is unloaded. With these options it is possible for
2154 instance to restore the paths unset at load time or to set other paths:
2155
2156 .. parsed-literal::
2157
2158 :ps:`$` module display bar/1.0
2159 -------------------------------------------------------------------
2160 :sgrhi:`/path/to/modulefiles/bar/1.0`:
2161
2162 :sgrcm:`module` unuse --prepend-on-unload /path/to/dir1
2163 :sgrcm:`module` use /path/to/dir2
2164 -------------------------------------------------------------------
2165 :ps:`$` module use
2166 Search path for module files (in search order):
2167 :sgrmp:`/path/to/dir1`
2168 :sgrmp:`/path/to/modulefiles`
2169 :ps:`$` module bar/1.0
2170 :ps:`$` module use
2171 Search path for module files (in search order):
2172 :sgrmp:`/path/to/dir2`
2173 :sgrmp:`/path/to/modulefiles`
2174 :ps:`$` module unload bar/1.0
2175 :ps:`$` module use
2176 Search path for module files (in search order):
2177 :sgrmp:`/path/to/dir1`
2178 :sgrmp:`/path/to/modulefiles`
2179
2180 Following the same trend, the ``--unset-on-unload`` and ``--noop-on-unload``
2181 options are added to the :mfcmd:`unsetenv` modulefile command to be able to
2182 choose between unsetting variable, setting a value or performing no operation
2183 when modulefile is unloaded.
2184
2185 Reducing number of I/O operations
2186 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2187
2188 A new configuration option named :mconfig:`mcookie_check` is introduced to
2189 control the verification made to files to determine if they are modulefiles.
2190 By default this configuration option is set to ``always`` and when searching
2191 for modulefiles within enabled modulepaths each file below these directories
2192 is opened to check if it starts with the Modules magic cookie (i.e.,
2193 ``#%Module`` file signature).
2194
2195 These historical checks lead to a large number of I/O operations on large
2196 module setup like in the below example where a total of 1098 modulefiles are
2197 available:
2198
2199 .. parsed-literal::
2200
2201 :ps:`$` module -o "" avail -t | wc -l
2202 1098
2203 :ps:`$` module config mcookie_check always
2204 :ps:`$` strace -f -e open,openat,read,close -c $MODULES_CMD bash avail
2205 ...
2206 % time seconds usecs/call calls errors syscall
2207 ------ ----------- ----------- --------- --------- ----------------
2208 44.29 0.044603 25 1734 166 open
2209 34.15 0.034389 16 2056 close
2210 11.87 0.011949 24 483 5 openat
2211 9.69 0.009761 4 2146 read
2212 ------ ----------- ----------- --------- --------- ----------------
2213 100.00 0.100702 6419 171 total
2214
2215 For each file, 3 I/O operations (``open``, ``read`` and ``close``) are
2216 achieved to determine if it is a modulefile and include it in search results.
2217 When modulefiles are located in a shared filesystem concurrently accessed by
2218 hundreds of users, a ``module avail`` command may take some time to finish.
2219
2220 When setting the :mconfig:`mcookie_check` configuration option to the ``eval``
2221 value, files are not checked anymore when searching for modulefiles, only when
2222 evaluating them. All files under modulepaths are considered modulefiles, so
2223 the content of these directories must be carefully checked to use this
2224 :mconfig:`mcookie_check` mode which lead to a significant reduction of I/O
2225 operations:
2226
2227 .. parsed-literal::
2228
2229 :ps:`$` module config mcookie_check eval
2230 :ps:`$` strace -f -e open,openat,read,close -c $MODULES_CMD bash avail
2231 ...
2232 % time seconds usecs/call calls errors syscall
2233 ------ ----------- ----------- --------- --------- ----------------
2234 30.56 0.013717 14 944 close
2235 28.76 0.012911 21 612 156 open
2236 26.41 0.011857 24 483 5 openat
2237 14.26 0.006403 6 1034 read
2238 ------ ----------- ----------- --------- --------- ----------------
2239 100.00 0.044888 3073 161 total
2240
2241 A substantial reduction of execution time may be noticed depending on the
2242 storage setup used to host the modulepath directories. A special care should
2243 be given to the content of these directories to ensure they only contain
2244 modulefiles (see :envvar:`MODULES_MCOOKIE_CHECK`).
2245
2246 See the :ref:`reduce-io-load` cookbook recipe to learn additional features of
2247 Modules that could be leveraged to lower the number of I/O operations.
2248
2249 Shell command completion
2250 ^^^^^^^^^^^^^^^^^^^^^^^^
2251
2252 New modulefile commands :mfcmd:`complete` and :mfcmd:`uncomplete` are added to
2253 get the ability to respectively define and unset command completion. *bash*,
2254 *tcsh* and *fish* shells are supported.
2255
2256 .. parsed-literal::
2257
2258 :ps:`>` module display foo
2259 -------------------------------------------------------------------
2260 :sgrhi:`/path/to/modulefiles/foo/1.0`:
2261
2262 :sgrcm:`append-path` PATH /path/to/foo-1.0/bin
2263 :sgrcm:`complete` fish foo {-s V -l version --description 'Command version'}
2264 :sgrcm:`complete` fish foo {-s h -l help --description 'Command help'}
2265 -------------------------------------------------------------------
2266 :ps:`>` module load foo
2267 :ps:`>` foo -<TAB>
2268 -h --help (Command help) -V --version (Command version)
2269
2270 :subcmd:`sh-to-mod` sub-command and :mfcmd:`source-sh` modulefile command have
2271 also been updated to track shell completion changes.
2272
2273 .. parsed-literal::
2274
2275 :ps:`$` module sh-to-mod bash /path/to/foo-1.0/share/setup-env.sh
2276 #%Module
2277 :sgrcm:`complete` bash foo {-o default -F _foo}
2278 :sgrcm:`append-path` PATH /path/to/foo-1.0/bin
2279 :sgrcm:`set-function` _foo {
2280 ...bash completion code...}
2281
2282 Lmod Tcl modulefile support
2283 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2284
2285 With this new version, Modules now supports Tcl modulefiles written for
2286 `Lmod`_, the alternative :command:`module` implementation developed in Lua.
2287 Such modulefiles can be safely evaluated by Modules without raising error.
2288
2289 Support has been added for the following modulefile commands introduced by
2290 Lmod: :mfcmd:`depends-on`, :mfcmd:`prereq-any`, :mfcmd:`always-load`,
2291 :mfcmd:`module load-any<module>`, :mfcmd:`pushenv`, :mfcmd:`require-fullname`
2292 and :mfcmd:`family`.
2293
2294 The :ref:`Compatibility with Lmod Tcl modulefile` section in the
2295 :ref:`modulefile(5)` man page describes the differences existing between the
2296 two implementations.
2297
2298 Note that when processing a :mfcmd:`family` command, the
2299 :envvar:`LMOD_FAMILY_\<NAME\>` environment variable is defined by Modules to
2300 be compatible with existing modulefiles or scripts relying on such variable.
2301
2302 .. _More tagging capabilities:
2303
2304 More tagging capabilities
2305 ^^^^^^^^^^^^^^^^^^^^^^^^^
2306
2307 The new :option:`--tag` option helps to define extra tags onto a loading
2308 module. These tags comes in addition to those inherited from the module state
2309 or those associated with the :mfcmd:`module-tag` modulefile command.
2310
2311 The :option:`--tag` option is available on :subcmd:`load`, :subcmd:`load-any`,
2312 :subcmd:`switch` and :subcmd:`try-load` sub-commands and on
2313 :mfcmd:`always-load`, :mfcmd:`depends-on`, :mfcmd:`module`, :mfcmd:`prereq`,
2314 :mfcmd:`prereq-all` and :mfcmd:`prereq-any` modulefile commands.
2315
2316 Informational messages of module evaluation have been updated to mention in
2317 module denomination the tags applying to it, as it is done in :subcmd:`list`
2318 sub-command output:
2319
2320 .. parsed-literal::
2321
2322 :ps:`$` module load -v --tag=sticky:bar foo/1.0
2323 Loading :sgrshi:`foo/1.0` <bar>
2324
2325 In case the designated module is already loaded, the additional tags are added
2326 to the list of tags already applied to this module.
2327
2328 .. parsed-literal::
2329
2330 :ps:`$` module list
2331 Currently Loaded Modulefiles:
2332 1) foo/1.0
2333 :ps:`$` module load -v --tag=sticky:bar foo/1.0
2334 Tagging :sgrshi:`foo/1.0` <bar>
2335
2336 The ``keep-loaded`` tag is introduced in this version. It avoids an
2337 auto-loaded module to get automatically unloaded when its dependent modules
2338 are unloaded. This new tag can be set with the :mfcmd:`module-tag` modulefile
2339 command or when module is loaded with the :mfcmd:`always-load` modulefile
2340 command. Default :mconfig:`tag_abbrev` configuration option has been updated
2341 to add the ``kL`` abbreviation for ``keep-loaded`` tag. Default dark and light
2342 color palettes have been updated too.
2343
2344 .. parsed-literal::
2345
2346 :ps:`$` module show bar/1.0
2347 -------------------------------------------------------------------
2348 :sgrhi:`/path/to/modulefiles/bar/1.0`:
2349
2350 :sgrcm:`always-load` foo/1.0
2351 -------------------------------------------------------------------
2352 :ps:`$` module load bar/1.0
2353 Loading :sgrhi:`bar/1.0`
2354 :sgrin:`Loading requirement`: foo/1.0
2355 :ps:`$` module unload bar/1.0
2356 :ps:`$` module list
2357 Currently Loaded Modulefiles:
2358 1) :sgral:`foo`:sgrkl:`/1.0`
2359
2360 Key:
2361 :sgral:`auto-loaded` :sgrkl:`keep-loaded`
2362
2363 When saving a collection, the tags defined with :option:`--tag` option are
2364 recorded to set them again when collection is restored. Tags resulting from
2365 module load state, like ``auto-loaded`` and ``keep-loaded``, are also
2366 recorded.
2367
2368 .. parsed-literal::
2369
2370 :ps:`$` module load --tag=sticky bar/1.0
2371 Loading :sgrshi:`bar/1.0`
2372 :sgrin:`Loading requirement`: foo/1.0
2373 :ps:`$` module save
2374 :ps:`$` module saveshow
2375 -------------------------------------------------------------------
2376 :sgrhi:`/home/user/.module/default`:
2377
2378 :sgrcm:`module` use --append /path/to/modulefiles
2379 :sgrcm:`module` load --tag=auto-loaded:keep-loaded foo
2380 :sgrcm:`module` load --tag=sticky bar
2381
2382 -------------------------------------------------------------------
2383
2384 The :mconfig:`collection_pin_tag` configuration option is added to record in
2385 collection all tags set on loaded modules. This configuration option is
2386 disabled by default.
2387
2388 .. note:: Collection saved now starts with a ``#%Module5.1`` file signature if
2389 :option:`--tag` option is recorded in it. Such collection could only be
2390 handled by Modules version 5.1 and above.
2391
2392 v5.0
2393 ----
2394
2395 With this new major version the :command:`module` experience has been updated
2396 to benefit by default from all the advanced behaviors and features developed
2397 over the Modules 4 versions. Modules 5.0 also introduces some breaking
2398 changes to improve the consistency of the whole solution. See the :ref:`5.0
2399 release notes<5.0 release notes>` for a complete list of the changes between
2400 Modules v4.8 and v5.0. The :ref:`changes<Modules 5 changes>` document gives an
2401 in-depth view of the modified behaviors.
2402
2403 Upgraded default configuration
2404 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2405
2406 Release after release, new advanced features were added on Modules 4. They
2407 were set off by default to avoid breaking change during the version 4 cycle.
2408 With the move to a new major release, these features are set on to improve by
2409 default usages for everybody. These features enabled by default are:
2410
2411 * :ref:`v42-automated-module-handling-mode` which provides the automatic
2412 dependency resolution when loading and unloading modules
2413
2414 .. parsed-literal::
2415
2416 :ps:`$` module load foo/1.0
2417 Loading :sgrhi:`foo/1.0`
2418 :sgrin:`Loading requirement`: bar/1.0
2419
2420 * `Extended default`_ that enables to select a module when the first number
2421 in its version is specified
2422
2423 .. parsed-literal::
2424
2425 :ps:`$` module load -v foo/1
2426 Loading :sgrhi:`foo/1.2.3`
2427
2428
2429 * :ref:`Advanced module version specifiers`, an improved syntax to designate
2430 module version in range or list and module variant
2431
2432 .. parsed-literal::
2433
2434 :ps:`$` module load foo@:2.2 %gcc11
2435 Loading :sgrhi:`foo/2.1`:sgrse:`{`:sgrva:`%gcc11`:sgrse:`}`
2436 :sgrin:`Loading requirement`: bar/1.2\ :sgrse:`{`:sgrva:`-debug`:sgrse:`:`:sgrva:`%gcc11`:sgrse:`}`
2437
2438 * `Colored output`_ to graphically enhance parts of the produced output to
2439 improve readability
2440
2441 .. parsed-literal::
2442
2443 :ps:`$` ml av
2444 ------------------ :sgrdi:`/path/to/modulefiles` ------------------
2445 :sgrde:`bar/1.0` bar/2.0 :sgrf:`foo/1.0` :sgrs:`foo/2.0` :sgrali:`foo/2.2`
2446
2447 Key:
2448 :sgrdi:`modulepath` :sgrali:`module-alias` :sgrs:`sticky`
2449 :sgrde:`default-version` :sgrf:`forbidden`
2450
2451 * :ref:`Insensitive case` search that matches modules using a different
2452 character case than the one expressed in search query
2453
2454 .. parsed-literal::
2455
2456 :ps:`$` ml av liba
2457 ------------------ :sgrdi:`/path/to/modulefiles` ------------------
2458 LibA/1.0 LibA/2.0
2459
2460 Some other features that were enabled by default during the Modules 4 cycle
2461 have been turned off as they may not be useful for basic usages. Among other
2462 things setting off the following features makes the definition of the
2463 :command:`module` function simpler. Even if off by default, these features
2464 can now be enabled once Modules is installed through the :file:`initrc`
2465 configuration file.
2466
2467 * *Set shell startup* files to ensure the :command:`module` command is defined
2468 once shell has been initialized. See the :mconfig:`set_shell_startup`
2469 configuration option to activate.
2470
2471 * *Quarantine mechanism* that protects the :command:`module` command
2472 run-time environment from side effect coming from the current environment
2473 definition. See the :mconfig:`quarantine_support` configuration option to
2474 activate.
2475
2476 * *Silent shell debug* which disables the debugging property set on current
2477 shell session for the duration of the :command:`module` command. See the
2478 :mconfig:`silent_shell_debug` configuration option to activate.
2479
2480 Removing compatibility version
2481 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2482
2483 The ability to co-install version 3.2 of Modules along newer version is
2484 discontinued. The installation option ``--enable-compat-version``,
2485 :command:`switchml` shell function and :envvar:`MODULES_USE_COMPAT_VERSION`
2486 environment variables are thus removed.
2487
2488 The interesting features of Modules 3.2 that were missing in the
2489 initial Modules 4 release in 2017 have been reintroduced progressively (like
2490 :subcmd:`clear` sub-command or :option:`--icase` search). With Modules 5.0,
2491 the :subcmd:`refresh` sub-command is even changed to the behavior it had on
2492 Modules 3.2. So it is a good time for the big jump.
2493
2494 If you are still using Modules 3.2, please refer to the :ref:`changes`
2495 document that describes the differences of this version compared to the newer
2496 releases.
2497
2498 Improving Modules initialization
2499 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2500
2501 Modules initialization files are now installed by default in the *etc*
2502 directory designated by the :instopt:`--etcdir` installation option. The
2503 initialization configuration file is named :file:`initrc` in this directory,
2504 and the modulepath-specific configuration file is named :file:`modulespath`.
2505 When both files exist, now they are both evaluated instead of just the
2506 :file:`modulespath` file.
2507
2508 Modules magic cookie (i.e., ``#%Module`` file signature) is now required at
2509 the start of :file:`initrc`. An error is produced if the magic cookie is
2510 missing or if the optional version number placed after the cookie string is
2511 higher than the version of the :file:`modulecmd.tcl` script in use.
2512
2513 Note that :file:`initrc` configuration file can host more than
2514 :subcmd:`module use<use>` and :subcmd:`module load<load>` commands.
2515 :command:`module` configuration can also be achieved with this file through
2516 the use of :subcmd:`module config<config>` commands.
2517
2518 Modules initialization has been enhanced for situations where a module
2519 environment is found already defined. In this case the loaded modules are
2520 automatically refreshed which is useful to re-apply the non-persistent
2521 environment configuration (i.e., shell alias and function that are not
2522 exported to the sub-shell). For instance when starting a sub-shell session it
2523 ensures that the loaded environment is fully inherited from parent shell:
2524
2525 .. parsed-literal::
2526
2527 :ps:`$` ml show foo/1.0
2528 -------------------------------------------------------------------
2529 :sgrhi:`/path/to/modulefiles/foo/1.0`:
2530
2531 :sgrcm:`set-alias` foo {echo foo}
2532 -------------------------------------------------------------------
2533 :ps:`$` ml foo/1.0
2534 :ps:`$` alias foo
2535 alias foo='echo foo'
2536 :ps:`$` bash
2537 :ps:`$` ml
2538 Currently Loaded Modulefiles:
2539 1) foo/1.0
2540 :ps:`$` alias foo
2541 alias foo='echo foo'
2542
2543 Simplifying path-like variable reference counting
2544 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2545
2546 The reference counting mechanism used for path-like environment variable
2547 enables to determine if a path entry has been added several times (by several
2548 loaded modules for instance) to know whether or not this path entry should be
2549 unset when unloading a module. Entry is not removed if multiple loaded modules
2550 rely on it.
2551
2552 The mechanism is not applied anymore to the Modules-specific path variables
2553 (like :envvar:`LOADEDMODULES`) as an element entry in these variables cannot
2554 be added multiple times without duplication. For instance, a given module name
2555 and version cannot be added twice in :envvar:`LOADEDMODULES` as this module is
2556 only loaded once. With this change a thinner environment is produced by
2557 :command:`module`. An exception is made for :envvar:`MODULEPATH` environment
2558 variable where the mechanism still applies.
2559
2560 .. parsed-literal::
2561
2562 :ps:`$` $MODULES_CMD bash load foo/2.0
2563 _LMFILES_=/path/to/modulefiles/foo/2.0; export _LMFILES_;
2564 LOADEDMODULES=foo/2.0; export LOADEDMODULES;
2565 __MODULES_LMTAG=foo/2.0\&mytag; export __MODULES_LMTAG;
2566 test 0;
2567
2568 Reference counting mechanism has also been simplified for entries in path-like
2569 variable that are only referred once. For such entries no entry is set in the
2570 reference counting variable (which are now called
2571 :envvar:`__MODULES_SHARE_\<VAR\>`). A reference count entry is set only if the
2572 entry in the path-like variable is referred more than one time.
2573
2574 .. parsed-literal::
2575
2576 :ps:`$` ml foo/3.0
2577 :ps:`$` echo $PATHVAR
2578 /path/to/dir1
2579 :ps:`$` echo $__MODULES_SHARE_PATHVAR
2580
2581 :ps:`$` ml bar/1.0
2582 :ps:`$` echo $PATHVAR
2583 /path/to/dir1
2584 :ps:`$` echo $__MODULES_SHARE_PATHVAR
2585 /path/to/dir1:2
2586 :ps:`$` ml -foo/3.0
2587 :ps:`$` echo $PATHVAR
2588 /path/to/dir1
2589 :ps:`$` echo $__MODULES_SHARE_PATHVAR
2590
2591 :ps:`$`
2592
2593 When the :subcmd:`use` and :subcmd:`unuse` module sub-commands are not called
2594 during a modulefile evaluation, the reference counter associated with each
2595 entry in :envvar:`MODULEPATH` environment variable is ignored. In such
2596 context, a :subcmd:`module use<use>` will not increase the reference counter
2597 of a path entry already defined and a :subcmd:`module unuse<unuse>` will
2598 remove specified path whatever its reference counter value. Same change applies
2599 for :subcmd:`append-path`, :subcmd:`prepend-path` and :subcmd:`remove-path`
2600 module sub-commands when called from the command-line.
2601
2602 .. parsed-literal::
2603
2604 :ps:`$` echo $MODULEPATH
2605 /path/to/modulefiles
2606 :ps:`$` echo $__MODULES_SHARE_MODULEPATH
2607 /path/to/modulefiles:2
2608 :ps:`$` ml use /path/to/modulefiles
2609 :ps:`$` echo $__MODULES_SHARE_MODULEPATH
2610 /path/to/modulefiles:2
2611 :ps:`$` ml unuse /path/to/modulefiles
2612 :ps:`$` echo $MODULEPATH
2613
2614 :ps:`$` echo $__MODULES_SHARE_MODULEPATH
2615
2616 :ps:`$`
2617
2618
2619 v4.8
2620 ----
2621
2622 This new version is backward-compatible with previous version 4 releases. It
2623 fixes bugs but also introduces new functionalities that are described in this
2624 section. See the :ref:`4.8 release notes<4.8 release notes>` for a complete
2625 list of the changes between Modules v4.7 and v4.8.
2626
2627 .. _Editing modulefiles:
2628
2629 Editing modulefiles
2630 ^^^^^^^^^^^^^^^^^^^
2631
2632 :subcmd:`edit` sub-command is introduced to give the ability to open
2633 modulefiles in a text editor. Modulefiles can be specified like with any other
2634 sub-command: using regular, symbolic or aliased names or using advanced
2635 version specifiers.
2636
2637 .. parsed-literal::
2638
2639 :ps:`$` ml edit foo
2640
2641 :subcmd:`edit` sub-command resolves the path toward the designated modulefile
2642 then call configured text editor to open this modulefile with it. Below, the
2643 modulefile is opened with the ``vi`` command:
2644
2645 .. parsed-literal::
2646
2647 #%Module
2648 module-whatis [module-info name]
2649 setenv PATH /path/to/foo-1.0/bin
2650 ~
2651 ~
2652 ~
2653 "/path/to/modulefiles/foo/1.0" 3L, 42B 1,1 All
2654
2655 The :mconfig:`editor` configuration option controls the editor command to use.
2656 This option can be configured at installation time with the
2657 :instopt:`--with-editor` installation option. If not set, :mconfig:`editor`
2658 configuration option is set by default to ``vi``.
2659
2660 :mconfig:`editor` configuration option can be changed with the
2661 :subcmd:`config` sub-command. Which sets the :envvar:`MODULES_EDITOR`
2662 environment variable.
2663
2664 The :envvar:`VISUAL` or the :envvar:`EDITOR` environment variables override
2665 the default value of :mconfig:`editor` configuration option but are overridden
2666 by the :envvar:`MODULES_EDITOR` environment variable.
2667
2668 Using version range in version list
2669 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2670
2671 The :ref:`Advanced module version specifiers` mechanism has been improved to
2672 allow the use of version range (*@:version*, *@vers1:vers2* or *@version:*)
2673 within version list (*@version1,version2,...*).
2674
2675 It is now possible to write for instance ``mod@:1.2,1.4:1.6,1.8:`` to
2676 designate all versions of module *mod*, except versions *1.3* and *1.7*.
2677
2678 This improvement is available where the advanced version specifier syntax is
2679 supported. Thus it can be either used from the command-line or when writing
2680 modulefiles, for instance to hide or tag modules or to declare requirements.
2681
2682 Try module load with no complain if not found
2683 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2684
2685 Add the :subcmd:`try-load` sub-command that tries to load the modulefile
2686 passed as argument, like the :subcmd:`load` sub-command, but does not raise an
2687 error if this modulefile cannot be found.
2688
2689 .. parsed-literal::
2690
2691 :ps:`$` module load unknown
2692 :sgrer:`ERROR`: Unable to locate a modulefile for 'unknown'
2693 :ps:`$` echo $?
2694 1
2695 :ps:`$` module try-load unknown
2696 :ps:`$` echo $?
2697 0
2698 :ps:`$` module list
2699 No Modulefiles Currently Loaded.
2700
2701 This sub-command first introduced by the `Lmod`_ project is added to Modules
2702 to improve the compatibility between the two ``module`` implementations.
2703
2704 :subcmd:`try-load` is also available within modulefile context to continue the
2705 evaluation of a modulefile in case no module is found in its attempt to load
2706 another modulefile
2707
2708 .. parsed-literal::
2709
2710 :ps:`$` module display foo/1.0
2711 -------------------------------------------------------------------
2712 :sgrhi:`/path/to/modulefiles/foo/1.0`:
2713
2714 :sgrcm:`module` try-load unknown/1.0
2715 -------------------------------------------------------------------
2716 :ps:`$` module load foo/1.0
2717 :ps:`$` module list
2718 Currently Loaded Modulefiles:
2719 1) foo/1.0
2720
2721 Module variants
2722 ^^^^^^^^^^^^^^^
2723
2724 :ref:`Module variants` is a new mechanism that allows to pass arguments to
2725 evaluated modulefiles in order to achieve different environment variable or
2726 module requirement setup with a single modulefile.
2727
2728 Variant specification relies on the :ref:`Advanced module version specifiers`
2729 mechanism, which leverages the `variant syntax`_ of the `Spack`_ package
2730 manager:
2731
2732 .. _variant syntax: https://spack.readthedocs.io/en/latest/basic_usage.html#variants
2733
2734 .. parsed-literal::
2735
2736 :ps:`$` module config advanced_version_spec 1
2737 :ps:`$` module load -v bar/1.2 toolchain=a -debug
2738 Loading :sgrhi:`bar/1.2`:sgrse:`{`:sgrva:`-debug`:sgrse:`:`:sgrva:`toolchain=a`:sgrse:`}`
2739
2740 Variants are defined in modulefile with the :mfcmd:`variant` command, which
2741 defines the variant type and its accepted values:
2742
2743 .. code-block:: tcl
2744
2745 #%Module4.8
2746 variant toolchain a b c
2747 variant --boolean --default off debug
2748
2749 # select software build depending on variant values
2750 set suffix -[getvariant toolchain]
2751 if {$ModuleVariant(debug)} {
2752 append suffix -dbg
2753 }
2754
2755 prepend-path PATH /path/to/bar-1.2$suffix/bin
2756 prepend-path LD_LIBRARY_PATH /path/to/bar-1.2$suffix/lib
2757
2758 The *bar/1.2* modulefile defines a ``toolchain`` variant, which accepts the
2759 ``a``, ``b`` and ``c`` values, and a ``debug`` Boolean variant, which is set
2760 ``off`` by default. Once these two variants are declared, their value
2761 specified on module designation are instantiated in the :mfvar:`ModuleVariant`
2762 array variable which could also be queried with the :mfcmd:`getvariant`
2763 modulefile command. Selected variant values enable to define a specific
2764 installation build path for the *bar/1.2* software.
2765
2766 If a variant is not specified when designating module and if this variant is
2767 not declared with a default value, an error is obtained:
2768
2769 .. parsed-literal::
2770
2771 :ps:`$` module purge
2772 :ps:`$` module load :noparse:`[email protected]`
2773 Loading :sgrhi:`bar/1.2`
2774 :sgrer:`ERROR`: No value specified for variant 'toolchain'
2775 Allowed values are: a b c
2776
2777 Once module is loaded, selected variants are reported on the :subcmd:`list`
2778 sub-command output:
2779
2780 .. parsed-literal::
2781
2782 :ps:`$` module load :noparse:`[email protected]` toolchain=b
2783 :ps:`$` module list
2784 Currently Loaded Modulefiles:
2785 1) bar/1.2\ :sgrse:`{`:sgrva:`-debug`:sgrse:`:`:sgrva:`toolchain=b`:sgrse:`}`
2786
2787 Key:
2788 :sgrse:`{`:sgrva:`-variant`:sgrse:`}`\=\ :sgrse:`{`:sgrva:`variant=off`:sgrse:`}` :sgrse:`{`:sgrva:`variant=value`:sgrse:`}`
2789
2790 .. note:: The default value of the :instopt:`--with-list-output` installation
2791 option has been updated to include variant information.
2792
2793 Variant specification could be used where the :ref:`Advanced module version
2794 specifiers` is supported. For instance a module may express a dependency over
2795 a specific module variant:
2796
2797 .. parsed-literal::
2798
2799 :ps:`$` module show foo/2.1 toolchain=c
2800 -------------------------------------------------------------------
2801 :sgrhi:`/path/to/modulefiles/foo/2.1`:
2802
2803 :sgrcm:`variant` toolchain a b c
2804 :sgrcm:`prereq` :noparse:`[email protected] toolchain=`:sgrva:`{toolchain}`
2805 :sgrcm:`prepend-path` PATH /path/to/foo-2.1-:sgrva:`{toolchain}`/bin
2806 :sgrcm:`prepend-path` LD_LIBRARY_PATH /path/to/foo-2.1-:sgrva:`{toolchain}`/lib
2807 -------------------------------------------------------------------
2808
2809 In this example, *foo/2.1* module depends on *bar/1.2* and the same toolchain
2810 variant should be selected for both modules in order to load two software
2811 builds that are compatible between each other.
2812
2813 .. parsed-literal::
2814
2815 :ps:`$` module purge
2816 :ps:`$` module config auto_handling 1
2817 :ps:`$` module load foo/2.1 toolchain=a
2818 Loading :sgrhi:`foo/2.1`:sgrse:`{`:sgrva:`toolchain=a`:sgrse:`}`
2819 :sgrin:`Loading requirement`: bar/1.2\ :sgrse:`{`:sgrva:`-debug`:sgrse:`:`:sgrva:`toolchain=a`:sgrse:`}`
2820
2821 Variant shortcuts
2822 ^^^^^^^^^^^^^^^^^
2823
2824 The :mconfig:`variant_shortcut` configuration option is added to define
2825 shortcut characters for easily specifying variants. Instead of writing the
2826 variant name to specify it in module designation (e.g., *name=value*), the
2827 shortcut associated to this variant could be used (i.e., *<shortcut>value*):
2828
2829 .. parsed-literal::
2830
2831 :ps:`$` module purge
2832 :ps:`$` module config variant_shortcut toolchain=%
2833 :ps:`$` module load foo/2.1 %a
2834 Loading :sgrhi:`foo/2.1`:sgrse:`{`:sgrva:`%a`:sgrse:`}`
2835 :sgrin:`Loading requirement`: bar/1.2\ :sgrse:`{`:sgrva:`-debug`:sgrse:`:`:sgrva:`%a`:sgrse:`}`
2836
2837 Configured shortcuts are also used to report the loaded variant on
2838 :subcmd:`list` sub-command output (shortcuts are explained in key section):
2839
2840 .. parsed-literal::
2841
2842 :ps:`$` module list
2843 Currently Loaded Modulefiles:
2844 1) :sgral:`bar/1.2`\ :sgrse:`{`:sgrva:`-debug`:sgrse:`:`:sgrva:`%a`:sgrse:`}` 2) foo/2.1\ :sgrse:`{`:sgrva:`%a`:sgrse:`}`
2845
2846 Key:
2847 :sgral:`auto-loaded` :sgrse:`{`:sgrva:`-variant`:sgrse:`}`\=\ :sgrse:`{`:sgrva:`variant=off`:sgrse:`}` :sgrse:`{`:sgrva:`%value`:sgrse:`}`\=\ :sgrse:`{`:sgrva:`toolchain=value`:sgrse:`}` :sgrse:`{`:sgrva:`variant=value`:sgrse:`}`
2848
2849
2850 v4.7
2851 ----
2852
2853 This new version is backward-compatible with previous version 4 releases. It
2854 fixes bugs but also introduces new functionalities that are described in this
2855 section. See the :ref:`4.7 release notes<4.7 release notes>` for a complete
2856 list of the changes between Modules v4.6 and v4.7.
2857
2858 Determining module implementation and version
2859 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2860
2861 New Modules variables are introduced to determine during the evaluation of a
2862 modulefile or a modulerc what *module* implementation is currently in use. The
2863 :mfvar:`ModuleTool` variable corresponds to the name of the *module*
2864 implementation and is set to ``Modules`` for this project. The
2865 :mfvar:`ModuleToolVersion` variable corresponds to the version number of the
2866 implementation (e.g. ``4.7.0``).
2867
2868 With these new variables it is possible to precisely know what *module*
2869 command is in use then adapt modulefile code to handle a specific behavior or
2870 leverage a new feature.
2871
2872 The modulefile command :mfcmd:`versioncmp` is also introduced to provide a
2873 simple way to compare two version strings and return if first version string
2874 is less than, equal to or greater than second one.
2875
2876 .. code-block:: tcl
2877
2878 if {[info exists ModuleTool] && $ModuleTool eq {Modules}
2879 && [versioncmp $ModuleToolVersion 4.7] >= 0} {
2880 # here some code specific for Modules 4.7 and later versions
2881 }
2882
2883 The :mfvar:`ModuleTool` and :mfvar:`ModuleToolVersion` variables and the
2884 :mfcmd:`versioncmp` modulefile command are supported by the `Lmod`_ project
2885 starting version ``8.4.8``.
2886
2887 Symbolic version to designate module loaded version
2888 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2889
2890 When the `Advanced module version specifiers`_ is enabled, the ``loaded``
2891 symbolic version may be used to designate the currently loaded version of
2892 specified module.
2893
2894 .. parsed-literal::
2895
2896 :ps:`$` ml display :noparse:`foo@loaded`
2897 -------------------------------------------------------------------
2898 :sgrhi:`/path/to/modulefiles/foo/1.0`:
2899
2900 :sgrcm:`module-whatis` foo/1.0
2901 -------------------------------------------------------------------
2902
2903 If no version of specified module can be found loaded, an error is returned.
2904
2905 .. parsed-literal::
2906
2907 :ps:`$` ml display :noparse:`foo@loaded`
2908 :sgrer:`ERROR`: No loaded version found for 'foo' module
2909
2910 Module tags
2911 ^^^^^^^^^^^
2912
2913 Module tags are piece of information that can be associated to individual
2914 modulefiles. Tags could be purely informational or may lead to specific
2915 behaviors.
2916
2917 Module tags may be inherited from the module state set by a modulefile command
2918 or consequence of a module action. Tags may also be associated to modules by
2919 using the new :mfcmd:`module-tag` modulefile command.
2920
2921 Module tags are reported along the module they are associated to on
2922 :subcmd:`avail` and :subcmd:`list` sub-command results. Tags could be reported
2923 either:
2924
2925 * along the module name, all tags set within angle brackets, each tag
2926 separated from the others with a colon character (e.g.,
2927 ``foo/1.2 <tag1:tag2>``).
2928
2929 .. parsed-literal::
2930
2931 :ps:`$` cat /path/to/modulefiles/foo/.modulerc
2932 #%Module
2933 module-tag mytag foo
2934 module-tag othertag foo/1.0
2935 :ps:`$` ml av
2936 --------------- :sgrdi:`/path/to/modulefiles` ---------------
2937 foo/1.0 <mytag:othertag> foo/2.0 <mytag>
2938 :ps:`$` ml foo/1.0
2939 :ps:`$` ml
2940 Currently Loaded Modulefiles:
2941 1) foo/1.0 <mytag:othertag>
2942
2943 * graphically rendered over the module name for each tag associated to a
2944 Select Graphic Rendition (SGR) code in the color palette (see
2945 :envvar:`MODULES_COLORS`)
2946
2947 .. parsed-literal::
2948
2949 :ps:`$` # set SGR code to report 'mytag' with blue background color
2950 :ps:`$` ml config colors "hi=1:di=94:L=90;47:mytag=102"
2951 :ps:`$` ml av
2952 --------------- :sgrdi:`/path/to/modulefiles` ---------------
2953 :sgrl:`foo`:sgrss:`/1.0` <othertag> :sgrss:`foo/2.0`
2954 :ps:`$` ml
2955 Currently Loaded Modulefiles:
2956 1) :sgrl:`foo`:sgrss:`/1.0` <othertag>
2957
2958 The :mconfig:`tag_abbrev` configuration option is available to define
2959 abbreviated strings for module tags and then use these abbreviations instead
2960 of tag names when reporting tags on :subcmd:`avail` and :subcmd:`list` command
2961 results.
2962
2963 .. parsed-literal::
2964
2965 :ps:`$` # add abbreviation for 'othertag' tag
2966 :ps:`$` ml config tag_abbrev loaded=L:othertag=oT
2967 :ps:`$` ml av
2968 --------------- :sgrdi:`/path/to/modulefiles` ---------------
2969 :sgrl:`foo`:sgrss:`/1.0` <oT> :sgrss:`foo/2.0`
2970 :ps:`$` ml
2971 Currently Loaded Modulefiles:
2972 1) :sgrl:`foo`:sgrss:`/1.0` <oT>
2973
2974 When a SGR code is set for a tag in the color palette, this graphical
2975 rendition is applied by default over the module name and the tag name or its
2976 abbreviation is not displayed. If tag name or abbreviation is added to the
2977 :mconfig:`tag_color_name` configuration option, graphical rendering is applied
2978 to the tag name or abbreviation rather than over the module name they are
2979 attached to.
2980
2981 .. parsed-literal::
2982
2983 :ps:`$` # add SGR code for 'oT' tag and set rendition over tag name
2984 :ps:`$` ml config colors "hi=1:di=94:L=90;47:mytag=44:oT=41"
2985 :ps:`$` ml config tag_color_name oT
2986 :ps:`$` ml av
2987 --------------- :sgrdi:`/path/to/modulefiles` ---------------
2988 :sgrl:`foo`:sgrss:`/1.0` <:sgrf:`oT`> :sgrss:`foo/2.0`
2989 :ps:`$` ml
2990 Currently Loaded Modulefiles:
2991 1) :sgrl:`foo`:sgrss:`/1.0` <:sgrf:`oT`>
2992
2993 Tags inherited from module state, consequence of a module action or set by
2994 using :mfcmd:`module-tag` but that have a special meaning currently are:
2995
2996 .. only:: html
2997
2998 +------------------+-------------------------+------------------------+-------+------------------------+
2999 | Tag | Description | Set with | Abbr. | Color |
3000 | | | | | |
3001 +==================+=========================+========================+=======+========================+
3002 | auto-loaded | Module has been loaded | Inherited | aL | .. parsed-literal:: |
3003 | | automatically | | | |
3004 | | | | | :sgral:`mod/1.0` |
3005 +------------------+-------------------------+------------------------+-------+------------------------+
3006 | forbidden | Module cannot be loaded | Inherited from | F | .. parsed-literal:: |
3007 | | | :mfcmd:`module-forbid` | | |
3008 | | | | | :sgrf:`mod/1.0` |
3009 +------------------+-------------------------+------------------------+-------+------------------------+
3010 | hidden | Module is not visible | Inherited from | H | .. parsed-literal:: |
3011 | | on :subcmd:`avail` | :mfcmd:`module-hide` | | |
3012 | | | | | :sgrh:`mod/1.0` |
3013 +------------------+-------------------------+------------------------+-------+------------------------+
3014 | hidden-loaded | See `Hiding loaded | Inherited from | H | .. parsed-literal:: |
3015 | | modules`_ | :mfcmd:`module-hide` | | |
3016 | | | | | :sgrh:`mod/1.0` |
3017 +------------------+-------------------------+------------------------+-------+------------------------+
3018 | loaded | Module is currently | Inherited | L | .. parsed-literal:: |
3019 | | loaded | | | |
3020 | | | | | :sgrl:`mod/1.0` |
3021 +------------------+-------------------------+------------------------+-------+------------------------+
3022 | nearly-forbidden | Module will soon not be | Inherited from | nL | .. parsed-literal:: |
3023 | | able to load anymore | :mfcmd:`module-forbid` | | |
3024 | | | | | :sgrnf:`mod/1.0` |
3025 +------------------+-------------------------+------------------------+-------+------------------------+
3026 | sticky | See `Sticky modules`_ | :mfcmd:`module-tag` | S | .. parsed-literal:: |
3027 | | | | | |
3028 | | | | | :sgrs:`mod/1.0` |
3029 +------------------+-------------------------+------------------------+-------+------------------------+
3030 | super-sticky | See `Sticky modules`_ | :mfcmd:`module-tag` | sS | .. parsed-literal:: |
3031 | | | | | |
3032 | | | | | :sgrss:`mod/1.0` |
3033 +------------------+-------------------------+------------------------+-------+------------------------+
3034
3035 .. only:: not html
3036
3037 +------------------+-------------------------+------------------------+-------+
3038 | Tag | Description | Set with | Abbr. |
3039 | | | | |
3040 +==================+=========================+========================+=======+
3041 | auto-loaded | Module has been loaded | Inherited | aL |
3042 | | automatically | | |
3043 | | | | |
3044 +------------------+-------------------------+------------------------+-------+
3045 | forbidden | Module cannot be loaded | Inherited from | F |
3046 | | | :mfcmd:`module-forbid` | |
3047 | | | | |
3048 +------------------+-------------------------+------------------------+-------+
3049 | hidden | Module is not visible | Inherited from | H |
3050 | | on :subcmd:`avail` | :mfcmd:`module-hide` | |
3051 | | | | |
3052 +------------------+-------------------------+------------------------+-------+
3053 | hidden-loaded | See `Hiding loaded | Inherited from | H |
3054 | | modules`_ | :mfcmd:`module-hide` | |
3055 | | | | |
3056 +------------------+-------------------------+------------------------+-------+
3057 | loaded | Module is currently | Inherited | L |
3058 | | loaded | | |
3059 | | | | |
3060 +------------------+-------------------------+------------------------+-------+
3061 | nearly-forbidden | Module will soon not be | Inherited from | nL |
3062 | | able to load anymore | :mfcmd:`module-forbid` | |
3063 | | | | |
3064 +------------------+-------------------------+------------------------+-------+
3065 | sticky | See `Sticky modules`_ | :mfcmd:`module-tag` | S |
3066 | | | | |
3067 | | | | |
3068 +------------------+-------------------------+------------------------+-------+
3069 | super-sticky | See `Sticky modules`_ | :mfcmd:`module-tag` | sS |
3070 | | | | |
3071 | | | | |
3072 +------------------+-------------------------+------------------------+-------+
3073
3074 Hiding loaded modules
3075 ^^^^^^^^^^^^^^^^^^^^^
3076
3077 The ``--hidden-loaded`` option has been added to the :mfcmd:`module-hide`
3078 modulefile command and it indicates that designated hidden modules remain
3079 hidden after being loaded.
3080
3081 .. parsed-literal::
3082
3083 :ps:`$` cat /path/to/modulefiles/foo/1.0
3084 #%Module
3085 module load bar
3086 :ps:`$` cat /path/to/modulefiles/bar/.modulerc
3087 #%Module4.7
3088 module-hide --soft --hidden-loaded bar
3089
3090 In this example, *foo* depends on *bar* which is set soft hidden and hidden
3091 once loaded. As a consequence, automated load of *bar* module will not be
3092 reported and *bar/1.0* will not appear in loaded module list by default:
3093
3094 .. parsed-literal::
3095
3096 :ps:`$` ml foo
3097 :ps:`$` ml
3098 Currently Loaded Modulefiles:
3099 1) foo/1.0
3100
3101 However *bar/1.0* is loaded. Hidden loaded modules can be unveiled with the
3102 :option:`--all`/:option:`-a` option set on the :subcmd:`list` sub-command.
3103 ``hidden-loaded`` tag (abbreviated by default to ``H`` when colored output is
3104 disabled) applies to such modules.
3105
3106 .. parsed-literal::
3107
3108 :ps:`$` ml -a
3109 Currently Loaded Modulefiles:
3110 1) :sgral:`bar`:sgrh:`/1.0` 2) foo/1.0
3111
3112 To also get the informational messages about hidden loaded module automated
3113 load or unload, the new verbosity level ``verbose2`` can be used (with
3114 :option:`-vv` option for instance):
3115
3116 .. parsed-literal::
3117
3118 :ps:`$` ml purge
3119 :ps:`$` ml -vv foo
3120 Loading bar/1.0
3121
3122 Loading :sgrhi:`foo/1.0`
3123 :sgrin:`Loading requirement`: bar/1.0
3124
3125 Sticky modules
3126 ^^^^^^^^^^^^^^
3127
3128 Module stickiness is introduced, in a similar fashion than on the `Lmod`_
3129 project, to allow to glue modules to the loaded environment. A sticky module
3130 cannot be unloaded, unless if the unload action is forced or if the module
3131 reloads after being unloaded.
3132
3133 A modulefile is declared *sticky* by applying it the ``sticky`` tag with the
3134 :mfcmd:`module-tag` modulefile command.
3135
3136 .. parsed-literal::
3137
3138 :ps:`$` cat /path/to/modulefiles/foo/.modulerc
3139 #%Module4.7
3140 module-tag sticky foo/1.0
3141 :ps:`$` ml
3142 Currently Loaded Modulefiles:
3143 1) :sgrs:`foo/1.0`
3144 :ps:`$` ml -foo
3145 Unloading :sgrhi:`foo/1.0`
3146 :sgrer:`ERROR`: Unload of sticky module 'foo/1.0' skipped
3147 :ps:`$` ml
3148 Currently Loaded Modulefiles:
3149 1) :sgrs:`foo/1.0`
3150 :ps:`$` ml --force -foo
3151 Unloading :sgrhi:`foo/1.0`
3152 :sgrwa:`WARNING`: Unload of sticky module 'foo/1.0' forced
3153 :ps:`$` ml
3154 No Modulefiles Currently Loaded.
3155
3156
3157 Modulefile can also be defined ``super-sticky`` by applying the corresponding
3158 module tag. *Super-sticky* module cannot be unloaded even if the unload action
3159 is forced. It can only be unloaded if the module reloads afterward.
3160
3161 .. parsed-literal::
3162
3163 :ps:`$` cat /path/to/modulefiles/bar/.modulerc
3164 #%Module4.7
3165 module-tag super-sticky bar/1.0
3166 :ps:`$` ml
3167 Currently Loaded Modulefiles:
3168 1) :sgrss:`bar/1.0`
3169 :ps:`$` ml purge
3170 Unloading :sgrhi:`bar/1.0`
3171 :sgrer:`ERROR`: Unload of super-sticky module 'bar/1.0' skipped
3172 :ps:`$` ml purge -f
3173 Unloading :sgrhi:`bar/1.0`
3174 :sgrer:`ERROR`: Unload of super-sticky module 'bar/1.0' skipped
3175 :ps:`$` ml
3176 Currently Loaded Modulefiles:
3177 1) :sgrss:`bar/1.0`
3178
3179 Modulefiles targeted by a ``sticky`` or a ``super-sticky`` tag are colored on
3180 :subcmd:`avail` and :subcmd:`list` sub-command outputs to indicate such tag
3181 applies. If colored output is disabled a tag abbreviation is reported along
3182 module designation (respectively ``S`` and ``sS``).
3183
3184 In case the stickiness applies to the generic module name (and does not target
3185 a specific module version or version-set), one version of the sticky or
3186 super-sticky module can be swapped by another version of this same module:
3187
3188 .. parsed-literal::
3189
3190 :ps:`$` cat /path/to/modulefiles/baz/.modulerc
3191 #%Module4.7
3192 module-tag sticky baz
3193 :ps:`$` ml
3194 Currently Loaded Modulefiles:
3195 1) :sgrs:`baz/2.0`
3196 :ps:`$` ml switch baz/1.0
3197 :ps:`$` ml
3198 Currently Loaded Modulefiles:
3199 1) :sgrs:`baz/1.0`
3200
3201 Explaining avail/list output
3202 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3203
3204 A *Key* section is added at the end of the :subcmd:`avail` and :subcmd:`list`
3205 sub-commands output to give hints on the meaning of the graphical rendition
3206 applied to elements or what the elements set in parentheses or chevrons along
3207 module name stand for.
3208
3209 .. parsed-literal::
3210
3211 :ps:`$` ml av
3212 ------------------ :sgrdi:`/path/to/modulefiles` ------------------
3213 :sgrs:`foo`:sgrl:`/1.0` <oT> :sgrde:`foo/2.0` :sgrali:`foo/3.0`
3214
3215 Key:
3216 :sgrl:`loaded` :sgrde:`default-version` :sgrs:`sticky` <oT>=othertag
3217 :sgrdi:`modulepath` :sgrali:`module-alias` <module-tag>
3218
3219 Configuring avail/list output
3220 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3221
3222 New configuration options are introduced to control what content to output in
3223 addition to modules names on the regular and terse output modes of the
3224 :subcmd:`avail` and :subcmd:`list` sub-commands.
3225
3226 These new configuration options named :mconfig:`avail_output`,
3227 :mconfig:`avail_terse_output`, :mconfig:`list_output` and
3228 :mconfig:`list_terse_output` can be updated using the :subcmd:`config`
3229 sub-command or set at installation time respectively with the
3230 :instopt:`--with-avail-output`, :instopt:`--with-avail-terse-output`,
3231 :instopt:`--with-list-output` and :instopt:`--with-list-terse-output`
3232 configure options.
3233
3234 The four options accept a colon separated list of elements as value. Accepted
3235 elements for the :subcmd:`avail`-related options are: ``modulepath``,
3236 ``alias``, ``dirwsym``, ``sym``, ``tag`` and ``key``. Accepted elements for
3237 the :subcmd:`list`-related options are: ``header``, ``idx``, ``sym``, ``tag``
3238 and ``key``.
3239
3240 In the following example, default output configuration for the :subcmd:`avail`
3241 sub-command is checked then module tags and key section are removed to get a
3242 simpler output:
3243
3244 .. parsed-literal::
3245
3246 :ps:`$` ml config avail_output
3247 Modules Release 4.7.0 (2021-02-19)
3248
3249 - Config. name ---------.- Value (set by if default overridden) ---------------
3250 avail_output modulepath:alias:dirwsym:sym:tag:key
3251 :ps:`$` ml av
3252 ------------------ :sgrdi:`/path/to/modulefiles` ------------------
3253 :sgrde:`bar/1.0` bar/2.0 :sgrf:`foo/1.0` :sgrs:`foo/2.0` :sgrali:`foo/2.2`
3254
3255 Key:
3256 :sgrdi:`modulepath` :sgrali:`module-alias` :sgrs:`sticky`
3257 :sgrde:`default-version` :sgrf:`forbidden`
3258 :ps:`$` ml config avail_output modulepath:alias:dirwsym:sym
3259 :ps:`$` ml av
3260 ------------------ :sgrdi:`/path/to/modulefiles` ------------------
3261 :sgrde:`bar/1.0` bar/2.0 foo/1.0 foo/2.0 :sgrali:`foo/2.2`
3262
3263 The :option:`--output`/:option:`-o` switches are added to define a specific
3264 output configuration for the duration of the associated command line. The
3265 following example shows how to limit the content reported on a module
3266 :subcmd:`list` to the loaded index and the symbolic versions in addition to
3267 the module names:
3268
3269 .. parsed-literal::
3270
3271 :ps:`$` ml
3272 Currently Loaded Modulefiles:
3273 1) :sgrde:`bar/1.0` 2) :sgrs:`foo/2.0`
3274
3275 Key:
3276 :sgrde:`default-version` :sgrs:`sticky`
3277 :ps:`$` ml -o idx:sym
3278 1) :sgrde:`bar/1.0` 2) foo/2.0
3279
3280 When the new configuration options or command line switches are set to an
3281 empty value, the module names are the sole information reported:
3282
3283 .. parsed-literal::
3284
3285 :ps:`$` ml -t -o ""
3286 bar/1.0
3287 foo/2.0
3288
3289 In case the ``modulepath`` element is withdrawn from the :subcmd:`avail`
3290 sub-command output configuration, the available modules from all enabled
3291 modulepaths are reported as a single list:
3292
3293 .. parsed-literal::
3294
3295 :ps:`$` ml av
3296 --------------- :sgrdi:`/path/to/other/modulefiles` ---------------
3297 baz/1.0 baz/2.0
3298
3299 ------------------ :sgrdi:`/path/to/modulefiles` ------------------
3300 :sgrde:`bar/1.0` bar/2.0 :sgrf:`foo/1.0` :sgrs:`foo/2.0` :sgrali:`foo/2.2`
3301
3302 Key:
3303 :sgrdi:`modulepath` :sgrali:`module-alias` :sgrs:`sticky`
3304 :sgrde:`default-version` :sgrf:`forbidden`
3305 :ps:`$` ml av --output=alias:tag
3306 bar/1.0 baz/1.0 :sgrf:`foo/1.0` :sgrali:`foo/2.2`
3307 bar/2.0 baz/2.0 :sgrs:`foo/2.0`
3308
3309 .. note:: The ``avail_report_dir_sym`` and ``avail_report_mfile_sym`` locked
3310 configuration options have been removed. Their behaviors can now be
3311 obtained by respectively adding the ``dirwsym`` and ``sym`` elements to the
3312 :mconfig:`avail_output` or :mconfig:`avail_terse_output` configuration
3313 options.
3314
3315 v4.6
3316 ----
3317
3318 This new version is backward-compatible with previous version 4 releases. It
3319 fixes bugs but also introduces new functionalities that are described in this
3320 section. See the :ref:`4.6 release notes<4.6 release notes>` for a complete
3321 list of the changes between Modules v4.5 and v4.6.
3322
3323 .. _sh-to-mod_sub-command:
3324
3325 sh-to-mod sub-command
3326 ^^^^^^^^^^^^^^^^^^^^^
3327
3328 The :subcmd:`sh-to-mod` sub-command is added to output as a modulefile content
3329 the environment changes done by the evaluation of a shell script passed as
3330 argument. :subcmd:`sh-to-mod` is especially useful for software providing a
3331 shell script for their enablement in shell session: it can convert these
3332 scripts into modulefiles.
3333
3334 Say for instance, a *foo* software has been installed and it provides a
3335 ``foo-setup.sh`` script to activate *foo* software in user environment:
3336
3337 .. parsed-literal::
3338
3339 :ps:`$` cat /path/to/foo-1.2/foo-setup.sh
3340 #!/bin/sh
3341 export FOOENV="$1"
3342 export PATH=/path/to/foo-1.2/bin:$PATH
3343 alias foo='foobin -q -l'
3344
3345 Calling ``module sh-to-mod`` on this shell script outputs the environment
3346 changes it performs as a modulefile content:
3347
3348 .. parsed-literal::
3349
3350 :ps:`$` module sh-to-mod sh /path/to/foo-1.2/foo-setup.sh arg1
3351 #%Module
3352 prepend-path PATH /path/to/foo-1.2/bin
3353 set-alias foo {foobin -q -l}
3354 setenv FOOENV arg1
3355
3356 Changes on environment variables, shell aliases, shell functions and current
3357 working directory are tracked. The following shells are supported: *sh*,
3358 *dash*, *csh*, *tcsh*, *bash*, *ksh*, *ksh93*, *zsh* and *fish*.
3359
3360 :subcmd:`sh-to-mod` acts as a full replacement for the standalone
3361 :command:`createmodule.sh` and :command:`createmodule.py` scripts. However
3362 those two scripts are currently still provided for compatibility purpose.
3363
3364 .. _source-sh_modulefile_command:
3365
3366 source-sh modulefile command
3367 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3368
3369 The :mfcmd:`source-sh` modulefile command is introduced to source environment
3370 changes done by the evaluation of a shell script passed as argument. With
3371 newly introduced :subcmd:`sh-to-mod` sub-command resulting environment changes
3372 done by script are output as modulefile commands. :mfcmd:`source-sh` applies
3373 those modulefile commands as if they were directly written in loading
3374 modulefile.
3375
3376 :mfcmd:`source-sh` is useful for software providing a shell script for their
3377 enablement. If you want to enable such software with :command:`module` yet
3378 using shell script provided by software for this task, just write a modulefile
3379 using :mfcmd:`source-sh` command to call the shell script.
3380
3381 Keeping the same example used to describe :subcmd:`sh-to-mod` sub-command:
3382 *foo* software provides a ``foo-setup.sh`` script for its activation. Create a
3383 modulefile ``foo/1.2`` that calls this script:
3384
3385 .. parsed-literal::
3386
3387 :ps:`$` cat /path/to/modulefiles/foo/1.2
3388 #%Module4.6
3389 source-sh sh /path/to/foo-1.2/foo-setup.sh arg1
3390
3391 Displaying this modulefile indicates the environment changes done by script:
3392
3393 .. parsed-literal::
3394
3395 :ps:`$` module display foo/1.2
3396 -------------------------------------------------------------------
3397 :sgrhi:`/path/to/modulefiles/foo/1.2`:
3398
3399 :sgrcm:`prepend-path` PATH /path/to/foo-1.2/bin
3400 :sgrcm:`set-alias` foo {foobin -q -l}
3401 :sgrcm:`setenv` FOOENV arg1
3402 -------------------------------------------------------------------
3403
3404 Loading the modulefile applies the environment changes seen above:
3405
3406 .. parsed-literal::
3407
3408 :ps:`$` module load -v foo/1.2
3409 Loading :sgrhi:`foo/1.2`
3410 :ps:`$` echo $FOOENV
3411 arg1
3412 :ps:`$` alias foo
3413 alias foo='foobin -q -l'
3414
3415 Track of these changes is kept in user environment to be able to undo them
3416 when modulefile is unloaded:
3417
3418 .. parsed-literal::
3419
3420 :ps:`$` module unload -v foo/1.2
3421 Unloading :sgrhi:`foo/1.2`
3422 :ps:`$` echo $FOOENV
3423
3424 :ps:`$` alias foo
3425 bash: alias: foo: not found
3426
3427 Changes on environment variables, shell aliases, shell functions and current
3428 working directory are tracked. The following shells are supported: *sh*,
3429 *dash*, *csh*, *tcsh*, *bash*, *ksh*, *ksh93*, *zsh* and *fish*.
3430
3431 Querying user's name and groups membership
3432 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3433
3434 Two new sub-commands are introduced for the :mfcmd:`module-info` modulefile
3435 command: ``username`` and ``usergroups``. They respectively fetch the name of
3436 the user currently running :file:`modulecmd.tcl` or the name of all the groups
3437 this user is member of.
3438
3439 These two new modulefile commands can help to adapt code to specific users or
3440 groups. Like for instance to instantiate a modulefile for each group the user
3441 is member of:
3442
3443 .. parsed-literal::
3444
3445 :ps:`$` cat /path/to/modulefiles/foo/.modulerc
3446 #%Module4.6
3447 foreach grp [module-info usergroups] {
3448 module-virtual foo/$grp .common
3449 }
3450 :ps:`$` id -G -n
3451 grp1 grp2 grp3
3452 :ps:`$` module avail
3453 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3454 foo/grp1 foo/grp2 foo/grp3
3455
3456 ``username`` and ``usergroups`` sub-commands of :mfcmd:`module-info`
3457 modulefile command are only supported on Unix platform.
3458
3459 Hiding modules
3460 ^^^^^^^^^^^^^^
3461
3462 The newly introduced :mfcmd:`module-hide` modulefile command enables to
3463 dynamically hide modulefiles, module aliases or symbolic versions specified to
3464 it:
3465
3466 .. parsed-literal::
3467
3468 :ps:`$` cat /path/to/modulefiles/bar/.modulerc
3469 #%Module4.6
3470 module-version bar/1.0 old
3471 # hide 'old' symbolic version
3472 module-hide bar/old
3473 # hide all version 2 and above
3474 module-hide bar@2:
3475 :ps:`$` cat /path/to/modulefiles/.modulerc
3476 #%Module4.6
3477 # hide all versions of foo module
3478 module-hide foo
3479
3480 :mfcmd:`module-hide` commands should be placed in module rc files and can
3481 leverage the `Advanced module version specifiers`_ syntax as shown in the
3482 above example.
3483
3484 Hidden modules are excluded from available module search or module selection
3485 unless query refers to hidden module by its exact name:
3486
3487 .. parsed-literal::
3488
3489 :ps:`$` ml av
3490 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3491 bar/1.0 bar/2.0
3492 :ps:`$` module load -v foo
3493 :sgrer:`ERROR`: Unable to locate a modulefile for 'foo'
3494 :ps:`$` module load -v foo/1.0
3495 Loading foo/1.0
3496 :ps:`$` module avail bar/old
3497 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3498 bar/1.0\ :sgrse:`(`:sgrsy:`old`:sgrse:`)`
3499
3500 :mfcmd:`module-hide` command accepts a ``--soft`` option to apply a lighter of
3501 hiding to modules:
3502
3503 .. parsed-literal::
3504
3505 :ps:`$` cat /path/to/modulefiles/qux/.modulerc
3506 #%Module4.6
3507 # softly hide all qux modules
3508 module-hide --soft qux
3509
3510 The soft hiding mode enables to hide modules from full availability listing
3511 yet keeping the ability to select such module for load without having to use
3512 module exact name:
3513
3514 .. parsed-literal::
3515
3516 :ps:`$` ml av
3517 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3518 bar/1.0 bar/2.0
3519 :ps:`$` ml av qux
3520 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3521 qux/1.0 qux/2.0
3522 :ps:`$` module load -v qux
3523 Loading :sgrhi:`qux/2.0`
3524
3525 Alternatively, a ``--hard`` option can be set on :mfcmd:`module-hide` command
3526 to ensure designated modules do not unveil even if referred by their exact
3527 name:
3528
3529 .. parsed-literal::
3530
3531 :ps:`$` cat /path/to/modulefiles/qux/.modulerc
3532 #%Module4.6
3533 # softly hide all qux modules
3534 module-hide --soft qux
3535 # set highest version of qux hard hidden
3536 module-hide --hard qux/3.0
3537 :ps:`$` ml av qux/3.0
3538 :ps:`$` ml qux/3.0
3539 :sgrer:`ERROR`: Unable to locate a modulefile for 'qux/3.0'
3540
3541 Some users or groups can be set unaffected by hiding mechanism with
3542 the ``--not-user`` or ``--not-group`` options:
3543
3544 .. parsed-literal::
3545
3546 :ps:`$` cat /path/to/modulefiles/quuz/.modulerc
3547 #%Module4.6
3548 # hiding does not apply to grp1 and grp2 groups
3549 module-hide --not-group {grp1 grp2} quuz
3550
3551 .. parsed-literal::
3552
3553 :ps:`$` id --groups --name
3554 grp1 grp7
3555 :ps:`$` ml av quuz
3556 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3557 quuz/1.0 quuz/2.0
3558 :ps:`$` ml -v quuz
3559 Loading :sgrhi:`quuz/2.0`
3560
3561 Hiding mechanism can also be set effective only before or after a given date
3562 time with the ``--before`` and ``--after`` options. Accepted date time format
3563 is ``YYYY-MM-DD[THH:MM]``.
3564
3565 .. parsed-literal::
3566
3567 :ps:`$` cat /path/to/modulefiles/fum/.modulerc
3568 #%Module4.6
3569 # hide only before a given date
3570 module-hide --hard --before 2020-09-01T12:00 fum/1.0
3571 # hide only after a given date
3572 module-hide --hard --after 2020-09-01 fum/2.0
3573
3574 .. parsed-literal::
3575
3576 :ps:`$` date
3577 Fri 04 Sep 2020 06:21:48 AM CEST
3578 :ps:`$` ml av fum
3579 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3580 fum/1.0
3581
3582 Hidden modules can be included in available module searches if option
3583 :option:`--all`/:option:`-a` is set on :subcmd:`avail`, :subcmd:`aliases`,
3584 :subcmd:`whatis` or :subcmd:`search` sub-commands. Hard hidden modules are
3585 unaffected by this option and stay hidden.
3586
3587 .. parsed-literal::
3588
3589 :ps:`$` ml av -a
3590 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3591 bar/1.0\ :sgrse:`(`:sgrsy:`old`:sgrse:`)` foo/1.0 fum/1.0 quuz/2.0 qux/2.0
3592 bar/2.0 foo/2.0 quuz/1.0 qux/1.0
3593
3594 Forbidding use of modules
3595 ^^^^^^^^^^^^^^^^^^^^^^^^^
3596
3597 The :mfcmd:`module-forbid` modulefile command is added to dynamically forbid
3598 the evaluation of modulefiles it specifies. When forbidden, a module cannot be
3599 loaded and an access error is returned when an attempt is made to evaluate it.
3600
3601 .. parsed-literal::
3602
3603 :ps:`$` cat /path/to/modulefiles/foo/.modulerc
3604 #%Module4.6
3605 module-forbid foo@1:
3606 :ps:`$` ml foo/1.0
3607 :sgrer:`ERROR`: Access to module 'foo/1.0' is denied
3608 :ps:`$` ml
3609 No Modulefiles Currently Loaded.
3610
3611 :mfcmd:`module-forbid` statements can be coupled with :mfcmd:`module-hide`
3612 statements to hide modules in addition to forbid their use.
3613 :mfcmd:`module-forbid` supports the ``--not-user``, ``--not-group``,
3614 ``--before`` and ``--after`` options to still allow some users or forbid
3615 modules before or after a given date time.
3616
3617 An additional error message can be defined with the ``--message`` option
3618 to guide for instance users when they try to evaluate a forbidden module:
3619
3620 .. parsed-literal::
3621
3622 :ps:`$` cat /path/to/modulefiles/bar/.modulerc
3623 #%Module4.6
3624 module-forbid --message {Software bar/1.0 is decommissioned, please now use\\
3625 bar/2.0} --after 2020-09-01 bar/1.0
3626 :ps:`$` ml bar/1.0
3627 :sgrer:`ERROR`: Access to module 'bar/1.0' is denied
3628 Software bar/1.0 is decommissioned, please now use bar/2.0
3629
3630 When an evaluated module will soon be forbidden, a message is returned to the
3631 user to warn them of the near limit. An additional warning message can also be
3632 defined here with the ``--nearly-message`` option to guide users.
3633
3634 .. parsed-literal::
3635
3636 :ps:`$` cat /path/to/modulefiles/qux/.modulerc
3637 #%Module4.6
3638 module-forbid --nearly-message {Version 1.0 will soon expire, please now use\\
3639 version 2.0} --after 2020-09-15 qux/1.0
3640 :ps:`$` date
3641 Tue 08 Sep 2020 06:49:43 AM CEST
3642 :ps:`$` ml qux/1.0
3643 Loading :sgrhi:`qux/1.0`
3644 :sgrwa:`WARNING`: Access to module will be denied starting '2020-09-15'
3645 Version 1.0 will soon expire, please now use version 2.0
3646
3647 The range of time the *nearly forbidden* warning appears can be controlled
3648 with the :mconfig:`nearly_forbidden_days` configuration option, whose value
3649 equals to the number of days prior the module starts to be forbidden. This
3650 configuration is set to ``14`` (days) by default and this value can be
3651 controlled at :file:`configure` time with
3652 :instopt:`--with-nearly-forbidden-days` option. When the
3653 :mconfig:`nearly_forbidden_days` configuration is set through the
3654 :subcmd:`config` sub-command, the :envvar:`MODULES_NEARLY_FORBIDDEN_DAYS`
3655 environment variable is set.
3656
3657 Tracing module execution
3658 ^^^^^^^^^^^^^^^^^^^^^^^^
3659
3660 The ``trace`` verbosity is introduced between the ``verbose`` and ``debug``
3661 levels to report details on module searches, resolutions, selections and
3662 evaluations. Trace mode can be enabled by setting the :mconfig:`verbosity`
3663 config to the ``trace`` value or by using the :option:`-T`/:option:`--trace`
3664 command-line switches.
3665
3666 To specifically render trace messages, the ``tr`` key is added to the color
3667 palette with a default value of ``2`` (decreased intensity).
3668
3669 .. parsed-literal::
3670
3671 :ps:`$` ml -T foo
3672 :sgrtr:`Evaluate modulerc: '/path/to/modulefiles/.modulerc'`
3673 :sgrtr:`Get modules: {foo} matching 'foo' in '/path/to/modulefiles'`
3674 :sgrtr:`Resolve: 'foo' into 'bar'`
3675 :sgrtr:`Get modules: {bar bar/1.0} matching 'bar' in '/path/to/modulefiles'`
3676 :sgrtr:`Select module: 'bar/1.0' (/path/to/modulefiles/bar/1.0) matching 'bar/1.0'`
3677
3678 Loading :sgrhi:`bar/1.0`
3679 :sgrtr:`Evaluate modulefile: '/path/to/modulefiles/bar/1.0' as 'bar/1.0'`
3680
3681
3682 v4.5
3683 ----
3684
3685 This new version is backward-compatible with previous version 4 releases. It
3686 fixes bugs but also introduces new functionalities that are described in this
3687 section. See the :ref:`4.5 release notes<4.5 release notes>` for a complete
3688 list of the changes between Modules v4.4 and v4.5.
3689
3690 ml command
3691 ^^^^^^^^^^
3692
3693 The ``ml`` command is added to Modules. ``ml`` is a frontend to the ``module``
3694 command that reduces the number of characters to type to trigger module
3695 actions.
3696
3697 With no argument provided ``ml`` is equivalent to ``module list``, ``ml foo``
3698 corresponds to ``module load foo`` and ``ml -foo`` means ``module unload
3699 foo``:
3700
3701 .. parsed-literal::
3702
3703 :ps:`$` ml foo
3704 :ps:`$` ml
3705 Currently Loaded Modulefiles:
3706 1) foo/2
3707 :ps:`$` ml -foo
3708 :ps:`$` ml
3709 No Modulefiles Currently Loaded.
3710
3711 Multiple modules to either load or unload can be combined on a single command.
3712 The unloads are first processed then the loads.
3713
3714 ``ml`` accepts all command-line switches and sub-commands accepted by
3715 ``module`` command:
3716
3717 .. parsed-literal::
3718
3719 :ps:`$` ml avail -t foo
3720 foo/1
3721 foo/2
3722
3723 This handy interface has been originally developed by the `Lmod`_ project.
3724 Having this command line interface also supported on Modules helps to provide
3725 a similar user experience whatever the module implementation used.
3726
3727 .. _Lmod: https://github.com/TACC/Lmod
3728
3729 JSON format output
3730 ^^^^^^^^^^^^^^^^^^
3731
3732 The ``-j`` and ``--json`` command line switches are added for the ``avail``,
3733 ``list``, ``savelist``, ``whatis`` and ``search`` module sub-commands. When
3734 set, the output result of these sub-commands is rendered in `JSON`_ format:
3735
3736 .. parsed-literal::
3737
3738 :ps:`$` module avail --json bar | python -mjson.tool
3739 {
3740 "/path/to/modulefiles": {
3741 "bar/2.3": {
3742 "name": "bar/2.3",
3743 "pathname": "/path/to/modulefiles/bar/2.3",
3744 "symbols": [
3745 "default"
3746 ],
3747 "type": "modulefile"
3748 },
3749 "bar/3.4": {
3750 "name": "bar/3.4",
3751 "pathname": "/path/to/modulefiles/bar/3.4",
3752 "symbols": [],
3753 "type": "modulefile"
3754 }
3755 }
3756 }
3757 :ps:`$` ml whatis -j foo/1.2.3 | python -mjson.tool
3758 {
3759 "/path/to/modulefiles": {
3760 "foo/1.2.3": {
3761 "name": "foo/1.2.3",
3762 "whatis": [
3763 "The foo/1.2.3 modulefile"
3764 ]
3765 }
3766 }
3767 }
3768
3769 .. _JSON: https://tools.ietf.org/html/rfc8259
3770
3771 Improved Windows support
3772 ^^^^^^^^^^^^^^^^^^^^^^^^
3773
3774 A new option to the ``./configure`` script named
3775 :instopt:`--enable-windows-support` is introduced to install additional files
3776 relative to the enablement of Modules on the Windows platform. When set, this
3777 option installs ``module.cmd``, ``ml.cmd`` and ``envml.cmd`` scripts in
3778 ``bindir`` and initialization script ``cmd.cmd`` in ``initdir``. With these
3779 four files the Modules installation may be used from either a Unix or a
3780 Windows platform.
3781
3782 ``module.cmd``, ``ml.cmd`` and ``envml.cmd`` scripts respectively provide the
3783 ``module``, ``ml`` and ``envml`` commands for Windows ``cmd`` terminal shell,
3784 relying on ``modulecmd.tcl`` script which was already able to produce shell
3785 code for this Windows shell. Initialization script ``cmd.cmd`` adds the
3786 directory of ``module.cmd``, ``ml.cmd`` and ``envml.cmd`` to ``PATH``.
3787
3788 These Windows-specific files are relocatable: ``module.cmd``, ``ml.cmd`` and
3789 ``envml.cmd`` scripts expect to find initialization script ``cmd.cmd`` in the
3790 ``init`` directory next to them (to setup Modules-specific variables in
3791 current environment) and ``cmd.cmd`` expects ``modulecmd.tcl`` to be found in
3792 ``libexec`` directory and the 3 commands in ``bin`` directory next to it.
3793
3794 Starting from this ``4.5`` release a distribution zipball is published to
3795 install Modules on Windows. This zip archive ships an install and an uninstall
3796 scripts (``INSTALL.bat`` and ``UNINSTALL.bat``). The zipball can be built
3797 locally from Modules sources by running ``make dist-win``.
3798
3799 The :ref:`INSTALL-win` document describes how to install Modules on Windows
3800 from the distribution zipball.
3801
3802 Error stack trace
3803 ^^^^^^^^^^^^^^^^^
3804
3805 Error messages will now embed a stack trace for unknown errors to help
3806 localize the root cause of issues. This change applies to modulefile
3807 evaluation:
3808
3809 .. parsed-literal::
3810
3811 Loading :sgrhi:`foo/1.2`
3812 :sgrme:`Module ERROR`: add-path cannot handle path equals to separator string
3813 while executing
3814 "append-path PATH :"
3815 (file "/path/to/modulefiles/foo/1.2" line 24)
3816 Please contact <:noparse:`root@localhost`>
3817
3818 A stack trace is also returned when an unknown error occurs in
3819 ``modulecmd.tcl`` script, which facilitates issue report and analysis:
3820
3821 .. parsed-literal::
3822
3823 :ps:`$` module load bar
3824 :sgrer:`ERROR`: invalid command name "badcommand"
3825 while executing
3826 "badcommand"
3827 (procedure "module" line 14)
3828 invoked from within
3829 "module load bar"
3830 ("eval" body line 1)
3831 invoked from within
3832 "eval $execcmdlist"
3833 Please report this issue at :noparse:`https://github.com/envmodules/modules/issues`
3834
3835 .. _Automatic default and latest syms:
3836
3837 Automatic default and latest symbolic versions
3838 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3839
3840 When the implicit default mechanism and the `Advanced module version
3841 specifiers`_ are both enabled, a ``default`` and a ``latest`` symbolic
3842 versions are automatically defined for each module name.
3843
3844 This new feature gives the ability to select the highest version available for
3845 a module, without knowing beforehand this version name:
3846
3847 .. parsed-literal::
3848
3849 :ps:`$` module load -v :noparse:`foo@latest`
3850 Loading foo/1.10
3851
3852 The symbolic versions are automatically defined unless a symbolic version, an
3853 alias or a regular module version already exists for these ``default`` or
3854 ``latest`` version names.
3855
3856
3857 v4.4
3858 ----
3859
3860 This new version is backward-compatible with previous version 4 releases. It
3861 fixes bugs but also introduces new functionalities that are described in this
3862 section. See the :ref:`4.4 release notes<4.4 release notes>` for a complete
3863 list of the changes between Modules v4.3 and v4.4.
3864
3865 .. warning:: Modules configuration option handling has been reworked
3866 internally to provide a unified way for all options to get initialized,
3867 retrieved or set. Existing site-specific configuration script should be
3868 reviewed to make use of the new ``getConf``, ``setConf``, ``unsetConf``
3869 and ``lappendConf`` procedures to manipulate configuration options.
3870
3871 .. _Insensitive case:
3872
3873 Specify modules in a case insensitive manner
3874 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3875
3876 The ability to match module name in a case insensitive manner has been added.
3877 This feature can be enabled at different level with the following values set
3878 to the ``icase`` configuration option:
3879
3880 * ``never``: a case sensitive match is applied in any cases
3881 * ``search``: a case insensitive match is applied to the ``avail``, ``whatis``
3882 and ``paths`` sub-commands
3883 * ``always``: a case insensitive match is applied to search contexts and also
3884 to the other module sub-commands and modulefile Tcl commands for the module
3885 specification they receive as argument.
3886
3887 It can help for instance to load a module without knowing the case used to
3888 name its relative modulefile:
3889
3890 .. parsed-literal::
3891
3892 :ps:`$` module config icase always
3893 :ps:`$` module load -v mysoftware
3894 Loading :sgrhi:`MySoftware/1.0`
3895
3896 Insensitive case match activation can be controlled at configure time with
3897 the ``--with-icase`` option, which could be passed any of the above activation
3898 levels. This option could be superseded with the ``MODULES_ICASE`` environment
3899 variable, which could be set through the **config** sub-command with the
3900 ``icase`` option. Command-line switch **--icase** supersedes in turns any
3901 other icase configurations. When this command-line switch is passed, ``icase``
3902 mode equals ``always``.
3903
3904 Extended default
3905 ^^^^^^^^^^^^^^^^
3906
3907 The extended default mechanism has been introduced to help selecting a module
3908 when only the first numbers in its version are specified. Starting portion of
3909 the version, part separated from the rest of the version string by a ``.``
3910 character, could be used to refer to a more precise version number.
3911
3912 This mechanism is activated through the new configuration option
3913 ``extended_default``. It enables to refer to a module named ``foo/1.2.3`` as
3914 ``foo/1.2`` or ``foo/1``:
3915
3916 .. parsed-literal::
3917
3918 :ps:`$` module config extended_default 1
3919 :ps:`$` module load -v foo/1
3920 Loading :sgrhi:`foo/1.2.3`
3921
3922 When multiple versions match partial version specified and only one module
3923 should be selected, the default version (whether implicitly or explicitly
3924 defined) among matches is returned. The following example shows that
3925 ``foo/1.1.1``, the *foo* module default version, is selected when it matches
3926 query. Elsewhere the highest version (also called the latest version or the
3927 implicit default) among matching modules is returned:
3928
3929 .. parsed-literal::
3930
3931 :ps:`$` module av foo
3932 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3933 foo/1.1.1\ :sgrse:`(`:sgrsy:`default`:sgrse:`)` foo/1.2.1 foo/1.10
3934 foo/1.1.10 foo/1.2.3
3935 :ps:`$` module load -v foo/1.1
3936 Loading :sgrhi:`foo/1.1.1`
3937 :ps:`$` module purge
3938 :ps:`$` module load -v foo/1.2
3939 Loading :sgrhi:`foo/1.2.3`
3940 :ps:`$` module purge
3941 :ps:`$` module load -v foo/1
3942 Loading :sgrhi:`foo/1.1.1`
3943
3944 In case ``implicit_default`` option is disabled and no explicit default is
3945 found among matches, an error is returned:
3946
3947 .. parsed-literal::
3948
3949 :ps:`$` module config implicit_default 0
3950 :ps:`$` module load -v foo/1.2
3951 :sgrer:`ERROR`: No default version defined for 'foo/1.2'
3952
3953 When it is enabled, extended default applies everywhere a module could be
3954 specified, which means it could be used with any module sub-command or any
3955 modulefile Tcl command receiving a module specification as argument. It may
3956 help for instance to declare dependencies between modules:
3957
3958 .. parsed-literal::
3959
3960 :ps:`$` module show bar/3
3961 ----------------------------------------------------------
3962 :sgrhi:`/path/to/modulefiles/bar/3.4`:
3963
3964 :sgrcm:`prereq` foo/1.2
3965 ----------------------------------------------------------
3966 :ps:`$` module load --auto bar/3
3967 Loading :sgrhi:`bar/3.4`
3968 :sgrin:`Loading requirement`: foo/1.2.3
3969
3970 Extended default activation can be controlled at configure time with the
3971 ``--enable-extended-default`` option. This option could be superseded with the
3972 ``MODULES_EXTENDED_DEFAULT`` environment variable, which could be set through
3973 the **config** sub-command with the ``extended_default`` option.
3974
3975 Advanced module version specifiers
3976 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3977
3978 The ability to specify finer constraints on module version has been added to
3979 Modules. It enables to filter the module selection to a given version list or
3980 range by specifying after the module name a version constraint prefixed by the
3981 ``@`` character.
3982
3983 This new feature leverages the `version specifier syntax`_ of the `Spack`_
3984 package manager as this syntax covers all the needs for a fine-grained
3985 selection of module versions. It copes very well with command-line typing, by
3986 avoiding characters having a special meaning on shells. Moreover the users of
3987 Spack that also are users of Modules may already be familiar with this syntax.
3988
3989 .. _version specifier syntax: https://spack.readthedocs.io/en/latest/basic_usage.html#version-specifier
3990 .. _Spack: https://github.com/spack/spack
3991
3992 The mechanism introduced here is called *advanced module version specifier*
3993 and it can be activated through the new configuration option
3994 ``advanced_version_spec``. Constraints can be expressed to refine the
3995 selection of module version to:
3996
3997 * a single version with the ``@version`` syntax, for instance ``[email protected]``
3998 syntax will select module ``foo/1.2.3``
3999 * a list of versions with the ``@version1,version2,...`` syntax, for instance
4000 ``[email protected],1.10`` will match modules ``foo/1.2.3`` and ``foo/1.10``
4001 * a range of versions with the ``@version1:``, ``@:version2`` and
4002 ``@version1:version2`` syntaxes, for instance ``[email protected]:`` will select all
4003 versions of module ``foo`` greater than or equal to ``1.2``, ``foo@:1.3``
4004 will select all versions less than or equal to ``1.3`` and ``[email protected]:1.3``
4005 matches all versions between ``1.2`` and ``1.3`` including ``1.2`` and
4006 ``1.3`` versions
4007
4008 This new feature enables for instance to list available versions of module
4009 ``foo`` higher or equal to ``1.2``:
4010
4011 .. parsed-literal::
4012
4013 :ps:`$` module config advanced_version_spec 1
4014 :ps:`$` module av foo
4015 --------------- :sgrdi:`/path/to/modulefiles` ---------------
4016 foo/1.1.1\ :sgrse:`(`:sgrsy:`default`:sgrse:`)` foo/1.2.1 foo/1.10
4017 foo/1.1.10 foo/1.2.3
4018 :ps:`$` module av :noparse:`[email protected]:`
4019 --------------- :sgrdi:`/path/to/modulefiles` ---------------
4020 foo/1.2.1 foo/1.2.3 foo/1.10
4021
4022 Then choose to load for instance a version higher than or equal to ``1.2`` and
4023 less than or equal to ``1.3``. Default version is selected if it corresponds
4024 to a version included in the range, elsewhere the highest version (also called
4025 latest version or implicit default) is selected:
4026
4027 .. parsed-literal::
4028
4029 :ps:`$` module load -v :noparse:`[email protected]:1.3`
4030 Loading :sgrhi:`foo/1.2.3`
4031
4032 In case ``implicit_default`` option is disabled and no explicit default is
4033 found among version specifier matches, an error is returned:
4034
4035 .. parsed-literal::
4036
4037 :ps:`$` module config implicit_default 0
4038 :ps:`$` module load -v :noparse:`[email protected]:1.3`
4039 :sgrer:`ERROR`: No default version defined for ':noparse:`[email protected]:1.3`'
4040
4041 When advanced module version specifier is enabled, it applies everywhere a
4042 module could be specified, which means it could be used with any module
4043 sub-command or any modulefile Tcl command receiving a module specification
4044 as argument. It may help for instance to declare smoother dependencies between
4045 modules:
4046
4047 .. parsed-literal::
4048
4049 :ps:`$` module show bar@:2
4050 ----------------------------------------------------------
4051 :sgrhi:`/path/to/modulefiles/bar/2.3`:
4052
4053 :sgrcm:`prereq` :noparse:`[email protected],1.2.1`
4054 ----------------------------------------------------------
4055 :ps:`$` module load --auto bar@:2
4056 Loading :sgrhi:`bar/2.3`
4057 :sgrin:`Loading requirement`: foo/1.2.1
4058
4059 Advanced specification of single version or list of versions may benefit from
4060 the activation of the `Extended default`_ mechanism (range of versions
4061 natively handles abbreviated versions):
4062
4063 .. parsed-literal::
4064
4065 :ps:`$` module config extended_default 1
4066 :ps:`$` module load -v :noparse:`[email protected]`
4067 Loading :sgrhi:`foo/1.2.3`
4068 :ps:`$` module unload -v foo @1.2,1.5
4069 Unloading :sgrhi:`foo/1.2.3`
4070
4071 Advanced module version specifier activation can be controlled at configure
4072 time with the ``--enable-advanced-version-spec`` option. This option could be
4073 superseded with the ``MODULES_ADVANCED_VERSION_SPEC`` environment variable,
4074 which could be set through the **config** sub-command with the
4075 ``advanced_version_spec`` option.
4076
4077
4078 v4.3
4079 ----
4080
4081 This new version is backward-compatible with previous version 4 releases. It
4082 fixes bugs but also introduces new functionalities that are described in this
4083 section. See the :ref:`4.3 release notes<4.3 release notes>` for a complete
4084 list of the changes between Modules v4.2 and v4.3.
4085
4086 Modulepath rc file
4087 ^^^^^^^^^^^^^^^^^^
4088
4089 A ``.modulerc`` file found at the root of an enabled modulepath directory is
4090 now evaluated when modulepath is walked through to locate modulefiles. This
4091 modulepath rc file gives for instance the ability to define module alias whose
4092 name does not correspond to any module directory in this modulepath. Thus this
4093 kind of module alias would not be found unless if it is defined at the
4094 modulepath global scope.
4095
4096 Further I/O operations optimization
4097 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4098
4099 Additional work has been performed to save a significant number of filesystem
4100 I/O operations made to search and evaluate modulefiles.
4101
4102 When fully read, the content of a modulefile is now cached in memory to avoid
4103 new I/O operations in case this modulefile should be read one more time during
4104 the same module command evaluation.
4105
4106 Except for ``path``, ``paths``, ``list``, ``avail`` and ``aliases`` module
4107 commands always fully read a modulefile whether its full content is needed or
4108 just its header to verify its validity. This way modulefiles are only read
4109 once on commands that first check modulefile validity then read again valid
4110 files to get their full content.
4111
4112 Last but not least, Modules Tcl extension library is introduced to extend the
4113 Tcl language in order to provide more optimized I/O commands to read a file or
4114 a directory content than native Tcl commands do. This library is built and
4115 enabled in ``modulecmd.tcl`` script with ``--enable-libtclenvmodules``
4116 configure argument (it is enabled by default). As this library is written in
4117 C, it must be compiled and ``--with-tcl`` or ``--with-tclinclude`` configure
4118 arguments may be used to indicate where to find Tcl development files.
4119
4120 Modules Tcl extension library greatly reduces the number of filesystem I/O
4121 operations by removing unneeded ``ioctl``, ``fcntl`` and ``lstat`` system
4122 calls done (by Tcl ``open`` command) to read each modulefile. Directory
4123 content read is also improved by fetching hidden and regular files in one
4124 pass. Moreover ``.modulerc`` and ``.version`` read access is tested only if
4125 these files are found in the directory.
4126
4127 Colored output
4128 ^^^^^^^^^^^^^^
4129
4130 The ability to graphically enhance some part of the produced output has been
4131 added to improve readability. Among others, error, warning and info message
4132 prefixes can be colored as well as modulepath, module alias and symbolic
4133 version.
4134
4135 Color mode can be set to ``never``, ``auto`` or ``always``. When color mode is
4136 set to ``auto``, output is colored only if the standard error output channel
4137 is attached to a terminal.
4138
4139 Default color mode could be controlled at configure time with the
4140 ``--enable-color`` and the ``--disable-color`` option, which respectively
4141 correspond to the ``auto`` and ``never`` color mode. This default mode could
4142 be superseded with the ``CLICOLOR``, ``CLICOLOR_FORCE`` and ``MODULES_COLOR``
4143 environment variables and the ``--color`` command-line switch.
4144
4145 Color to apply to each element can be controlled with the ``MODULES_COLORS``
4146 environment variable or the ``--with-dark-background-colors`` and
4147 ``--with-light-background-colors`` configure options. These variable and
4148 options take as value a colon-separated list in the same fashion ``LS_COLORS``
4149 does. In this list, output item that should be highlighted is designated by
4150 a key which is associated to a `Select Graphic Rendition (SGR) code`_.
4151
4152 .. _Select Graphic Rendition (SGR) code: https://en.wikipedia.org/wiki/ANSI_escape_code#Select_Graphic_Rendition_parameters
4153
4154 The ``MODULES_TERM_BACKGROUND`` environment variable and the
4155 ``--with-terminal-background`` configure option help Modules to determine if
4156 the color set for dark background or the color set for light background should
4157 be used to color output in case no specific color set is defined with the
4158 ``MODULES_COLORS`` variable.
4159
4160 Output items able to be colorized and their relative key are: highlighted
4161 element (``hi``), debug information (``db``), tag separator (``se``); Error
4162 (``er``), warning (``wa``), module error (``me``) and info (``in``) message
4163 prefixes; Modulepath (``mp``), directory (``di``), module alias (``al``),
4164 module symbolic version (``sy``), module ``default`` version (``de``) and
4165 modulefile command (``cm``).
4166
4167 For instance the default color set for a terminal with dark background is
4168 defined to:
4169
4170 .. parsed-literal::
4171
4172 :noparse:`hi=1:db=2:se=2:er=91:wa=93:me=95:in=94:mp=1;94:di=94:al=96:sy=95:de=4:cm=92`
4173
4174 When colored output is enabled and a specific graphical rendition is defined
4175 for module *default* version, the ``default`` symbol is omitted and instead
4176 the defined graphical rendition is applied to the relative modulefile. When
4177 colored output is enabled and a specific graphical rendition is defined for
4178 module alias, the ``@`` symbol is omitted.
4179
4180 ``CLICOLOR`` and ``CLICOLOR_FORCE`` environment variables are also honored to
4181 define color mode. The ``never`` mode is set if ``CLICOLOR`` equals to ``0``.
4182 If ``CLICOLOR`` is set to another value, it corresponds to the ``auto`` mode.
4183 The ``always`` mode is set if ``CLICOLOR_FORCE`` is set to a value different
4184 than ``0``. Color mode set with these two variables is superseded by mode set
4185 with ``MODULES_COLOR`` environment variable.
4186
4187 Configure modulecmd with config sub-command
4188 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4189
4190 The **config** sub-command has been added to ``module`` to help getting or
4191 setting the **modulecmd.tcl** options. With no additional command-line
4192 argument, this sub-command reports the current value of all existing options
4193 with a mention to indicate if this value has been overridden from a
4194 command-line switch or from an environment variable.
4195
4196 See the description of this sub-command in the :ref:`module(1)` man page for a
4197 complete reference on existing configuration options.
4198
4199 Most of the options can be altered by passing the option name and a value to
4200 the sub-command. Setting an option by this mean overrides its default value,
4201 set at installation time in **modulecmd.tcl** script, by defining the
4202 environment variable which supersedes this default.:
4203
4204 .. parsed-literal::
4205
4206 :ps:`$` module config auto_handling 1
4207 :ps:`$` module config auto_handling
4208 Modules Release 4.3.0 (2019-07-26)
4209
4210 - Config. name ---------.- Value (set by if default overridden) ---------------
4211 auto_handling 1 (env-var)
4212
4213 Setting options with ``module config`` could be done in the Modules
4214 initialization RC file to change default value of options when ``module``
4215 command is initialized.
4216
4217 When command-line switch ``--reset`` and an option name is passed to the
4218 **config** sub-command, it restores default value for configuration option by
4219 unsetting related environment variable.
4220
4221 With command-line switch ``--dump-state``, the **config** sub-command reports,
4222 in addition to currently set options, the current state of **modulecmd.tcl**
4223 script and Modules-related environment variables. Providing the output of the
4224 ``module config --dump-state`` command when submitting an issue to the Modules
4225 project will help to analyze the situation.
4226
4227 Control module command verbosity
4228 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4229
4230 The ability to control message verbosity has been added so ``module`` command
4231 can be configured whether it should display more or less information.
4232 Available verbosity levels from the least to the most verbose are:
4233
4234 * ``silent``: turn off error, warning and informational messages but does not
4235 affect module command output result.
4236 * ``concise``: enable error and warning messages but disable informational
4237 messages.
4238 * ``normal``: turn on informational messages, like a report of the additional
4239 module evaluations triggered by loading or unloading modules, aborted
4240 evaluation issues or a report of each module evaluation occurring during a
4241 **restore** or **source** sub-commands.
4242 * ``verbose``: add additional informational messages, like a systematic report
4243 of the loading or unloading module evaluations.
4244 * ``debug``: print debugging messages about module command execution.
4245
4246 Default verbosity level can be controlled at configure time with the
4247 ``--with-verbosity`` option, which could be passed any of the above level
4248 names. This default verbosity level could be superseded with the
4249 ``MODULES_VERBOSITY`` environment variable, which could be set through the
4250 **config** sub-command with the ``verbosity`` option. Command-line switches
4251 **--silent**, **--verbose** and **--debug** supersede in turns any other
4252 verbosity configuration to respectively set module command silent, verbose or
4253 in debug mode.
4254
4255 Other new sub-commands, command-line switches and environment variables
4256 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4257
4258 * The **avail** sub-command gets two new command-line switches: **--indepth**
4259 and **--no-indepth**. These options control whether search results should
4260 recursively include or not modulefiles from directories matching search
4261 query. Shell completion scripts have been updated to complete available
4262 modulefiles in the no in depth mode.
4263
4264 * The **MODULES_AVAIL_INDEPTH** environment variable defines if the **avail**
4265 sub-command should include or exclude by default the modulefiles from
4266 directories matching search query. Its value is superseded by the use of the
4267 **--indepth** and **--no-indepth** command-line switches.
4268
4269 * The **clear** sub-command, which was available on Modules version 3.2, has
4270 been reintroduced. This sub-command resets the Modules runtime information
4271 but does not apply further changes to the environment at all. This
4272 sub-command now leverages the **--force** command-line switch to skip its
4273 confirmation dialog.
4274
4275 * The **MODULES_SITECONFIG** environment variable defines an additional
4276 siteconfig script which is loaded if it exists after the siteconfig script
4277 configured at build time in ``modulecmd.tcl``. This ability is enabled by
4278 default and could be disabled with configure option
4279 ``--with-locked-configs=extra_siteconfig``.
4280
4281 * The **MODULES_UNLOAD_MATCH_ORDER** environment variable sets whether the
4282 firstly or the lastly loaded module should be selected for unload when
4283 multiple loaded modules match unload request. Configure option
4284 ``--with-unload-match-order`` defines this setting which can be superseded
4285 by the environment variable. By default, lastly loaded module is selected
4286 and it is recommended to keep this behavior when used modulefiles express
4287 dependencies between each other.
4288
4289 * The **MODULES_IMPLICIT_DEFAULT** environment variable sets whether an
4290 implicit default version should be defined for modules with no default
4291 version explicitly defined. When enabled, which is the default behavior, a
4292 module version is automatically selected (latest one) when the generic
4293 name of the module is passed. When implicit default is disabled and no
4294 default version is explicitly defined for a module, the name of this module
4295 to evaluate should be fully qualified elsewhere an error is returned.
4296 Configure option ``--enable-implicit-default`` defines this setting which
4297 can be superseded by the environment variable. This superseding mechanism
4298 can be disabled with configure option
4299 ``--with-locked-configs=implicit_default``.
4300
4301 * The **MODULES_SEARCH_MATCH** environment variable defines the matching style
4302 to perform when searching for available modules. With **starts_with** value,
4303 modules whose name begins by search query string are returned. When search
4304 match style is set to **contains**, modules returned are those whose fully
4305 qualified name contains search query string. Configure option
4306 ``--with-search-match`` defines this setting which can be superseded by the
4307 environment variable, which in turns can be superseded by the
4308 **--starts-with** and **--contains** command-line switches of **avail**
4309 module sub-command.
4310
4311 * The **MODULES_SET_SHELL_STARTUP** environment variable controls whether or
4312 not shell startup file should be set to ensure ``module`` command is defined
4313 once shell has been initialized. When enabled, the ``ENV`` and ``BASH_ENV``
4314 environment variables are set, when ``module`` function is defined, to the
4315 Modules bourne shell initialization script. Configure options
4316 ``--enable-set-shell-startup`` and ``--disable-set-shell-startup`` define
4317 this setting which can be superseded by the environment variable.
4318
4319 * When initializing the ``module`` command in a shell session, initialization
4320 configuration files stored in the defined configuration directory are taken
4321 into account if present instead of the configuration files stored in the
4322 initialization script directory. When they are stored in the configuration
4323 directory, these configuration files are named ``initrc`` and
4324 ``modulespath`` instead of respectively ``modulerc`` and ``.modulespath``.
4325 The location of the installation of those files can be controlled with
4326 configure option ``--with-initconf-in``, which accepts ``etcdir`` and
4327 ``initdir`` values.
4328
4329 * The **MODULES_WA_277** environment variable helps to define an alternative
4330 ``module`` alias on Tcsh shell when set to *1*. It workarounds an issue on
4331 Tcsh history mechanism occurring with default ``module`` command alias:
4332 erroneous history entries are recorded each time the ``module`` command is
4333 called. However the alternative definition of the module alias weakens shell
4334 evaluation of the code produced by modulefiles. Characters with special
4335 meaning for Tcsh shell (like *{* and *}*) may not be used anymore in shell
4336 alias definition elsewhere the evaluation of the code produced by
4337 modulefiles will return a syntax error.
4338
4339
4340 v4.2
4341 ----
4342
4343 This new version is backward-compatible with previous version 4 releases. It
4344 fixes bugs but also introduces new functionalities that are described in this
4345 section. See the :ref:`4.2 release notes<4.2 release notes>` for a complete
4346 list of the changes between Modules v4.1 and v4.2.
4347
4348 .. _v42-conflict-constraints-consistency:
4349
4350 Modulefile conflict constraints consistency
4351 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4352
4353 With the **conflict** modulefile command, a given modulefile can list the
4354 other modulefiles it conflicts with. To load this modulefile, the modulefiles
4355 it conflicts with cannot be loaded.
4356
4357 This constraint was until now satisfied when loading the modulefile declaring
4358 the **conflict** but it vanished as soon as this modulefile was loaded. In the
4359 following example ``a`` modulefile declares a conflict with ``b``:
4360
4361 .. parsed-literal::
4362
4363 :ps:`$` module load b a
4364 WARNING: a cannot be loaded due to a conflict.
4365 HINT: Might try "module unload b" first.
4366 :ps:`$` module list
4367 Currently Loaded Modulefiles:
4368 1) b
4369 :ps:`$` module purge
4370 :ps:`$` module load a b
4371 :ps:`$` module list
4372 Currently Loaded Modulefiles:
4373 1) a 2) b
4374
4375 Consistency of the declared **conflict** is now ensured to satisfy this
4376 constraint even after the load of the modulefile declaring it. This is
4377 achieved by keeping track of the conflict constraints of the loaded
4378 modulefiles in an environment variable called ``MODULES_LMCONFLICT``:
4379
4380 .. parsed-literal::
4381
4382 :ps:`$` module load a b
4383 ERROR: WARNING: b cannot be loaded due to a conflict.
4384 HINT: Might try "module unload a" first.
4385 :ps:`$` module list
4386 Currently Loaded Modulefiles:
4387 1) a
4388
4389 An environment variable is used to keep track of this conflict information to
4390 proceed the same way than used to keep track of the loaded modulefiles with
4391 the ``LOADEDMODULES`` environment variable.
4392
4393 In case a conflict constraint toward a modulefile is set by an already loaded
4394 modulefile, loading the conflicting modulefile will lead to a load evaluation
4395 attempt in order for this modulefile to get the chance to solve the constraint
4396 violation. If at the end of the load evaluation, the conflict has not been
4397 solved, modulefile load will be discarded.
4398
4399 .. warning:: On versions ``4.2.0`` and ``4.2.1``, a conflict constraint set by
4400 an already loaded modulefile forbade the load of the conflicting
4401 modulefile. This has been changed starting version ``4.2.2`` to better cope
4402 with behaviors of previous Modules version: an evaluation attempt of the
4403 conflicting modulefile is made to give it the opportunity to solve this
4404 conflict by using **module unload** modulefile command.
4405
4406 .. _v42-prereq-constraints-consistency:
4407
4408 Modulefile prereq constraints consistency
4409 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4410
4411 With the **prereq** modulefile command, a given modulefile can list the
4412 other modulefiles it pre-requires. To load this modulefile, the modulefiles it
4413 pre-requires must be loaded prior its own load.
4414
4415 This constraint was until now satisfied when loading the modulefile declaring
4416 the **prereq** but, as for the declared **conflict**, it vanished as soon as
4417 this modulefile was loaded. In the following example ``c`` modulefile declares
4418 a prereq on ``a``:
4419
4420 .. parsed-literal::
4421
4422 :ps:`$` module load c
4423 WARNING: c cannot be loaded due to missing prereq.
4424 HINT: the following module must be loaded first: a
4425 :ps:`$` module list
4426 No Modulefiles Currently Loaded.
4427 :ps:`$` module load a c
4428 :ps:`$` module list
4429 Currently Loaded Modulefiles:
4430 1) a 2) c
4431 :ps:`$` module unload a
4432 :ps:`$` module list
4433 Currently Loaded Modulefiles:
4434 1) c
4435
4436 Consistency of the declared **prereq** is now ensured to satisfy this
4437 constraint even after the load of the modulefile declaring it. This is
4438 achieved, like for the conflict consistency, by keeping track of the prereq
4439 constraints of the loaded modulefiles in an environment variable called
4440 ``MODULES_LMPREREQ``:
4441
4442 .. parsed-literal::
4443
4444 :ps:`$` module load a c
4445 :ps:`$` module list
4446 Currently Loaded Modulefiles:
4447 1) a 2) c
4448 :ps:`$` module unload a
4449 ERROR: WARNING: a cannot be unloaded due to a prereq.
4450 HINT: Might try "module unload c" first.
4451 :ps:`$` module list
4452 Currently Loaded Modulefiles:
4453 1) a 2) c
4454
4455 .. _v42-by-passing-module-constraints:
4456
4457 By-passing module defined constraints
4458 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4459
4460 The ability to by-pass a **conflict** or a **prereq** constraint defined by
4461 modulefiles is introduced with the ``--force`` command line switch (``-f`` for
4462 short notation) for the **load**, **unload** and **switch** sub-commands.
4463
4464 With this new command line switch, a given modulefile is loaded even if it
4465 conflicts with other loaded modulefiles or even if the modulefiles it
4466 pre-requires are not loaded. Some example reusing the same modulefiles ``a``,
4467 ``b`` and ``c`` than above:
4468
4469 .. parsed-literal::
4470
4471 :ps:`$` module load b
4472 :ps:`$` module load --force a
4473 WARNING: a conflicts with b
4474 :ps:`$` module list
4475 Currently Loaded Modulefiles:
4476 1) b 2) a
4477 :ps:`$` module purge
4478 :ps:`$` module load --force c
4479 WARNING: c requires a loaded
4480 :ps:`$` module list
4481 Currently Loaded Modulefiles:
4482 1) c
4483
4484 ``--force`` also enables to unload a modulefile required by another loaded
4485 modulefiles:
4486
4487 .. parsed-literal::
4488
4489 :ps:`$` module load a c
4490 :ps:`$` module list
4491 Currently Loaded Modulefiles:
4492 1) a 2) c
4493 :ps:`$` module unload --force a
4494 WARNING: a is required by c
4495 :ps:`$` module list
4496 Currently Loaded Modulefiles:
4497 1) c
4498
4499 In a situation where some of the loaded modulefiles have unsatisfied
4500 constraints corresponding to the **prereq** and **conflict** they declare, the
4501 **save** and **reload** sub-commands do not perform and return an error.
4502
4503 .. _v42-automated-module-handling-mode:
4504
4505 Automated module handling mode
4506 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4507
4508 An automatic management of the dependencies between modulefiles has been added
4509 and it is called *automated module handling mode*. This new mode consists in
4510 additional actions triggered when loading or unloading a modulefile to satisfy
4511 the constraints it declares.
4512
4513 When loading a modulefile, following actions are triggered:
4514
4515 * Requirement Load (ReqLo): load of the modulefiles declared as a **prereq**
4516 of the loading modulefile.
4517
4518 * Dependent Reload (DepRe): reload of the modulefiles declaring a **prereq**
4519 onto loaded modulefile or declaring a **prereq** onto a modulefile part of
4520 this reloading batch.
4521
4522 When unloading a modulefile, following actions are triggered:
4523
4524 * Dependent Unload (DepUn): unload of the modulefiles declaring a non-optional
4525 **prereq** onto unloaded modulefile or declaring a non-optional **prereq**
4526 onto a modulefile part of this unloading batch. A **prereq** modulefile is
4527 considered optional if the **prereq** definition order is made of multiple
4528 modulefiles and at least one alternative modulefile is loaded.
4529
4530 * Useless Requirement Unload (UReqUn): unload of the **prereq** modulefiles
4531 that have been automatically loaded for either the unloaded modulefile, an
4532 unloaded dependent modulefile or a modulefile part of this useless
4533 requirement unloading batch. Modulefiles are added to this unloading batch
4534 only if they are not required by any other loaded modulefiles.
4535 ``MODULES_LMNOTUASKED`` environment variable helps to keep track of these
4536 automatically loaded modulefiles and to distinguish them from modulefiles
4537 asked by user.
4538
4539 * Dependent Reload (DepRe): reload of the modulefiles declaring a **conflict**
4540 or an optional **prereq** onto either the unloaded modulefile, an unloaded
4541 dependent or an unloaded useless requirement or declaring a **prereq** onto
4542 a modulefile part of this reloading batch.
4543
4544 In case a loaded modulefile has some of its declared constraints unsatisfied
4545 (pre-required modulefile not loaded or conflicting modulefile loaded for
4546 instance), this loaded modulefile is excluded from the automatic reload
4547 actions described above.
4548
4549 For the specific case of the **switch** sub-command, where a modulefile is
4550 unloaded to then load another modulefile. Dependent modulefiles to Unload are
4551 merged into the Dependent modulefiles to Reload that are reloaded after the
4552 load of the switched-to modulefile.
4553
4554 This automated module handling mode integrates concepts (like the Dependent
4555 Reload mechanism) of the Flavours_ extension, which was designed for Modules
4556 compatibility version. As a whole, automated module handling mode can be seen
4557 as a generalization and as an expansion of the Flavours_ concepts.
4558
4559 .. _Flavours: https://sourceforge.net/projects/flavours/
4560
4561 This new feature can be controlled at build time with the
4562 ``--enable-auto-handling`` configure option. This default configuration can be
4563 superseded at run-time with the ``MODULES_AUTO_HANDLING`` environment variable
4564 or the command line switches ``--auto`` and ``--no-auto``.
4565
4566 By default, automated module handling mode is disabled and will stay so until
4567 the next major release version (5.0) where it will be enabled by default. This
4568 new feature is currently considered experimental and the set of triggered
4569 actions will be refined over the next feature releases.
4570
4571 .. _v42-consistency-module-load-unload-commands:
4572
4573 Consistency of module load/unload commands in modulefile
4574 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4575
4576 With the **module load** modulefile command, a given modulefile can
4577 automatically load a modulefile it pre-requires. Similarly with the **module
4578 unload** modulefile command, a given modulefile can automatically unload a
4579 modulefile it conflicts with.
4580
4581 Both commands imply additional actions on the loaded environment (loading or
4582 unloading extra modulefiles) that should cope with the constraints defined by
4583 the loaded environment.
4584
4585 Additionally **module load** and **module unload** modulefile commands express
4586 themselves constraints on loaded environment that should stay satisfied to
4587 ensure consistency.
4588
4589 To ensure the consistency of **module load** modulefile command once the
4590 modulefile defining it has been loaded, this command is assimilated to a
4591 **prereq** command. Thus the defined constraint is recorded in the
4592 ``MODULES_LMPREREQ`` environment variable. Same approach is used for **module
4593 unload** modulefile command which is assimilated to a **conflict** command.
4594 Thus the defined constraint is recorded in the ``MODULES_LMCONFLICT``
4595 environment variable.
4596
4597 To ensure the consistency of the loaded environment, the additional actions of
4598 the **module load** and **module unload** modulefile commands have been
4599 adapted in particular situations:
4600
4601 * When unloading modulefile, **module load** command will unload the
4602 modulefile it targets only if no other loaded modulefile requires it and if
4603 this target has not been explicitly loaded by user.
4604
4605 * When unloading modulefile, **module unload** command does nothing as the
4606 relative conflict registered at load time ensure environment consistency and
4607 will forbid conflicting modulefile load.
4608
4609 Please note that loading and unloading results may differ than from previous
4610 Modules version now that consistency is checked:
4611
4612 * Modulefile targeted by a **module load** modulefile command may not be able
4613 to load due to a registered conflict in the currently loaded environment.
4614 Which in turn will break the load of the modulefile declaring the **module
4615 load** command.
4616
4617 * Modulefile targeted by a **module unload** modulefile command may not be
4618 able to unload due to a registered prereq in the loaded environment. Which
4619 in turn will break the load of the modulefile declaring the **module
4620 unload** command.
4621
4622 * If automated module handling mode is enabled, **module load** modulefile
4623 command is interpreted when unloading modulefile as part of the Useless
4624 Requirement Unload (UReqUn) mechanism not through modulefile evaluation.
4625 As a consequence, an error occurring when unloading the modulefile targeted
4626 by the **module load** command does not break the unload of the modulefile
4627 declaring this command. Moreover unload of the **module load** targets is
4628 done in the reverse loaded order, not in the **module load** command
4629 definition order.
4630
4631 .. _v42-alias-symbolic-name-consistency:
4632
4633 Modulefile alias and symbolic modulefile name consistency
4634 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4635
4636 With the **module-alias** and **module-version** modulefile commands,
4637 alternative names can be given to a modulefile. When these names are used to
4638 load for instance a modulefile, they are resolved to the modulefile they
4639 target which is then processed for the load action.
4640
4641 Until now, the alias and symbolic version names were correctly resolved for
4642 the **load** and **unload** actions and also for the querying sub-commands
4643 (like **avail** or **whatis**). However this alternative name information
4644 vanishes once the modulefile it resolves to is loaded. As a consequence there
4645 was no consistency over these alternative designations. In the following
4646 example ``f`` modulefile declares a conflict on ``e`` alias which resolves to
4647 ``d`` modulefile:
4648
4649 .. parsed-literal::
4650
4651 :ps:`$` module load e
4652 :ps:`$` module list
4653 Currently Loaded Modulefiles:
4654 1) d
4655 :ps:`$` module info-loaded e
4656 :ps:`$` module load f
4657 :ps:`$` module list
4658 Currently Loaded Modulefiles:
4659 1) d 2) f
4660
4661 Consistency of the alternative names set on a modulefile with **module-alias**
4662 and **module-version** commands is now ensured to enable modulefile commands
4663 **prereq**, **conflict**, **is-loaded** and **module-info loaded** using these
4664 alternative designations as argument. This consistency is achieved, like for
4665 the conflict and prereq consistencies, by keeping track of the alternative
4666 names of the loaded modulefiles in an environment variable called
4667 ``MODULES_LMALTNAME``:
4668
4669 .. parsed-literal::
4670
4671 :ps:`$` module load e
4672 :ps:`$` module list
4673 Currently Loaded Modulefiles:
4674 1) d
4675 :ps:`$` module info-loaded e
4676 d
4677 :ps:`$` module load f
4678 WARNING: f cannot be loaded due to a conflict.
4679 HINT: Might try "module unload e" first.
4680 :ps:`$` module list
4681 Currently Loaded Modulefiles:
4682 1) d
4683
4684 .. _v42-variable-change-through-modulefile-evaluation:
4685
4686 Environment variable change through modulefile evaluation context
4687 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4688
4689 All environment variable edition commands (``setenv``, ``unsetenv``,
4690 ``append-path``, ``prepend-path`` and ``remove-path``) have been updated to:
4691
4692 * Reflect environment variable value change on the environment of the current
4693 modulefile Tcl interpreter. So using ``$env(VAR)`` will return the currently
4694 defined value for environment variable ``VAR``, not the one found prior
4695 modulefile evaluation.
4696 * Clear environment variable content instead of unsetting it on the
4697 environment of the current modulefile Tcl interpreter to avoid raising
4698 error about accessing an undefined element in ``$env()``. Code is still
4699 produced to purely unset environment variable in shell environment.
4700
4701 Exception is made for the ``whatis`` evaluation mode: environment variables
4702 targeted by variable edition commands are not set to the defined value in the
4703 evaluation context during this ``whatis`` evaluation. These variables are
4704 only initialized to an empty value if undefined. This exception is made to
4705 save performances on this global evaluation mode.
4706
4707 .. _v42-versioned-magic-cookie:
4708
4709 Express Modules compatibility of modulefile with versioned magic cookie
4710 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4711
4712 Any modulefile should start with the ``#%Module`` file signature (also called
4713 the Modules magic cookie) and sometimes a version number may be placed right
4714 after this string. Until now this version number corresponded to a modulefile
4715 format version but it was never checked.
4716
4717 Starting with this new Modules release, this version number reflects the
4718 minimum version of Modules required to interpret the modulefile. If the
4719 version number is set along the magic cookie string it is now checked and the
4720 modulefile is interpreted only if Modules version is greater or equal to this
4721 version number. For instance, if a modulefile begins with the ``#%Module4.3``
4722 string, it can only be evaluated by Modules version 4.3 and above. Elsewhere
4723 the modulefile is ignored like files without the ``#%Module`` magic cookie
4724 set.
4725
4726 .. _v42-module-message-report:
4727
4728 Improved module message report
4729 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4730
4731 Module sub-commands like ``load``, ``unload`` or ``switch``, may perform
4732 multiple load or unload modulefile evaluations in a row. Also these kind of
4733 evaluation modes may sometimes trigger additional load or unload evaluations,
4734 when for instance a modulefile contains a ``module load`` command.
4735
4736 To improve the readability of the module messages produced relatively to
4737 a load or an unload evaluation, these messages are now stacked under a
4738 *Loading* or an *Unloading* message block that gathers all the messages
4739 produced for a given modulefile evaluation:
4740
4741 .. parsed-literal::
4742
4743 :ps:`$` module load --no-auto foo
4744 Loading foo/1.2
4745 ERROR: foo/1.2 cannot be loaded due to missing prereq.
4746 HINT: the following module must be loaded first: bar/4.5
4747
4748 In addition, foreground ``load``, ``unload``, ``switch`` and ``restore``
4749 actions (i.e., asked on the command-line) now report a summary of the
4750 additional load and unload evaluations that were eventually triggered in
4751 the process:
4752
4753 .. parsed-literal::
4754
4755 :ps:`$` module load --auto foo
4756 Loading foo/1.2
4757 Loading requirement: bar/4.5
4758
4759 New modulefile commands
4760 ^^^^^^^^^^^^^^^^^^^^^^^
4761
4762 2 new modulefile Tcl commands have been introduced:
4763
4764 * **set-function**: define a shell function on sh-kind and fish shells.
4765 * **unset-function**: unset a shell function on sh-kind and fish shells.
4766
4767
4768 v4.1
4769 ----
4770
4771 This new version is backward-compatible with previous version 4 releases. It
4772 fixes bugs but also introduces new functionalities that are described in this
4773 section. See the :ref:`4.1 release notes<4.1 release notes>` for a complete
4774 list of the changes between Modules v4.0 and v4.1.
4775
4776 .. _Virtual modules:
4777
4778 Virtual modules
4779 ^^^^^^^^^^^^^^^
4780
4781 A virtual module stands for a module name associated to a modulefile. The
4782 modulefile is the script interpreted when loading or unloading the virtual
4783 module which appears or can be found with its virtual name.
4784
4785 The **module-virtual** modulefile command is introduced to give the ability
4786 to define these virtual modules. This new command takes a module name as
4787 first argument and a modulefile location as second argument::
4788
4789 module-virtual app/1.2.3 /path/to/virtualmod/app
4790
4791 With this feature it is now possible to dynamically define modulefiles
4792 depending on the context.
4793
4794 Extend module command with site-specific Tcl code
4795 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4796
4797 ``module`` command can now be extended with site-specific Tcl
4798 code. ``modulecmd.tcl`` now looks at a **siteconfig.tcl** file in an
4799 ``etcdir`` defined at configure time (by default ``$prefix/etc``). If
4800 it finds this Tcl script file, it is sourced within ``modulecmd.tcl`` at the
4801 beginning of the main procedure code.
4802
4803 ``siteconfig.tcl`` enables to supersede any global variable or procedure
4804 definitions made in ``modulecmd.tcl`` with site-specific code. A module
4805 sub-command can for instance be redefined to make it fit local needs
4806 without having to touch the main ``modulecmd.tcl``.
4807
4808 .. _Quarantine mechanism:
4809
4810 Quarantine mechanism to protect module execution
4811 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4812
4813 To protect the module command run-time environment from side effect
4814 coming from the current environment definition a quarantine mechanism
4815 is introduced. This mechanism, sets within module function definition
4816 and shell initialization script, modifies the ``modulecmd.tcl`` run-time
4817 environment to sanitize it.
4818
4819 The mechanism is piloted by environment variables. First of all
4820 ``MODULES_RUN_QUARANTINE``, a space-separated list of environment variable
4821 names. Every variable found in ``MODULES_RUN_QUARANTINE`` will be set in
4822 quarantine during the ``modulecmd.tcl`` run-time. Their value will be set
4823 empty or set to the value of the corresponding ``MODULES_RUNENV_<VAR>``
4824 environment variable if defined. Once ``modulecmd.tcl`` is started it
4825 restores quarantine variables to their original values.
4826
4827 ``MODULES_RUN_QUARANTINE`` and ``MODULES_RUNENV_<VAR>`` environment variables
4828 can be defined at build time by using the following configure option::
4829
4830 --with-quarantine-vars='VARNAME[=VALUE] ...'
4831
4832 Quarantine mechanism is available for all supported shells except ``csh``
4833 and ``tcsh``.
4834
4835 Pager support
4836 ^^^^^^^^^^^^^
4837
4838 The informational messages Modules sends on the *stderr* channel may
4839 sometimes be quite long. This is especially the case for the avail
4840 sub-command when hundreds of modulefiles are handled. To improve the
4841 readability of those messages, *stderr* output can now be piped into a
4842 paging command.
4843
4844 This new feature can be controlled at build time with the ``--with-pager``
4845 and ``--with-pager-opts`` configure options. Default pager command is set
4846 to ``less`` and its relative options are by default ``-eFKRX``. Default
4847 configuration can be supersedes at run-time with ``MODULES_PAGER`` environment
4848 variables or command-line switches (``--no-pager``, ``--paginate``).
4849
4850 .. warning:: On version ``4.1.0``, the ``PAGER`` environment variable was
4851 taken in consideration to supersede pager configuration at run-time. Since
4852 version ``4.1.1``, ``PAGER`` environment variable is ignored to avoid side
4853 effects coming from the system general pager configuration.
4854
4855 Module function to return value in scripting languages
4856 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4857
4858 On Tcl, Perl, Python, Ruby, CMake and R scripting shells, module function
4859 was not returning value and until now an occurred error led to raising a
4860 fatal exception.
4861
4862 To make ``module`` function more friendly to use on these scripting shells
4863 it now returns a value. False in case of error, true if everything goes well.
4864
4865 As a consequence, returned value of a module sub-command can be checked. For
4866 instance in Python::
4867
4868 if module('load', 'foo'):
4869 # success
4870 else:
4871 # failure
4872
4873 New modulefile commands
4874 ^^^^^^^^^^^^^^^^^^^^^^^
4875
4876 4 new modulefile Tcl commands have been introduced:
4877
4878 * **is-saved**: returns true or false whether a collection, corresponding to
4879 currently set collection target, exists or not.
4880 * **is-used**: returns true or false whether a given directory is currently
4881 enabled in ``MODULEPATH``.
4882 * **is-avail**: returns true or false whether a given modulefile exists in
4883 currently enabled module paths.
4884 * **module-info loaded**: returns the exact name of the modulefile currently
4885 loaded corresponding to the name argument.
4886
4887 Multiple collections, paths or modulefiles can be passed respectively to
4888 ``is-saved``, ``is-used`` and ``is-avail`` in which case true is returned if
4889 at least one argument matches condition (acts as a OR boolean operation). No
4890 argument may be passed to ``is-loaded``, ``is-saved`` and ``is-used``
4891 commands to return if anything is respectively loaded, saved or used.
4892
4893 If no loaded modulefile matches the ``module-info loaded`` query, an empty
4894 string is returned.
4895
4896 New module sub-commands
4897 ^^^^^^^^^^^^^^^^^^^^^^^
4898
4899 Modulefile-specific commands are sometimes wished to be used outside of a
4900 modulefile context. Especially for the commands managing path variables
4901 or commands querying current environment context. So the following
4902 modulefile-specific commands have been made reachable as module sub-commands
4903 with same arguments and properties as if called from within a modulefile:
4904
4905 * **append-path**
4906 * **prepend-path**
4907 * **remove-path**
4908 * **is-loaded**
4909 * **info-loaded**
4910
4911 The ``is-loaded`` sub-command returns a boolean value. Small Python example::
4912
4913 if module('is-loaded', 'app'):
4914 print 'app is loaded'
4915 else:
4916 print 'app not loaded'
4917
4918 ``info-loaded`` returns a string value and is the sub-command counterpart
4919 of the ``module-info loaded`` modulefile command:
4920
4921 .. parsed-literal::
4922
4923 :ps:`$` module load app/0.8
4924 :ps:`$` module info-loaded app
4925 app/0.8
4926
4927
4928 v4.0
4929 ----
4930
4931 Major evolution occurs with this v4.0 release as the traditional *module*
4932 command implemented in C is replaced by the native Tcl version. This full
4933 Tcl rewrite of the Modules package was started in 2002 and has now reached
4934 maturity to take over the binary version. This flavor change enables to
4935 refine and push forward the *module* concept.
4936
4937 This document provides an outlook of what is changing when migrating from
4938 v3.2 to v4.0 by first describing the introduced new features. Both v3.2
4939 and v4.0 are quite similar and transition to the new major version should
4940 be smooth. Slights differences may be noticed in a few use-cases. So the
4941 second part of the document will help to learn about them by listing the
4942 features that have been discontinued in this new major release or the
4943 features where a behavior change can be noticed.
4944
4945 New features
4946 ^^^^^^^^^^^^
4947
4948 On its overall this major release brings a lot more robustness to the
4949 *module* command with now more than 4000 non-regression tests crafted
4950 to ensure correct operations over the time. This version 4.0 also comes
4951 with fair amount of improved functionalities. The major new features are
4952 described in this section.
4953
4954 Additional shells supported
4955 """""""""""""""""""""""""""
4956
4957 Modules v4 introduces support for **fish**, **lisp**, **tcl** and **R**
4958 code output.
4959
4960 Non-zero exit code in case of error
4961 """""""""""""""""""""""""""""""""""
4962
4963 All module sub-commands will now return a non-zero exit code in case of error
4964 whereas Modules v3.2 always returned zero exit code even if issue occurred.
4965
4966 Output redirect
4967 """""""""""""""
4968
4969 Traditionally the *module* command output text that should be seen by the
4970 user on *stderr* since shell commands are output to *stdout* to change
4971 shell's environment. Now on *sh*, *bash*, *ksh*, *zsh* and *fish* shells,
4972 output text is redirected to *stdout* after shell command evaluation if
4973 shell is in interactive mode.
4974
4975 Filtering avail output
4976 """"""""""""""""""""""
4977
4978 Results obtained from the **avail** sub-command can now be filtered to only
4979 get the default version of each module name with use of the **--default**
4980 or **-d** command line switch. Default version is either the explicitly
4981 set default version or the highest numerically sorted modulefile or module
4982 alias if no default version set.
4983
4984 It is also possible to filter results to only get the highest numerically
4985 sorted version of each module name with use of the **--latest** or **-L**
4986 command line switch.
4987
4988 Extended support for module alias and symbolic version
4989 """"""""""""""""""""""""""""""""""""""""""""""""""""""
4990
4991 Module aliases are now included in the result of the **avail**, **whatis**
4992 and **apropos** sub-commands. They are displayed in the module path
4993 section where they are defined or in a *global/user modulerc* section for
4994 aliases set in user's or global ``modulerc`` file. A **@** symbol is added
4995 in parenthesis next to their name to distinguish them from modulefiles.
4996
4997 Search may be performed with an alias or a symbolic version-name passed
4998 as argument on **avail**, **whatis** and **apropos** sub-commands.
4999
5000 Modules v4 resolves module alias or symbolic version passed to **unload**
5001 command to then remove the loaded modulefile pointed by the mentioned
5002 alias or symbolic version.
5003
5004 A symbolic version sets on a module alias is now propagated toward the
5005 resolution path to also apply to the relative modulefile if it still
5006 correspond to the same module name.
5007
5008 Hiding modulefiles
5009 """"""""""""""""""
5010
5011 Visibility of modulefiles can be adapted by use of file mode bits or file
5012 ownership. If a modulefile should only be used by a given subset of persons,
5013 its mode an ownership can be tailored to provide read rights to this group of
5014 people only. In this situation, module only reports the modulefile, during an
5015 **avail** command for instance, if this modulefile can be read by the current
5016 user.
5017
5018 These hidden modulefiles are simply ignored when walking through the
5019 modulepath content. Access issues (permission denied) occur only when trying
5020 to access directly a hidden modulefile or when accessing a symbol or an alias
5021 targeting a hidden modulefile.
5022
5023 Improved modulefiles location
5024 """""""""""""""""""""""""""""
5025
5026 When looking for an implicit default in a modulefile directory, aliases
5027 are now taken into account in addition to modulefiles and directories to
5028 determine the highest numerically sorted element.
5029
5030 Modules v4 resolves module alias or symbolic version when it points to a
5031 modulefile located in another modulepath.
5032
5033 Access issues (permission denied) are now distinguished from find issues
5034 (cannot locate) when trying to access directly a directory or a modulefile
5035 as done on **load**, **display** or **whatis** commands. In addition,
5036 on this kind of access not readable ``.modulerc`` or ``.version`` files are
5037 ignored rather producing a missing magic cookie error.
5038
5039 Module collection
5040 """""""""""""""""
5041
5042 Modules v4 introduces support for module *collections*. Collections
5043 describe a sequence of **module use** then **module load** commands that
5044 are interpreted by Modules to set the user environment as described by this
5045 sequence. When a collection is activated, with the **restore** sub-command,
5046 modulepaths and loaded modules are unused or unloaded if they are not part
5047 or if they are not ordered the same way as in the collection.
5048
5049 Collections are generated by the **save** sub-command that dumps the current
5050 user environment state in terms of modulepaths and loaded modules. By default
5051 collections are saved under the ``$HOME/.module`` directory. Collections
5052 can be listed with **savelist** sub-command, displayed with **saveshow**
5053 and removed with **saverm**.
5054
5055 Collections may be valid for a given target if they are suffixed. In this
5056 case these collections can only be restored if their suffix correspond
5057 to the current value of the ``MODULES_COLLECTION_TARGET`` environment
5058 variable. Saving collection registers the target footprint by suffixing
5059 the collection filename with ``.$MODULES_COLLECTION_TARGET``.
5060
5061 Path variable element counter
5062 """""""""""""""""""""""""""""
5063
5064 Modules 4 provides path element counting feature which increases a
5065 reference counter each time a given path entry is added to a given
5066 path-like environment variable. As consequence a path entry element is
5067 removed from a path-like variable only if the related element counter is
5068 equal to 1. If this counter is greater than 1, path element is kept in
5069 variable and reference counter is decreased by 1.
5070
5071 This feature allows shared usage of particular path elements. For instance,
5072 modulefiles can append ``/usr/local/bin`` to ``PATH``, which is not unloaded
5073 until all the modulefiles that loaded it unload too.
5074
5075 Optimized I/O operations
5076 """"""""""""""""""""""""
5077
5078 Substantial work has been done to reduce the number of I/O operations
5079 done during global modulefile analysis commands like **avail** or
5080 **whatis**. ``stat``, ``open``, ``read`` and ``close`` I/O operations have
5081 been cut down to the minimum required when walking through the modulepath
5082 directories to check if files are modulefiles or to resolve module aliases.
5083
5084 Interpretation of modulefiles and modulerc are handled by the minimum
5085 required Tcl interpreters. Which means a configured Tcl interpreter is
5086 reused as much as possible between each modulefile interpretation or
5087 between each modulerc interpretation.
5088
5089 .. _Sourcing modulefiles:
5090
5091 Sourcing modulefiles
5092 """"""""""""""""""""
5093
5094 Modules 4 introduces the possibility to **source** a modulefile rather
5095 loading it. When it is sourced, a modulefile is interpreted into the shell
5096 environment but then it is not marked loaded in shell environment which
5097 differ from **load** sub-command.
5098
5099 This functionality is used in shell initialization scripts once **module**
5100 function is defined. There the ``etc/modulerc`` modulefile is sourced to
5101 setup the initial state of the environment, composed of *module use*
5102 and *module load* commands.
5103
5104
5105 Removed features and substantial behavior changes
5106 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
5107
5108 Following sections provide list of Modules v3.2 features that are
5109 discontinued on Modules v4 or features with a substantial behavior change
5110 that should be taken in consideration when migrating to v4.
5111
5112 Package initialization
5113 """"""""""""""""""""""
5114
5115 ``MODULESBEGINENV`` environment snapshot functionality is not supported
5116 anymore on Modules v4. Modules collection mechanism should be used instead to
5117 **save** and **restore** sets of enabled modulepaths and loaded modulefiles.
5118
5119 Command line switches
5120 """""""""""""""""""""
5121
5122 Some command line switches are not supported anymore on v4.0. When still
5123 using them, a warning message is displayed and the command is ran with these
5124 unsupported switches ignored. Following command line switches are concerned:
5125
5126 * ``--force``, ``-f``
5127 * ``--human``
5128 * ``--verbose``, ``-v``
5129 * ``--silent``, ``-s``
5130 * ``--create``, ``-c``
5131 * ``--icase``, ``-i``
5132 * ``--userlvl`` lvl, ``-u`` lvl
5133
5134 Module sub-commands
5135 """""""""""""""""""
5136
5137 During an **help** sub-command, Modules v4 does not redirect output made
5138 on stdout in *ModulesHelp* Tcl procedure to stderr. Moreover when running
5139 **help**, version 4 interprets all the content of the modulefile, then call
5140 the *ModulesHelp* procedure if it exists, whereas Modules 3.2 only interprets
5141 the *ModulesHelp* procedure and not the rest of the modulefile content.
5142
5143 When **load** is asked on an already loaded modulefiles, Modules v4 ignores
5144 this new load order whereas v3.2 refreshed shell alias definitions found
5145 in this modulefile.
5146
5147 When **switching** on version 4 an *old* modulefile by a *new* one,
5148 no error is raised if *old* modulefile is not currently loaded. In this
5149 situation v3.2 threw an error and abort switch action. Additionally on
5150 **switch** sub-command, *new* modulefile does not keep the position held
5151 by *old* modulefile in loaded modules list on Modules v4 as it was the
5152 case on v3.2. Same goes for path-like environment variables: replaced
5153 path component is appended to the end or prepended to the beginning of
5154 the relative path-like variable, not appended or prepended relatively to
5155 the position hold by the swapped path component.
5156
5157 During a **switch** command, version 4 interprets the swapped-out modulefile
5158 in *unload* mode, so the sub-modulefiles loaded, with ``module load``
5159 order in the swapped-out modulefile are also unloaded during the switch.
5160
5161 Modules 4 provides path element counting feature which increases a reference
5162 counter each time a given path entry is added to a given environment
5163 variable. This feature also applies to the ``MODULEPATH`` environment
5164 variable. As consequence a modulepath entry element is removed from the
5165 modulepath enabled list only if the related element counter is equal to 1.
5166 When **unusing** a modulepath if its reference counter is greater than 1,
5167 modulepath is kept enabled and reference counter is decreased by 1.
5168
5169 On Modules 3.2 paths composing the ``MODULEPATH`` environment variable
5170 may contain reference to environment variable. These variable references
5171 are resolved dynamically when ``MODULEPATH`` is looked at during module
5172 sub-command action. This feature has been discontinued on Modules v4.
5173
5174 Following Modules sub-commands are not supported anymore on v4.0:
5175
5176 * ``clear``
5177 * ``update``
5178
5179
5180 Modules specific Tcl commands
5181 """""""""""""""""""""""""""""
5182
5183 Modules v4 provides path element counting feature which increases a reference
5184 counter each time a given path entry is added to a given environment
5185 variable. As a consequence a path entry element is not always removed
5186 from a path-like variable when calling to ``remove-path`` or calling to
5187 ``append-path`` or ``append-path`` at unloading time. The path element is
5188 removed only if its related element counter is equal to 1. If this counter
5189 is greater than 1, path element is kept in variable and reference counter
5190 is decreased by 1.
5191
5192 On Modules v4, **module-info mode** returns during an **unload** sub-command
5193 the ``unload`` value instead of ``remove`` on Modules v3.2. However if *mode*
5194 is tested against ``remove`` value, true will be returned. During a **switch**
5195 sub-command on Modules v4, ``unload`` then ``load`` is returned instead of
5196 ``switch1`` then ``switch2`` then ``switch3`` on Modules v3.2. However if
5197 *mode* is tested against ``switch`` value, true will be returned.
5198
5199 When using **set-alias**, Modules v3.2 defines a shell function when
5200 variables are in use in alias value on Bourne shell derivatives, Modules
5201 4 always defines a shell alias never a shell function.
5202
5203 Some Modules specific Tcl commands are not supported anymore on v4.0. When
5204 still using them, a warning message is displayed and these unsupported Tcl
5205 commands are ignored. Following Modules specific Tcl commands are concerned:
5206
5207 * ``module-info flags``
5208 * ``module-info trace``
5209 * ``module-info tracepat``
5210 * ``module-info user``
5211 * ``module-log``
5212 * ``module-trace``
5213 * ``module-user``
5214 * ``module-verbosity``
5215
5216
5217 Further reading
5218 ^^^^^^^^^^^^^^^
5219
5220 To get a complete list of the differences between Modules v3.2 and v4,
5221 please read the :ref:`changes` document.
5222
5223 A significant number of issues reported for v3.2 have been closed on v4.
5224 List of these closed issues can be found at:
5225
5226 https://github.com/envmodules/modules/milestone/1?closed=1