Debian Codegraph @

MIGRATING.rst

raw
1 .. _MIGRATING:
2
3 New features
4 ============
5
6 This document describes the major changes occurring between versions of
7 Modules. It provides an overview of the new features and changed behaviors
8 that will be encountered when upgrading.
9
10
11 v5.6
12 ----
13
14 This new version is backward-compatible with previous version 5 release. It
15 fixes bugs but also introduces new functionalities that are described in this
16 section. See the :ref:`5.6 release notes<5.6 release notes>` for a complete
17 list of the changes between Modules v5.5 and v5.6.
18
19 spider sub-command
20 ^^^^^^^^^^^^^^^^^^
21
22 Support has been added for the :subcmd:`spider` sub-command introduced by the
23 `Lmod`_ project. This sub-command lists available modules found in enabled
24 modulepaths and recursively found in modulepaths enabled by available modules.
25
26 .. parsed-literal::
27
28 :ps:`$` module avail
29 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
30 foo/1 foo/2
31 :ps:`$` module show foo/1
32 -------------------------------------------------------------------
33 :sgrhi:`/path/to/modulefiles/foo/1`:
34
35 :sgrcm:`conflict` foo
36 :sgrcm:`module` use /path/to/modulefiles.2
37 -------------------------------------------------------------------
38 :ps:`$` module show foo/2
39 -------------------------------------------------------------------
40 :sgrhi:`/path/to/modulefiles/foo/2`:
41
42 :sgrcm:`conflict` foo
43 :sgrcm:`append-path` MODULEPATH /path/to/modulefiles.3
44 -------------------------------------------------------------------
45 :ps:`$` module spider
46 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
47 foo/1 foo/2
48
49 -------------- :sgrdi:`/path/to/modulefiles.2` (via foo/1) --------------
50 bar/1 bar/2 bar/3
51
52 -------------- :sgrdi:`/path/to/modulefiles.3` (via foo/2) --------------
53 bar/3 bar/4
54
55 -------------- :sgrdi:`/path/to/modulefiles.3` (via bar/1) --------------
56 qux/1 qux/2
57
58 The :subcmd:`spider` sub-command relies on the :ref:`Extra match search`
59 mechanism to scan modulefiles and identify the modulepaths they enable. Since
60 all modulefiles are evaluated during this scan, it is recommended to build
61 and use a :ref:`Module cache` to improve search performance.
62
63 The output of :subcmd:`spider` is similar to that of the :subcmd:`avail`
64 sub-command and supports the same set of options and queries.
65
66 .. parsed-literal::
67
68 :ps:`$` module spider -t bar@2:
69 :sgrdi:`/path/to/modulefiles.2`:
70 :sgrhi:`bar/2`
71 bar/3
72
73 :sgrdi:`/path/to/modulefiles.3`:
74 bar/3
75 bar/4
76
77 The :subcmd:`spider` sub-command has its own configuration options to control
78 the content of its output: :mconfig:`spider_indepth`,
79 :mconfig:`spider_output`, and :mconfig:`spider_terse_output`.
80
81 These settings make it possible to define output for :subcmd:`spider` that
82 differs from the :subcmd:`avail` sub-command.
83
84 .. parsed-literal::
85
86 :ps:`$` module config spider_output -modulepath
87 :ps:`$` module config spider_indepth 0
88 :ps:`$` module spider
89 :sgrdi:`bar`/ :sgrdi:`foo`/ :sgrdi:`qux`/
90
91 When a modulepath is enabled by a specific module, that modulepath is
92 considered available *via* that module. When the module is loaded, this *via*
93 information is stored in the :envvar:`__MODULES_LMUSE` environment variable to
94 track which modules enable which modulepaths.
95
96 By default, the *via* information is included in the standard output of the
97 :subcmd:`spider` sub-command. Additionally, the JSON output for both the
98 :subcmd:`avail` and :subcmd:`spider` sub-commands has been updated to include
99 *via* details for each modulefile.
100
101 .. parsed-literal::
102
103 :ps:`$` module spider -j bar@2:
104 {"/path/to/modulefiles.2": {
105 "bar/2": { "name": "bar/2", "type": "modulefile", "symbols": [], "tags": [], "pathname": "/path/to/modulefiles.2/bar/2", "via": "foo/1"}
106 "bar/2": { "name": "bar/3", "type": "modulefile", "symbols": [], "tags": [], "pathname": "/path/to/modulefiles.2/bar/3", "via": "foo/1"}
107 },
108 "/path/to/modulefiles.3": {
109 "bar/3": { "name": "bar/3", "type": "modulefile", "symbols": [], "tags": [], "pathname": "/path/to/modulefiles.3/bar/3", "via": "foo/2"},
110 "bar/4": { "name": "bar/4", "type": "modulefile", "symbols": [], "tags": [], "pathname": "/path/to/modulefiles.3/bar/4", "via": "foo/2"}
111 }}
112
113 .. _Requiring via module:
114
115 Requiring *via* module
116 ^^^^^^^^^^^^^^^^^^^^^^
117
118 A loaded module that enables a modulepath is considered the *via* module of
119 other loaded modules whose modulefiles are stored in this modulepath. The
120 :mconfig:`require_via` configuration option is introduced to consider a *via*
121 module a requirement for the loaded module stored in the modulepath it
122 enables.
123
124 A *module hierarchy* mechanism, as introduced by the Lmod_ project, is now
125 supported with this feature. It allows organizing modulefiles through a
126 primary modulepath, where loading certain modules enables additional
127 modulepaths. :mconfig:`require_via` maintains a link between modulefiles in
128 these additional paths and the *via* module that activated them.
129
130 The :mconfig:`require_via` setting is disabled by default to maintain
131 compatibility with previous Modules 5 releases. However, users are encouraged
132 to enable this feature if they want to use the *module hierarchy* mechanism.
133
134 .. parsed-literal::
135
136 :ps:`$` module config require_via 1
137
138 When unloading a *via* module, all the modules stored in its enabled
139 modulepath are automatically unloaded thanks to the *Dependent Unload*
140 mechanism of the :ref:`Automated module handling mode<MODULES_AUTO_HANDLING>`.
141
142 .. parsed-literal::
143
144 :ps:`$` module load foo/1 bar/2
145 :ps:`$` module avail
146 -------------- :sgrdi:`/path/to/modulefiles.2` (via foo/1) --------------
147 bar/1 :sgrl:`bar/2` bar/3
148
149 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
150 :sgrl:`foo/1` foo/2
151
152 Key:
153 :sgrl:`loaded` :sgrdi:`modulepath`
154 :ps:`$` module unload foo
155 Unloading :sgrhi:`foo/1`
156 :sgrin:`Unloading dependent`: bar/2
157
158 When a *via* module is replaced by another, any modulepaths it enabled are
159 updated accordingly. If the newly enabled modulepath provides alternatives to
160 modules previously loaded from the original path, those modules are reloaded
161 from the new location. If no alternatives are found, the dependent modules are
162 unloaded.
163
164 .. parsed-literal::
165
166 :ps:`$` module config conflict_unload 1
167 :ps:`$` module load foo/1 bar/3
168 :ps:`$` module load foo/2
169 Loading :sgrhi:`foo/2`
170 :sgrin:`Unloading dependent`: bar/3
171 :sgrin:`Unloading conflict`: foo/1
172 :sgrin:`Reloading dependent`: bar/3
173 :ps:`$` module avail
174 -------------- :sgrdi:`/path/to/modulefiles.3` (via foo/2) --------------
175 :sgrl:`bar/3` bar/4
176
177 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
178 foo/1 :sgrl:`foo/2`
179
180 Key:
181 :sgrl:`loaded` :sgrdi:`modulepath`
182
183 Always see hidden modules
184 ^^^^^^^^^^^^^^^^^^^^^^^^^
185
186 The ``hidden`` element is a new allowed value for the :mconfig:`avail_output`,
187 :mconfig:`avail_terse_output`, :mconfig:`list_output`,
188 :mconfig:`list_terse_output`, :mconfig:`spider_output` and
189 :mconfig:`spider_terse_output` configuration options. When set into the value
190 list of these options hidden modules are shown on the output of corresponding
191 commands (:subcmd:`avail`, :subcmd:`list` and :subcmd:`spider`).
192
193 .. parsed-literal::
194
195 --------------- :sgrdi:`/path/to/modulefiles` ---------------
196 bar/1.0
197 :ps:`$` module config avail_output +hidden
198 :ps:`$` ml av
199 --------------- :sgrdi:`/path/to/modulefiles` ---------------
200 bar/1.0 foo/1.0 <H>
201
202 When set, this is equivalent to always adding the :option:`--all` command-line
203 option.
204
205 Issue warning upon module evaluation
206 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
207
208 The mfcmd:`module-warn` modulefile Tcl command is introduced to define warning
209 message to print when loading specified modulefiles.
210
211 .. parsed-literal::
212
213 :ps:`$` cat /path/to/modulefiles/foo/1.0
214 #%Module
215 module-warn --message {Please consider using "bar" module instead} foo
216 :ps:`$` module load foo
217 Loading :sgrhi:`foo/1.0`
218 :sgrwa:`WARNING`: Please consider using "bar" module instead
219
220 The warning message is printed when targeted modulefile is evaluated in
221 *load*, *display*, *help* or *test* mode.
222
223 :mfcmd:`module-warn` supports the same kind of options than
224 :mfcmd:`module-tag` or :mfcmd:`module-hide`: it is possible to specify a list
225 of affected or unaffected users or groups. Warning may also be only effective
226 before or after a particular date.
227
228 The ``warning`` tag is set on modules targeted by a mfcmd:`module-warn`
229 command.
230
231 Simplified way to define module help message
232 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
233
234 Until now module help message was defined through a specific Tcl procedure
235 that outputs text to *stderr* channel:
236
237 .. code-block:: tcl
238
239 #%Module
240 proc ModulesHelp {} {
241 puts stderr {Module Name: foo, Version: 1.0}
242 puts stderr {This modulefile does this and that}
243 }
244
245 A new modulefile Tcl command is introduced, :mfcmd:`module-help`, to simplify
246 a bit the definition of help message:
247
248 .. code-block:: tcl
249
250 #%Module
251 module-help {Module Name: foo, Version: 1.0}
252 module-help {This modulefile does this and that}
253
254 With this modulefile command there is no need to define the ``ModulesHelp``
255 procedure anymore. As :mfcmd:`module-help` is reported on module *display*
256 evaluation, help message will also be visible there.
257
258 Support for newline character in environment variable value
259 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
260
261 It is now possible on *sh*, *bash*, *ksh*, *zsh* and *fish* shells to set
262 value for environment variable that contains newline character:
263
264 .. parsed-literal::
265
266 :ps:`$` cat /path/to/modulefiles/bar/1.0
267 #%Module
268 setenv BAR "multi\nline\nvalue"
269 :ps:`$` module load bar
270 :ps:`$` echo "$BAR"
271 multi
272 line
273 value
274
275 Generated shell code on these shells has been changed to enclose value within
276 single quotes rather escaping each special characters:
277
278 .. parsed-literal::
279
280 :ps:`$` module mod-to-sh bash bar/1.0
281 BAR='multi
282 line
283 value'; export BAR;
284
285 Declare additional elements provided by a module
286 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
287
288 New :mfcmd:`provide` modulefile command is added to declare additional
289 elements included in a modulefile. When the modulefile is loaded, each
290 argument specified in the :mfcmd:`provide` command is registered as an alias
291 for the module.
292
293
294 .. parsed-literal::
295
296 :ps:`$` module show foo
297 -------------------------------------------------------------------
298 :sgrhi:`/path/to/modulefiles/foo/1`:
299
300 :sgrcm:`provide` liba/1.2 libb/2.1
301 :sgrcm:`conflict` foo
302 :sgrcm:`conflict` liba libb
303 -------------------------------------------------------------------
304 :ps:`$` module load foo
305
306 It is advised to define :mfcmd:`conflict` over the name of these additional
307 elements to ensure only one provider is found loaded in user environment.
308
309 .. parsed-literal::
310
311 :ps:`$` module show bar/1
312 -------------------------------------------------------------------
313 :sgrhi:`/path/to/modulefiles/bar/1`:
314
315 :sgrcm:`provide` liba/1.2 libb/2.1
316 :sgrcm:`conflict` bar
317 :sgrcm:`conflict` liba libb
318 -------------------------------------------------------------------
319 :ps:`$` module load bar
320 Loading :sgrhi:`bar/1`
321 :sgrer:`ERROR`: Module cannot be loaded due to a conflict.
322 HINT: Might try "module unload foo/1" first.
323
324 Once a module with a provided alias is loaded, it fulfills any requirements
325 declared by other modules for that name and version, just like any other type
326 of module alias.
327
328 .. parsed-literal::
329
330 :ps:`$` module show qux
331 -------------------------------------------------------------------
332 :sgrhi:`/path/to/modulefiles/qux/1`:
333
334 :sgrcm:`prereq` liba/1.2
335 -------------------------------------------------------------------
336 :ps:`$` module load qux
337 :ps:`$` module list
338 Currently Loaded Modulefiles:
339 1) foo/1 2) qux/1
340
341 Module aliases defined via the :mfcmd:`provide` command are not available
342 during module resolution, as modulefiles are not evaluated in that context.
343 Therefore, it is recommended to define a global module alias in a ``modulerc``
344 file using the :mfcmd:`module-alias` command to specify a default target for
345 the alias.
346
347 .. parsed-literal::
348
349 :ps:`$` cat /path/to/modulefiles/.modulerc
350 #%Module
351 module-alias liba/1.2 bar/1
352 :ps:`$` module purge
353 :ps:`$` module load qux
354 Loading :sgrhi:`qux/1`
355 :sgrin:`Loading requirement`: bar/1
356
357 The :mfcmd:`provide` modulefile command also serves as an alias for the
358 :mfcmd:`extensions` command, ensuring compatibility with modulefiles written
359 for `Lmod`_.
360
361 By default an :subcmd:`avail` command includes module aliases defined in
362 ``modulerc`` files. To also get aliases defined within modulefiles with
363 :mfcmd:`provide` or :mfcmd:`family`, output element ``provided-alias`` has to
364 be included. It is supported on :mconfig:`avail_output`,
365 :mconfig:`avail_terse_output`, :mconfig:`spider_output` and
366 :mconfig:`spider_terse_output` configuration options.
367
368 Having ``provided-alias`` included in output triggers :ref:`Extra match
369 search` mechanism that scans all modulefiles to find these aliases. It is thus
370 advised to have :ref:`Module cache` built and used to improve search speed.
371
372 .. parsed-literal::
373
374 :ps:`$` module avail
375 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
376 bar/1 foo/1 :sgrali:`liba/1.2` qux/1
377
378 Key:
379 :sgrdi:`modulepath` :sgrali:`module-alias`
380 :ps:`$` module avail -o +provided-alias
381 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
382 bar/1 foo/1 :sgrali:`liba/1.2` :sgrali:`libb/2.1` qux/1
383
384 Key:
385 :sgrdi:`modulepath` :sgrali:`module-alias`
386
387 The ``provided-alias`` :ref:`Extra specifier` is also introduced to find
388 modules that define such alias with :mfcmd:`provide` or :mfcmd:`family`
389 modulefile commands.
390
391 .. parsed-literal::
392
393 :ps:`$` module avail provided-alias:liba/1.2
394 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
395 bar/1 foo/1
396
397 Key:
398 :sgrdi:`modulepath`
399
400
401 v5.5
402 ----
403
404 This new version is backward-compatible with previous version 5 release. It
405 fixes bugs but also introduces new functionalities that are described in this
406 section. See the :ref:`5.5 release notes<5.5 release notes>` for a complete
407 list of the changes between Modules v5.4 and v5.5.
408
409 .. _Logging activity:
410
411 Logging activity
412 ^^^^^^^^^^^^^^^^
413
414 The ability to log module command activity is now available out of the box. It
415 could be enabled on previous versions by using a specific
416 :file:`siteconfig.tcl` configuration script as described in the
417 :ref:`log-module-command` cookbook recipe.
418
419 Integrated logging feature relies on two configuration options:
420
421 * :mconfig:`logger`, the command run to transmit messages to the log system
422 * :mconfig:`logged_events`, list of module event to log
423
424 :mconfig:`logger` option relies on the :command:`logger` command by default,
425 which is usually available and already installed on most systems.
426
427 The :mconfig:`logged_events` option is empty by default, so no content is sent
428 to logs by default. It recognizes the following events:
429
430 * ``requested_cmd``: record module commands typed by users
431 * ``requested_eval``: record modulefile evaluations requested by users
432 * ``auto_eval``: record modulefile evaluations automatically triggered
433
434 .. parsed-literal::
435
436 :ps:`$` module config logged_events +requested_cmd:requested_eval
437 :ps:`$` ml av
438 --------------- :sgrdi:`/path/to/modulefiles` ---------------
439 bar/1.0 foo/1.0 qux/1.0
440 :ps:`$` module load bar
441 Loading :sgrhi:`bar/1.0`
442 :sgrin:`Loading requirement`: qux/1.0
443 :ps:`$` module purge
444
445 In the above example the module command and modulefile evaluations directly
446 requested by users are sent to the system log:
447
448 .. parsed-literal::
449
450 :ps:`$` journalctl -q -t modules
451 Apr 29 07:47:42 hostname modules[3777797]: user="username" command="avail" arguments=""
452 Apr 29 07:48:10 hostname modules[3777876]: user="username" command="load" arguments="bar"
453 Apr 29 07:48:10 hostname modules[3777876]: user="username" mode="load" module="bar/1.0" specified="bar" modulefile="/path/to/modulefiles/bar/1.0" requested="1"
454 Apr 29 07:48:17 hostname modules[3777914]: user="username" command="purge" arguments=""
455 Apr 29 07:48:17 hostname modules[3777914]: user="username" mode="unload" module="bar/1.0" specified="bar/1.0" modulefile="/path/to/modulefiles/bar/1.0" requested="1"
456
457 Some messages can also be sent during modulefile evaluation by using the
458 ``log`` channel of the :mfcmd:`puts` command:
459
460 .. parsed-literal::
461
462 :ps:`$` cat /path/to/modulefiles/foo/1.0
463 #%Module
464 puts log {some message sent to log}
465 :ps:`$` ml foo/1.0
466 :ps:`$` journalctl -q -t modules
467 Apr 29 07:47:51 hostname modules[3777835]: user="username" command="ml" arguments="foo/1.0"
468 Apr 29 07:47:51 hostname modules[3777835]: some message sent to log
469 Apr 29 07:47:51 hostname modules[3777835]: user="username" mode="load" module="foo/1.0" specified="foo/1.0" modulefile="/path/to/modulefiles/foo/1.0" requested="1"
470
471 Negating extra match search criteria
472 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
473
474 The ``not:`` prefix string is introduced for extra specifier and variant
475 criteria. When this prefix is used, the search criteria where it is applied is
476 negated.
477
478 .. parsed-literal::
479
480 :ps:`$` module config avail_output +variant
481 :ps:`$` module config variant_shortcut toolchain=%
482 :ps:`$` module avail
483 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
484 bar/1.0{:sgrva:`%x86_64`} foo/2.0{:sgrva:`%x86_64`,\ :sgrva:`arm64`}
485 bar/2.0{:sgrva:`%x86_64`,\ :sgrva:`arm64`} qux/1.0{:sgrva:`%x86_64`}
486 foo/1.0{:sgrva:`%x86_64`} qux/2.0{:sgrva:`%x86_64`,\ :sgrva:`arm64`}
487 :ps:`$` module avail not:envvar:FOO,BAR
488 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
489 qux/1.0{:sgrva:`%x86_64`} qux/2.0{:sgrva:`%x86_64`,\ :sgrva:`arm64`}
490
491 The above search query returns all modules not defining ``FOO`` and ``BAR``
492 environment variables.
493
494 .. parsed-literal::
495
496 :ps:`$` module avail not:envvar:BAR not:%arm64
497 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
498 foo/1.0{:sgrva:`%x86_64`} qux/1.0{:sgrva:`%x86_64`}
499
500 The above example returns all modules not defining ``BAR`` environment
501 variable and not defining ``toolchain`` variant or defining this variant
502 without ``arm64`` among the possible values.
503
504 This prefix is recognized on module search context (i.e., :subcmd:`avail`,
505 :subcmd:`whatis` and :subcmd:`paths` sub-commands).
506
507 PowerShell support
508 ^^^^^^^^^^^^^^^^^^
509
510 Support for PowerShell (``pwsh``) is added. This shell may be used with
511 Modules on either Unix or Windows platforms. All main :ref:`kind of
512 environment changes are supported<Shell support>`.
513
514 A ``envmodule`` shell function is configured on PowerShell, as ``module`` is a
515 reserved keyword on this shell.
516
517 The Windows-specific distribution zipball of Modules has been updated to also
518 contain the required files to setup PowerShell support.
519
520 .. _Conflict unload MIGRATING:
521
522 Conflict unload
523 ^^^^^^^^^^^^^^^
524
525 Automated module handling is enhanced with the addition of a new mechanism:
526 *Conflict Unload*. When loading a module, this new mechanism automatically
527 unloads conflicting modules and their dependent.
528
529 .. parsed-literal::
530
531 :ps:`$` module load foo/1
532 Loading :sgrhi:`foo/1`:sgrse:`{`:sgrvahi:`-debug`:sgrse:`}`
533 :sgrin:`Loading requirement`: qux/1 bar/1
534 :ps:`$` module load foo/2
535 Loading :sgrhi:`foo/2`
536 :sgrer:`ERROR`: Module cannot be loaded due to a conflict.
537 HINT: Might try "module unload foo/1" first.
538 :ps:`$` module config conflict_unload 1
539 :ps:`$` module load foo/2
540 Loading :sgrhi:`foo/2`:sgrse:`{`:sgrvahi:`-debug`:sgrse:`}`
541 :sgrin:`Unloading conflict`: foo/1\ :sgrse:`{`:sgrva:`-debug`:sgrse:`}`
542
543 As seen in the above example, *Conflict Unload* is controlled by the
544 :mconfig:`conflict_unload` configuration option. It is disabled by default
545 as it changes behaviors of the :envvar:`automated module handling
546 mode<MODULES_AUTO_HANDLING>`. But everyone is encouraged to enable this new
547 option to benefit from an highly automated experience. This option can be
548 changed at installation time with :instopt:`--enable-conflict-unload`
549 configure script option.
550
551 *Conflict Unload* handles all kind of conflicts: conflicts defined by already
552 loaded modules, conflict declared by loading module through :mfcmd:`conflict`,
553 :mfcmd:`family` or :mfcmd:`module unload<module>` commands. It also supports
554 the unload of a module that is loaded again but with different variant values.
555
556 .. parsed-literal::
557
558 :ps:`$` module load foo/2 +debug
559 Loading :sgrhi:`foo/2`:sgrse:`{`:sgrvahi:`+debug`:sgrse:`}`
560 :sgrin:`Unloading conflict`: foo/2\ :sgrse:`{`:sgrva:`-debug`:sgrse:`}`
561
562 Along with *Conflict Unload*, the *Useless Requirement Unload* mechanism is
563 added to load evaluation. With it, auto loaded requirements of modules
564 unloaded by *Conflict Unload* mechanism are automatically unloaded.
565
566 .. parsed-literal::
567
568 :ps:`$` module load foo/3
569 Loading :sgrhi:`foo/3`:sgrse:`{`:sgrvahi:`-debug`:sgrse:`}`
570 :sgrin:`Unloading conflict`: foo/2\ :sgrse:`{`:sgrva:`+debug`:sgrse:`}` bar/1
571 :sgrin:`Loading requirement`: bar/2
572 :sgrin:`Unloading useless requirement`: qux/1
573
574 Dependent modules of *Conflict Unload* modules are considered *Dependent
575 Reload* modules. They are unloaded prior unloading the *Conflict Unload*
576 module that depend on them. Reload occurs after loading main module and only
577 if the loaded environment satisfies the dependencies of the module to reload.
578
579 .. parsed-literal::
580
581 :ps:`$` module load bar/3
582 Loading :sgrhi:`bar/3`
583 :sgrin:`Loading requirement`: qux/2
584 :ps:`$` module load qux/1
585 Loading :sgrhi:`qux/1`
586 :sgrin:`Unloading dependent`: bar/3
587 :sgrin:`Unloading conflict`: qux/2
588 :sgrin:`Reloading dependent`: bar/3
589
590 Improved automated module handling mechanisms
591 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
592
593 Along with the introduction of *Conflict Unload* mechanism, some of the
594 existing automated module handling mechanisms have been adapted and enhanced.
595
596 *Useless Requirement Unload* (UReqUn) mechanism is moved during a
597 :subcmd:`switch` sub-command from the end of the unload phase to the end of
598 the load phase. Both *UReqUn* modules coming from unload and load phases are
599 this way treated jointly. If *UReqUn* modules resulting from the switch unload
600 phase are detected as conflict to the switched-on module, it is advised to
601 enable the :mconfig:`conflict_unload` mechanism to automatically handle them.
602
603 .. parsed-literal::
604
605 :ps:`$` module switch foo/3 foo/4
606 Loading :sgrhi:`foo/4`
607 :sgrer:`ERROR`: Module cannot be loaded due to a conflict.
608 HINT: Might try "module unload bar/2" first.
609
610 Switching from :sgrhi:`foo/3`:sgrse:`{`:sgrvahi:`-debug`:sgrse:`}` to :sgrhi:`foo/4`
611 :sgrer:`ERROR`: Load of switched-on foo/4 failed
612 :ps:`$` module config conflict_unload 1
613 :ps:`$` module switch foo/3 foo/4
614 Switching from :sgrhi:`foo/3`:sgrse:`{`:sgrvahi:`-debug`:sgrse:`}` to :sgrhi:`foo/4`
615 :sgrin:`Unloading conflict`: bar/2
616
617 *Dependent Reload* (DepRe) modules that also are *UReqUn* modules are now
618 unloaded during the *DepRe* unload phase instead of during the *UReqUn*
619 process. As a consequence these modules are now unloaded prior main module
620 action, as they are a dependent of this main module. This kind of *UReqUn*
621 modules may not be a dependency of a module unloaded in the current
622 processing.
623
624 Modules from *Dependent Unload* (DepUn) and *Dependent Reload* (DepRe)
625 mechanisms are now mixed together to proceed to their unload. They are this
626 way unloaded in their reverse loading order. So if a module is part of the
627 *DepUn* process and one of its requirement is part of the *DepRe* process, the
628 requirement is now unloaded after its dependent.
629
630 Reload of *Dependent Reload* (DepRe) module is now skipped if this module is
631 not found loadable in environment. Either because its requirements are not
632 loaded or a conflict is spotted. Such module is considered *Dependent Unload*
633 rather *Dependent Reload*. If *DepRe* module is sticky, reload is attempted
634 anyway.
635
636 Reload of all *Dependent Reload* (DepRe) modules now occurs after main module
637 evaluation rather after each sub module evaluation that triggered the
638 *Dependent Reload* unload phase.
639
640 When a loading module requires a *Dependent Unload* (DepUn) module, the unload
641 of this module was forced when it was triggered by a Conflict Unload or a
642 switched-off module. The unload of such *DepUn* module is not forced anymore
643 by default and an error is raised if a loading module requiring the *DepUn*
644 module is detected.
645
646 When unloading *Dependent Reload* modules, do not force by default this unload
647 unless asked with :option:`--force` option. As a result an error is obtained
648 when trying to reload Dependent Reload modules that conflict with other loaded
649 modules. This error may be by-passed with :option:`--force` command-line
650 option.
651
652 .. _Specific modulepath for requirements:
653
654 Specific modulepath for requirements
655 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
656
657 A ``--modulepath`` option is introduced on the :mfcmd:`always-load`,
658 :mfcmd:`depends-on`, :mfcmd:`prereq`, :mfcmd:`prereq-all` and
659 :mfcmd:`prereq-any` modulefile commands. This option indicates that
660 requirement should be specifically searched in the provided list of
661 modulepaths. Such mechanism helps to ensure loaded requirements are those
662 expected and they are not coming from an unexpected module tree.
663
664 In the following example, ``foo`` module requires ``bar``. It expects the
665 ``bar`` module from the same modulepath than ``foo``. User's environment has
666 enabled another modulepath that also provides a ``bar`` module. Use of
667 ``--modulepath`` option guides the requirement load mechanism to the expected
668 module.
669
670 .. parsed-literal::
671
672 :ps:`$` module avail
673 --------------------- :sgrdi:`/path/to/othermfiles` ---------------------
674 bar/1
675
676 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
677 bar/1 foo/1 foo/2
678 :ps:`$` module show foo/1
679 -------------------------------------------------------------------
680 :sgrhi:`/path/to/modulefiles/foo/1`:
681
682 :sgrcm:`prereq` --modulepath .. bar/1
683 -------------------------------------------------------------------
684 :ps:`$` module load foo/1
685 Loading :sgrhi:`foo/1`
686 :sgrin:`Loading requirement`: bar/1
687 :ps:`$` echo $_LMFILES_
688 /path/to/modulefiles/bar/1:/path/to/modulefiles/foo/1
689
690 Specific modulepath set may guide to a directory not necessarily enabled
691 (i.e., not defined in :envvar:`MODULEPATH` environment variable):
692
693 .. parsed-literal::
694
695 :ps:`$` module show foo/2
696 -------------------------------------------------------------------
697 :sgrhi:`/path/to/modulefiles/foo/2`:
698
699 :sgrcm:`prereq` --modulepath /path/to/alternatefiles bar/2
700 -------------------------------------------------------------------
701 :ps:`$` module load foo/2
702 Loading :sgrhi:`foo/2`
703 :sgrin:`Loading requirement`: bar/2
704 :ps:`$` module list
705 Currently Loaded Modulefiles:
706 1) bar/2 2) foo/2
707 :ps:`$` echo $_LMFILES_
708 /path/to/alternatefiles/bar/2:/path/to/modulefiles/foo/2
709
710 If a required module is already loaded from a modulepath that does not
711 correspond to the requirement definition, an error is obtained. However if
712 the *Conflict Unload* mechanism described above is enabled, the conflicting
713 module will automatically be unloaded.
714
715 .. parsed-literal::
716
717 :ps:`$` module purge
718 :ps:`$` module load bar/1
719 :ps:`$` echo $_LMFILES_
720 /path/to/othermfiles/bar/1
721 :ps:`$` module load foo/1
722 Loading :sgrhi:`bar/1`
723 :sgrer:`ERROR`: Module already loaded from a different modulepath
724
725 Loading :sgrhi:`foo/1`
726 :sgrer:`ERROR`: Load of requirement bar/1 (specific path) failed
727 :ps:`$` module config conflict_unload 1
728 :ps:`$` module load foo/1
729 Loading :sgrhi:`foo/1`
730 :sgrin:`Unloading conflict`: bar/1
731 :sgrin:`Loading requirement`: bar/1
732 :ps:`$` echo $_LMFILES_
733 /path/to/modulefiles/bar/1:/path/to/modulefiles/foo/1
734
735
736 v5.4
737 ----
738
739 This new version is backward-compatible with previous version 5 release. It
740 fixes bugs but also introduces new functionalities that are described in this
741 section. See the :ref:`5.4 release notes<5.4 release notes>` for a complete
742 list of the changes between Modules v5.3 and v5.4.
743
744 Enhancing extra specifiers
745 ^^^^^^^^^^^^^^^^^^^^^^^^^^
746
747 Extra specifier ``tag`` is introduced to search modules based on tags applied.
748 It applies to :subcmd:`avail`, :subcmd:`whatis` and :subcmd:`paths`
749 sub-commands. Tag abbreviation may also be used as extra specifier name.
750
751 .. parsed-literal::
752
753 :ps:`$` module avail tag:sticky
754 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
755 :sgrs:`bar/1`
756
757 Multiple names can now be specified on one extra specifier criterion to select
758 modules matching any of these names.
759
760 .. parsed-literal::
761
762 :ps:`$` module avail tag:S,F
763 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
764 :sgrs:`bar/1` :sgrf:`qux/1`
765
766 Multiple values can now be specified on one variant criterion to select
767 modules providing any of these variant values.
768
769 .. parsed-literal::
770
771 :ps:`$` module config variant_shortcut toolchain=%
772 :ps:`$` module avail %a,c
773 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
774 baz/1{:sgrvahi:`%c`} foo/1{:sgrvahi:`%a`,\ :sgrva:`b`,\ :sgrvahi:`c`}
775 :ps:`$` module avail %a,c %b
776 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
777 foo/1{:sgrvahi:`%a`,\ :sgrvahi:`b`,\ :sgrvahi:`c`}
778
779 Purging sticky modules
780 ^^^^^^^^^^^^^^^^^^^^^^
781
782 The :mconfig:`sticky_purge` configuration option is added to define the
783 behavior of :subcmd:`purge` sub-command when unloading a sticky or
784 super-sticky module.
785
786 By default an ``error`` is raised. :mconfig:`sticky_purge` can be changed to
787 emit a ``warning`` message instead or to be ``silent``.
788
789 .. parsed-literal::
790
791 :ps:`$` module purge
792 Unloading :sgrshi:`foo/1.0`
793 :sgrer:`ERROR`: Unload of sticky module skipped
794 :ps:`$` module config sticky_purge warning
795 :ps:`$` module purge
796 Unloading :sgrshi:`foo/1.0`
797 :sgrwa:`WARNING`: Unload of sticky module skipped
798 :ps:`$` module config sticky_purge silent
799 :ps:`$` module purge
800 :ps:`$` module list
801 Currently Loaded Modulefiles:
802 1) :sgrs:`foo/1.0`
803
804 Key:
805 :sgrs:`sticky`
806
807
808 Specific modulepath labels
809 ^^^^^^^^^^^^^^^^^^^^^^^^^^
810
811 :mfcmd:`modulepath-label` command is introduced to define a label to use to
812 designate modulepath in module :subcmd:`avail` output. This new command should
813 be used in global or modulepath-specific rc files.
814
815 .. parsed-literal::
816
817 :ps:`$` cat /path/to/modulefiles/.modulerc
818 #%Module
819 modulepath-label . Tools
820 :ps:`$` module avail foo
821 ---------------------- :sgrdi:`Tools` ----------------------
822 foo/1.0 foo/2.0
823
824 Unique module name loaded
825 ^^^^^^^^^^^^^^^^^^^^^^^^^
826
827 The configuration option :mconfig:`unique_name_loaded` is introduced to allow
828 only one module loaded per module name. It is similar to the *One name rule*
829 feature introduced by the `Lmod`_ project.
830
831 When enabled, :mconfig:`unique_name_loaded` produces a conflict definition
832 at the start of modulefile evaluation for each actual and alternative module
833 root name. For instance when loading a module named ``foo/sub/1.0`` with a
834 ``qux/1.0`` alias, it produces a conflict against ``foo`` and ``qux``.
835
836 When loading a module that shares a name with an already loaded module, an
837 error is raised due to the conflict definition. This error aborts the module
838 load evaluation.
839
840 .. parsed-literal::
841
842 :ps:`$` cat /path/to/modulefiles/bar/1.0
843 #%Module
844 :ps:`$` cat /path/to/modulefiles/bar/2.0
845 #%Module
846 :ps:`$` module config unique_name_loaded 1
847 :ps:`$` module load bar/1.0
848 :ps:`$` module load bar/2.0
849 Loading :sgrhi:`bar/2.0`
850 :sgrer:`ERROR`: Module cannot be loaded due to a conflict.
851 HINT: Might try "module unload bar" first.
852
853 :mconfig:`unique_name_loaded` is disabled by default. It can be changed with
854 module :subcmd:`config` sub-command or at installation time with
855 :instopt:`--enable-unique-name-loaded` configure script option.
856
857 Cache sourced files
858 ^^^^^^^^^^^^^^^^^^^
859
860 You may have gathered common Tcl code files used by all your modulefiles in
861 Tcl files that are sourced with :manpage:`source(n)` Tcl command. When loading
862 multiple modulefiles at once, these sourced files may be read for every loaded
863 modules, which may induce some I/O load.
864
865 The configuration option :mconfig:`source_cache` is introduced to cache files
866 evaluated through ``source`` Tcl command in order to only read them once
867 during a :file:`modulecmd.tcl` execution. When this option is enabled, sourced
868 file read also benefits from optimized I/O calls of Modules Tcl extension
869 library.
870
871 In the following example, 20 modules are loaded, each of them sources the same
872 Tcl file:
873
874 .. parsed-literal::
875
876 :ps:`$` syscall_list=close,fcntl,ioctl,newfstatat,openat,read,readlink
877 :ps:`$` strace -f -e $syscall_list -c $MODULES_CMD bash load foo/20
878 % time seconds usecs/call calls errors syscall
879 ------ ----------- ----------- --------- --------- ----------------
880 31.48 0.000781 1 466 read
881 28.58 0.000709 5 131 16 openat
882 14.19 0.000352 2 142 3 newfstatat
883 9.71 0.000241 1 124 close
884 8.02 0.000199 0 212 211 readlink
885 4.55 0.000113 1 67 47 ioctl
886 3.47 0.000086 1 60 fcntl
887 ------ ----------- ----------- --------- --------- ----------------
888 100.00 0.002481 2 1202 277 total
889
890 When sourced file is cached, it is only read once which saves a substantial
891 amount of I/O operations:
892
893 .. parsed-literal::
894
895 :ps:`$` module config source_cache 1
896 :ps:`$` strace -f -e $syscall_list -c $MODULES_CMD bash load foo/20
897 % time seconds usecs/call calls errors syscall
898 ------ ----------- ----------- --------- --------- ----------------
899 34.41 0.000523 4 112 16 openat
900 27.37 0.000416 0 428 read
901 11.84 0.000180 1 122 3 newfstatat
902 11.51 0.000175 1 105 close
903 7.11 0.000108 2 47 27 ioctl
904 5.99 0.000091 0 92 91 readlink
905 1.78 0.000027 0 40 fcntl
906 ------ ----------- ----------- --------- --------- ----------------
907 100.00 0.001520 1 946 137 total
908
909 .. _Abort on error:
910
911 Abort on error
912 ^^^^^^^^^^^^^^
913
914 When an error occurs during the evaluation of a modulefile, several module
915 sub-commands, like :subcmd:`load` or :subcmd:`unload`, continue their
916 processing. It means these sub-commands do not stop if they have multiple
917 modulefiles to evaluate. This is named the *continue on error* behavior.
918
919 The :mconfig:`abort_on_error` configuration option is introduced to abort, for
920 the sub-commands listed in its value, the evaluation of multiple modulefiles
921 if one fails to evaluate.
922
923 The *abort on error* behavior is applied by default on :command:`ml` command,
924 :subcmd:`reload` sub-command and the unload phase of :subcmd:`switch`
925 sub-command. This behavior may also be turned on for :subcmd:`load`,
926 :subcmd:`mod-to-sh`, :subcmd:`purge`, :subcmd:`switch`, :subcmd:`try-load` and
927 :subcmd:`unload` sub-commands. If a command is removed from
928 :mconfig:`abort_on_error`'s value, it applies the *continue on error*
929 behavior.
930
931 In the following example, *abort on error* behavior is applied to
932 :subcmd:`load` sub-command to stop evaluation instead of loading the remaining
933 modulefile in the sequence if first modulefile fails to evaluate.
934
935 .. parsed-literal::
936
937 :ps:`$` module load foo bar
938 Loading :sgrhi:`foo/1`
939 :sgrer:`ERROR`: Module evaluation aborted
940 :ps:`$` module list
941 Currently Loaded Modulefiles:
942 1) bar/1
943 :ps:`$` module purge
944 :ps:`$` module config abort_on_error +load
945 :ps:`$` module load foo bar
946 Loading :sgrhi:`foo/1`
947 :sgrer:`ERROR`: Module evaluation aborted
948 :ps:`$` module list
949 No Modulefiles Currently Loaded.
950
951 When enabled for :subcmd:`switch` sub-command, whole action aborts if
952 switched-on modulefile fails to load. By default, :subcmd:`switch` only aborts
953 if switched-off modulefile fails to unload.
954
955 .. parsed-literal::
956
957 :ps:`$` module load bar
958 :ps:`$` module switch bar foo
959 Loading :sgrhi:`foo/1`
960 :sgrer:`ERROR`: Module evaluation aborted
961
962 Switching from :sgrhi:`bar/1` to :sgrhi:`foo/1`
963 :sgrwa:`WARNING`: Load of switched-on foo/1 failed
964 :ps:`$` module list
965 No Modulefiles Currently Loaded.
966 :ps:`$` module load bar
967 :ps:`$` module config abort_on_error +switch
968 :ps:`$` module switch bar foo
969 Loading :sgrhi:`foo/1`
970 :sgrer:`ERROR`: Module evaluation aborted
971
972 Switching from :sgrhi:`bar/1` to :sgrhi:`foo/1`
973 :sgrwa:`WARNING`: Load of switched-on foo/1 failed
974 :ps:`$` module list
975 Currently Loaded Modulefiles:
976 1) bar/1
977
978 :mconfig:`abort_on_error` configure option has no impact on :mfcmd:`module`
979 commands defined in modulefiles. When :option:`--force` option is set,
980 sub-commands apply the *continue on error* behavior.
981
982 When :instopt:`--enable-new-features` installation option is set, the
983 *abort on error* behavior is also enabled on :subcmd:`load` and
984 :subcmd:`switch` sub-commands.
985
986 Improve error reporting
987 ^^^^^^^^^^^^^^^^^^^^^^^
988
989 Module evaluation error reports have been tweaked to appear where these errors
990 occur rather reporting them on the message block of the main action. Such
991 change helps to better understand the module evaluation flow.
992
993 .. parsed-literal::
994
995 :ps:`$` module load baz
996 Loading :sgrhi:`baz/1`
997 :sgrin:`Loading requirement`: foo/1
998 :ps:`$` module switch foo bar
999 Unloading :sgrhi:`baz/1`
1000 :sgrme:`Module ERROR`: invalid command name "bad_command"
1001 while executing
1002 "bad_command"
1003 (file "/path/to/modulefiles/baz/1" line 4)
1004 Please contact <root@localhost>
1005
1006 Unloading :sgrhi:`foo/1`
1007 :sgrer:`ERROR`: Unload of dependent baz/1 failed
1008
1009 Switching from :sgrhi:`foo/1` to :sgrhi:`bar`
1010 :sgrer:`ERROR`: Unload of switched-off foo/1 failed
1011
1012 :subcmd:`switch` sub-command specific error messages are not reported when
1013 :mfcmd:`module switch<module>` command is run from a modulefile (like *unload
1014 of switched-off module failed*, *load of switched-on module failed* messages
1015 or *Switching* block message). On :subcmd:`switch` sub-command, load failure
1016 of switched-on module is now reported as an error.
1017
1018 .. parsed-literal::
1019
1020 :ps:`$` module load foo
1021 :ps:`$` module switch foo qux
1022 Loading :sgrhi:`qux/1`
1023 :sgrme:`Module ERROR`: invalid command name "bad_command"
1024 while executing
1025 "bad_command"
1026 (file "/path/to/modulefiles/qux/1" line 2)
1027 Please contact <root@localhost>
1028
1029 Switching from :sgrhi:`foo/1` to :sgrhi:`qux/1`
1030 :sgrer:`ERROR`: Load of switched-on qux/1 failed
1031
1032 Conflict error messages now describe the loaded module name and version that
1033 conflicts rather the generic conflict module specification.
1034
1035 .. parsed-literal::
1036
1037 :ps:`$` cat /path/to/modulefiles/bar/1
1038 #%Module
1039 conflict foo
1040 :ps:`$` module load foo
1041 :ps:`$` module load bar
1042 Loading :sgrhi:`bar/1`
1043 :sgrer:`ERROR`: Module cannot be loaded due to a conflict.
1044 HINT: Might try "module unload foo/1" first.
1045
1046 New options for source-sh modulefile command
1047 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1048
1049 Support for new *shell* mode named ``bash-eval`` is added to
1050 :mfcmd:`source-sh` modulefile command. With this new mode, the generated
1051 output of the bash shell script is evaluated to get the environment changes
1052 instead of sourcing this script.
1053
1054 .. parsed-literal::
1055
1056 :ps:`$` cat /path/to/modulefiles/foo/foo-output.sh
1057 #!/bin/bash
1058 cat <<EOF
1059 export FOO=value;
1060 alias foo_alias='echo foo_alias;'
1061 foo_function() {
1062 echo foo_function;
1063 }
1064 EOF
1065 :ps:`$` cat /path/to/modulefiles/foo/1
1066 #%Module
1067 set foo_script [file dirname $ModulesCurrentModulefile]/foo-output.sh
1068 :sgrhi:`source-sh bash-eval` $foo_script
1069
1070 :ps:`$` module show foo/1
1071 -------------------------------------------------------------------
1072 :sgrhi:`/path/to/modulefiles/foo/1`:
1073
1074 :sgrcm:`set-alias` foo_alias {echo foo_alias;}
1075 :sgrcm:`set-function` foo_function {
1076 echo foo_function}
1077 :sgrcm:`setenv` FOO value
1078 -------------------------------------------------------------------
1079 :ps:`$` module load foo/1
1080 :ps:`$` foo_alias
1081 foo_alias
1082
1083 ``bash-eval`` *shell* support is also available on :subcmd:`sh-to-mod`
1084 sub-command.
1085
1086 A new option is added to :mfcmd:`source-sh` modulefile command: ``--ignore``.
1087 It filters shell elements changed by script, not to get these changes applied
1088 when loading modulefile. This option accepts a list, separated by colon
1089 character, of shell elements: ``envvar``, ``alias``, ``function``,
1090 ``complete`` or ``chdir``.
1091
1092 .. parsed-literal::
1093
1094 :ps:`$` module purge
1095 :ps:`$` cat /path/to/modulefiles/foo/2
1096 #%Module
1097 set foo_script [file dirname $ModulesCurrentModulefile]/foo-output.sh
1098 :sgrhi:`source-sh --ignore alias:function bash-eval` $foo_script
1099
1100 :ps:`$` module show foo/2
1101 -------------------------------------------------------------------
1102 :sgrhi:`/path/to/modulefiles/foo/2`:
1103
1104 :sgrcm:`setenv` FOO value
1105 -------------------------------------------------------------------
1106 :ps:`$` module load foo/2
1107 :ps:`$` foo_alias
1108 bash: foo_alias: command not found
1109
1110 This ``--ignore`` option can be used with any *shell* supported by
1111 :mfcmd:`source-sh`.
1112
1113
1114 v5.3
1115 ----
1116
1117 This new version is backward-compatible with previous version 5 release. It
1118 fixes bugs but also introduces new functionalities that are described in this
1119 section. See the :ref:`5.3 release notes<5.3 release notes>` for a complete
1120 list of the changes between Modules v5.2 and v5.3.
1121
1122 Module cache
1123 ^^^^^^^^^^^^
1124
1125 A module cache file can be created under each modulepath directory with new
1126 :subcmd:`cachebuild` sub-command. Cache file is named :file:`.modulecache` and
1127 contains in one file all modulefiles and modulercs found in modulepath
1128 directory.
1129
1130 When cache file is available, module search mechanism uses this file rather
1131 walking through the content of modulepath directory. I/O operations are saved
1132 this way which reduces search processing time.
1133
1134 When searching for available modules without cache, each file contained
1135 in enabled modulepaths is opened to check if it is a modulefile or not. Such
1136 checks lead to a large number of I/O operations on large module setup like in
1137 the below example where a total of 1051 modulefiles are available:
1138
1139 .. parsed-literal::
1140
1141 :ps:`$` module -o "" avail -t | wc -l
1142 1051
1143 :ps:`$` syscall_list=access,close,getdents64,newfstatat,openat,read
1144 :ps:`$` strace -f -e $syscall_list -c $MODULES_CMD bash avail
1145 % time seconds usecs/call calls errors syscall
1146 ------ ----------- ----------- --------- --------- ----------------
1147 31.09 0.003776 2 1424 9 openat
1148 28.56 0.003469 2 1649 3 newfstatat
1149 14.08 0.001710 1 1421 close
1150 11.85 0.001439 3 460 getdents64
1151 10.88 0.001321 0 1505 read
1152 3.54 0.000430 4 107 5 access
1153 ------ ----------- ----------- --------- --------- ----------------
1154 100.00 0.012145 1 6566 17 total
1155
1156 After building cache file for every enabled modulepaths in this example setup,
1157 a lot of I/O operations are saved when searching for available modules:
1158
1159 .. parsed-literal::
1160
1161 :ps:`$` module cachebuild
1162 Creating :sgrhi:`/path/to/modulefiles/.modulecache`
1163 Creating :sgrhi:`/path/to/modulefiles.2/.modulecache`
1164 Creating :sgrhi:`/path/to/modulefiles.3/.modulecache`
1165 :ps:`$` module config cache_buffer_bytes 1000000
1166 :ps:`$` strace -f -e $syscall_list -c $MODULES_CMD bash avail
1167 % time seconds usecs/call calls errors syscall
1168 ------ ----------- ----------- --------- --------- ----------------
1169 70.19 0.000544 2 255 read
1170 13.16 0.000102 2 38 9 openat
1171 8.90 0.000069 1 35 close
1172 6.06 0.000047 1 31 2 newfstatat
1173 1.68 0.000013 1 11 2 access
1174 ------ ----------- ----------- --------- --------- ----------------
1175 100.00 0.000775 2 370 13 total
1176
1177 A significant execution time drop may be noticed, especially if modulepath
1178 directories are stored on heavily loaded network filesystem.
1179
1180 To further optimize I/O operation count, the :mconfig:`cache_buffer_bytes`
1181 configuration option can be set like in the above example to use fewer number
1182 of ``read`` operation to load cache file content.
1183
1184 To build cache file, user should be granted write access on modulepath
1185 directory. Modulefiles or directories that are not accessible for everyone are
1186 not recorded in cache. An indication is saved instead to test these limited
1187 access elements when cache is loaded to determine if they are available to
1188 currently running user.
1189
1190 Cache file can be ignored with :option:`--ignore-cache` command line switch or
1191 more permanently with :mconfig:`ignore_cache` configuration option.
1192
1193 Cache file is valid indefinitely by default but :mconfig:`cache_expiry_secs`
1194 configuration option can be used to define the number of seconds a cache file
1195 is considered valid after being generated. Expired cache file is ignored.
1196
1197 Cache file of enabled modulepaths can be deleted all at once with
1198 :subcmd:`cacheclear` sub-command.
1199
1200 Querying available module variants
1201 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1202
1203 A new mechanism named :ref:`Extra match search` is introduced to evaluate
1204 modulefiles during a module search to find those matching an extra query on a
1205 variant value, a dependency or an environment variable definition.
1206
1207 During this specific evaluation, modulefiles are interpreted in *scan* mode to
1208 collect the different Tcl modulefile commands they use. Special care should be
1209 given when writing modulefiles to ensure they cope with such evaluation mode.
1210
1211 :ref:`Extra match search` mechanism is available on :subcmd:`avail`,
1212 :subcmd:`whatis` and :subcmd:`paths` sub-commands.
1213
1214 With this new mechanism, it is possible to list all available variant defined
1215 in modulefiles with their associated values:
1216
1217 .. parsed-literal::
1218
1219 :ps:`$` module config avail_output modulepath:alias:dirwsym:sym:tag:key:variant
1220 :ps:`$` module config variant_shortcut toolchain=%
1221 :ps:`$` module avail
1222 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1223 bar/1.0{:sgrva:`%a`,\ :sgrva:`b`} foo/1.0{:sgrvade:`%a`} qux/1.0{:sgrva:`%a`,\ :sgrva:`b`}
1224 bar/2.0{:sgrvade:`%b`} foo/2.0{:sgrva:`%a`,\ :sgrva:`b`,\ :sgrva:`c`} qux/2.0{:sgrva:`%b`,\ :sgrva:`c`}
1225
1226 Key:
1227 :sgrdi:`modulepath` {:sgrva:`%value`}={:sgrva:`toolchain=value`}
1228 :sgrde:`default-version` {:sgrva:`variant=value`}
1229
1230 You can also search for modules defining a specific variant value:
1231
1232 .. parsed-literal::
1233
1234 :ps:`$` module avail %a
1235 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1236 bar/1.0{:sgrvahi:`%a`,\ :sgrva:`b`} foo/1.0{:sgrvadehi:`%a`} foo/2.0{:sgrvahi:`%a`,\ :sgrva:`b`,\ :sgrva:`c`} qux/1.0{:sgrvahi:`%a`,\ :sgrva:`b`}
1237
1238 Key:
1239 :sgrdi:`modulepath` {:sgrva:`%value`}={:sgrva:`toolchain=value`}
1240 :sgrde:`default-version` {:sgrva:`variant=value`}
1241
1242 .. note:: As extra match search implies additional modulefile evaluations, it
1243 is advised to build and use `Module cache`_ to improve search speed.
1244
1245 Two new elements, ``variant`` and ``variantifspec``, are added to the allowed
1246 value list of :mconfig:`avail_output` and :mconfig:`avail_terse_output`
1247 configuration options. The latter is set in the default value list of both
1248 options. When ``variant`` is set, variants and their possible values are
1249 reported along module they are associated to. When ``variantifspec`` is set,
1250 available variants are reported only if a variant is specified in search
1251 query.
1252
1253 .. parsed-literal::
1254
1255 :ps:`$` module config --reset avail_output
1256 :ps:`$` module config avail_output
1257 Modules Release 5.3.0 (2023-05-14)
1258
1259 - :sgrhi:`Config. name` ---------.- :sgrhi:`Value (set by if default overridden)` ---------------
1260 avail_output modulepath:alias:dirwsym:sym:tag:variantifspec:key
1261 :ps:`$` module avail foo
1262 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1263 :sgrhi:`foo`/1.0 :sgrhi:`foo`/2.0
1264
1265 Key:
1266 :sgrdi:`modulepath`
1267 :ps:`$` module avail foo %a
1268 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1269 :sgrhi:`foo`/1.0{:sgrvadehi:`%a`} :sgrhi:`foo`/2.0{:sgrvahi:`%a`,\ :sgrva:`b`,\ :sgrva:`c`}
1270
1271 Key:
1272 :sgrdi:`modulepath` {:sgrva:`%value`}={:sgrva:`toolchain=value`}
1273 :sgrde:`default-version` {:sgrva:`variant=value`}
1274
1275 Extra specifiers
1276 ^^^^^^^^^^^^^^^^
1277
1278 Extra specifiers are introduced to query content of modulefiles. They can be
1279 specified with *element:name* syntax as part of module specification on module
1280 search commands (:subcmd:`avail`, :subcmd:`paths` and and :subcmd:`whatis`).
1281
1282 Extra specifiers trigger :ref:`Extra match search` mechanism when found in
1283 module specification. Available modulefiles are evaluated in *scan* mode to
1284 collect the different Tcl modulefile commands they use.
1285
1286 .. parsed-literal::
1287
1288 :ps:`$` module avail variant:toolchain
1289 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1290 bar/1.0 bar/2.0 foo/1.0 foo/2.0 qux/1.0 qux/2.0
1291
1292 In the above example, all modulefiles defining a ``toolchain`` variant are
1293 returned.
1294
1295 Most Tcl modulefile commands can be queried with extra specifiers:
1296 ``variant``, ``setenv``, ``unsetenv``, ``append-path``, ``prepend-path``,
1297 ``remove-path``, ``pushenv``, ``complete``, ``uncomplete``, ``set-alias``,
1298 ``unset-alias``, ``set-function``, ``unset-function``, ``chdir``, ``family``,
1299 ``prereq``, ``prereq-any``, ``prereq-all``, ``depends-on``, ``always-load``,
1300 ``load``, ``load-any``, ``try-load``, ``switch``, ``switch-on``,
1301 ``switch-off``, ``conflict`` and ``unload``. Commands that handle environment
1302 variables may be aliased ``envvar``. Commands that define a module requirement
1303 may be aliased ``require`` and those that define a module incompatibility may
1304 be aliased ``incompat``.
1305
1306 When several extra specifiers are set in query, modules returned are those
1307 matching both conditions. In the following example, all modulefiles
1308 interacting with :envvar:`PATH` environment variable and requiring ``foo``
1309 module are returned.
1310
1311 .. parsed-literal::
1312
1313 :ps:`$` module avail envvar:PATH require:foo
1314 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1315 bar/1.0 bar/2.0
1316
1317 Extra specifiers related to module requirement or incompatibility may leverage
1318 the :ref:`Advanced module version specifiers` syntax. On following example,
1319 modulefiles returned are those defining a requirement on ``foo`` module with
1320 version higher or equal to ``1.2`` and variant ``toolchain=a`` selected.
1321
1322 .. parsed-literal::
1323
1324 :ps:`$` module avail "require:[email protected]: toolchain=a"
1325 --------------------- :sgrdi:`/path/to/modulefiles` ---------------------
1326 bar/1.0
1327
1328 .. note:: Module aliases or symbolic versions used either in modulefile
1329 definitions or as extra specifier values are not resolved.
1330
1331 Append or subtract elements to current option value
1332 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1333
1334 For command-line switches and configuration options whose value is a
1335 colon-separated list, it is now possible to append or subtract elements to the
1336 current value. With a ``+`` prefix elements are appended and with a ``-``
1337 prefix elements are subtracted.
1338
1339 This new feature applies to :option:`--output`/:option:`-o` command-line
1340 switches and :mconfig:`avail_output`, :mconfig:`avail_terse_output`,
1341 :mconfig:`list_output`, :mconfig:`list_terse_output`, :mconfig:`colors`,
1342 :mconfig:`protected_envvars`, :mconfig:`shells_with_ksh_fpath`,
1343 :mconfig:`tag_abbrev`, :mconfig:`tag_color_name`, :mconfig:`variant_shortcut`
1344 configuration options.
1345
1346 For instance, to output available *foo* modules without modulepath and tag
1347 information:
1348
1349 .. parsed-literal::
1350
1351 :ps:`$` module -o -modulepath:tag avail foo
1352 :sgrhi:`foo`/1.0 :sgrhi:`foo`/2.0
1353
1354 On following example, terse output mode of *list* sub-command is permanently
1355 updated to report variant information:
1356
1357 .. parsed-literal::
1358
1359 :ps:`$` module config list_terse_output
1360 Modules Release 5.3.0 (2023-05-14)
1361
1362 - :sgrhi:`Config. name` ---------.- :sgrhi:`Value (set by if default overridden)` ---------------
1363 list_terse_output header
1364 :ps:`$` module config list_terse_output +variant
1365 :ps:`$` module config list_terse_output
1366 Modules Release 5.3.0 (2023-05-14)
1367
1368 - :sgrhi:`Config. name` ---------.- :sgrhi:`Value (set by if default overridden)` ---------------
1369 list_terse_output header:variant (env-var)
1370
1371
1372 v5.2
1373 ----
1374
1375 This new version is backward-compatible with previous version 5 release. It
1376 fixes bugs but also introduces new functionalities that are described in this
1377 section. See the :ref:`5.2 release notes<5.2 release notes>` for a complete
1378 list of the changes between Modules v5.1 and v5.2.
1379
1380 Optional requirements
1381 ^^^^^^^^^^^^^^^^^^^^^
1382
1383 The ``--optional`` option has been added to the :mfcmd:`prereq`,
1384 :mfcmd:`prereq-all`, :mfcmd:`depends-on` and :mfcmd:`always-load` modulefile
1385 commands to indicate that specified requirement is optional. An automatic
1386 load attempt is also performed for optional requirements. If requirement is
1387 not found or cannot be loaded, the dependency is yet considered satisfied as
1388 it is optional.
1389
1390 .. parsed-literal::
1391
1392 :ps:`$` module show foo
1393 -------------------------------------------------------------------
1394 :sgrhi:`/path/to/modulefiles/foo`:
1395
1396 :sgrcm:`prereq` --optional bar
1397 -------------------------------------------------------------------
1398 :ps:`$` module load foo
1399 Loading :sgrhi:`foo`
1400 :sgrin:`Loading requirement`: bar
1401
1402
1403 If the optional requirement is unloaded or loaded afterward, the dependent
1404 module is automatically reloaded thanks to the :mconfig:`auto_handling`
1405 mechanism.
1406
1407 .. parsed-literal::
1408
1409 :ps:`$` module unload bar
1410 Unloading :sgrhi:`bar`
1411 :sgrin:`Unloading dependent`: foo
1412 :sgrin:`Reloading dependent`: foo
1413 :ps:`$` module list
1414 Currently Loaded Modulefiles:
1415 1) foo
1416 :ps:`$` module load bar
1417 Loading :sgrhi:`bar`
1418 :sgrin:`Unloading dependent`: foo
1419 :sgrin:`Reloading dependent`: foo
1420
1421 Modules loaded by other modules with the :mfcmd:`module try-load<module>`
1422 command are now considered optional requirements. Dependent module can be
1423 loaded without the *try-load* modules, but now it gets automatically reloaded
1424 if *try-load* module is loaded afterward, to take it into account.
1425
1426 Linting modulefiles
1427 ^^^^^^^^^^^^^^^^^^^
1428
1429 Static analysis of modulefile, modulerc and global/user rc is now possible
1430 with :subcmd:`lint` sub-command. It relies on an external program defined with
1431 :mconfig:`tcl_linter` configuration option. Modules or files specified on the
1432 command-line are resolved then passed to the Tcl linter program.
1433
1434 .. parsed-literal::
1435
1436 :ps:`$` cat /path/to/modulefiles/foo/1.0
1437 #%Module
1438 if {"str" eq} {
1439 else {
1440 }
1441 :ps:`$` module lint foo/1.0
1442 Linting :sgrhi:`/path/to/modulefiles/foo/1.0`
1443 :sgrer:`ERROR line 2`: Could not complete statement.
1444 One close brace would complete the first line
1445 One close brace would complete at end of line 4.
1446 One close brace would complete the script body at line 5.
1447 Assuming completeness for further processing.
1448 :sgrer:`ERROR line 2`: Bad expression: missing operand at _@_
1449 in expression ""str" eq_@_"
1450 :sgrwa:`WARNING line 3`: Unknown command "else"
1451 :sgrin:`NOTICE line 4`: Close brace not aligned with line 3 (1 0)
1452
1453 `Nagelfar`_ is the Tcl linter recommended for Modules and set by default. This
1454 default can be changed at installation time with :instopt:`--with-tcl-linter`
1455 and :instopt:`--with-tcl-linter-opts` options. It can also be configured later
1456 on through :mconfig:`tcl_linter` config option.
1457
1458 Specific syntax databases and plugins for `Nagelfar`_ are added by Modules to
1459 precisely lint modulefile commands syntax in addition to regular Tcl syntax.
1460 The installation of these specific files is controlled with
1461 :instopt:`--enable-nagelfar-addons` option (enabled by default). Their
1462 location is controlled by the :instopt:`--nagelfardatadir` option.
1463
1464 .. parsed-literal::
1465
1466 :ps:`$` module lint bar@:1 /path/to/modulefiles/.modulerc
1467 Linting :sgrhi:`/path/to/modulefiles/.modulerc`
1468 :sgrer:`ERROR line 35`: Wrong number of arguments (3) to "module-alias"
1469 :sgrer:`ERROR line 41`: Wrong number of arguments (3) to "module-virtual"
1470
1471 Linting :sgrhi:`/path/to/modulefiles/bar/1.2`
1472 :sgrwa:`WARNING line 19`: Unknown command "unk"
1473 :ps:`$` module lint ~/.modulerc
1474 Linting :sgrhi:`/home/user/.modulerc`
1475 :sgrwa:`WARNING line 2`: Command "setenv" should not be be used in global rc file
1476
1477 :subcmd:`lint` sub-command outputs messages returned by the Tcl linter
1478 program. Nagelfar produces NOTICE, WARNING and ERROR messages. If linter does
1479 not report a thing, :subcmd:`lint` sub-command will be silent, unless if the
1480 :option:`--verbose`/:option:`-v` is set.
1481
1482 .. parsed-literal::
1483
1484 :ps:`$` module lint /path/to/modulefiles/bar/.version bar/1.4
1485 :ps:`$` module lint -v /path/to/modulefiles/bar/.version bar/1.4
1486 Linting :sgrhi:`/path/to/modulefiles/bar/.version`
1487 Linting :sgrhi:`/path/to/modulefiles/bar/1.4`
1488
1489 When no file is specified to :subcmd:`lint` sub-command, all the global/user
1490 rc files and all the modulefiles and modulercs from enabled modulepaths are
1491 analyzed. If the :option:`--all`/:option:`-a` option is set, all hidden
1492 modulefiles are also linted.
1493
1494 .. parsed-literal::
1495
1496 :ps:`$` module lint
1497 Linting :sgrhi:`/home/user/.modulerc`
1498 :sgrwa:`WARNING line 2`: Command "setenv" should not be be used in global rc file
1499
1500 Linting :sgrhi:`/path/to/modulefiles/.modulerc`
1501 :sgrer:`ERROR line 35`: Wrong number of arguments (3) to "module-alias"
1502 :sgrer:`ERROR line 41`: Wrong number of arguments (3) to "module-virtual"
1503
1504 Linting :sgrhi:`/path/to/modulefiles/bar/1.2`
1505 :sgrwa:`WARNING line 19`: Unknown command "unk"
1506 ...
1507
1508 To use `Nagelfar`_ as Tcl linter for Modules, this open source tool has to be
1509 installed on your system. When installing from tarball distribution, make sure
1510 :command:`nagelfar.tcl` command is found through a :envvar:`PATH` lookup or
1511 that :mconfig:`tcl_linter` option is set to its full path location.
1512 `Nagelfar`_ is also made available as a RPM package in `EPEL and Fedora
1513 repositories`_.
1514
1515 .. _Nagelfar: http://nagelfar.sourceforge.net/
1516 .. _EPEL and Fedora repositories: https://src.fedoraproject.org/rpms/nagelfar
1517
1518 mod-to-sh sub-command
1519 ^^^^^^^^^^^^^^^^^^^^^
1520
1521 New sub-command is added to translate modulefile into shell code:
1522 :subcmd:`mod-to-sh`. It evaluates modulefiles passed as argument and produces
1523 code for specified shell.
1524
1525 .. parsed-literal::
1526
1527 :ps:`$` cat /path/to/modulefiles/foo
1528 #%Module
1529 setenv FOO value
1530 set-function foo {echo foo}
1531 :ps:`$` module mod-to-sh bash foo
1532 FOO=value; export FOO;
1533 foo () { echo foo; }; export -f foo;
1534
1535 Designated modulefiles are evaluated as if they were loading. But instead of
1536 producing shell code that is evaluated in current shell session,
1537 :command:`module` command outputs this shell code.
1538
1539 .. parsed-literal::
1540
1541 :ps:`$` cat /path/to/modulefiles/bar
1542 #%Module
1543 setenv BAR othervalue
1544 set-alias bar {echo bar}
1545 :ps:`$` module mod-to-sh fish foo bar
1546 set -xg FOO value;
1547 set -xg BAR othervalue;
1548 alias bar echo\ bar;
1549 function foo; echo foo; end;
1550 :ps:`$` module list
1551 No Modulefiles Currently Loaded.
1552
1553 All shells supported by :file:`modulecmd.tcl` script are supported by
1554 :subcmd:`mod-to-sh`.
1555
1556 .. parsed-literal::
1557
1558 :ps:`$` module mod-to-sh python foo bar
1559 import os
1560 os.environ['FOO'] = 'value'
1561 os.environ['BAR'] = 'othervalue'
1562
1563 .. _Initial environment:
1564
1565 Initial environment
1566 ^^^^^^^^^^^^^^^^^^^
1567
1568 When Modules initializes, it evaluates the :file:`initrc` and
1569 :file:`modulespath` configuration files to enable default modulepaths and load
1570 default modules. Initial environment corresponds to the environment state
1571 after this initialization.
1572
1573 Initial environment is now saved in an environment variable
1574 (:envvar:`__MODULES_LMINIT`) in current shell session to remember what are the
1575 initial modulepaths and initial modules with their tags and variants if any.
1576
1577 :subcmd:`reset` sub-command is introduced, in a similar fashion than on
1578 `Lmod`_, to restore the initial environment. Here, :subcmd:`reset` relies on
1579 the :ref:`collection<collections>` mechanism based and the content of
1580 :envvar:`__MODULES_LMINIT`. Currently enabled modulepaths and loaded modules
1581 are respectively unused and unloaded to use the modulepaths and load the
1582 modules with tags and variants as described by initial environment.
1583
1584 .. parsed-literal::
1585
1586 :ps:`$` module list
1587 Currently Loaded Modulefiles:
1588 1) foo/1.0 2) bar/1.0
1589 :ps:`$` module switch bar/1.0 qux/1.0
1590 :ps:`$` module reset
1591 Unloading :sgrhi:`qux/1.0`
1592 Loading :sgrhi:`bar/1.0`
1593
1594 :subcmd:`restore` sub-command has been adapted to reinitialize the environment
1595 to its initial state when no collection name is provided and no *default*
1596 collection exists or if ``__init__`` virtual collection name is provided.
1597
1598 It is possible to view the content of the initial environment with
1599 :subcmd:`saveshow` sub-command. It is displayed when no argument is provided
1600 and no collection exists or if ``__init__`` name is provided.
1601
1602 .. parsed-literal::
1603
1604 :ps:`$` module saveshow __init__
1605 -------------------------------------------------------------------
1606 :sgrhi:`initial environment`:
1607
1608 :sgrcm:`module` use --append /path/to/modulefiles
1609 :sgrcm:`module` load foo/1.0
1610 :sgrcm:`module` load bar/1.0
1611
1612 -------------------------------------------------------------------
1613
1614 Users have the ability to define what is their initial environment state and
1615 thus adapt the behavior of :subcmd:`reset` sub-command with
1616 :mconfig:`reset_target_state` configuration option. Default value is
1617 ``__init__`` and it corresponds to the behavior described above. When set to
1618 ``__purge__``, a :subcmd:`purge` command is performed when resetting. Any
1619 other value corresponds to the name of a collection to restore.
1620
1621 .. parsed-literal::
1622
1623 :ps:`$` module config reset_target_state __purge__
1624 :ps:`$` module reset
1625 Unloading bar/1.0
1626 Unloading foo/1.0
1627
1628 .. _Stashing environment:
1629
1630 Stashing environment
1631 ^^^^^^^^^^^^^^^^^^^^
1632
1633 The ability to stash current environment is added with the introduction of the
1634 :subcmd:`stash` sub-command. When called current environment is saved in a
1635 *stash* collection then initial environment is restored.
1636
1637 .. parsed-literal::
1638
1639 :ps:`$` module list
1640 Currently Loaded Modulefiles:
1641 1) bar/2.0 2) foo/2.0
1642 :ps:`$` module stash
1643 Unloading :sgrhi:`foo/2.0`
1644 Unloading :sgrhi:`bar/2.0`
1645 :ps:`$` module list
1646 No Modulefiles Currently Loaded.
1647
1648 Sub-commands are added to specifically handle stash collections. Their names
1649 are prefixed with *stash*, like :subcmd:`stashlist` to list existing stash
1650 collections or :subcmd:`stashshow` to display their content.
1651
1652 .. parsed-literal::
1653
1654 :ps:`$` module stashlist
1655 Stash collection list:
1656 0) stash-1665377597432 1) stash-1664946764252
1657 :ps:`$` module stashshow
1658 -------------------------------------------------------------------
1659 :sgrhi:`/home/user/.module/stash-1665377597432:`
1660
1661 :sgrcm:`module` use --append /path/to/modulefiles
1662 :sgrcm:`module` load bar
1663 :sgrcm:`module` load foo
1664
1665 -------------------------------------------------------------------
1666
1667 Stash collections can be designated on sub-commands by their collection name
1668 or stash index. Most recent stash collection has index ``0``, the one after is
1669 designated with index ``1``, and so on. When no stash collection is specified,
1670 most recent one is assumed.
1671
1672 .. parsed-literal::
1673
1674 :ps:`$` module stashshow 1
1675 -------------------------------------------------------------------
1676 :sgrhi:`/home/user/.module/stash-1664946764252:`
1677
1678 :sgrcm:`module` use --append /path/to/modulefiles
1679 :sgrcm:`module` load foobar
1680
1681 -------------------------------------------------------------------
1682
1683 Stash collections are restored with the :subcmd:`stashpop` sub-command. It
1684 changes the user environment to match the stash definition, then it deletes
1685 the stash collection file.
1686
1687 .. parsed-literal::
1688
1689 :ps:`$` module stashpop
1690 Loading :sgrhi:`bar/2.0`
1691 Loading :sgrhi:`foo/2.0`
1692 :ps:`$` module stashlist
1693 Stash collection list:
1694 0) stash-1664946764252
1695
1696 Stash collections can be deleted one by one with :subcmd:`stashrm` sub-command
1697 or all together with :subcmd:`stashclear`.
1698
1699 .. parsed-literal::
1700
1701 :ps:`$` module stashrm
1702 :ps:`$` module stashlist
1703 No stash collection.
1704
1705 Siteconfig hook variables
1706 ^^^^^^^^^^^^^^^^^^^^^^^^^
1707
1708 Several Tcl variables are introduced for :ref:`Site-specific configuration`
1709 script to define specific commands and variables in the evaluation context of
1710 modulefiles and modulercs. These commands and variables setup in
1711 :file:`siteconfig.tcl` can be used in modulefile or modulerc. Sites can easily
1712 extend modulefile and modulerc syntax with specific elements.
1713
1714 :sitevar:`modulefile_extra_cmds` variable defines a list of commands to expose
1715 in the modulefile evaluation context and the associated procedure to run when
1716 this command is called. This variable has to be defined in
1717 :file:`siteconfig.tcl` located for instance at |file etcdir_siteconfig|.
1718
1719 In the following example :sitevar:`modulefile_extra_cmds` is used to define
1720 the ``sys`` command and bound it to the ``sys`` procedure that is also defined
1721 in :file:`siteconfig.tcl`.
1722
1723 .. code-block:: tcl
1724
1725 proc sys {mode} {
1726 switch -- $mode {
1727 name { return myhost-$::tcl_platform(machine) }
1728 default { error "Unknown mode '$mode'" }
1729 }
1730 }
1731 set modulefile_extra_cmds {sys sys}
1732
1733 Once :file:`siteconfig.tcl` is setup, the ``sys`` command can be called by
1734 modulefiles. In the following example it is used to determine the application
1735 path.
1736
1737 .. parsed-literal::
1738
1739 :ps:`$` cat /path/to/modulefiles/foo/1.2
1740 #%Module
1741 append-path PATH /path/to/apps/foo-1.2/[sys name]/bin
1742
1743 :ps:`$` module show foo/1.2
1744 -------------------------------------------------------------------
1745 :sgrhi:`/path/to/modulefiles/foo/1.2`:
1746
1747 :sgrcm:`append-path` PATH /path/to/apps/foo-1.2/myhost-x86_64/bin
1748 -------------------------------------------------------------------
1749
1750 :sitevar:`modulerc_extra_cmds` follows the same approach than
1751 :sitevar:`modulefile_extra_cmds` and makes specific commands available during
1752 modulerc evaluation.
1753
1754 :sitevar:`modulefile_extra_vars` variable defines a list of variables to
1755 expose in the modulefile evaluation context and their associated value. This
1756 variable has to be defined in :file:`siteconfig.tcl`.
1757
1758 In the following example :sitevar:`modulefile_extra_vars` is used to define
1759 the ``APP_ROOT`` variable with ``/path/to/apps`` as value.
1760
1761 .. code-block:: tcl
1762
1763 set modulefile_extra_vars {APP_ROOT /path/to/apps}
1764
1765 Once :file:`siteconfig.tcl` is setup, the ``APP_ROOT`` variable can be used in
1766 modulefiles.
1767
1768 .. parsed-literal::
1769
1770 :ps:`$` cat /path/to/modulefiles/bar/2.1
1771 #%Module
1772 append-path PATH $APP_ROOT/bar-2.1/[sys name]/bin
1773
1774 :ps:`$` module show bar/2.1
1775 -------------------------------------------------------------------
1776 :sgrhi:`/path/to/modulefiles/bar/2.1`:
1777
1778 :sgrcm:`append-path` PATH /path/to/apps/bar-2.1/myhost-x86_64/bin
1779 -------------------------------------------------------------------
1780
1781 :sitevar:`modulerc_extra_vars` follows the same approach than
1782 :sitevar:`modulefile_extra_vars` and makes specific variables available during
1783 modulerc evaluation.
1784
1785
1786 v5.1
1787 ----
1788
1789 This new version is backward-compatible with previous version 5 release. It
1790 fixes bugs but also introduces new functionalities that are described in this
1791 section. See the :ref:`5.1 release notes<5.1 release notes>` for a complete
1792 list of the changes between Modules v5.0 and v5.1.
1793
1794 Control output redirection
1795 ^^^^^^^^^^^^^^^^^^^^^^^^^^
1796
1797 Since version 4.0, the :command:`module` function is initialized differently
1798 on *sh*, *bash*, *ksh*, *zsh* and *fish* shells when their session is found
1799 interactive. In such situation :command:`module` redirects its output from
1800 *stderr* to *stdout*. Once initialized the redirection behavior is inherited
1801 in sub-sessions.
1802
1803 The :mconfig:`redirect_output` configuration option is introduced in version
1804 5.1, to supersede the default behavior set at initialization time.
1805
1806 .. parsed-literal::
1807
1808 :ps:`$` module load unknown >/dev/null
1809 :ps:`$` module config redirect_output 0
1810 :ps:`$` module load unknown >/dev/null
1811 :sgrer:`ERROR`: Unable to locate a modulefile for 'unknown'
1812
1813 The :option:`--redirect` and :option:`--no-redirect` command-line switches are
1814 also added to change the output redirection behavior for a single command:
1815
1816 .. parsed-literal::
1817
1818 :ps:`$` module load unknown --redirect >/dev/null
1819 :ps:`$` module load unknown --no-redirect >/dev/null
1820 :sgrer:`ERROR`: Unable to locate a modulefile for 'unknown'
1821
1822 .. _Change modulefile command behavior:
1823
1824 Change modulefile command behavior
1825 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1826
1827 Depending on the evaluation mode of the modulefile (e.g. *load*, *unload*,
1828 *display*, etc) commands have different behavior. Most common example is the
1829 :mfcmd:`setenv` command that sets an environment variable when modulefile is
1830 loaded and unsets it when it is unloaded. A different behavior may be wished
1831 sometimes for commands. This is why options are introduced for some modulefile
1832 commands to control what happens on particular evaluation mode.
1833
1834 The ``--return-value`` option is added to the :mfcmd:`getenv` and
1835 :mfcmd:`getvariant` modulefile commands to ensure that the value of the
1836 designated environment variable or variant is returned even if modulefile is
1837 evaluated in *display* mode:
1838
1839 .. parsed-literal::
1840
1841 :ps:`$` cat /path/to/modulefiles/foo/1.0
1842 #%Module5.1
1843 if {[getenv --return-value VAR] eq {}} {
1844 setenv VAR value
1845 }
1846 :ps:`$` module display foo/1.0
1847 -------------------------------------------------------------------
1848 :sgrhi:`/path/to/modulefiles/foo/1.0`:
1849
1850 :sgrcm:`setenv` VAR value
1851 -------------------------------------------------------------------
1852
1853 ``--remove-on-unload``, ``--append-on-unload``, ``--prepend-on-unload`` and
1854 ``--noop-on-unload`` options are added to the :mfcmd:`remove-path` and
1855 :mfcmd:`module unuse<module>` modulefile commands to control the behavior
1856 applied when modulefile is unloaded. With these options it is possible for
1857 instance to restore the paths unset at load time or to set other paths:
1858
1859 .. parsed-literal::
1860
1861 :ps:`$` module display bar/1.0
1862 -------------------------------------------------------------------
1863 :sgrhi:`/path/to/modulefiles/bar/1.0`:
1864
1865 :sgrcm:`module` unuse --prepend-on-unload /path/to/dir1
1866 :sgrcm:`module` use /path/to/dir2
1867 -------------------------------------------------------------------
1868 :ps:`$` module use
1869 Search path for module files (in search order):
1870 :sgrmp:`/path/to/dir1`
1871 :sgrmp:`/path/to/modulefiles`
1872 :ps:`$` module bar/1.0
1873 :ps:`$` module use
1874 Search path for module files (in search order):
1875 :sgrmp:`/path/to/dir2`
1876 :sgrmp:`/path/to/modulefiles`
1877 :ps:`$` module unload bar/1.0
1878 :ps:`$` module use
1879 Search path for module files (in search order):
1880 :sgrmp:`/path/to/dir1`
1881 :sgrmp:`/path/to/modulefiles`
1882
1883 Following the same trend, the ``--unset-on-unload`` and ``--noop-on-unload``
1884 options are added to the :mfcmd:`unsetenv` modulefile command to be able to
1885 choose between unsetting variable, setting a value or performing no operation
1886 when modulefile is unloaded.
1887
1888 Reducing number of I/O operations
1889 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1890
1891 A new configuration option named :mconfig:`mcookie_check` is introduced to
1892 control the verification made to files to determine if they are modulefiles.
1893 By default this configuration option is set to ``always`` and when searching
1894 for modulefiles within enabled modulepaths each file below these directories
1895 is opened to check if it starts with the Modules magic cookie (i.e.,
1896 ``#%Module`` file signature).
1897
1898 These historical checks lead to a large number of I/O operations on large
1899 module setup like in the below example where a total of 1098 modulefiles are
1900 available:
1901
1902 .. parsed-literal::
1903
1904 :ps:`$` module -o "" avail -t | wc -l
1905 1098
1906 :ps:`$` module config mcookie_check always
1907 :ps:`$` strace -f -e open,openat,read,close -c $MODULES_CMD bash avail
1908 ...
1909 % time seconds usecs/call calls errors syscall
1910 ------ ----------- ----------- --------- --------- ----------------
1911 44.29 0.044603 25 1734 166 open
1912 34.15 0.034389 16 2056 close
1913 11.87 0.011949 24 483 5 openat
1914 9.69 0.009761 4 2146 read
1915 ------ ----------- ----------- --------- --------- ----------------
1916 100.00 0.100702 6419 171 total
1917
1918 For each file, 3 I/O operations (``open``, ``read`` and ``close``) are
1919 achieved to determine if it is a modulefile and include it in search results.
1920 When modulefiles are located in a shared filesystem concurrently accessed by
1921 hundreds of users, a ``module avail`` command may take some time to finish.
1922
1923 When setting the :mconfig:`mcookie_check` configuration option to the ``eval``
1924 value, files are not checked anymore when searching for modulefiles, only when
1925 evaluating them. All files under modulepaths are considered modulefiles, so
1926 the content of these directories must be carefully checked to use this
1927 :mconfig:`mcookie_check` mode which lead to a significant reduction of I/O
1928 operations:
1929
1930 .. parsed-literal::
1931
1932 :ps:`$` module config mcookie_check eval
1933 :ps:`$` strace -f -e open,openat,read,close -c $MODULES_CMD bash avail
1934 ...
1935 % time seconds usecs/call calls errors syscall
1936 ------ ----------- ----------- --------- --------- ----------------
1937 30.56 0.013717 14 944 close
1938 28.76 0.012911 21 612 156 open
1939 26.41 0.011857 24 483 5 openat
1940 14.26 0.006403 6 1034 read
1941 ------ ----------- ----------- --------- --------- ----------------
1942 100.00 0.044888 3073 161 total
1943
1944 A substantial reduction of execution time may be noticed depending on the
1945 storage setup used to host the modulepath directories. A special care should
1946 be given to the content of these directories to ensure they only contain
1947 modulefiles (see :envvar:`MODULES_MCOOKIE_CHECK`).
1948
1949 See the :ref:`reduce-io-load` cookbook recipe to learn additional features of
1950 Modules that could be leveraged to lower the number of I/O operations.
1951
1952 Shell command completion
1953 ^^^^^^^^^^^^^^^^^^^^^^^^
1954
1955 New modulefile commands :mfcmd:`complete` and :mfcmd:`uncomplete` are added to
1956 get the ability to respectively define and unset command completion. *bash*,
1957 *tcsh* and *fish* shells are supported.
1958
1959 .. parsed-literal::
1960
1961 :ps:`>` module display foo
1962 -------------------------------------------------------------------
1963 :sgrhi:`/path/to/modulefiles/foo/1.0`:
1964
1965 :sgrcm:`append-path` PATH /path/to/foo-1.0/bin
1966 :sgrcm:`complete` fish foo {-s V -l version --description 'Command version'}
1967 :sgrcm:`complete` fish foo {-s h -l help --description 'Command help'}
1968 -------------------------------------------------------------------
1969 :ps:`>` module load foo
1970 :ps:`>` foo -<TAB>
1971 -h --help (Command help) -V --version (Command version)
1972
1973 :subcmd:`sh-to-mod` sub-command and :mfcmd:`source-sh` modulefile command have
1974 also been updated to track shell completion changes.
1975
1976 .. parsed-literal::
1977
1978 :ps:`$` module sh-to-mod bash /path/to/foo-1.0/share/setup-env.sh
1979 #%Module
1980 :sgrcm:`complete` bash foo {-o default -F _foo}
1981 :sgrcm:`append-path` PATH /path/to/foo-1.0/bin
1982 :sgrcm:`set-function` _foo {
1983 ...bash completion code...}
1984
1985 Lmod Tcl modulefile support
1986 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
1987
1988 With this new version, Modules now supports Tcl modulefiles written for
1989 `Lmod`_, the alternative :command:`module` implementation developed in Lua.
1990 Such modulefiles can be safely evaluated by Modules without raising error.
1991
1992 Support has been added for the following modulefile commands introduced by
1993 Lmod: :mfcmd:`depends-on`, :mfcmd:`prereq-any`, :mfcmd:`always-load`,
1994 :mfcmd:`module load-any<module>`, :mfcmd:`pushenv`, :mfcmd:`require-fullname`
1995 and :mfcmd:`family`.
1996
1997 The :ref:`Compatibility with Lmod Tcl modulefile` section in the
1998 :ref:`modulefile(5)` man page describes the differences existing between the
1999 two implementations.
2000
2001 Note that when processing a :mfcmd:`family` command, the
2002 :envvar:`LMOD_FAMILY_\<NAME\>` environment variable is defined by Modules to
2003 be compatible with existing modulefiles or scripts relying on such variable.
2004
2005 .. _More tagging capabilities:
2006
2007 More tagging capabilities
2008 ^^^^^^^^^^^^^^^^^^^^^^^^^
2009
2010 The new :option:`--tag` option helps to define extra tags onto a loading
2011 module. These tags comes in addition to those inherited from the module state
2012 or those associated with the :mfcmd:`module-tag` modulefile command.
2013
2014 The :option:`--tag` option is available on :subcmd:`load`, :subcmd:`load-any`,
2015 :subcmd:`switch` and :subcmd:`try-load` sub-commands and on
2016 :mfcmd:`always-load`, :mfcmd:`depends-on`, :mfcmd:`module`, :mfcmd:`prereq`,
2017 :mfcmd:`prereq-all` and :mfcmd:`prereq-any` modulefile commands.
2018
2019 Informational messages of module evaluation have been updated to mention in
2020 module denomination the tags applying to it, as it is done in :subcmd:`list`
2021 sub-command output:
2022
2023 .. parsed-literal::
2024
2025 :ps:`$` module load -v --tag=sticky:bar foo/1.0
2026 Loading :sgrshi:`foo/1.0` <bar>
2027
2028 In case the designated module is already loaded, the additional tags are added
2029 to the list of tags already applied to this module.
2030
2031 .. parsed-literal::
2032
2033 :ps:`$` module list
2034 Currently Loaded Modulefiles:
2035 1) foo/1.0
2036 :ps:`$` module load -v --tag=sticky:bar foo/1.0
2037 Tagging :sgrshi:`foo/1.0` <bar>
2038
2039 The ``keep-loaded`` tag is introduced in this version. It avoids an
2040 auto-loaded module to get automatically unloaded when its dependent modules
2041 are unloaded. This new tag can be set with the :mfcmd:`module-tag` modulefile
2042 command or when module is loaded with the :mfcmd:`always-load` modulefile
2043 command. Default :mconfig:`tag_abbrev` configuration option has been updated
2044 to add the ``kL`` abbreviation for ``keep-loaded`` tag. Default dark and light
2045 color palettes have been updated too.
2046
2047 .. parsed-literal::
2048
2049 :ps:`$` module show bar/1.0
2050 -------------------------------------------------------------------
2051 :sgrhi:`/path/to/modulefiles/bar/1.0`:
2052
2053 :sgrcm:`always-load` foo/1.0
2054 -------------------------------------------------------------------
2055 :ps:`$` module load bar/1.0
2056 Loading :sgrhi:`bar/1.0`
2057 :sgrin:`Loading requirement`: foo/1.0
2058 :ps:`$` module unload bar/1.0
2059 :ps:`$` module list
2060 Currently Loaded Modulefiles:
2061 1) :sgral:`foo`:sgrkl:`/1.0`
2062
2063 Key:
2064 :sgral:`auto-loaded` :sgrkl:`keep-loaded`
2065
2066 When saving a collection, the tags defined with :option:`--tag` option are
2067 recorded to set them again when collection is restored. Tags resulting from
2068 module load state, like ``auto-loaded`` and ``keep-loaded``, are also
2069 recorded.
2070
2071 .. parsed-literal::
2072
2073 :ps:`$` module load --tag=sticky bar/1.0
2074 Loading :sgrshi:`bar/1.0`
2075 :sgrin:`Loading requirement`: foo/1.0
2076 :ps:`$` module save
2077 :ps:`$` module saveshow
2078 -------------------------------------------------------------------
2079 :sgrhi:`/home/user/.module/default`:
2080
2081 :sgrcm:`module` use --append /path/to/modulefiles
2082 :sgrcm:`module` load --tag=auto-loaded:keep-loaded foo
2083 :sgrcm:`module` load --tag=sticky bar
2084
2085 -------------------------------------------------------------------
2086
2087 The :mconfig:`collection_pin_tag` configuration option is added to record in
2088 collection all tags set on loaded modules. This configuration option is
2089 disabled by default.
2090
2091 .. note:: Collection saved now starts with a ``#%Module5.1`` file signature if
2092 :option:`--tag` option is recorded in it. Such collection could only be
2093 handled by Modules version 5.1 and above.
2094
2095 v5.0
2096 ----
2097
2098 With this new major version the :command:`module` experience has been updated
2099 to benefit by default from all the advanced behaviors and features developed
2100 over the Modules 4 versions. Modules 5.0 also introduces some breaking
2101 changes to improve the consistency of the whole solution. See the :ref:`5.0
2102 release notes<5.0 release notes>` for a complete list of the changes between
2103 Modules v4.8 and v5.0. The :ref:`changes<Modules 5 changes>` document gives an
2104 in-depth view of the modified behaviors.
2105
2106 Upgraded default configuration
2107 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2108
2109 Release after release, new advanced features were added on Modules 4. They
2110 were set off by default to avoid breaking change during the version 4 cycle.
2111 With the move to a new major release, these features are set on to improve by
2112 default usages for everybody. These features enabled by default are:
2113
2114 * :ref:`v42-automated-module-handling-mode` which provides the automatic
2115 dependency resolution when loading and unloading modules
2116
2117 .. parsed-literal::
2118
2119 :ps:`$` module load foo/1.0
2120 Loading :sgrhi:`foo/1.0`
2121 :sgrin:`Loading requirement`: bar/1.0
2122
2123 * `Extended default`_ that enables to select a module when the first number
2124 in its version is specified
2125
2126 .. parsed-literal::
2127
2128 :ps:`$` module load -v foo/1
2129 Loading :sgrhi:`foo/1.2.3`
2130
2131
2132 * :ref:`Advanced module version specifiers`, an improved syntax to designate
2133 module version in range or list and module variant
2134
2135 .. parsed-literal::
2136
2137 :ps:`$` module load foo@:2.2 %gcc11
2138 Loading :sgrhi:`foo/2.1`:sgrse:`{`:sgrva:`%gcc11`:sgrse:`}`
2139 :sgrin:`Loading requirement`: bar/1.2\ :sgrse:`{`:sgrva:`-debug`:sgrse:`:`:sgrva:`%gcc11`:sgrse:`}`
2140
2141 * `Colored output`_ to graphically enhance parts of the produced output to
2142 improve readability
2143
2144 .. parsed-literal::
2145
2146 :ps:`$` ml av
2147 ------------------ :sgrdi:`/path/to/modulefiles` ------------------
2148 :sgrde:`bar/1.0` bar/2.0 :sgrf:`foo/1.0` :sgrs:`foo/2.0` :sgrali:`foo/2.2`
2149
2150 Key:
2151 :sgrdi:`modulepath` :sgrali:`module-alias` :sgrs:`sticky`
2152 :sgrde:`default-version` :sgrf:`forbidden`
2153
2154 * :ref:`Insensitive case` search that matches modules using a different
2155 character case than the one expressed in search query
2156
2157 .. parsed-literal::
2158
2159 :ps:`$` ml av liba
2160 ------------------ :sgrdi:`/path/to/modulefiles` ------------------
2161 LibA/1.0 LibA/2.0
2162
2163 Some other features that were enabled by default during the Modules 4 cycle
2164 have been turned off as they may not be useful for basic usages. Among other
2165 things setting off the following features makes the definition of the
2166 :command:`module` function simpler. Even if off by default, these features
2167 can now be enabled once Modules is installed through the :file:`initrc`
2168 configuration file.
2169
2170 * *Set shell startup* files to ensure the :command:`module` command is defined
2171 once shell has been initialized. See the :mconfig:`set_shell_startup`
2172 configuration option to activate.
2173
2174 * *Quarantine mechanism* that protects the :command:`module` command
2175 run-time environment from side effect coming from the current environment
2176 definition. See the :mconfig:`quarantine_support` configuration option to
2177 activate.
2178
2179 * *Silent shell debug* which disables the debugging property set on current
2180 shell session for the duration of the :command:`module` command. See the
2181 :mconfig:`silent_shell_debug` configuration option to activate.
2182
2183 Removing compatibility version
2184 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2185
2186 The ability to co-install version 3.2 of Modules along newer version is
2187 discontinued. The installation option ``--enable-compat-version``,
2188 :command:`switchml` shell function and :envvar:`MODULES_USE_COMPAT_VERSION`
2189 environment variables are thus removed.
2190
2191 The interesting features of Modules 3.2 that were missing in the
2192 initial Modules 4 release in 2017 have been reintroduced progressively (like
2193 :subcmd:`clear` sub-command or :option:`--icase` search). With Modules 5.0,
2194 the :subcmd:`refresh` sub-command is even changed to the behavior it had on
2195 Modules 3.2. So it is a good time for the big jump.
2196
2197 If you are still using Modules 3.2, please refer to the :ref:`changes`
2198 document that describes the differences of this version compared to the newer
2199 releases.
2200
2201 Improving Modules initialization
2202 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2203
2204 Modules initialization files are now installed by default in the *etc*
2205 directory designated by the :instopt:`--etcdir` installation option. The
2206 initialization configuration file is named :file:`initrc` in this directory,
2207 and the modulepath-specific configuration file is named :file:`modulespath`.
2208 When both files exist, now they are both evaluated instead of just the
2209 :file:`modulespath` file.
2210
2211 Modules magic cookie (i.e., ``#%Module`` file signature) is now required at
2212 the start of :file:`initrc`. An error is produced if the magic cookie is
2213 missing or if the optional version number placed after the cookie string is
2214 higher than the version of the :file:`modulecmd.tcl` script in use.
2215
2216 Note that :file:`initrc` configuration file can host more than
2217 :subcmd:`module use<use>` and :subcmd:`module load<load>` commands.
2218 :command:`module` configuration can also be achieved with this file through
2219 the use of :subcmd:`module config<config>` commands.
2220
2221 Modules initialization has been enhanced for situations where a module
2222 environment is found already defined. In this case the loaded modules are
2223 automatically refreshed which is useful to re-apply the non-persistent
2224 environment configuration (i.e., shell alias and function that are not
2225 exported to the sub-shell). For instance when starting a sub-shell session it
2226 ensures that the loaded environment is fully inherited from parent shell:
2227
2228 .. parsed-literal::
2229
2230 :ps:`$` ml show foo/1.0
2231 -------------------------------------------------------------------
2232 :sgrhi:`/path/to/modulefiles/foo/1.0`:
2233
2234 :sgrcm:`set-alias` foo {echo foo}
2235 -------------------------------------------------------------------
2236 :ps:`$` ml foo/1.0
2237 :ps:`$` alias foo
2238 alias foo='echo foo'
2239 :ps:`$` bash
2240 :ps:`$` ml
2241 Currently Loaded Modulefiles:
2242 1) foo/1.0
2243 :ps:`$` alias foo
2244 alias foo='echo foo'
2245
2246 Simplifying path-like variable reference counting
2247 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2248
2249 The reference counting mechanism used for path-like environment variable
2250 enables to determine if a path entry has been added several times (by several
2251 loaded modules for instance) to know whether or not this path entry should be
2252 unset when unloading a module. Entry is not removed if multiple loaded modules
2253 rely on it.
2254
2255 The mechanism is not applied anymore to the Modules-specific path variables
2256 (like :envvar:`LOADEDMODULES`) as an element entry in these variables cannot
2257 be added multiple times without duplication. For instance, a given module name
2258 and version cannot be added twice in :envvar:`LOADEDMODULES` as this module is
2259 only loaded once. With this change a thinner environment is produced by
2260 :command:`module`. An exception is made for :envvar:`MODULEPATH` environment
2261 variable where the mechanism still applies.
2262
2263 .. parsed-literal::
2264
2265 :ps:`$` $MODULES_CMD bash load foo/2.0
2266 _LMFILES_=/path/to/modulefiles/foo/2.0; export _LMFILES_;
2267 LOADEDMODULES=foo/2.0; export LOADEDMODULES;
2268 __MODULES_LMTAG=foo/2.0\&mytag; export __MODULES_LMTAG;
2269 test 0;
2270
2271 Reference counting mechanism has also been simplified for entries in path-like
2272 variable that are only referred once. For such entries no entry is set in the
2273 reference counting variable (which are now called
2274 :envvar:`__MODULES_SHARE_\<VAR\>`). A reference count entry is set only if the
2275 entry in the path-like variable is referred more than one time.
2276
2277 .. parsed-literal::
2278
2279 :ps:`$` ml foo/3.0
2280 :ps:`$` echo $PATHVAR
2281 /path/to/dir1
2282 :ps:`$` echo $__MODULES_SHARE_PATHVAR
2283
2284 :ps:`$` ml bar/1.0
2285 :ps:`$` echo $PATHVAR
2286 /path/to/dir1
2287 :ps:`$` echo $__MODULES_SHARE_PATHVAR
2288 /path/to/dir1:2
2289 :ps:`$` ml -foo/3.0
2290 :ps:`$` echo $PATHVAR
2291 /path/to/dir1
2292 :ps:`$` echo $__MODULES_SHARE_PATHVAR
2293
2294 :ps:`$`
2295
2296 When the :subcmd:`use` and :subcmd:`unuse` module sub-commands are not called
2297 during a modulefile evaluation, the reference counter associated with each
2298 entry in :envvar:`MODULEPATH` environment variable is ignored. In such
2299 context, a :subcmd:`module use<use>` will not increase the reference counter
2300 of a path entry already defined and a :subcmd:`module unuse<unuse>` will
2301 remove specified path whatever its reference counter value. Same change applies
2302 for :subcmd:`append-path`, :subcmd:`prepend-path` and :subcmd:`remove-path`
2303 module sub-commands when called from the command-line.
2304
2305 .. parsed-literal::
2306
2307 :ps:`$` echo $MODULEPATH
2308 /path/to/modulefiles
2309 :ps:`$` echo $__MODULES_SHARE_MODULEPATH
2310 /path/to/modulefiles:2
2311 :ps:`$` ml use /path/to/modulefiles
2312 :ps:`$` echo $__MODULES_SHARE_MODULEPATH
2313 /path/to/modulefiles:2
2314 :ps:`$` ml unuse /path/to/modulefiles
2315 :ps:`$` echo $MODULEPATH
2316
2317 :ps:`$` echo $__MODULES_SHARE_MODULEPATH
2318
2319 :ps:`$`
2320
2321
2322 v4.8
2323 ----
2324
2325 This new version is backward-compatible with previous version 4 releases. It
2326 fixes bugs but also introduces new functionalities that are described in this
2327 section. See the :ref:`4.8 release notes<4.8 release notes>` for a complete
2328 list of the changes between Modules v4.7 and v4.8.
2329
2330 .. _Editing modulefiles:
2331
2332 Editing modulefiles
2333 ^^^^^^^^^^^^^^^^^^^
2334
2335 :subcmd:`edit` sub-command is introduced to give the ability to open
2336 modulefiles in a text editor. Modulefiles can be specified like with any other
2337 sub-command: using regular, symbolic or aliased names or using advanced
2338 version specifiers.
2339
2340 .. parsed-literal::
2341
2342 :ps:`$` ml edit foo
2343
2344 :subcmd:`edit` sub-command resolves the path toward the designated modulefile
2345 then call configured text editor to open this modulefile with it. Below, the
2346 modulefile is opened with the ``vi`` command:
2347
2348 .. parsed-literal::
2349
2350 #%Module
2351 module-whatis [module-info name]
2352 setenv PATH /path/to/foo-1.0/bin
2353 ~
2354 ~
2355 ~
2356 "/path/to/modulefiles/foo/1.0" 3L, 42B 1,1 All
2357
2358 The :mconfig:`editor` configuration option controls the editor command to use.
2359 This option can be configured at installation time with the
2360 :instopt:`--with-editor` installation option. If not set, :mconfig:`editor`
2361 configuration option is set by default to ``vi``.
2362
2363 :mconfig:`editor` configuration option can be changed with the
2364 :subcmd:`config` sub-command. Which sets the :envvar:`MODULES_EDITOR`
2365 environment variable.
2366
2367 The :envvar:`VISUAL` or the :envvar:`EDITOR` environment variables override
2368 the default value of :mconfig:`editor` configuration option but are overridden
2369 by the :envvar:`MODULES_EDITOR` environment variable.
2370
2371 Using version range in version list
2372 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2373
2374 The :ref:`Advanced module version specifiers` mechanism has been improved to
2375 allow the use of version range (*@:version*, *@vers1:vers2* or *@version:*)
2376 within version list (*@version1,version2,...*).
2377
2378 It is now possible to write for instance ``mod@:1.2,1.4:1.6,1.8:`` to
2379 designate all versions of module *mod*, except versions *1.3* and *1.7*.
2380
2381 This improvement is available where the advanced version specifier syntax is
2382 supported. Thus it can be either used from the command-line or when writing
2383 modulefiles, for instance to hide or tag modules or to declare requirements.
2384
2385 Try module load with no complain if not found
2386 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2387
2388 Add the :subcmd:`try-load` sub-command that tries to load the modulefile
2389 passed as argument, like the :subcmd:`load` sub-command, but does not raise an
2390 error if this modulefile cannot be found.
2391
2392 .. parsed-literal::
2393
2394 :ps:`$` module load unknown
2395 :sgrer:`ERROR`: Unable to locate a modulefile for 'unknown'
2396 :ps:`$` echo $?
2397 1
2398 :ps:`$` module try-load unknown
2399 :ps:`$` echo $?
2400 0
2401 :ps:`$` module list
2402 No Modulefiles Currently Loaded.
2403
2404 This sub-command first introduced by the `Lmod`_ project is added to Modules
2405 to improve the compatibility between the two ``module`` implementations.
2406
2407 :subcmd:`try-load` is also available within modulefile context to continue the
2408 evaluation of a modulefile in case no module is found in its attempt to load
2409 another modulefile
2410
2411 .. parsed-literal::
2412
2413 :ps:`$` module display foo/1.0
2414 -------------------------------------------------------------------
2415 :sgrhi:`/path/to/modulefiles/foo/1.0`:
2416
2417 :sgrcm:`module` try-load unknown/1.0
2418 -------------------------------------------------------------------
2419 :ps:`$` module load foo/1.0
2420 :ps:`$` module list
2421 Currently Loaded Modulefiles:
2422 1) foo/1.0
2423
2424 Module variants
2425 ^^^^^^^^^^^^^^^
2426
2427 :ref:`Module variants` is a new mechanism that allows to pass arguments to
2428 evaluated modulefiles in order to achieve different environment variable or
2429 module requirement setup with a single modulefile.
2430
2431 Variant specification relies on the :ref:`Advanced module version specifiers`
2432 mechanism, which leverages the `variant syntax`_ of the `Spack`_ package
2433 manager:
2434
2435 .. _variant syntax: https://spack.readthedocs.io/en/latest/basic_usage.html#variants
2436
2437 .. parsed-literal::
2438
2439 :ps:`$` module config advanced_version_spec 1
2440 :ps:`$` module load -v bar/1.2 toolchain=a -debug
2441 Loading :sgrhi:`bar/1.2`:sgrse:`{`:sgrva:`-debug`:sgrse:`:`:sgrva:`toolchain=a`:sgrse:`}`
2442
2443 Variants are defined in modulefile with the :mfcmd:`variant` command, which
2444 defines the variant type and its accepted values:
2445
2446 .. code-block:: tcl
2447
2448 #%Module4.8
2449 variant toolchain a b c
2450 variant --boolean --default off debug
2451
2452 # select software build depending on variant values
2453 set suffix -[getvariant toolchain]
2454 if {$ModuleVariant(debug)} {
2455 append suffix -dbg
2456 }
2457
2458 prepend-path PATH /path/to/bar-1.2$suffix/bin
2459 prepend-path LD_LIBRARY_PATH /path/to/bar-1.2$suffix/lib
2460
2461 The *bar/1.2* modulefile defines a ``toolchain`` variant, which accepts the
2462 ``a``, ``b`` and ``c`` values, and a ``debug`` Boolean variant, which is set
2463 ``off`` by default. Once these two variants are declared, their value
2464 specified on module designation are instantiated in the :mfvar:`ModuleVariant`
2465 array variable which could also be queried with the :mfcmd:`getvariant`
2466 modulefile command. Selected variant values enable to define a specific
2467 installation build path for the *bar/1.2* software.
2468
2469 If a variant is not specified when designating module and if this variant is
2470 not declared with a default value, an error is obtained:
2471
2472 .. parsed-literal::
2473
2474 :ps:`$` module purge
2475 :ps:`$` module load :noparse:`[email protected]`
2476 Loading :sgrhi:`bar/1.2`
2477 :sgrer:`ERROR`: No value specified for variant 'toolchain'
2478 Allowed values are: a b c
2479
2480 Once module is loaded, selected variants are reported on the :subcmd:`list`
2481 sub-command output:
2482
2483 .. parsed-literal::
2484
2485 :ps:`$` module load :noparse:`[email protected]` toolchain=b
2486 :ps:`$` module list
2487 Currently Loaded Modulefiles:
2488 1) bar/1.2\ :sgrse:`{`:sgrva:`-debug`:sgrse:`:`:sgrva:`toolchain=b`:sgrse:`}`
2489
2490 Key:
2491 :sgrse:`{`:sgrva:`-variant`:sgrse:`}`\=\ :sgrse:`{`:sgrva:`variant=off`:sgrse:`}` :sgrse:`{`:sgrva:`variant=value`:sgrse:`}`
2492
2493 .. note:: The default value of the :instopt:`--with-list-output` installation
2494 option has been updated to include variant information.
2495
2496 Variant specification could be used where the :ref:`Advanced module version
2497 specifiers` is supported. For instance a module may express a dependency over
2498 a specific module variant:
2499
2500 .. parsed-literal::
2501
2502 :ps:`$` module show foo/2.1 toolchain=c
2503 -------------------------------------------------------------------
2504 :sgrhi:`/path/to/modulefiles/foo/2.1`:
2505
2506 :sgrcm:`variant` toolchain a b c
2507 :sgrcm:`prereq` :noparse:`[email protected] toolchain=`:sgrva:`{toolchain}`
2508 :sgrcm:`prepend-path` PATH /path/to/foo-2.1-:sgrva:`{toolchain}`/bin
2509 :sgrcm:`prepend-path` LD_LIBRARY_PATH /path/to/foo-2.1-:sgrva:`{toolchain}`/lib
2510 -------------------------------------------------------------------
2511
2512 In this example, *foo/2.1* module depends on *bar/1.2* and the same toolchain
2513 variant should be selected for both modules in order to load two software
2514 builds that are compatible between each other.
2515
2516 .. parsed-literal::
2517
2518 :ps:`$` module purge
2519 :ps:`$` module config auto_handling 1
2520 :ps:`$` module load foo/2.1 toolchain=a
2521 Loading :sgrhi:`foo/2.1`:sgrse:`{`:sgrva:`toolchain=a`:sgrse:`}`
2522 :sgrin:`Loading requirement`: bar/1.2\ :sgrse:`{`:sgrva:`-debug`:sgrse:`:`:sgrva:`toolchain=a`:sgrse:`}`
2523
2524 Variant shortcuts
2525 ^^^^^^^^^^^^^^^^^
2526
2527 The :mconfig:`variant_shortcut` configuration option is added to define
2528 shortcut characters for easily specifying variants. Instead of writing the
2529 variant name to specify it in module designation (e.g., *name=value*), the
2530 shortcut associated to this variant could be used (i.e., *<shortcut>value*):
2531
2532 .. parsed-literal::
2533
2534 :ps:`$` module purge
2535 :ps:`$` module config variant_shortcut toolchain=%
2536 :ps:`$` module load foo/2.1 %a
2537 Loading :sgrhi:`foo/2.1`:sgrse:`{`:sgrva:`%a`:sgrse:`}`
2538 :sgrin:`Loading requirement`: bar/1.2\ :sgrse:`{`:sgrva:`-debug`:sgrse:`:`:sgrva:`%a`:sgrse:`}`
2539
2540 Configured shortcuts are also used to report the loaded variant on
2541 :subcmd:`list` sub-command output (shortcuts are explained in key section):
2542
2543 .. parsed-literal::
2544
2545 :ps:`$` module list
2546 Currently Loaded Modulefiles:
2547 1) :sgral:`bar/1.2`\ :sgrse:`{`:sgrva:`-debug`:sgrse:`:`:sgrva:`%a`:sgrse:`}` 2) foo/2.1\ :sgrse:`{`:sgrva:`%a`:sgrse:`}`
2548
2549 Key:
2550 :sgral:`auto-loaded` :sgrse:`{`:sgrva:`-variant`:sgrse:`}`\=\ :sgrse:`{`:sgrva:`variant=off`:sgrse:`}` :sgrse:`{`:sgrva:`%value`:sgrse:`}`\=\ :sgrse:`{`:sgrva:`toolchain=value`:sgrse:`}` :sgrse:`{`:sgrva:`variant=value`:sgrse:`}`
2551
2552
2553 v4.7
2554 ----
2555
2556 This new version is backward-compatible with previous version 4 releases. It
2557 fixes bugs but also introduces new functionalities that are described in this
2558 section. See the :ref:`4.7 release notes<4.7 release notes>` for a complete
2559 list of the changes between Modules v4.6 and v4.7.
2560
2561 Determining module implementation and version
2562 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2563
2564 New Modules variables are introduced to determine during the evaluation of a
2565 modulefile or a modulerc what *module* implementation is currently in use. The
2566 :mfvar:`ModuleTool` variable corresponds to the name of the *module*
2567 implementation and is set to ``Modules`` for this project. The
2568 :mfvar:`ModuleToolVersion` variable corresponds to the version number of the
2569 implementation (e.g. ``4.7.0``).
2570
2571 With these new variables it is possible to precisely know what *module*
2572 command is in use then adapt modulefile code to handle a specific behavior or
2573 leverage a new feature.
2574
2575 The modulefile command :mfcmd:`versioncmp` is also introduced to provide a
2576 simple way to compare two version strings and return if first version string
2577 is less than, equal to or greater than second one.
2578
2579 .. code-block:: tcl
2580
2581 if {[info exists ModuleTool] && $ModuleTool eq {Modules}
2582 && [versioncmp $ModuleToolVersion 4.7] >= 0} {
2583 # here some code specific for Modules 4.7 and later versions
2584 }
2585
2586 The :mfvar:`ModuleTool` and :mfvar:`ModuleToolVersion` variables and the
2587 :mfcmd:`versioncmp` modulefile command are supported by the `Lmod`_ project
2588 starting version ``8.4.8``.
2589
2590 Symbolic version to designate module loaded version
2591 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2592
2593 When the `Advanced module version specifiers`_ is enabled, the ``loaded``
2594 symbolic version may be used to designate the currently loaded version of
2595 specified module.
2596
2597 .. parsed-literal::
2598
2599 :ps:`$` ml display :noparse:`foo@loaded`
2600 -------------------------------------------------------------------
2601 :sgrhi:`/path/to/modulefiles/foo/1.0`:
2602
2603 :sgrcm:`module-whatis` foo/1.0
2604 -------------------------------------------------------------------
2605
2606 If no version of specified module can be found loaded, an error is returned.
2607
2608 .. parsed-literal::
2609
2610 :ps:`$` ml display :noparse:`foo@loaded`
2611 :sgrer:`ERROR`: No loaded version found for 'foo' module
2612
2613 Module tags
2614 ^^^^^^^^^^^
2615
2616 Module tags are piece of information that can be associated to individual
2617 modulefiles. Tags could be purely informational or may lead to specific
2618 behaviors.
2619
2620 Module tags may be inherited from the module state set by a modulefile command
2621 or consequence of a module action. Tags may also be associated to modules by
2622 using the new :mfcmd:`module-tag` modulefile command.
2623
2624 Module tags are reported along the module they are associated to on
2625 :subcmd:`avail` and :subcmd:`list` sub-command results. Tags could be reported
2626 either:
2627
2628 * along the module name, all tags set within angle brackets, each tag
2629 separated from the others with a colon character (e.g.,
2630 ``foo/1.2 <tag1:tag2>``).
2631
2632 .. parsed-literal::
2633
2634 :ps:`$` cat /path/to/modulefiles/foo/.modulerc
2635 #%Module
2636 module-tag mytag foo
2637 module-tag othertag foo/1.0
2638 :ps:`$` ml av
2639 --------------- :sgrdi:`/path/to/modulefiles` ---------------
2640 foo/1.0 <mytag:othertag> foo/2.0 <mytag>
2641 :ps:`$` ml foo/1.0
2642 :ps:`$` ml
2643 Currently Loaded Modulefiles:
2644 1) foo/1.0 <mytag:othertag>
2645
2646 * graphically rendered over the module name for each tag associated to a
2647 Select Graphic Rendition (SGR) code in the color palette (see
2648 :envvar:`MODULES_COLORS`)
2649
2650 .. parsed-literal::
2651
2652 :ps:`$` # set SGR code to report 'mytag' with blue background color
2653 :ps:`$` ml config colors "hi=1:di=94:L=90;47:mytag=102"
2654 :ps:`$` ml av
2655 --------------- :sgrdi:`/path/to/modulefiles` ---------------
2656 :sgrl:`foo`:sgrss:`/1.0` <othertag> :sgrss:`foo/2.0`
2657 :ps:`$` ml
2658 Currently Loaded Modulefiles:
2659 1) :sgrl:`foo`:sgrss:`/1.0` <othertag>
2660
2661 The :mconfig:`tag_abbrev` configuration option is available to define
2662 abbreviated strings for module tags and then use these abbreviations instead
2663 of tag names when reporting tags on :subcmd:`avail` and :subcmd:`list` command
2664 results.
2665
2666 .. parsed-literal::
2667
2668 :ps:`$` # add abbreviation for 'othertag' tag
2669 :ps:`$` ml config tag_abbrev loaded=L:othertag=oT
2670 :ps:`$` ml av
2671 --------------- :sgrdi:`/path/to/modulefiles` ---------------
2672 :sgrl:`foo`:sgrss:`/1.0` <oT> :sgrss:`foo/2.0`
2673 :ps:`$` ml
2674 Currently Loaded Modulefiles:
2675 1) :sgrl:`foo`:sgrss:`/1.0` <oT>
2676
2677 When a SGR code is set for a tag in the color palette, this graphical
2678 rendition is applied by default over the module name and the tag name or its
2679 abbreviation is not displayed. If tag name or abbreviation is added to the
2680 :mconfig:`tag_color_name` configuration option, graphical rendering is applied
2681 to the tag name or abbreviation rather than over the module name they are
2682 attached to.
2683
2684 .. parsed-literal::
2685
2686 :ps:`$` # add SGR code for 'oT' tag and set rendition over tag name
2687 :ps:`$` ml config colors "hi=1:di=94:L=90;47:mytag=44:oT=41"
2688 :ps:`$` ml config tag_color_name oT
2689 :ps:`$` ml av
2690 --------------- :sgrdi:`/path/to/modulefiles` ---------------
2691 :sgrl:`foo`:sgrss:`/1.0` <:sgrf:`oT`> :sgrss:`foo/2.0`
2692 :ps:`$` ml
2693 Currently Loaded Modulefiles:
2694 1) :sgrl:`foo`:sgrss:`/1.0` <:sgrf:`oT`>
2695
2696 Tags inherited from module state, consequence of a module action or set by
2697 using :mfcmd:`module-tag` but that have a special meaning currently are:
2698
2699 .. only:: html
2700
2701 +------------------+-------------------------+------------------------+-------+------------------------+
2702 | Tag | Description | Set with | Abbr. | Color |
2703 | | | | | |
2704 +==================+=========================+========================+=======+========================+
2705 | auto-loaded | Module has been loaded | Inherited | aL | .. parsed-literal:: |
2706 | | automatically | | | |
2707 | | | | | :sgral:`mod/1.0` |
2708 +------------------+-------------------------+------------------------+-------+------------------------+
2709 | forbidden | Module cannot be loaded | Inherited from | F | .. parsed-literal:: |
2710 | | | :mfcmd:`module-forbid` | | |
2711 | | | | | :sgrf:`mod/1.0` |
2712 +------------------+-------------------------+------------------------+-------+------------------------+
2713 | hidden | Module is not visible | Inherited from | H | .. parsed-literal:: |
2714 | | on :subcmd:`avail` | :mfcmd:`module-hide` | | |
2715 | | | | | :sgrh:`mod/1.0` |
2716 +------------------+-------------------------+------------------------+-------+------------------------+
2717 | hidden-loaded | See `Hiding loaded | Inherited from | H | .. parsed-literal:: |
2718 | | modules`_ | :mfcmd:`module-hide` | | |
2719 | | | | | :sgrh:`mod/1.0` |
2720 +------------------+-------------------------+------------------------+-------+------------------------+
2721 | loaded | Module is currently | Inherited | L | .. parsed-literal:: |
2722 | | loaded | | | |
2723 | | | | | :sgrl:`mod/1.0` |
2724 +------------------+-------------------------+------------------------+-------+------------------------+
2725 | nearly-forbidden | Module will soon not be | Inherited from | nL | .. parsed-literal:: |
2726 | | able to load anymore | :mfcmd:`module-forbid` | | |
2727 | | | | | :sgrnf:`mod/1.0` |
2728 +------------------+-------------------------+------------------------+-------+------------------------+
2729 | sticky | See `Sticky modules`_ | :mfcmd:`module-tag` | S | .. parsed-literal:: |
2730 | | | | | |
2731 | | | | | :sgrs:`mod/1.0` |
2732 +------------------+-------------------------+------------------------+-------+------------------------+
2733 | super-sticky | See `Sticky modules`_ | :mfcmd:`module-tag` | sS | .. parsed-literal:: |
2734 | | | | | |
2735 | | | | | :sgrss:`mod/1.0` |
2736 +------------------+-------------------------+------------------------+-------+------------------------+
2737
2738 .. only:: not html
2739
2740 +------------------+-------------------------+------------------------+-------+
2741 | Tag | Description | Set with | Abbr. |
2742 | | | | |
2743 +==================+=========================+========================+=======+
2744 | auto-loaded | Module has been loaded | Inherited | aL |
2745 | | automatically | | |
2746 | | | | |
2747 +------------------+-------------------------+------------------------+-------+
2748 | forbidden | Module cannot be loaded | Inherited from | F |
2749 | | | :mfcmd:`module-forbid` | |
2750 | | | | |
2751 +------------------+-------------------------+------------------------+-------+
2752 | hidden | Module is not visible | Inherited from | H |
2753 | | on :subcmd:`avail` | :mfcmd:`module-hide` | |
2754 | | | | |
2755 +------------------+-------------------------+------------------------+-------+
2756 | hidden-loaded | See `Hiding loaded | Inherited from | H |
2757 | | modules`_ | :mfcmd:`module-hide` | |
2758 | | | | |
2759 +------------------+-------------------------+------------------------+-------+
2760 | loaded | Module is currently | Inherited | L |
2761 | | loaded | | |
2762 | | | | |
2763 +------------------+-------------------------+------------------------+-------+
2764 | nearly-forbidden | Module will soon not be | Inherited from | nL |
2765 | | able to load anymore | :mfcmd:`module-forbid` | |
2766 | | | | |
2767 +------------------+-------------------------+------------------------+-------+
2768 | sticky | See `Sticky modules`_ | :mfcmd:`module-tag` | S |
2769 | | | | |
2770 | | | | |
2771 +------------------+-------------------------+------------------------+-------+
2772 | super-sticky | See `Sticky modules`_ | :mfcmd:`module-tag` | sS |
2773 | | | | |
2774 | | | | |
2775 +------------------+-------------------------+------------------------+-------+
2776
2777 Hiding loaded modules
2778 ^^^^^^^^^^^^^^^^^^^^^
2779
2780 The ``--hidden-loaded`` option has been added to the :mfcmd:`module-hide`
2781 modulefile command and it indicates that designated hidden modules remain
2782 hidden after being loaded.
2783
2784 .. parsed-literal::
2785
2786 :ps:`$` cat /path/to/modulefiles/foo/1.0
2787 #%Module
2788 module load bar
2789 :ps:`$` cat /path/to/modulefiles/bar/.modulerc
2790 #%Module4.7
2791 module-hide --soft --hidden-loaded bar
2792
2793 In this example, *foo* depends on *bar* which is set soft hidden and hidden
2794 once loaded. As a consequence, automated load of *bar* module will not be
2795 reported and *bar/1.0* will not appear in loaded module list by default:
2796
2797 .. parsed-literal::
2798
2799 :ps:`$` ml foo
2800 :ps:`$` ml
2801 Currently Loaded Modulefiles:
2802 1) foo/1.0
2803
2804 However *bar/1.0* is loaded. Hidden loaded modules can be unveiled with the
2805 :option:`--all`/:option:`-a` option set on the :subcmd:`list` sub-command.
2806 ``hidden-loaded`` tag (abbreviated by default to ``H`` when colored output is
2807 disabled) applies to such modules.
2808
2809 .. parsed-literal::
2810
2811 :ps:`$` ml -a
2812 Currently Loaded Modulefiles:
2813 1) :sgral:`bar`:sgrh:`/1.0` 2) foo/1.0
2814
2815 To also get the informational messages about hidden loaded module automated
2816 load or unload, the new verbosity level ``verbose2`` can be used (with
2817 :option:`-vv` option for instance):
2818
2819 .. parsed-literal::
2820
2821 :ps:`$` ml purge
2822 :ps:`$` ml -vv foo
2823 Loading bar/1.0
2824
2825 Loading :sgrhi:`foo/1.0`
2826 :sgrin:`Loading requirement`: bar/1.0
2827
2828 Sticky modules
2829 ^^^^^^^^^^^^^^
2830
2831 Module stickiness is introduced, in a similar fashion than on the `Lmod`_
2832 project, to allow to glue modules to the loaded environment. A sticky module
2833 cannot be unloaded, unless if the unload action is forced or if the module
2834 reloads after being unloaded.
2835
2836 A modulefile is declared *sticky* by applying it the ``sticky`` tag with the
2837 :mfcmd:`module-tag` modulefile command.
2838
2839 .. parsed-literal::
2840
2841 :ps:`$` cat /path/to/modulefiles/foo/.modulerc
2842 #%Module4.7
2843 module-tag sticky foo/1.0
2844 :ps:`$` ml
2845 Currently Loaded Modulefiles:
2846 1) :sgrs:`foo/1.0`
2847 :ps:`$` ml -foo
2848 Unloading :sgrhi:`foo/1.0`
2849 :sgrer:`ERROR`: Unload of sticky module 'foo/1.0' skipped
2850 :ps:`$` ml
2851 Currently Loaded Modulefiles:
2852 1) :sgrs:`foo/1.0`
2853 :ps:`$` ml --force -foo
2854 Unloading :sgrhi:`foo/1.0`
2855 :sgrwa:`WARNING`: Unload of sticky module 'foo/1.0' forced
2856 :ps:`$` ml
2857 No Modulefiles Currently Loaded.
2858
2859
2860 Modulefile can also be defined ``super-sticky`` by applying the corresponding
2861 module tag. *Super-sticky* module cannot be unloaded even if the unload action
2862 is forced. It can only be unloaded if the module reloads afterward.
2863
2864 .. parsed-literal::
2865
2866 :ps:`$` cat /path/to/modulefiles/bar/.modulerc
2867 #%Module4.7
2868 module-tag super-sticky bar/1.0
2869 :ps:`$` ml
2870 Currently Loaded Modulefiles:
2871 1) :sgrss:`bar/1.0`
2872 :ps:`$` ml purge
2873 Unloading :sgrhi:`bar/1.0`
2874 :sgrer:`ERROR`: Unload of super-sticky module 'bar/1.0' skipped
2875 :ps:`$` ml purge -f
2876 Unloading :sgrhi:`bar/1.0`
2877 :sgrer:`ERROR`: Unload of super-sticky module 'bar/1.0' skipped
2878 :ps:`$` ml
2879 Currently Loaded Modulefiles:
2880 1) :sgrss:`bar/1.0`
2881
2882 Modulefiles targeted by a ``sticky`` or a ``super-sticky`` tag are colored on
2883 :subcmd:`avail` and :subcmd:`list` sub-command outputs to indicate such tag
2884 applies. If colored output is disabled a tag abbreviation is reported along
2885 module designation (respectively ``S`` and ``sS``).
2886
2887 In case the stickiness applies to the generic module name (and does not target
2888 a specific module version or version-set), one version of the sticky or
2889 super-sticky module can be swapped by another version of this same module:
2890
2891 .. parsed-literal::
2892
2893 :ps:`$` cat /path/to/modulefiles/baz/.modulerc
2894 #%Module4.7
2895 module-tag sticky baz
2896 :ps:`$` ml
2897 Currently Loaded Modulefiles:
2898 1) :sgrs:`baz/2.0`
2899 :ps:`$` ml switch baz/1.0
2900 :ps:`$` ml
2901 Currently Loaded Modulefiles:
2902 1) :sgrs:`baz/1.0`
2903
2904 Explaining avail/list output
2905 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2906
2907 A *Key* section is added at the end of the :subcmd:`avail` and :subcmd:`list`
2908 sub-commands output to give hints on the meaning of the graphical rendition
2909 applied to elements or what the elements set in parentheses or chevrons along
2910 module name stand for.
2911
2912 .. parsed-literal::
2913
2914 :ps:`$` ml av
2915 ------------------ :sgrdi:`/path/to/modulefiles` ------------------
2916 :sgrs:`foo`:sgrl:`/1.0` <oT> :sgrde:`foo/2.0` :sgrali:`foo/3.0`
2917
2918 Key:
2919 :sgrl:`loaded` :sgrde:`default-version` :sgrs:`sticky` <oT>=othertag
2920 :sgrdi:`modulepath` :sgrali:`module-alias` <module-tag>
2921
2922 Configuring avail/list output
2923 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2924
2925 New configuration options are introduced to control what content to output in
2926 addition to modules names on the regular and terse output modes of the
2927 :subcmd:`avail` and :subcmd:`list` sub-commands.
2928
2929 These new configuration options named :mconfig:`avail_output`,
2930 :mconfig:`avail_terse_output`, :mconfig:`list_output` and
2931 :mconfig:`list_terse_output` can be updated using the :subcmd:`config`
2932 sub-command or set at installation time respectively with the
2933 :instopt:`--with-avail-output`, :instopt:`--with-avail-terse-output`,
2934 :instopt:`--with-list-output` and :instopt:`--with-list-terse-output`
2935 configure options.
2936
2937 The four options accept a colon separated list of elements as value. Accepted
2938 elements for the :subcmd:`avail`-related options are: ``modulepath``,
2939 ``alias``, ``dirwsym``, ``sym``, ``tag`` and ``key``. Accepted elements for
2940 the :subcmd:`list`-related options are: ``header``, ``idx``, ``sym``, ``tag``
2941 and ``key``.
2942
2943 In the following example, default output configuration for the :subcmd:`avail`
2944 sub-command is checked then module tags and key section are removed to get a
2945 simpler output:
2946
2947 .. parsed-literal::
2948
2949 :ps:`$` ml config avail_output
2950 Modules Release 4.7.0 (2021-02-19)
2951
2952 - Config. name ---------.- Value (set by if default overridden) ---------------
2953 avail_output modulepath:alias:dirwsym:sym:tag:key
2954 :ps:`$` ml av
2955 ------------------ :sgrdi:`/path/to/modulefiles` ------------------
2956 :sgrde:`bar/1.0` bar/2.0 :sgrf:`foo/1.0` :sgrs:`foo/2.0` :sgrali:`foo/2.2`
2957
2958 Key:
2959 :sgrdi:`modulepath` :sgrali:`module-alias` :sgrs:`sticky`
2960 :sgrde:`default-version` :sgrf:`forbidden`
2961 :ps:`$` ml config avail_output modulepath:alias:dirwsym:sym
2962 :ps:`$` ml av
2963 ------------------ :sgrdi:`/path/to/modulefiles` ------------------
2964 :sgrde:`bar/1.0` bar/2.0 foo/1.0 foo/2.0 :sgrali:`foo/2.2`
2965
2966 The :option:`--output`/:option:`-o` switches are added to define a specific
2967 output configuration for the duration of the associated command line. The
2968 following example shows how to limit the content reported on a module
2969 :subcmd:`list` to the loaded index and the symbolic versions in addition to
2970 the module names:
2971
2972 .. parsed-literal::
2973
2974 :ps:`$` ml
2975 Currently Loaded Modulefiles:
2976 1) :sgrde:`bar/1.0` 2) :sgrs:`foo/2.0`
2977
2978 Key:
2979 :sgrde:`default-version` :sgrs:`sticky`
2980 :ps:`$` ml -o idx:sym
2981 1) :sgrde:`bar/1.0` 2) foo/2.0
2982
2983 When the new configuration options or command line switches are set to an
2984 empty value, the module names are the sole information reported:
2985
2986 .. parsed-literal::
2987
2988 :ps:`$` ml -t -o ""
2989 bar/1.0
2990 foo/2.0
2991
2992 In case the ``modulepath`` element is withdrawn from the :subcmd:`avail`
2993 sub-command output configuration, the available modules from all enabled
2994 modulepaths are reported as a single list:
2995
2996 .. parsed-literal::
2997
2998 :ps:`$` ml av
2999 --------------- :sgrdi:`/path/to/other/modulefiles` ---------------
3000 baz/1.0 baz/2.0
3001
3002 ------------------ :sgrdi:`/path/to/modulefiles` ------------------
3003 :sgrde:`bar/1.0` bar/2.0 :sgrf:`foo/1.0` :sgrs:`foo/2.0` :sgrali:`foo/2.2`
3004
3005 Key:
3006 :sgrdi:`modulepath` :sgrali:`module-alias` :sgrs:`sticky`
3007 :sgrde:`default-version` :sgrf:`forbidden`
3008 :ps:`$` ml av --output=alias:tag
3009 bar/1.0 baz/1.0 :sgrf:`foo/1.0` :sgrali:`foo/2.2`
3010 bar/2.0 baz/2.0 :sgrs:`foo/2.0`
3011
3012 .. note:: The ``avail_report_dir_sym`` and ``avail_report_mfile_sym`` locked
3013 configuration options have been removed. Their behaviors can now be
3014 obtained by respectively adding the ``dirwsym`` and ``sym`` elements to the
3015 :mconfig:`avail_output` or :mconfig:`avail_terse_output` configuration
3016 options.
3017
3018 v4.6
3019 ----
3020
3021 This new version is backward-compatible with previous version 4 releases. It
3022 fixes bugs but also introduces new functionalities that are described in this
3023 section. See the :ref:`4.6 release notes<4.6 release notes>` for a complete
3024 list of the changes between Modules v4.5 and v4.6.
3025
3026 .. _sh-to-mod_sub-command:
3027
3028 sh-to-mod sub-command
3029 ^^^^^^^^^^^^^^^^^^^^^
3030
3031 The :subcmd:`sh-to-mod` sub-command is added to output as a modulefile content
3032 the environment changes done by the evaluation of a shell script passed as
3033 argument. :subcmd:`sh-to-mod` is especially useful for software providing a
3034 shell script for their enablement in shell session: it can convert these
3035 scripts into modulefiles.
3036
3037 Say for instance, a *foo* software has been installed and it provides a
3038 ``foo-setup.sh`` script to activate *foo* software in user environment:
3039
3040 .. parsed-literal::
3041
3042 :ps:`$` cat /path/to/foo-1.2/foo-setup.sh
3043 #!/bin/sh
3044 export FOOENV="$1"
3045 export PATH=/path/to/foo-1.2/bin:$PATH
3046 alias foo='foobin -q -l'
3047
3048 Calling ``module sh-to-mod`` on this shell script outputs the environment
3049 changes it performs as a modulefile content:
3050
3051 .. parsed-literal::
3052
3053 :ps:`$` module sh-to-mod sh /path/to/foo-1.2/foo-setup.sh arg1
3054 #%Module
3055 prepend-path PATH /path/to/foo-1.2/bin
3056 set-alias foo {foobin -q -l}
3057 setenv FOOENV arg1
3058
3059 Changes on environment variables, shell aliases, shell functions and current
3060 working directory are tracked. The following shells are supported: *sh*,
3061 *dash*, *csh*, *tcsh*, *bash*, *ksh*, *ksh93*, *zsh* and *fish*.
3062
3063 :subcmd:`sh-to-mod` acts as a full replacement for the standalone
3064 :command:`createmodule.sh` and :command:`createmodule.py` scripts. However
3065 those two scripts are currently still provided for compatibility purpose.
3066
3067 .. _source-sh_modulefile_command:
3068
3069 source-sh modulefile command
3070 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3071
3072 The :mfcmd:`source-sh` modulefile command is introduced to source environment
3073 changes done by the evaluation of a shell script passed as argument. With
3074 newly introduced :subcmd:`sh-to-mod` sub-command resulting environment changes
3075 done by script are output as modulefile commands. :mfcmd:`source-sh` applies
3076 those modulefile commands as if they were directly written in loading
3077 modulefile.
3078
3079 :mfcmd:`source-sh` is useful for software providing a shell script for their
3080 enablement. If you want to enable such software with :command:`module` yet
3081 using shell script provided by software for this task, just write a modulefile
3082 using :mfcmd:`source-sh` command to call the shell script.
3083
3084 Keeping the same example used to describe :subcmd:`sh-to-mod` sub-command:
3085 *foo* software provides a ``foo-setup.sh`` script for its activation. Create a
3086 modulefile ``foo/1.2`` that calls this script:
3087
3088 .. parsed-literal::
3089
3090 :ps:`$` cat /path/to/modulefiles/foo/1.2
3091 #%Module4.6
3092 source-sh sh /path/to/foo-1.2/foo-setup.sh arg1
3093
3094 Displaying this modulefile indicates the environment changes done by script:
3095
3096 .. parsed-literal::
3097
3098 :ps:`$` module display foo/1.2
3099 -------------------------------------------------------------------
3100 :sgrhi:`/path/to/modulefiles/foo/1.2`:
3101
3102 :sgrcm:`prepend-path` PATH /path/to/foo-1.2/bin
3103 :sgrcm:`set-alias` foo {foobin -q -l}
3104 :sgrcm:`setenv` FOOENV arg1
3105 -------------------------------------------------------------------
3106
3107 Loading the modulefile applies the environment changes seen above:
3108
3109 .. parsed-literal::
3110
3111 :ps:`$` module load -v foo/1.2
3112 Loading :sgrhi:`foo/1.2`
3113 :ps:`$` echo $FOOENV
3114 arg1
3115 :ps:`$` alias foo
3116 alias foo='foobin -q -l'
3117
3118 Track of these changes is kept in user environment to be able to undo them
3119 when modulefile is unloaded:
3120
3121 .. parsed-literal::
3122
3123 :ps:`$` module unload -v foo/1.2
3124 Unloading :sgrhi:`foo/1.2`
3125 :ps:`$` echo $FOOENV
3126
3127 :ps:`$` alias foo
3128 bash: alias: foo: not found
3129
3130 Changes on environment variables, shell aliases, shell functions and current
3131 working directory are tracked. The following shells are supported: *sh*,
3132 *dash*, *csh*, *tcsh*, *bash*, *ksh*, *ksh93*, *zsh* and *fish*.
3133
3134 Querying user's name and groups membership
3135 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3136
3137 Two new sub-commands are introduced for the :mfcmd:`module-info` modulefile
3138 command: ``username`` and ``usergroups``. They respectively fetch the name of
3139 the user currently running :file:`modulecmd.tcl` or the name of all the groups
3140 this user is member of.
3141
3142 These two new modulefile commands can help to adapt code to specific users or
3143 groups. Like for instance to instantiate a modulefile for each group the user
3144 is member of:
3145
3146 .. parsed-literal::
3147
3148 :ps:`$` cat /path/to/modulefiles/foo/.modulerc
3149 #%Module4.6
3150 foreach grp [module-info usergroups] {
3151 module-virtual foo/$grp .common
3152 }
3153 :ps:`$` id -G -n
3154 grp1 grp2 grp3
3155 :ps:`$` module avail
3156 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3157 foo/grp1 foo/grp2 foo/grp3
3158
3159 ``username`` and ``usergroups`` sub-commands of :mfcmd:`module-info`
3160 modulefile command are only supported on Unix platform.
3161
3162 Hiding modules
3163 ^^^^^^^^^^^^^^
3164
3165 The newly introduced :mfcmd:`module-hide` modulefile command enables to
3166 dynamically hide modulefiles, module aliases or symbolic versions specified to
3167 it:
3168
3169 .. parsed-literal::
3170
3171 :ps:`$` cat /path/to/modulefiles/bar/.modulerc
3172 #%Module4.6
3173 module-version bar/1.0 old
3174 # hide 'old' symbolic version
3175 module-hide bar/old
3176 # hide all version 2 and above
3177 module-hide bar@2:
3178 :ps:`$` cat /path/to/modulefiles/.modulerc
3179 #%Module4.6
3180 # hide all versions of foo module
3181 module-hide foo
3182
3183 :mfcmd:`module-hide` commands should be placed in module rc files and can
3184 leverage the `Advanced module version specifiers`_ syntax as shown in the
3185 above example.
3186
3187 Hidden modules are excluded from available module search or module selection
3188 unless query refers to hidden module by its exact name:
3189
3190 .. parsed-literal::
3191
3192 :ps:`$` ml av
3193 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3194 bar/1.0 bar/2.0
3195 :ps:`$` module load -v foo
3196 :sgrer:`ERROR`: Unable to locate a modulefile for 'foo'
3197 :ps:`$` module load -v foo/1.0
3198 Loading foo/1.0
3199 :ps:`$` module avail bar/old
3200 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3201 bar/1.0\ :sgrse:`(`:sgrsy:`old`:sgrse:`)`
3202
3203 :mfcmd:`module-hide` command accepts a ``--soft`` option to apply a lighter of
3204 hiding to modules:
3205
3206 .. parsed-literal::
3207
3208 :ps:`$` cat /path/to/modulefiles/qux/.modulerc
3209 #%Module4.6
3210 # softly hide all qux modules
3211 module-hide --soft qux
3212
3213 The soft hiding mode enables to hide modules from full availability listing
3214 yet keeping the ability to select such module for load without having to use
3215 module exact name:
3216
3217 .. parsed-literal::
3218
3219 :ps:`$` ml av
3220 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3221 bar/1.0 bar/2.0
3222 :ps:`$` ml av qux
3223 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3224 qux/1.0 qux/2.0
3225 :ps:`$` module load -v qux
3226 Loading :sgrhi:`qux/2.0`
3227
3228 Alternatively, a ``--hard`` option can be set on :mfcmd:`module-hide` command
3229 to ensure designated modules do not unveil even if referred by their exact
3230 name:
3231
3232 .. parsed-literal::
3233
3234 :ps:`$` cat /path/to/modulefiles/qux/.modulerc
3235 #%Module4.6
3236 # softly hide all qux modules
3237 module-hide --soft qux
3238 # set highest version of qux hard hidden
3239 module-hide --hard qux/3.0
3240 :ps:`$` ml av qux/3.0
3241 :ps:`$` ml qux/3.0
3242 :sgrer:`ERROR`: Unable to locate a modulefile for 'qux/3.0'
3243
3244 Some users or groups can be set unaffected by hiding mechanism with
3245 the ``--not-user`` or ``--not-group`` options:
3246
3247 .. parsed-literal::
3248
3249 :ps:`$` cat /path/to/modulefiles/quuz/.modulerc
3250 #%Module4.6
3251 # hiding does not apply to grp1 and grp2 groups
3252 module-hide --not-group {grp1 grp2} quuz
3253
3254 .. parsed-literal::
3255
3256 :ps:`$` id --groups --name
3257 grp1 grp7
3258 :ps:`$` ml av quuz
3259 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3260 quuz/1.0 quuz/2.0
3261 :ps:`$` ml -v quuz
3262 Loading :sgrhi:`quuz/2.0`
3263
3264 Hiding mechanism can also be set effective only before or after a given date
3265 time with the ``--before`` and ``--after`` options. Accepted date time format
3266 is ``YYYY-MM-DD[THH:MM]``.
3267
3268 .. parsed-literal::
3269
3270 :ps:`$` cat /path/to/modulefiles/fum/.modulerc
3271 #%Module4.6
3272 # hide only before a given date
3273 module-hide --hard --before 2020-09-01T12:00 fum/1.0
3274 # hide only after a given date
3275 module-hide --hard --after 2020-09-01 fum/2.0
3276
3277 .. parsed-literal::
3278
3279 :ps:`$` date
3280 Fri 04 Sep 2020 06:21:48 AM CEST
3281 :ps:`$` ml av fum
3282 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3283 fum/1.0
3284
3285 Hidden modules can be included in available module searches if option
3286 :option:`--all`/:option:`-a` is set on :subcmd:`avail`, :subcmd:`aliases`,
3287 :subcmd:`whatis` or :subcmd:`search` sub-commands. Hard hidden modules are
3288 unaffected by this option and stay hidden.
3289
3290 .. parsed-literal::
3291
3292 :ps:`$` ml av -a
3293 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3294 bar/1.0\ :sgrse:`(`:sgrsy:`old`:sgrse:`)` foo/1.0 fum/1.0 quuz/2.0 qux/2.0
3295 bar/2.0 foo/2.0 quuz/1.0 qux/1.0
3296
3297 Forbidding use of modules
3298 ^^^^^^^^^^^^^^^^^^^^^^^^^
3299
3300 The :mfcmd:`module-forbid` modulefile command is added to dynamically forbid
3301 the evaluation of modulefiles it specifies. When forbidden, a module cannot be
3302 loaded and an access error is returned when an attempt is made to evaluate it.
3303
3304 .. parsed-literal::
3305
3306 :ps:`$` cat /path/to/modulefiles/foo/.modulerc
3307 #%Module4.6
3308 module-forbid foo@1:
3309 :ps:`$` ml foo/1.0
3310 :sgrer:`ERROR`: Access to module 'foo/1.0' is denied
3311 :ps:`$` ml
3312 No Modulefiles Currently Loaded.
3313
3314 :mfcmd:`module-forbid` statements can be coupled with :mfcmd:`module-hide`
3315 statements to hide modules in addition to forbid their use.
3316 :mfcmd:`module-forbid` supports the ``--not-user``, ``--not-group``,
3317 ``--before`` and ``--after`` options to still allow some users or forbid
3318 modules before or after a given date time.
3319
3320 An additional error message can be defined with the ``--message`` option
3321 to guide for instance users when they try to evaluate a forbidden module:
3322
3323 .. parsed-literal::
3324
3325 :ps:`$` cat /path/to/modulefiles/bar/.modulerc
3326 #%Module4.6
3327 module-forbid --message {Software bar/1.0 is decommissioned, please now use\\
3328 bar/2.0} --after 2020-09-01 bar/1.0
3329 :ps:`$` ml bar/1.0
3330 :sgrer:`ERROR`: Access to module 'bar/1.0' is denied
3331 Software bar/1.0 is decommissioned, please now use bar/2.0
3332
3333 When an evaluated module will soon be forbidden, a message is returned to the
3334 user to warn them of the near limit. An additional warning message can also be
3335 defined here with the ``--nearly-message`` option to guide users.
3336
3337 .. parsed-literal::
3338
3339 :ps:`$` cat /path/to/modulefiles/qux/.modulerc
3340 #%Module4.6
3341 module-forbid --nearly-message {Version 1.0 will soon expire, please now use\\
3342 version 2.0} --after 2020-09-15 qux/1.0
3343 :ps:`$` date
3344 Tue 08 Sep 2020 06:49:43 AM CEST
3345 :ps:`$` ml qux/1.0
3346 Loading :sgrhi:`qux/1.0`
3347 :sgrwa:`WARNING`: Access to module will be denied starting '2020-09-15'
3348 Version 1.0 will soon expire, please now use version 2.0
3349
3350 The range of time the *nearly forbidden* warning appears can be controlled
3351 with the :mconfig:`nearly_forbidden_days` configuration option, whose value
3352 equals to the number of days prior the module starts to be forbidden. This
3353 configuration is set to ``14`` (days) by default and this value can be
3354 controlled at :file:`configure` time with
3355 :instopt:`--with-nearly-forbidden-days` option. When the
3356 :mconfig:`nearly_forbidden_days` configuration is set through the
3357 :subcmd:`config` sub-command, the :envvar:`MODULES_NEARLY_FORBIDDEN_DAYS`
3358 environment variable is set.
3359
3360 Tracing module execution
3361 ^^^^^^^^^^^^^^^^^^^^^^^^
3362
3363 The ``trace`` verbosity is introduced between the ``verbose`` and ``debug``
3364 levels to report details on module searches, resolutions, selections and
3365 evaluations. Trace mode can be enabled by setting the :mconfig:`verbosity`
3366 config to the ``trace`` value or by using the :option:`-T`/:option:`--trace`
3367 command-line switches.
3368
3369 To specifically render trace messages, the ``tr`` key is added to the color
3370 palette with a default value of ``2`` (decreased intensity).
3371
3372 .. parsed-literal::
3373
3374 :ps:`$` ml -T foo
3375 :sgrtr:`Evaluate modulerc: '/path/to/modulefiles/.modulerc'`
3376 :sgrtr:`Get modules: {foo} matching 'foo' in '/path/to/modulefiles'`
3377 :sgrtr:`Resolve: 'foo' into 'bar'`
3378 :sgrtr:`Get modules: {bar bar/1.0} matching 'bar' in '/path/to/modulefiles'`
3379 :sgrtr:`Select module: 'bar/1.0' (/path/to/modulefiles/bar/1.0) matching 'bar/1.0'`
3380
3381 Loading :sgrhi:`bar/1.0`
3382 :sgrtr:`Evaluate modulefile: '/path/to/modulefiles/bar/1.0' as 'bar/1.0'`
3383
3384
3385 v4.5
3386 ----
3387
3388 This new version is backward-compatible with previous version 4 releases. It
3389 fixes bugs but also introduces new functionalities that are described in this
3390 section. See the :ref:`4.5 release notes<4.5 release notes>` for a complete
3391 list of the changes between Modules v4.4 and v4.5.
3392
3393 ml command
3394 ^^^^^^^^^^
3395
3396 The ``ml`` command is added to Modules. ``ml`` is a frontend to the ``module``
3397 command that reduces the number of characters to type to trigger module
3398 actions.
3399
3400 With no argument provided ``ml`` is equivalent to ``module list``, ``ml foo``
3401 corresponds to ``module load foo`` and ``ml -foo`` means ``module unload
3402 foo``:
3403
3404 .. parsed-literal::
3405
3406 :ps:`$` ml foo
3407 :ps:`$` ml
3408 Currently Loaded Modulefiles:
3409 1) foo/2
3410 :ps:`$` ml -foo
3411 :ps:`$` ml
3412 No Modulefiles Currently Loaded.
3413
3414 Multiple modules to either load or unload can be combined on a single command.
3415 The unloads are first processed then the loads.
3416
3417 ``ml`` accepts all command-line switches and sub-commands accepted by
3418 ``module`` command:
3419
3420 .. parsed-literal::
3421
3422 :ps:`$` ml avail -t foo
3423 foo/1
3424 foo/2
3425
3426 This handy interface has been originally developed by the `Lmod`_ project.
3427 Having this command line interface also supported on Modules helps to provide
3428 a similar user experience whatever the module implementation used.
3429
3430 .. _Lmod: https://github.com/TACC/Lmod
3431
3432 JSON format output
3433 ^^^^^^^^^^^^^^^^^^
3434
3435 The ``-j`` and ``--json`` command line switches are added for the ``avail``,
3436 ``list``, ``savelist``, ``whatis`` and ``search`` module sub-commands. When
3437 set, the output result of these sub-commands is rendered in `JSON`_ format:
3438
3439 .. parsed-literal::
3440
3441 :ps:`$` module avail --json bar | python -mjson.tool
3442 {
3443 "/path/to/modulefiles": {
3444 "bar/2.3": {
3445 "name": "bar/2.3",
3446 "pathname": "/path/to/modulefiles/bar/2.3",
3447 "symbols": [
3448 "default"
3449 ],
3450 "type": "modulefile"
3451 },
3452 "bar/3.4": {
3453 "name": "bar/3.4",
3454 "pathname": "/path/to/modulefiles/bar/3.4",
3455 "symbols": [],
3456 "type": "modulefile"
3457 }
3458 }
3459 }
3460 :ps:`$` ml whatis -j foo/1.2.3 | python -mjson.tool
3461 {
3462 "/path/to/modulefiles": {
3463 "foo/1.2.3": {
3464 "name": "foo/1.2.3",
3465 "whatis": [
3466 "The foo/1.2.3 modulefile"
3467 ]
3468 }
3469 }
3470 }
3471
3472 .. _JSON: https://tools.ietf.org/html/rfc8259
3473
3474 Improved Windows support
3475 ^^^^^^^^^^^^^^^^^^^^^^^^
3476
3477 A new option to the ``./configure`` script named
3478 :instopt:`--enable-windows-support` is introduced to install additional files
3479 relative to the enablement of Modules on the Windows platform. When set, this
3480 option installs ``module.cmd``, ``ml.cmd`` and ``envml.cmd`` scripts in
3481 ``bindir`` and initialization script ``cmd.cmd`` in ``initdir``. With these
3482 four files the Modules installation may be used from either a Unix or a
3483 Windows platform.
3484
3485 ``module.cmd``, ``ml.cmd`` and ``envml.cmd`` scripts respectively provide the
3486 ``module``, ``ml`` and ``envml`` commands for Windows ``cmd`` terminal shell,
3487 relying on ``modulecmd.tcl`` script which was already able to produce shell
3488 code for this Windows shell. Initialization script ``cmd.cmd`` adds the
3489 directory of ``module.cmd``, ``ml.cmd`` and ``envml.cmd`` to ``PATH``.
3490
3491 These Windows-specific files are relocatable: ``module.cmd``, ``ml.cmd`` and
3492 ``envml.cmd`` scripts expect to find initialization script ``cmd.cmd`` in the
3493 ``init`` directory next to them (to setup Modules-specific variables in
3494 current environment) and ``cmd.cmd`` expects ``modulecmd.tcl`` to be found in
3495 ``libexec`` directory and the 3 commands in ``bin`` directory next to it.
3496
3497 Starting from this ``4.5`` release a distribution zipball is published to
3498 install Modules on Windows. This zip archive ships an install and an uninstall
3499 scripts (``INSTALL.bat`` and ``UNINSTALL.bat``). The zipball can be built
3500 locally from Modules sources by running ``make dist-win``.
3501
3502 The :ref:`INSTALL-win` document describes how to install Modules on Windows
3503 from the distribution zipball.
3504
3505 Error stack trace
3506 ^^^^^^^^^^^^^^^^^
3507
3508 Error messages will now embed a stack trace for unknown errors to help
3509 localize the root cause of issues. This change applies to modulefile
3510 evaluation:
3511
3512 .. parsed-literal::
3513
3514 Loading :sgrhi:`foo/1.2`
3515 :sgrme:`Module ERROR`: add-path cannot handle path equals to separator string
3516 while executing
3517 "append-path PATH :"
3518 (file "/path/to/modulefiles/foo/1.2" line 24)
3519 Please contact <:noparse:`root@localhost`>
3520
3521 A stack trace is also returned when an unknown error occurs in
3522 ``modulecmd.tcl`` script, which facilitates issue report and analysis:
3523
3524 .. parsed-literal::
3525
3526 :ps:`$` module load bar
3527 :sgrer:`ERROR`: invalid command name "badcommand"
3528 while executing
3529 "badcommand"
3530 (procedure "module" line 14)
3531 invoked from within
3532 "module load bar"
3533 ("eval" body line 1)
3534 invoked from within
3535 "eval $execcmdlist"
3536 Please report this issue at :noparse:`https://github.com/envmodules/modules/issues`
3537
3538 .. _Automatic default and latest syms:
3539
3540 Automatic default and latest symbolic versions
3541 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3542
3543 When the implicit default mechanism and the `Advanced module version
3544 specifiers`_ are both enabled, a ``default`` and a ``latest`` symbolic
3545 versions are automatically defined for each module name.
3546
3547 This new feature gives the ability to select the highest version available for
3548 a module, without knowing beforehand this version name:
3549
3550 .. parsed-literal::
3551
3552 :ps:`$` module load -v :noparse:`foo@latest`
3553 Loading foo/1.10
3554
3555 The symbolic versions are automatically defined unless a symbolic version, an
3556 alias or a regular module version already exists for these ``default`` or
3557 ``latest`` version names.
3558
3559
3560 v4.4
3561 ----
3562
3563 This new version is backward-compatible with previous version 4 releases. It
3564 fixes bugs but also introduces new functionalities that are described in this
3565 section. See the :ref:`4.4 release notes<4.4 release notes>` for a complete
3566 list of the changes between Modules v4.3 and v4.4.
3567
3568 .. warning:: Modules configuration option handling has been reworked
3569 internally to provide a unified way for all options to get initialized,
3570 retrieved or set. Existing site-specific configuration script should be
3571 reviewed to make use of the new ``getConf``, ``setConf``, ``unsetConf``
3572 and ``lappendConf`` procedures to manipulate configuration options.
3573
3574 .. _Insensitive case:
3575
3576 Specify modules in a case insensitive manner
3577 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3578
3579 The ability to match module name in a case insensitive manner has been added.
3580 This feature can be enabled at different level with the following values set
3581 to the ``icase`` configuration option:
3582
3583 * ``never``: a case sensitive match is applied in any cases
3584 * ``search``: a case insensitive match is applied to the ``avail``, ``whatis``
3585 and ``paths`` sub-commands
3586 * ``always``: a case insensitive match is applied to search contexts and also
3587 to the other module sub-commands and modulefile Tcl commands for the module
3588 specification they receive as argument.
3589
3590 It can help for instance to load a module without knowing the case used to
3591 name its relative modulefile:
3592
3593 .. parsed-literal::
3594
3595 :ps:`$` module config icase always
3596 :ps:`$` module load -v mysoftware
3597 Loading :sgrhi:`MySoftware/1.0`
3598
3599 Insensitive case match activation can be controlled at configure time with
3600 the ``--with-icase`` option, which could be passed any of the above activation
3601 levels. This option could be superseded with the ``MODULES_ICASE`` environment
3602 variable, which could be set through the **config** sub-command with the
3603 ``icase`` option. Command-line switch **--icase** supersedes in turns any
3604 other icase configurations. When this command-line switch is passed, ``icase``
3605 mode equals ``always``.
3606
3607 Extended default
3608 ^^^^^^^^^^^^^^^^
3609
3610 The extended default mechanism has been introduced to help selecting a module
3611 when only the first numbers in its version are specified. Starting portion of
3612 the version, part separated from the rest of the version string by a ``.``
3613 character, could be used to refer to a more precise version number.
3614
3615 This mechanism is activated through the new configuration option
3616 ``extended_default``. It enables to refer to a module named ``foo/1.2.3`` as
3617 ``foo/1.2`` or ``foo/1``:
3618
3619 .. parsed-literal::
3620
3621 :ps:`$` module config extended_default 1
3622 :ps:`$` module load -v foo/1
3623 Loading :sgrhi:`foo/1.2.3`
3624
3625 When multiple versions match partial version specified and only one module
3626 should be selected, the default version (whether implicitly or explicitly
3627 defined) among matches is returned. The following example shows that
3628 ``foo/1.1.1``, the *foo* module default version, is selected when it matches
3629 query. Elsewhere the highest version (also called the latest version or the
3630 implicit default) among matching modules is returned:
3631
3632 .. parsed-literal::
3633
3634 :ps:`$` module av foo
3635 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3636 foo/1.1.1\ :sgrse:`(`:sgrsy:`default`:sgrse:`)` foo/1.2.1 foo/1.10
3637 foo/1.1.10 foo/1.2.3
3638 :ps:`$` module load -v foo/1.1
3639 Loading :sgrhi:`foo/1.1.1`
3640 :ps:`$` module purge
3641 :ps:`$` module load -v foo/1.2
3642 Loading :sgrhi:`foo/1.2.3`
3643 :ps:`$` module purge
3644 :ps:`$` module load -v foo/1
3645 Loading :sgrhi:`foo/1.1.1`
3646
3647 In case ``implicit_default`` option is disabled and no explicit default is
3648 found among matches, an error is returned:
3649
3650 .. parsed-literal::
3651
3652 :ps:`$` module config implicit_default 0
3653 :ps:`$` module load -v foo/1.2
3654 :sgrer:`ERROR`: No default version defined for 'foo/1.2'
3655
3656 When it is enabled, extended default applies everywhere a module could be
3657 specified, which means it could be used with any module sub-command or any
3658 modulefile Tcl command receiving a module specification as argument. It may
3659 help for instance to declare dependencies between modules:
3660
3661 .. parsed-literal::
3662
3663 :ps:`$` module show bar/3
3664 ----------------------------------------------------------
3665 :sgrhi:`/path/to/modulefiles/bar/3.4`:
3666
3667 :sgrcm:`prereq` foo/1.2
3668 ----------------------------------------------------------
3669 :ps:`$` module load --auto bar/3
3670 Loading :sgrhi:`bar/3.4`
3671 :sgrin:`Loading requirement`: foo/1.2.3
3672
3673 Extended default activation can be controlled at configure time with the
3674 ``--enable-extended-default`` option. This option could be superseded with the
3675 ``MODULES_EXTENDED_DEFAULT`` environment variable, which could be set through
3676 the **config** sub-command with the ``extended_default`` option.
3677
3678 Advanced module version specifiers
3679 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3680
3681 The ability to specify finer constraints on module version has been added to
3682 Modules. It enables to filter the module selection to a given version list or
3683 range by specifying after the module name a version constraint prefixed by the
3684 ``@`` character.
3685
3686 This new feature leverages the `version specifier syntax`_ of the `Spack`_
3687 package manager as this syntax covers all the needs for a fine-grained
3688 selection of module versions. It copes very well with command-line typing, by
3689 avoiding characters having a special meaning on shells. Moreover the users of
3690 Spack that also are users of Modules may already be familiar with this syntax.
3691
3692 .. _version specifier syntax: https://spack.readthedocs.io/en/latest/basic_usage.html#version-specifier
3693 .. _Spack: https://github.com/spack/spack
3694
3695 The mechanism introduced here is called *advanced module version specifier*
3696 and it can be activated through the new configuration option
3697 ``advanced_version_spec``. Constraints can be expressed to refine the
3698 selection of module version to:
3699
3700 * a single version with the ``@version`` syntax, for instance ``[email protected]``
3701 syntax will select module ``foo/1.2.3``
3702 * a list of versions with the ``@version1,version2,...`` syntax, for instance
3703 ``[email protected],1.10`` will match modules ``foo/1.2.3`` and ``foo/1.10``
3704 * a range of versions with the ``@version1:``, ``@:version2`` and
3705 ``@version1:version2`` syntaxes, for instance ``[email protected]:`` will select all
3706 versions of module ``foo`` greater than or equal to ``1.2``, ``foo@:1.3``
3707 will select all versions less than or equal to ``1.3`` and ``[email protected]:1.3``
3708 matches all versions between ``1.2`` and ``1.3`` including ``1.2`` and
3709 ``1.3`` versions
3710
3711 This new feature enables for instance to list available versions of module
3712 ``foo`` higher or equal to ``1.2``:
3713
3714 .. parsed-literal::
3715
3716 :ps:`$` module config advanced_version_spec 1
3717 :ps:`$` module av foo
3718 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3719 foo/1.1.1\ :sgrse:`(`:sgrsy:`default`:sgrse:`)` foo/1.2.1 foo/1.10
3720 foo/1.1.10 foo/1.2.3
3721 :ps:`$` module av :noparse:`[email protected]:`
3722 --------------- :sgrdi:`/path/to/modulefiles` ---------------
3723 foo/1.2.1 foo/1.2.3 foo/1.10
3724
3725 Then choose to load for instance a version higher than or equal to ``1.2`` and
3726 less than or equal to ``1.3``. Default version is selected if it corresponds
3727 to a version included in the range, elsewhere the highest version (also called
3728 latest version or implicit default) is selected:
3729
3730 .. parsed-literal::
3731
3732 :ps:`$` module load -v :noparse:`[email protected]:1.3`
3733 Loading :sgrhi:`foo/1.2.3`
3734
3735 In case ``implicit_default`` option is disabled and no explicit default is
3736 found among version specifier matches, an error is returned:
3737
3738 .. parsed-literal::
3739
3740 :ps:`$` module config implicit_default 0
3741 :ps:`$` module load -v :noparse:`[email protected]:1.3`
3742 :sgrer:`ERROR`: No default version defined for ':noparse:`[email protected]:1.3`'
3743
3744 When advanced module version specifier is enabled, it applies everywhere a
3745 module could be specified, which means it could be used with any module
3746 sub-command or any modulefile Tcl command receiving a module specification
3747 as argument. It may help for instance to declare smoother dependencies between
3748 modules:
3749
3750 .. parsed-literal::
3751
3752 :ps:`$` module show bar@:2
3753 ----------------------------------------------------------
3754 :sgrhi:`/path/to/modulefiles/bar/2.3`:
3755
3756 :sgrcm:`prereq` :noparse:`[email protected],1.2.1`
3757 ----------------------------------------------------------
3758 :ps:`$` module load --auto bar@:2
3759 Loading :sgrhi:`bar/2.3`
3760 :sgrin:`Loading requirement`: foo/1.2.1
3761
3762 Advanced specification of single version or list of versions may benefit from
3763 the activation of the `Extended default`_ mechanism (range of versions
3764 natively handles abbreviated versions):
3765
3766 .. parsed-literal::
3767
3768 :ps:`$` module config extended_default 1
3769 :ps:`$` module load -v :noparse:`[email protected]`
3770 Loading :sgrhi:`foo/1.2.3`
3771 :ps:`$` module unload -v foo @1.2,1.5
3772 Unloading :sgrhi:`foo/1.2.3`
3773
3774 Advanced module version specifier activation can be controlled at configure
3775 time with the ``--enable-advanced-version-spec`` option. This option could be
3776 superseded with the ``MODULES_ADVANCED_VERSION_SPEC`` environment variable,
3777 which could be set through the **config** sub-command with the
3778 ``advanced_version_spec`` option.
3779
3780
3781 v4.3
3782 ----
3783
3784 This new version is backward-compatible with previous version 4 releases. It
3785 fixes bugs but also introduces new functionalities that are described in this
3786 section. See the :ref:`4.3 release notes<4.3 release notes>` for a complete
3787 list of the changes between Modules v4.2 and v4.3.
3788
3789 Modulepath rc file
3790 ^^^^^^^^^^^^^^^^^^
3791
3792 A ``.modulerc`` file found at the root of an enabled modulepath directory is
3793 now evaluated when modulepath is walked through to locate modulefiles. This
3794 modulepath rc file gives for instance the ability to define module alias whose
3795 name does not correspond to any module directory in this modulepath. Thus this
3796 kind of module alias would not be found unless if it is defined at the
3797 modulepath global scope.
3798
3799 Further I/O operations optimization
3800 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3801
3802 Additional work has been performed to save a significant number of filesystem
3803 I/O operations made to search and evaluate modulefiles.
3804
3805 When fully read, the content of a modulefile is now cached in memory to avoid
3806 new I/O operations in case this modulefile should be read one more time during
3807 the same module command evaluation.
3808
3809 Except for ``path``, ``paths``, ``list``, ``avail`` and ``aliases`` module
3810 commands always fully read a modulefile whether its full content is needed or
3811 just its header to verify its validity. This way modulefiles are only read
3812 once on commands that first check modulefile validity then read again valid
3813 files to get their full content.
3814
3815 Last but not least, Modules Tcl extension library is introduced to extend the
3816 Tcl language in order to provide more optimized I/O commands to read a file or
3817 a directory content than native Tcl commands do. This library is built and
3818 enabled in ``modulecmd.tcl`` script with ``--enable-libtclenvmodules``
3819 configure argument (it is enabled by default). As this library is written in
3820 C, it must be compiled and ``--with-tcl`` or ``--with-tclinclude`` configure
3821 arguments may be used to indicate where to find Tcl development files.
3822
3823 Modules Tcl extension library greatly reduces the number of filesystem I/O
3824 operations by removing unneeded ``ioctl``, ``fcntl`` and ``lstat`` system
3825 calls done (by Tcl ``open`` command) to read each modulefile. Directory
3826 content read is also improved by fetching hidden and regular files in one
3827 pass. Moreover ``.modulerc`` and ``.version`` read access is tested only if
3828 these files are found in the directory.
3829
3830 Colored output
3831 ^^^^^^^^^^^^^^
3832
3833 The ability to graphically enhance some part of the produced output has been
3834 added to improve readability. Among others, error, warning and info message
3835 prefixes can be colored as well as modulepath, module alias and symbolic
3836 version.
3837
3838 Color mode can be set to ``never``, ``auto`` or ``always``. When color mode is
3839 set to ``auto``, output is colored only if the standard error output channel
3840 is attached to a terminal.
3841
3842 Default color mode could be controlled at configure time with the
3843 ``--enable-color`` and the ``--disable-color`` option, which respectively
3844 correspond to the ``auto`` and ``never`` color mode. This default mode could
3845 be superseded with the ``CLICOLOR``, ``CLICOLOR_FORCE`` and ``MODULES_COLOR``
3846 environment variables and the ``--color`` command-line switch.
3847
3848 Color to apply to each element can be controlled with the ``MODULES_COLORS``
3849 environment variable or the ``--with-dark-background-colors`` and
3850 ``--with-light-background-colors`` configure options. These variable and
3851 options take as value a colon-separated list in the same fashion ``LS_COLORS``
3852 does. In this list, output item that should be highlighted is designated by
3853 a key which is associated to a `Select Graphic Rendition (SGR) code`_.
3854
3855 .. _Select Graphic Rendition (SGR) code: https://en.wikipedia.org/wiki/ANSI_escape_code#Select_Graphic_Rendition_parameters
3856
3857 The ``MODULES_TERM_BACKGROUND`` environment variable and the
3858 ``--with-terminal-background`` configure option help Modules to determine if
3859 the color set for dark background or the color set for light background should
3860 be used to color output in case no specific color set is defined with the
3861 ``MODULES_COLORS`` variable.
3862
3863 Output items able to be colorized and their relative key are: highlighted
3864 element (``hi``), debug information (``db``), tag separator (``se``); Error
3865 (``er``), warning (``wa``), module error (``me``) and info (``in``) message
3866 prefixes; Modulepath (``mp``), directory (``di``), module alias (``al``),
3867 module symbolic version (``sy``), module ``default`` version (``de``) and
3868 modulefile command (``cm``).
3869
3870 For instance the default color set for a terminal with dark background is
3871 defined to:
3872
3873 .. parsed-literal::
3874
3875 :noparse:`hi=1:db=2:se=2:er=91:wa=93:me=95:in=94:mp=1;94:di=94:al=96:sy=95:de=4:cm=92`
3876
3877 When colored output is enabled and a specific graphical rendition is defined
3878 for module *default* version, the ``default`` symbol is omitted and instead
3879 the defined graphical rendition is applied to the relative modulefile. When
3880 colored output is enabled and a specific graphical rendition is defined for
3881 module alias, the ``@`` symbol is omitted.
3882
3883 ``CLICOLOR`` and ``CLICOLOR_FORCE`` environment variables are also honored to
3884 define color mode. The ``never`` mode is set if ``CLICOLOR`` equals to ``0``.
3885 If ``CLICOLOR`` is set to another value, it corresponds to the ``auto`` mode.
3886 The ``always`` mode is set if ``CLICOLOR_FORCE`` is set to a value different
3887 than ``0``. Color mode set with these two variables is superseded by mode set
3888 with ``MODULES_COLOR`` environment variable.
3889
3890 Configure modulecmd with config sub-command
3891 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3892
3893 The **config** sub-command has been added to ``module`` to help getting or
3894 setting the **modulecmd.tcl** options. With no additional command-line
3895 argument, this sub-command reports the current value of all existing options
3896 with a mention to indicate if this value has been overridden from a
3897 command-line switch or from an environment variable.
3898
3899 See the description of this sub-command in the :ref:`module(1)` man page for a
3900 complete reference on existing configuration options.
3901
3902 Most of the options can be altered by passing the option name and a value to
3903 the sub-command. Setting an option by this mean overrides its default value,
3904 set at installation time in **modulecmd.tcl** script, by defining the
3905 environment variable which supersedes this default.:
3906
3907 .. parsed-literal::
3908
3909 :ps:`$` module config auto_handling 1
3910 :ps:`$` module config auto_handling
3911 Modules Release 4.3.0 (2019-07-26)
3912
3913 - Config. name ---------.- Value (set by if default overridden) ---------------
3914 auto_handling 1 (env-var)
3915
3916 Setting options with ``module config`` could be done in the Modules
3917 initialization RC file to change default value of options when ``module``
3918 command is initialized.
3919
3920 When command-line switch ``--reset`` and an option name is passed to the
3921 **config** sub-command, it restores default value for configuration option by
3922 unsetting related environment variable.
3923
3924 With command-line switch ``--dump-state``, the **config** sub-command reports,
3925 in addition to currently set options, the current state of **modulecmd.tcl**
3926 script and Modules-related environment variables. Providing the output of the
3927 ``module config --dump-state`` command when submitting an issue to the Modules
3928 project will help to analyze the situation.
3929
3930 Control module command verbosity
3931 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3932
3933 The ability to control message verbosity has been added so ``module`` command
3934 can be configured whether it should display more or less information.
3935 Available verbosity levels from the least to the most verbose are:
3936
3937 * ``silent``: turn off error, warning and informational messages but does not
3938 affect module command output result.
3939 * ``concise``: enable error and warning messages but disable informational
3940 messages.
3941 * ``normal``: turn on informational messages, like a report of the additional
3942 module evaluations triggered by loading or unloading modules, aborted
3943 evaluation issues or a report of each module evaluation occurring during a
3944 **restore** or **source** sub-commands.
3945 * ``verbose``: add additional informational messages, like a systematic report
3946 of the loading or unloading module evaluations.
3947 * ``debug``: print debugging messages about module command execution.
3948
3949 Default verbosity level can be controlled at configure time with the
3950 ``--with-verbosity`` option, which could be passed any of the above level
3951 names. This default verbosity level could be superseded with the
3952 ``MODULES_VERBOSITY`` environment variable, which could be set through the
3953 **config** sub-command with the ``verbosity`` option. Command-line switches
3954 **--silent**, **--verbose** and **--debug** supersede in turns any other
3955 verbosity configuration to respectively set module command silent, verbose or
3956 in debug mode.
3957
3958 Other new sub-commands, command-line switches and environment variables
3959 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3960
3961 * The **avail** sub-command gets two new command-line switches: **--indepth**
3962 and **--no-indepth**. These options control whether search results should
3963 recursively include or not modulefiles from directories matching search
3964 query. Shell completion scripts have been updated to complete available
3965 modulefiles in the no in depth mode.
3966
3967 * The **MODULES_AVAIL_INDEPTH** environment variable defines if the **avail**
3968 sub-command should include or exclude by default the modulefiles from
3969 directories matching search query. Its value is superseded by the use of the
3970 **--indepth** and **--no-indepth** command-line switches.
3971
3972 * The **clear** sub-command, which was available on Modules version 3.2, has
3973 been reintroduced. This sub-command resets the Modules runtime information
3974 but does not apply further changes to the environment at all. This
3975 sub-command now leverages the **--force** command-line switch to skip its
3976 confirmation dialog.
3977
3978 * The **MODULES_SITECONFIG** environment variable defines an additional
3979 siteconfig script which is loaded if it exists after the siteconfig script
3980 configured at build time in ``modulecmd.tcl``. This ability is enabled by
3981 default and could be disabled with configure option
3982 ``--with-locked-configs=extra_siteconfig``.
3983
3984 * The **MODULES_UNLOAD_MATCH_ORDER** environment variable sets whether the
3985 firstly or the lastly loaded module should be selected for unload when
3986 multiple loaded modules match unload request. Configure option
3987 ``--with-unload-match-order`` defines this setting which can be superseded
3988 by the environment variable. By default, lastly loaded module is selected
3989 and it is recommended to keep this behavior when used modulefiles express
3990 dependencies between each other.
3991
3992 * The **MODULES_IMPLICIT_DEFAULT** environment variable sets whether an
3993 implicit default version should be defined for modules with no default
3994 version explicitly defined. When enabled, which is the default behavior, a
3995 module version is automatically selected (latest one) when the generic
3996 name of the module is passed. When implicit default is disabled and no
3997 default version is explicitly defined for a module, the name of this module
3998 to evaluate should be fully qualified elsewhere an error is returned.
3999 Configure option ``--enable-implicit-default`` defines this setting which
4000 can be superseded by the environment variable. This superseding mechanism
4001 can be disabled with configure option
4002 ``--with-locked-configs=implicit_default``.
4003
4004 * The **MODULES_SEARCH_MATCH** environment variable defines the matching style
4005 to perform when searching for available modules. With **starts_with** value,
4006 modules whose name begins by search query string are returned. When search
4007 match style is set to **contains**, modules returned are those whose fully
4008 qualified name contains search query string. Configure option
4009 ``--with-search-match`` defines this setting which can be superseded by the
4010 environment variable, which in turns can be superseded by the
4011 **--starts-with** and **--contains** command-line switches of **avail**
4012 module sub-command.
4013
4014 * The **MODULES_SET_SHELL_STARTUP** environment variable controls whether or
4015 not shell startup file should be set to ensure ``module`` command is defined
4016 once shell has been initialized. When enabled, the ``ENV`` and ``BASH_ENV``
4017 environment variables are set, when ``module`` function is defined, to the
4018 Modules bourne shell initialization script. Configure options
4019 ``--enable-set-shell-startup`` and ``--disable-set-shell-startup`` define
4020 this setting which can be superseded by the environment variable.
4021
4022 * When initializing the ``module`` command in a shell session, initialization
4023 configuration files stored in the defined configuration directory are taken
4024 into account if present instead of the configuration files stored in the
4025 initialization script directory. When they are stored in the configuration
4026 directory, these configuration files are named ``initrc`` and
4027 ``modulespath`` instead of respectively ``modulerc`` and ``.modulespath``.
4028 The location of the installation of those files can be controlled with
4029 configure option ``--with-initconf-in``, which accepts ``etcdir`` and
4030 ``initdir`` values.
4031
4032 * The **MODULES_WA_277** environment variable helps to define an alternative
4033 ``module`` alias on Tcsh shell when set to *1*. It workarounds an issue on
4034 Tcsh history mechanism occurring with default ``module`` command alias:
4035 erroneous history entries are recorded each time the ``module`` command is
4036 called. However the alternative definition of the module alias weakens shell
4037 evaluation of the code produced by modulefiles. Characters with special
4038 meaning for Tcsh shell (like *{* and *}*) may not be used anymore in shell
4039 alias definition elsewhere the evaluation of the code produced by
4040 modulefiles will return a syntax error.
4041
4042
4043 v4.2
4044 ----
4045
4046 This new version is backward-compatible with previous version 4 releases. It
4047 fixes bugs but also introduces new functionalities that are described in this
4048 section. See the :ref:`4.2 release notes<4.2 release notes>` for a complete
4049 list of the changes between Modules v4.1 and v4.2.
4050
4051 .. _v42-conflict-constraints-consistency:
4052
4053 Modulefile conflict constraints consistency
4054 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4055
4056 With the **conflict** modulefile command, a given modulefile can list the
4057 other modulefiles it conflicts with. To load this modulefile, the modulefiles
4058 it conflicts with cannot be loaded.
4059
4060 This constraint was until now satisfied when loading the modulefile declaring
4061 the **conflict** but it vanished as soon as this modulefile was loaded. In the
4062 following example ``a`` modulefile declares a conflict with ``b``:
4063
4064 .. parsed-literal::
4065
4066 :ps:`$` module load b a
4067 WARNING: a cannot be loaded due to a conflict.
4068 HINT: Might try "module unload b" first.
4069 :ps:`$` module list
4070 Currently Loaded Modulefiles:
4071 1) b
4072 :ps:`$` module purge
4073 :ps:`$` module load a b
4074 :ps:`$` module list
4075 Currently Loaded Modulefiles:
4076 1) a 2) b
4077
4078 Consistency of the declared **conflict** is now ensured to satisfy this
4079 constraint even after the load of the modulefile declaring it. This is
4080 achieved by keeping track of the conflict constraints of the loaded
4081 modulefiles in an environment variable called ``MODULES_LMCONFLICT``:
4082
4083 .. parsed-literal::
4084
4085 :ps:`$` module load a b
4086 ERROR: WARNING: b cannot be loaded due to a conflict.
4087 HINT: Might try "module unload a" first.
4088 :ps:`$` module list
4089 Currently Loaded Modulefiles:
4090 1) a
4091
4092 An environment variable is used to keep track of this conflict information to
4093 proceed the same way than used to keep track of the loaded modulefiles with
4094 the ``LOADEDMODULES`` environment variable.
4095
4096 In case a conflict constraint toward a modulefile is set by an already loaded
4097 modulefile, loading the conflicting modulefile will lead to a load evaluation
4098 attempt in order for this modulefile to get the chance to solve the constraint
4099 violation. If at the end of the load evaluation, the conflict has not been
4100 solved, modulefile load will be discarded.
4101
4102 .. warning:: On versions ``4.2.0`` and ``4.2.1``, a conflict constraint set by
4103 an already loaded modulefile forbade the load of the conflicting
4104 modulefile. This has been changed starting version ``4.2.2`` to better cope
4105 with behaviors of previous Modules version: an evaluation attempt of the
4106 conflicting modulefile is made to give it the opportunity to solve this
4107 conflict by using **module unload** modulefile command.
4108
4109 .. _v42-prereq-constraints-consistency:
4110
4111 Modulefile prereq constraints consistency
4112 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4113
4114 With the **prereq** modulefile command, a given modulefile can list the
4115 other modulefiles it pre-requires. To load this modulefile, the modulefiles it
4116 pre-requires must be loaded prior its own load.
4117
4118 This constraint was until now satisfied when loading the modulefile declaring
4119 the **prereq** but, as for the declared **conflict**, it vanished as soon as
4120 this modulefile was loaded. In the following example ``c`` modulefile declares
4121 a prereq on ``a``:
4122
4123 .. parsed-literal::
4124
4125 :ps:`$` module load c
4126 WARNING: c cannot be loaded due to missing prereq.
4127 HINT: the following module must be loaded first: a
4128 :ps:`$` module list
4129 No Modulefiles Currently Loaded.
4130 :ps:`$` module load a c
4131 :ps:`$` module list
4132 Currently Loaded Modulefiles:
4133 1) a 2) c
4134 :ps:`$` module unload a
4135 :ps:`$` module list
4136 Currently Loaded Modulefiles:
4137 1) c
4138
4139 Consistency of the declared **prereq** is now ensured to satisfy this
4140 constraint even after the load of the modulefile declaring it. This is
4141 achieved, like for the conflict consistency, by keeping track of the prereq
4142 constraints of the loaded modulefiles in an environment variable called
4143 ``MODULES_LMPREREQ``:
4144
4145 .. parsed-literal::
4146
4147 :ps:`$` module load a c
4148 :ps:`$` module list
4149 Currently Loaded Modulefiles:
4150 1) a 2) c
4151 :ps:`$` module unload a
4152 ERROR: WARNING: a cannot be unloaded due to a prereq.
4153 HINT: Might try "module unload c" first.
4154 :ps:`$` module list
4155 Currently Loaded Modulefiles:
4156 1) a 2) c
4157
4158 .. _v42-by-passing-module-constraints:
4159
4160 By-passing module defined constraints
4161 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4162
4163 The ability to by-pass a **conflict** or a **prereq** constraint defined by
4164 modulefiles is introduced with the ``--force`` command line switch (``-f`` for
4165 short notation) for the **load**, **unload** and **switch** sub-commands.
4166
4167 With this new command line switch, a given modulefile is loaded even if it
4168 conflicts with other loaded modulefiles or even if the modulefiles it
4169 pre-requires are not loaded. Some example reusing the same modulefiles ``a``,
4170 ``b`` and ``c`` than above:
4171
4172 .. parsed-literal::
4173
4174 :ps:`$` module load b
4175 :ps:`$` module load --force a
4176 WARNING: a conflicts with b
4177 :ps:`$` module list
4178 Currently Loaded Modulefiles:
4179 1) b 2) a
4180 :ps:`$` module purge
4181 :ps:`$` module load --force c
4182 WARNING: c requires a loaded
4183 :ps:`$` module list
4184 Currently Loaded Modulefiles:
4185 1) c
4186
4187 ``--force`` also enables to unload a modulefile required by another loaded
4188 modulefiles:
4189
4190 .. parsed-literal::
4191
4192 :ps:`$` module load a c
4193 :ps:`$` module list
4194 Currently Loaded Modulefiles:
4195 1) a 2) c
4196 :ps:`$` module unload --force a
4197 WARNING: a is required by c
4198 :ps:`$` module list
4199 Currently Loaded Modulefiles:
4200 1) c
4201
4202 In a situation where some of the loaded modulefiles have unsatisfied
4203 constraints corresponding to the **prereq** and **conflict** they declare, the
4204 **save** and **reload** sub-commands do not perform and return an error.
4205
4206 .. _v42-automated-module-handling-mode:
4207
4208 Automated module handling mode
4209 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4210
4211 An automatic management of the dependencies between modulefiles has been added
4212 and it is called *automated module handling mode*. This new mode consists in
4213 additional actions triggered when loading or unloading a modulefile to satisfy
4214 the constraints it declares.
4215
4216 When loading a modulefile, following actions are triggered:
4217
4218 * Requirement Load (ReqLo): load of the modulefiles declared as a **prereq**
4219 of the loading modulefile.
4220
4221 * Dependent Reload (DepRe): reload of the modulefiles declaring a **prereq**
4222 onto loaded modulefile or declaring a **prereq** onto a modulefile part of
4223 this reloading batch.
4224
4225 When unloading a modulefile, following actions are triggered:
4226
4227 * Dependent Unload (DepUn): unload of the modulefiles declaring a non-optional
4228 **prereq** onto unloaded modulefile or declaring a non-optional **prereq**
4229 onto a modulefile part of this unloading batch. A **prereq** modulefile is
4230 considered optional if the **prereq** definition order is made of multiple
4231 modulefiles and at least one alternative modulefile is loaded.
4232
4233 * Useless Requirement Unload (UReqUn): unload of the **prereq** modulefiles
4234 that have been automatically loaded for either the unloaded modulefile, an
4235 unloaded dependent modulefile or a modulefile part of this useless
4236 requirement unloading batch. Modulefiles are added to this unloading batch
4237 only if they are not required by any other loaded modulefiles.
4238 ``MODULES_LMNOTUASKED`` environment variable helps to keep track of these
4239 automatically loaded modulefiles and to distinguish them from modulefiles
4240 asked by user.
4241
4242 * Dependent Reload (DepRe): reload of the modulefiles declaring a **conflict**
4243 or an optional **prereq** onto either the unloaded modulefile, an unloaded
4244 dependent or an unloaded useless requirement or declaring a **prereq** onto
4245 a modulefile part of this reloading batch.
4246
4247 In case a loaded modulefile has some of its declared constraints unsatisfied
4248 (pre-required modulefile not loaded or conflicting modulefile loaded for
4249 instance), this loaded modulefile is excluded from the automatic reload
4250 actions described above.
4251
4252 For the specific case of the **switch** sub-command, where a modulefile is
4253 unloaded to then load another modulefile. Dependent modulefiles to Unload are
4254 merged into the Dependent modulefiles to Reload that are reloaded after the
4255 load of the switched-to modulefile.
4256
4257 This automated module handling mode integrates concepts (like the Dependent
4258 Reload mechanism) of the Flavours_ extension, which was designed for Modules
4259 compatibility version. As a whole, automated module handling mode can be seen
4260 as a generalization and as an expansion of the Flavours_ concepts.
4261
4262 .. _Flavours: https://sourceforge.net/projects/flavours/
4263
4264 This new feature can be controlled at build time with the
4265 ``--enable-auto-handling`` configure option. This default configuration can be
4266 superseded at run-time with the ``MODULES_AUTO_HANDLING`` environment variable
4267 or the command line switches ``--auto`` and ``--no-auto``.
4268
4269 By default, automated module handling mode is disabled and will stay so until
4270 the next major release version (5.0) where it will be enabled by default. This
4271 new feature is currently considered experimental and the set of triggered
4272 actions will be refined over the next feature releases.
4273
4274 .. _v42-consistency-module-load-unload-commands:
4275
4276 Consistency of module load/unload commands in modulefile
4277 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4278
4279 With the **module load** modulefile command, a given modulefile can
4280 automatically load a modulefile it pre-requires. Similarly with the **module
4281 unload** modulefile command, a given modulefile can automatically unload a
4282 modulefile it conflicts with.
4283
4284 Both commands imply additional actions on the loaded environment (loading or
4285 unloading extra modulefiles) that should cope with the constraints defined by
4286 the loaded environment.
4287
4288 Additionally **module load** and **module unload** modulefile commands express
4289 themselves constraints on loaded environment that should stay satisfied to
4290 ensure consistency.
4291
4292 To ensure the consistency of **module load** modulefile command once the
4293 modulefile defining it has been loaded, this command is assimilated to a
4294 **prereq** command. Thus the defined constraint is recorded in the
4295 ``MODULES_LMPREREQ`` environment variable. Same approach is used for **module
4296 unload** modulefile command which is assimilated to a **conflict** command.
4297 Thus the defined constraint is recorded in the ``MODULES_LMCONFLICT``
4298 environment variable.
4299
4300 To ensure the consistency of the loaded environment, the additional actions of
4301 the **module load** and **module unload** modulefile commands have been
4302 adapted in particular situations:
4303
4304 * When unloading modulefile, **module load** command will unload the
4305 modulefile it targets only if no other loaded modulefile requires it and if
4306 this target has not been explicitly loaded by user.
4307
4308 * When unloading modulefile, **module unload** command does nothing as the
4309 relative conflict registered at load time ensure environment consistency and
4310 will forbid conflicting modulefile load.
4311
4312 Please note that loading and unloading results may differ than from previous
4313 Modules version now that consistency is checked:
4314
4315 * Modulefile targeted by a **module load** modulefile command may not be able
4316 to load due to a registered conflict in the currently loaded environment.
4317 Which in turn will break the load of the modulefile declaring the **module
4318 load** command.
4319
4320 * Modulefile targeted by a **module unload** modulefile command may not be
4321 able to unload due to a registered prereq in the loaded environment. Which
4322 in turn will break the load of the modulefile declaring the **module
4323 unload** command.
4324
4325 * If automated module handling mode is enabled, **module load** modulefile
4326 command is interpreted when unloading modulefile as part of the Useless
4327 Requirement Unload (UReqUn) mechanism not through modulefile evaluation.
4328 As a consequence, an error occurring when unloading the modulefile targeted
4329 by the **module load** command does not break the unload of the modulefile
4330 declaring this command. Moreover unload of the **module load** targets is
4331 done in the reverse loaded order, not in the **module load** command
4332 definition order.
4333
4334 .. _v42-alias-symbolic-name-consistency:
4335
4336 Modulefile alias and symbolic modulefile name consistency
4337 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4338
4339 With the **module-alias** and **module-version** modulefile commands,
4340 alternative names can be given to a modulefile. When these names are used to
4341 load for instance a modulefile, they are resolved to the modulefile they
4342 target which is then processed for the load action.
4343
4344 Until now, the alias and symbolic version names were correctly resolved for
4345 the **load** and **unload** actions and also for the querying sub-commands
4346 (like **avail** or **whatis**). However this alternative name information
4347 vanishes once the modulefile it resolves to is loaded. As a consequence there
4348 was no consistency over these alternative designations. In the following
4349 example ``f`` modulefile declares a conflict on ``e`` alias which resolves to
4350 ``d`` modulefile:
4351
4352 .. parsed-literal::
4353
4354 :ps:`$` module load e
4355 :ps:`$` module list
4356 Currently Loaded Modulefiles:
4357 1) d
4358 :ps:`$` module info-loaded e
4359 :ps:`$` module load f
4360 :ps:`$` module list
4361 Currently Loaded Modulefiles:
4362 1) d 2) f
4363
4364 Consistency of the alternative names set on a modulefile with **module-alias**
4365 and **module-version** commands is now ensured to enable modulefile commands
4366 **prereq**, **conflict**, **is-loaded** and **module-info loaded** using these
4367 alternative designations as argument. This consistency is achieved, like for
4368 the conflict and prereq consistencies, by keeping track of the alternative
4369 names of the loaded modulefiles in an environment variable called
4370 ``MODULES_LMALTNAME``:
4371
4372 .. parsed-literal::
4373
4374 :ps:`$` module load e
4375 :ps:`$` module list
4376 Currently Loaded Modulefiles:
4377 1) d
4378 :ps:`$` module info-loaded e
4379 d
4380 :ps:`$` module load f
4381 WARNING: f cannot be loaded due to a conflict.
4382 HINT: Might try "module unload e" first.
4383 :ps:`$` module list
4384 Currently Loaded Modulefiles:
4385 1) d
4386
4387 .. _v42-variable-change-through-modulefile-evaluation:
4388
4389 Environment variable change through modulefile evaluation context
4390 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4391
4392 All environment variable edition commands (``setenv``, ``unsetenv``,
4393 ``append-path``, ``prepend-path`` and ``remove-path``) have been updated to:
4394
4395 * Reflect environment variable value change on the environment of the current
4396 modulefile Tcl interpreter. So using ``$env(VAR)`` will return the currently
4397 defined value for environment variable ``VAR``, not the one found prior
4398 modulefile evaluation.
4399 * Clear environment variable content instead of unsetting it on the
4400 environment of the current modulefile Tcl interpreter to avoid raising
4401 error about accessing an undefined element in ``$env()``. Code is still
4402 produced to purely unset environment variable in shell environment.
4403
4404 Exception is made for the ``whatis`` evaluation mode: environment variables
4405 targeted by variable edition commands are not set to the defined value in the
4406 evaluation context during this ``whatis`` evaluation. These variables are
4407 only initialized to an empty value if undefined. This exception is made to
4408 save performances on this global evaluation mode.
4409
4410 .. _v42-versioned-magic-cookie:
4411
4412 Express Modules compatibility of modulefile with versioned magic cookie
4413 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4414
4415 Any modulefile should start with the ``#%Module`` file signature (also called
4416 the Modules magic cookie) and sometimes a version number may be placed right
4417 after this string. Until now this version number corresponded to a modulefile
4418 format version but it was never checked.
4419
4420 Starting with this new Modules release, this version number reflects the
4421 minimum version of Modules required to interpret the modulefile. If the
4422 version number is set along the magic cookie string it is now checked and the
4423 modulefile is interpreted only if Modules version is greater or equal to this
4424 version number. For instance, if a modulefile begins with the ``#%Module4.3``
4425 string, it can only be evaluated by Modules version 4.3 and above. Elsewhere
4426 the modulefile is ignored like files without the ``#%Module`` magic cookie
4427 set.
4428
4429 .. _v42-module-message-report:
4430
4431 Improved module message report
4432 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4433
4434 Module sub-commands like ``load``, ``unload`` or ``switch``, may perform
4435 multiple load or unload modulefile evaluations in a row. Also these kind of
4436 evaluation modes may sometimes trigger additional load or unload evaluations,
4437 when for instance a modulefile contains a ``module load`` command.
4438
4439 To improve the readability of the module messages produced relatively to
4440 a load or an unload evaluation, these messages are now stacked under a
4441 *Loading* or an *Unloading* message block that gathers all the messages
4442 produced for a given modulefile evaluation:
4443
4444 .. parsed-literal::
4445
4446 :ps:`$` module load --no-auto foo
4447 Loading foo/1.2
4448 ERROR: foo/1.2 cannot be loaded due to missing prereq.
4449 HINT: the following module must be loaded first: bar/4.5
4450
4451 In addition, foreground ``load``, ``unload``, ``switch`` and ``restore``
4452 actions (i.e., asked on the command-line) now report a summary of the
4453 additional load and unload evaluations that were eventually triggered in
4454 the process:
4455
4456 .. parsed-literal::
4457
4458 :ps:`$` module load --auto foo
4459 Loading foo/1.2
4460 Loading requirement: bar/4.5
4461
4462 New modulefile commands
4463 ^^^^^^^^^^^^^^^^^^^^^^^
4464
4465 2 new modulefile Tcl commands have been introduced:
4466
4467 * **set-function**: define a shell function on sh-kind and fish shells.
4468 * **unset-function**: unset a shell function on sh-kind and fish shells.
4469
4470
4471 v4.1
4472 ----
4473
4474 This new version is backward-compatible with previous version 4 releases. It
4475 fixes bugs but also introduces new functionalities that are described in this
4476 section. See the :ref:`4.1 release notes<4.1 release notes>` for a complete
4477 list of the changes between Modules v4.0 and v4.1.
4478
4479 .. _Virtual modules:
4480
4481 Virtual modules
4482 ^^^^^^^^^^^^^^^
4483
4484 A virtual module stands for a module name associated to a modulefile. The
4485 modulefile is the script interpreted when loading or unloading the virtual
4486 module which appears or can be found with its virtual name.
4487
4488 The **module-virtual** modulefile command is introduced to give the ability
4489 to define these virtual modules. This new command takes a module name as
4490 first argument and a modulefile location as second argument::
4491
4492 module-virtual app/1.2.3 /path/to/virtualmod/app
4493
4494 With this feature it is now possible to dynamically define modulefiles
4495 depending on the context.
4496
4497 Extend module command with site-specific Tcl code
4498 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4499
4500 ``module`` command can now be extended with site-specific Tcl
4501 code. ``modulecmd.tcl`` now looks at a **siteconfig.tcl** file in an
4502 ``etcdir`` defined at configure time (by default ``$prefix/etc``). If
4503 it finds this Tcl script file, it is sourced within ``modulecmd.tcl`` at the
4504 beginning of the main procedure code.
4505
4506 ``siteconfig.tcl`` enables to supersede any global variable or procedure
4507 definitions made in ``modulecmd.tcl`` with site-specific code. A module
4508 sub-command can for instance be redefined to make it fit local needs
4509 without having to touch the main ``modulecmd.tcl``.
4510
4511 .. _Quarantine mechanism:
4512
4513 Quarantine mechanism to protect module execution
4514 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4515
4516 To protect the module command run-time environment from side effect
4517 coming from the current environment definition a quarantine mechanism
4518 is introduced. This mechanism, sets within module function definition
4519 and shell initialization script, modifies the ``modulecmd.tcl`` run-time
4520 environment to sanitize it.
4521
4522 The mechanism is piloted by environment variables. First of all
4523 ``MODULES_RUN_QUARANTINE``, a space-separated list of environment variable
4524 names. Every variable found in ``MODULES_RUN_QUARANTINE`` will be set in
4525 quarantine during the ``modulecmd.tcl`` run-time. Their value will be set
4526 empty or set to the value of the corresponding ``MODULES_RUNENV_<VAR>``
4527 environment variable if defined. Once ``modulecmd.tcl`` is started it
4528 restores quarantine variables to their original values.
4529
4530 ``MODULES_RUN_QUARANTINE`` and ``MODULES_RUNENV_<VAR>`` environment variables
4531 can be defined at build time by using the following configure option::
4532
4533 --with-quarantine-vars='VARNAME[=VALUE] ...'
4534
4535 Quarantine mechanism is available for all supported shells except ``csh``
4536 and ``tcsh``.
4537
4538 Pager support
4539 ^^^^^^^^^^^^^
4540
4541 The informational messages Modules sends on the *stderr* channel may
4542 sometimes be quite long. This is especially the case for the avail
4543 sub-command when hundreds of modulefiles are handled. To improve the
4544 readability of those messages, *stderr* output can now be piped into a
4545 paging command.
4546
4547 This new feature can be controlled at build time with the ``--with-pager``
4548 and ``--with-pager-opts`` configure options. Default pager command is set
4549 to ``less`` and its relative options are by default ``-eFKRX``. Default
4550 configuration can be supersedes at run-time with ``MODULES_PAGER`` environment
4551 variables or command-line switches (``--no-pager``, ``--paginate``).
4552
4553 .. warning:: On version ``4.1.0``, the ``PAGER`` environment variable was
4554 taken in consideration to supersede pager configuration at run-time. Since
4555 version ``4.1.1``, ``PAGER`` environment variable is ignored to avoid side
4556 effects coming from the system general pager configuration.
4557
4558 Module function to return value in scripting languages
4559 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4560
4561 On Tcl, Perl, Python, Ruby, CMake and R scripting shells, module function
4562 was not returning value and until now an occurred error led to raising a
4563 fatal exception.
4564
4565 To make ``module`` function more friendly to use on these scripting shells
4566 it now returns a value. False in case of error, true if everything goes well.
4567
4568 As a consequence, returned value of a module sub-command can be checked. For
4569 instance in Python::
4570
4571 if module('load', 'foo'):
4572 # success
4573 else:
4574 # failure
4575
4576 New modulefile commands
4577 ^^^^^^^^^^^^^^^^^^^^^^^
4578
4579 4 new modulefile Tcl commands have been introduced:
4580
4581 * **is-saved**: returns true or false whether a collection, corresponding to
4582 currently set collection target, exists or not.
4583 * **is-used**: returns true or false whether a given directory is currently
4584 enabled in ``MODULEPATH``.
4585 * **is-avail**: returns true or false whether a given modulefile exists in
4586 currently enabled module paths.
4587 * **module-info loaded**: returns the exact name of the modulefile currently
4588 loaded corresponding to the name argument.
4589
4590 Multiple collections, paths or modulefiles can be passed respectively to
4591 ``is-saved``, ``is-used`` and ``is-avail`` in which case true is returned if
4592 at least one argument matches condition (acts as a OR boolean operation). No
4593 argument may be passed to ``is-loaded``, ``is-saved`` and ``is-used``
4594 commands to return if anything is respectively loaded, saved or used.
4595
4596 If no loaded modulefile matches the ``module-info loaded`` query, an empty
4597 string is returned.
4598
4599 New module sub-commands
4600 ^^^^^^^^^^^^^^^^^^^^^^^
4601
4602 Modulefile-specific commands are sometimes wished to be used outside of a
4603 modulefile context. Especially for the commands managing path variables
4604 or commands querying current environment context. So the following
4605 modulefile-specific commands have been made reachable as module sub-commands
4606 with same arguments and properties as if called from within a modulefile:
4607
4608 * **append-path**
4609 * **prepend-path**
4610 * **remove-path**
4611 * **is-loaded**
4612 * **info-loaded**
4613
4614 The ``is-loaded`` sub-command returns a boolean value. Small Python example::
4615
4616 if module('is-loaded', 'app'):
4617 print 'app is loaded'
4618 else:
4619 print 'app not loaded'
4620
4621 ``info-loaded`` returns a string value and is the sub-command counterpart
4622 of the ``module-info loaded`` modulefile command:
4623
4624 .. parsed-literal::
4625
4626 :ps:`$` module load app/0.8
4627 :ps:`$` module info-loaded app
4628 app/0.8
4629
4630
4631 v4.0
4632 ----
4633
4634 Major evolution occurs with this v4.0 release as the traditional *module*
4635 command implemented in C is replaced by the native Tcl version. This full
4636 Tcl rewrite of the Modules package was started in 2002 and has now reached
4637 maturity to take over the binary version. This flavor change enables to
4638 refine and push forward the *module* concept.
4639
4640 This document provides an outlook of what is changing when migrating from
4641 v3.2 to v4.0 by first describing the introduced new features. Both v3.2
4642 and v4.0 are quite similar and transition to the new major version should
4643 be smooth. Slights differences may be noticed in a few use-cases. So the
4644 second part of the document will help to learn about them by listing the
4645 features that have been discontinued in this new major release or the
4646 features where a behavior change can be noticed.
4647
4648 New features
4649 ^^^^^^^^^^^^
4650
4651 On its overall this major release brings a lot more robustness to the
4652 *module* command with now more than 4000 non-regression tests crafted
4653 to ensure correct operations over the time. This version 4.0 also comes
4654 with fair amount of improved functionalities. The major new features are
4655 described in this section.
4656
4657 Additional shells supported
4658 """""""""""""""""""""""""""
4659
4660 Modules v4 introduces support for **fish**, **lisp**, **tcl** and **R**
4661 code output.
4662
4663 Non-zero exit code in case of error
4664 """""""""""""""""""""""""""""""""""
4665
4666 All module sub-commands will now return a non-zero exit code in case of error
4667 whereas Modules v3.2 always returned zero exit code even if issue occurred.
4668
4669 Output redirect
4670 """""""""""""""
4671
4672 Traditionally the *module* command output text that should be seen by the
4673 user on *stderr* since shell commands are output to *stdout* to change
4674 shell's environment. Now on *sh*, *bash*, *ksh*, *zsh* and *fish* shells,
4675 output text is redirected to *stdout* after shell command evaluation if
4676 shell is in interactive mode.
4677
4678 Filtering avail output
4679 """"""""""""""""""""""
4680
4681 Results obtained from the **avail** sub-command can now be filtered to only
4682 get the default version of each module name with use of the **--default**
4683 or **-d** command line switch. Default version is either the explicitly
4684 set default version or the highest numerically sorted modulefile or module
4685 alias if no default version set.
4686
4687 It is also possible to filter results to only get the highest numerically
4688 sorted version of each module name with use of the **--latest** or **-L**
4689 command line switch.
4690
4691 Extended support for module alias and symbolic version
4692 """"""""""""""""""""""""""""""""""""""""""""""""""""""
4693
4694 Module aliases are now included in the result of the **avail**, **whatis**
4695 and **apropos** sub-commands. They are displayed in the module path
4696 section where they are defined or in a *global/user modulerc* section for
4697 aliases set in user's or global ``modulerc`` file. A **@** symbol is added
4698 in parenthesis next to their name to distinguish them from modulefiles.
4699
4700 Search may be performed with an alias or a symbolic version-name passed
4701 as argument on **avail**, **whatis** and **apropos** sub-commands.
4702
4703 Modules v4 resolves module alias or symbolic version passed to **unload**
4704 command to then remove the loaded modulefile pointed by the mentioned
4705 alias or symbolic version.
4706
4707 A symbolic version sets on a module alias is now propagated toward the
4708 resolution path to also apply to the relative modulefile if it still
4709 correspond to the same module name.
4710
4711 Hiding modulefiles
4712 """"""""""""""""""
4713
4714 Visibility of modulefiles can be adapted by use of file mode bits or file
4715 ownership. If a modulefile should only be used by a given subset of persons,
4716 its mode an ownership can be tailored to provide read rights to this group of
4717 people only. In this situation, module only reports the modulefile, during an
4718 **avail** command for instance, if this modulefile can be read by the current
4719 user.
4720
4721 These hidden modulefiles are simply ignored when walking through the
4722 modulepath content. Access issues (permission denied) occur only when trying
4723 to access directly a hidden modulefile or when accessing a symbol or an alias
4724 targeting a hidden modulefile.
4725
4726 Improved modulefiles location
4727 """""""""""""""""""""""""""""
4728
4729 When looking for an implicit default in a modulefile directory, aliases
4730 are now taken into account in addition to modulefiles and directories to
4731 determine the highest numerically sorted element.
4732
4733 Modules v4 resolves module alias or symbolic version when it points to a
4734 modulefile located in another modulepath.
4735
4736 Access issues (permission denied) are now distinguished from find issues
4737 (cannot locate) when trying to access directly a directory or a modulefile
4738 as done on **load**, **display** or **whatis** commands. In addition,
4739 on this kind of access not readable ``.modulerc`` or ``.version`` files are
4740 ignored rather producing a missing magic cookie error.
4741
4742 Module collection
4743 """""""""""""""""
4744
4745 Modules v4 introduces support for module *collections*. Collections
4746 describe a sequence of **module use** then **module load** commands that
4747 are interpreted by Modules to set the user environment as described by this
4748 sequence. When a collection is activated, with the **restore** sub-command,
4749 modulepaths and loaded modules are unused or unloaded if they are not part
4750 or if they are not ordered the same way as in the collection.
4751
4752 Collections are generated by the **save** sub-command that dumps the current
4753 user environment state in terms of modulepaths and loaded modules. By default
4754 collections are saved under the ``$HOME/.module`` directory. Collections
4755 can be listed with **savelist** sub-command, displayed with **saveshow**
4756 and removed with **saverm**.
4757
4758 Collections may be valid for a given target if they are suffixed. In this
4759 case these collections can only be restored if their suffix correspond
4760 to the current value of the ``MODULES_COLLECTION_TARGET`` environment
4761 variable. Saving collection registers the target footprint by suffixing
4762 the collection filename with ``.$MODULES_COLLECTION_TARGET``.
4763
4764 Path variable element counter
4765 """""""""""""""""""""""""""""
4766
4767 Modules 4 provides path element counting feature which increases a
4768 reference counter each time a given path entry is added to a given
4769 path-like environment variable. As consequence a path entry element is
4770 removed from a path-like variable only if the related element counter is
4771 equal to 1. If this counter is greater than 1, path element is kept in
4772 variable and reference counter is decreased by 1.
4773
4774 This feature allows shared usage of particular path elements. For instance,
4775 modulefiles can append ``/usr/local/bin`` to ``PATH``, which is not unloaded
4776 until all the modulefiles that loaded it unload too.
4777
4778 Optimized I/O operations
4779 """"""""""""""""""""""""
4780
4781 Substantial work has been done to reduce the number of I/O operations
4782 done during global modulefile analysis commands like **avail** or
4783 **whatis**. ``stat``, ``open``, ``read`` and ``close`` I/O operations have
4784 been cut down to the minimum required when walking through the modulepath
4785 directories to check if files are modulefiles or to resolve module aliases.
4786
4787 Interpretation of modulefiles and modulerc are handled by the minimum
4788 required Tcl interpreters. Which means a configured Tcl interpreter is
4789 reused as much as possible between each modulefile interpretation or
4790 between each modulerc interpretation.
4791
4792 .. _Sourcing modulefiles:
4793
4794 Sourcing modulefiles
4795 """"""""""""""""""""
4796
4797 Modules 4 introduces the possibility to **source** a modulefile rather
4798 loading it. When it is sourced, a modulefile is interpreted into the shell
4799 environment but then it is not marked loaded in shell environment which
4800 differ from **load** sub-command.
4801
4802 This functionality is used in shell initialization scripts once **module**
4803 function is defined. There the ``etc/modulerc`` modulefile is sourced to
4804 setup the initial state of the environment, composed of *module use*
4805 and *module load* commands.
4806
4807
4808 Removed features and substantial behavior changes
4809 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4810
4811 Following sections provide list of Modules v3.2 features that are
4812 discontinued on Modules v4 or features with a substantial behavior change
4813 that should be taken in consideration when migrating to v4.
4814
4815 Package initialization
4816 """"""""""""""""""""""
4817
4818 ``MODULESBEGINENV`` environment snapshot functionality is not supported
4819 anymore on Modules v4. Modules collection mechanism should be used instead to
4820 **save** and **restore** sets of enabled modulepaths and loaded modulefiles.
4821
4822 Command line switches
4823 """""""""""""""""""""
4824
4825 Some command line switches are not supported anymore on v4.0. When still
4826 using them, a warning message is displayed and the command is ran with these
4827 unsupported switches ignored. Following command line switches are concerned:
4828
4829 * ``--force``, ``-f``
4830 * ``--human``
4831 * ``--verbose``, ``-v``
4832 * ``--silent``, ``-s``
4833 * ``--create``, ``-c``
4834 * ``--icase``, ``-i``
4835 * ``--userlvl`` lvl, ``-u`` lvl
4836
4837 Module sub-commands
4838 """""""""""""""""""
4839
4840 During an **help** sub-command, Modules v4 does not redirect output made
4841 on stdout in *ModulesHelp* Tcl procedure to stderr. Moreover when running
4842 **help**, version 4 interprets all the content of the modulefile, then call
4843 the *ModulesHelp* procedure if it exists, whereas Modules 3.2 only interprets
4844 the *ModulesHelp* procedure and not the rest of the modulefile content.
4845
4846 When **load** is asked on an already loaded modulefiles, Modules v4 ignores
4847 this new load order whereas v3.2 refreshed shell alias definitions found
4848 in this modulefile.
4849
4850 When **switching** on version 4 an *old* modulefile by a *new* one,
4851 no error is raised if *old* modulefile is not currently loaded. In this
4852 situation v3.2 threw an error and abort switch action. Additionally on
4853 **switch** sub-command, *new* modulefile does not keep the position held
4854 by *old* modulefile in loaded modules list on Modules v4 as it was the
4855 case on v3.2. Same goes for path-like environment variables: replaced
4856 path component is appended to the end or prepended to the beginning of
4857 the relative path-like variable, not appended or prepended relatively to
4858 the position hold by the swapped path component.
4859
4860 During a **switch** command, version 4 interprets the swapped-out modulefile
4861 in *unload* mode, so the sub-modulefiles loaded, with ``module load``
4862 order in the swapped-out modulefile are also unloaded during the switch.
4863
4864 Modules 4 provides path element counting feature which increases a reference
4865 counter each time a given path entry is added to a given environment
4866 variable. This feature also applies to the ``MODULEPATH`` environment
4867 variable. As consequence a modulepath entry element is removed from the
4868 modulepath enabled list only if the related element counter is equal to 1.
4869 When **unusing** a modulepath if its reference counter is greater than 1,
4870 modulepath is kept enabled and reference counter is decreased by 1.
4871
4872 On Modules 3.2 paths composing the ``MODULEPATH`` environment variable
4873 may contain reference to environment variable. These variable references
4874 are resolved dynamically when ``MODULEPATH`` is looked at during module
4875 sub-command action. This feature has been discontinued on Modules v4.
4876
4877 Following Modules sub-commands are not supported anymore on v4.0:
4878
4879 * ``clear``
4880 * ``update``
4881
4882
4883 Modules specific Tcl commands
4884 """""""""""""""""""""""""""""
4885
4886 Modules v4 provides path element counting feature which increases a reference
4887 counter each time a given path entry is added to a given environment
4888 variable. As a consequence a path entry element is not always removed
4889 from a path-like variable when calling to ``remove-path`` or calling to
4890 ``append-path`` or ``append-path`` at unloading time. The path element is
4891 removed only if its related element counter is equal to 1. If this counter
4892 is greater than 1, path element is kept in variable and reference counter
4893 is decreased by 1.
4894
4895 On Modules v4, **module-info mode** returns during an **unload** sub-command
4896 the ``unload`` value instead of ``remove`` on Modules v3.2. However if *mode*
4897 is tested against ``remove`` value, true will be returned. During a **switch**
4898 sub-command on Modules v4, ``unload`` then ``load`` is returned instead of
4899 ``switch1`` then ``switch2`` then ``switch3`` on Modules v3.2. However if
4900 *mode* is tested against ``switch`` value, true will be returned.
4901
4902 When using **set-alias**, Modules v3.2 defines a shell function when
4903 variables are in use in alias value on Bourne shell derivatives, Modules
4904 4 always defines a shell alias never a shell function.
4905
4906 Some Modules specific Tcl commands are not supported anymore on v4.0. When
4907 still using them, a warning message is displayed and these unsupported Tcl
4908 commands are ignored. Following Modules specific Tcl commands are concerned:
4909
4910 * ``module-info flags``
4911 * ``module-info trace``
4912 * ``module-info tracepat``
4913 * ``module-info user``
4914 * ``module-log``
4915 * ``module-trace``
4916 * ``module-user``
4917 * ``module-verbosity``
4918
4919
4920 Further reading
4921 ^^^^^^^^^^^^^^^
4922
4923 To get a complete list of the differences between Modules v3.2 and v4,
4924 please read the :ref:`changes` document.
4925
4926 A significant number of issues reported for v3.2 have been closed on v4.
4927 List of these closed issues can be found at:
4928
4929 https://github.com/envmodules/modules/milestone/1?closed=1