Debian Codegraph @

doc/source/cookbook/hide-and-forbid-modules.rst

raw
1 .. _hide-and-forbid-modules:
2
3 Hide and forbid modules
4 =======================
5
6 :command:`module` is often used to give access to hundreds of different
7 software to a large variety of users, each of them provided in several
8 versions. In such situation users ends up facing thousands of modulefiles
9 which could be confusing.
10
11 On the other side, site's staff needs to decommission older software version
12 to ensure for instance that only the best matches for their hardware resources
13 remain. Such work needs specific care to avoid breaking the computing workflow
14 of users.
15
16 This recipe provides examples on how to hide or/and forbid modulefiles to
17 better cope with software life cycles and help both user and staff to get
18 their bearings in large environments.
19
20 Implementation
21 --------------
22
23 The :mfcmd:`module-hide` and :mfcmd:`module-forbid` modulefile commands block
24 respectively the visibility or the load evaluation of the modulefiles they
25 target. These two commands have a variety of options to inhibit their action
26 or display specific messages. Combining both commands enables to hide and
27 forbid specified modulefiles.
28
29 **Compatible with Modules v4.6+**
30
31 .. note::
32
33 Modules v4.7+ is required to use the ``--hidden-loaded`` option of
34 :mfcmd:`module-hide` command.
35
36 Usage examples
37 --------------
38
39 The use cases below describe situations where hiding or/and forbidding use of
40 modulefiles can be helpful.
41
42 The following set of modules will be used in the examples of this recipe. Each
43 use case will progressively add :mfcmd:`module-hide` and
44 :mfcmd:`module-forbid` statements in modulepath's :file:`.modulerc` file to
45 improve adapt module visibility or forbid access to them.
46
47 .. code-block:: console
48
49 $ module avail
50 -- ../example/hide-and-forbid-modules/modulefiles --
51 appA/1.0 bioappA/1.0 chemappA/1.0 deplibA/1.0
52 appA/2.0 bioappA/2.0 chemappA/2.0 deplibA/2.0
53 appB/1.0 bioappB/1.0 chemappB/1.0 deplibB/1.0
54 appB/2.0 bioappB/2.0 chemappB/2.0 deplibB/2.0
55
56 Limiting view to the useful software only
57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
58
59 With a large number of available modules it is interesting to reduce user's
60 visibility to only report the modules of interest.
61
62 Some modules may require other to be loaded as dependency. This is the case
63 for ``appA`` and ``appB`` which depend on ``deplibA`` or ``deplibB``. These
64 two dependency libraries are not interesting by themselves so they could be
65 hidden by default but they should stay findable for ``appA`` or ``appB``
66 modules to load their dependency.
67
68 :mfcmd:`module-hide --soft<module-hide>` could be used to address this need.
69 Designated module will be hidden unless searched. Following lines are added to
70 modulepath's :file:`.modulerc` file
71
72 .. code-block:: tcl
73
74 # hide modules only loaded as dependency
75 module-hide --soft deplibA
76 module-hide --soft --hidden-loaded deplibB
77
78 As a result ``deplibA`` and ``deplibB`` are not returned anymore on a global
79 :subcmd:`avail` sub-command::
80
81 $ module avail
82 -- ../example/hide-and-forbid-modules/modulefiles --
83 appA/1.0 appB/2.0 bioappB/1.0 chemappA/2.0
84 appA/2.0 bioappA/1.0 bioappB/2.0 chemappB/1.0
85 appB/1.0 bioappA/2.0 chemappA/1.0 chemappB/2.0
86
87 Yet these modules are still found when loading the modules of the application
88 requiring them::
89
90 $ module load appA
91 Loading appA/2.0
92 Loading requirement: deplibA/2.0
93
94 In some cases it is desirable to hide such dependency modules also when they
95 are loaded. This could be achieved by using the ``--hidden-loaded`` option of
96 the :mfcmd:`module-hide` command. Hidden loaded modules do not appear by
97 default on :subcmd:`list` sub-command output unless :option:`--all` is set.
98 Queries like :subcmd:`is-loaded` still detect such modules as loaded even if
99 hidden. In addition loading or unloading informational messages related to
100 these modules are not reported unless a :envvar:`verbosity
101 mode<MODULES_VERBOSITY>` higher than ``verbose`` is configured.
102
103 .. code-block:: console
104
105 $ module load appB
106 $ module list
107 Currently Loaded Modulefiles:
108 1) appB/2.0
109 $ module list --all
110 Currently Loaded Modulefiles:
111 1) deplibB/2.0 2) appB/2.0
112 $ module is-loaded deplibB
113 $ echo $?
114 0
115
116 Going further, among scientific applications some are only useful for a given
117 scientific field. A site may provide many software covering many scientific
118 fields but a user may only be concerned by one of these fields. In our example
119 software are provided for biology users (*bioappA* and *bioappB*) and other
120 software for chemistry users (*chemappA* and *chemappB*).
121
122 Say every biology users are part of a ``bio`` Unix group and every chemistry
123 users are member of a ``chem`` Unix group. Available software visibility could
124 be improved by only reporting the *bio* software to the *bio* users and the
125 *chem* software to the *chem* users.
126
127 .. code-block:: tcl
128
129 # hide modules not from user's scientific field
130 module-hide --soft --not-group bio bioappA bioappB
131 module-hide --soft --not-group chem chemappA chemappB
132
133 With the above statements put in modulepath's :file:`.modulerc` file, the
134 *bio* software are not seen anymore by *chem* users::
135
136 $ module avail
137 -- ../example/hide-and-forbid-modules/modulefiles --
138 appA/1.0 appB/1.0 chemappA/1.0 chemappB/1.0
139 appA/2.0 appB/2.0 chemappA/2.0 chemappB/2.0
140
141 However as they are softly hidden, these modules can still be seen if queried
142 or if ``--all`` option is used::
143
144 $ module avail bioappA
145 -- ../example/hide-and-forbid-modules/modulefiles --
146 bioappA/1.0 bioappA/2.0
147 $ module avail --all
148 -- ../example/hide-and-forbid-modules/modulefiles --
149 appA/1.0 bioappA/1.0 chemappA/1.0 deplibA/1.0
150 appA/2.0 bioappA/2.0 chemappA/2.0 deplibA/2.0
151 appB/1.0 bioappB/1.0 chemappB/1.0 deplibB/1.0
152 appB/2.0 bioappB/2.0 chemappB/2.0 deplibB/2.0
153
154 Software requiring administrative actions prior usage
155 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
156
157 Some software may require an administrative step to get allowed to use them,
158 like the signature of a user agreement. Such software should be reported among
159 the available modules but should not be used prior the administrative step
160 achieved.
161
162 In our example, the ``chemappA`` application requires a user charter to be
163 signed and then sent to the site staff. The access to the ``chemappA`` is
164 controlled by the ``chemappA`` Unix group: a user needs to be part of this
165 group to access and use the application.
166
167 .. code-block:: tcl
168
169 # forbid use of modules unless software user agreement is signed
170 set msg {User agreement for ChemAppA application must be validated to use it
171 1. Please connect to https://ChemAppA.example.org
172 2. Read the Term of Use and sign it
173 3. Send back this signed agreement to our service desk}
174 module-forbid --not-group chemappA --message $msg chemappA
175
176 With the above :mfcmd:`module-forbid` statement put in modulepath's
177 :file:`.modulerc` file, the software is still visible but its load is denied
178 unless if the user is part of the ``chemappA`` group.
179
180 .. code-block:: console
181
182 $ module avail chemappA
183 -- ../example/hide-and-forbid-modules/modulefiles --
184 chemappA/1.0 chemappA/2.0
185 $ module load chemappA
186 ERROR: Access to module 'chemappA/2.0' is denied
187 User agreement for ChemAppA application must be validated to use it
188 1. Please connect to https://ChemAppA.example.org
189 2. Read the Term of Use and sign it
190 3. Send back this signed agreement to our service desk
191
192 A specific message is provided through the ``--message`` option to guide the
193 user to complete the required administrative step. Once group membership is
194 acquired, the module can be seamlessly loaded.
195
196 .. code-block:: console
197
198 $ id --groups --name
199 chemappA chem
200 $ module load -v chemappA
201 Loading chemappA/2.0
202
203 .. note::
204
205 Do not forget to protect the access to the directory where the software is
206 installed to really ensure that only allowed users can use it.
207
208 Software limited to particular users
209 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
210
211 Some applications may be restricted to a limited set of users. For instance
212 because such application should not be disclosed or because it requires a
213 license that is paid only by a few users.
214
215 In our example, the ``appC`` application works with token-based licenses. The
216 only users that can use this software are those that have paid for a license
217 token. Other users should not access nor even see the availability of this
218 application. Users that have bought a license token are added to the ``appC``
219 Unix group.
220
221 .. code-block:: tcl
222
223 # fully hide and forbid modules unless user owns a license token
224 module-hide --hard --not-group appC appC
225 set msg {Access is restricted to owners of license token}
226 module-forbid --not-group appC --message $msg appC
227
228 The above statements have been added in modulepath's :file:`.modulerc` file.
229 The :mfcmd:`module-hide --hard<module-hide>` command is used to completely
230 remove visibility for non-authorized users.
231
232 .. code-block:: console
233
234 $ module avail
235 -- ../example/hide-and-forbid-modules/modulefiles --
236 appA/1.0 appB/1.0 chemappA/1.0 chemappB/1.0
237 appA/2.0 appB/2.0 chemappA/2.0 chemappB/2.0
238 $ module avail appC
239 $ module load appC
240 ERROR: Unable to locate a modulefile for 'appC'
241
242 The :mfcmd:`module-forbid` statement added for ``appC`` helps to get a clear
243 error message for the non-authorized users that are aware of the existence of
244 the module name and version (instead of getting a modulefile location error).
245
246 .. code-block:: console
247
248 $ module load appC/2.0
249 ERROR: Access to module 'appC/2.0' is denied
250 Access is restricted to owners of license token
251
252 Alternatively such restrictions on modulefiles can be achieved by adapting
253 file permission mode instead of adding statements in modulepath's
254 :file:`.modulerc` file.
255
256 .. code-block:: console
257
258 $ chmod 640 ../example/hide-and-forbid-modules/modulefiles/appC/*
259 $ chgrp appC ../example/hide-and-forbid-modules/modulefiles/appC/*
260
261 However restricting file permission mode does not enable to authorize several
262 Unix groups or users to access those modulefiles or to have specific error
263 messages unlike when :mfcmd:`module-hide` and :mfcmd:`module-forbid` commands
264 are used.
265
266 .. note::
267
268 Do not forget to protect the access to the directory where the software is
269 installed to really ensure that only the authorized users can use it.
270
271 Software life cycle
272 ^^^^^^^^^^^^^^^^^^^
273
274 When providing a new software version, it may be interesting to have a test
275 phase, for instance during one week, to make this new version checked by some
276 some pilot users prior the general availability.
277
278 In our example, we are currently November 13th and ``chempappB/2.0`` has been
279 installed two days ago. A test phase is ongoing and users that are member of
280 the ``pilot`` Unix group can already access this software prior its general
281 availability set for November 18th.
282
283 .. code-block:: tcl
284
285 # test new version of chemappB prior general availability
286 module-hide --before 2020-11-18 --not-group pilot chemappB/2.0
287
288 With the above statement added to modulepath's :file:`.modulerc` file, common
289 users will not see the new module until November 18th.
290
291 .. code-block:: console
292
293 $ date
294 Fri 13 Nov 2020 02:04:21 PM CET
295 $ module avail chemappB
296 -- ../example/hide-and-forbid-modules/modulefiles --
297 chemappB/1.0
298
299 At some point software need to be decommissioned to ensure for instance that
300 buggy or under-optimized versions are not used anymore by users.
301
302 In our example, ``appA/1.0`` needs to be removed as a newer version is
303 available for *appA* and this new version fixes a lot of issues that were
304 encountered with version 1.0. Decommission date for ``appA/1.0`` is planned on
305 November 18th.
306
307 .. code-block:: none
308
309 # decommission of old version of appA
310 set nearmsg {appA/1.0 will be decommissioned, please use appA/2.0}
311 set msg {appA/1.0 is decommissioned, please use appA/2.0}
312 module-hide --hard --after 2020-11-18 --not-group eol appA/1.0
313 module-forbid --after 2020-11-18 --not-group eol \
314 --nearly-message $nearmsg --message $msg appA/1.0
315
316 The above statements added to modulepath's :file:`.modulerc` file will remove
317 visibility and access to the decommissioned module on November 18th. Only
318 users part of the ``eol`` Unix group will still keep an access to the software
319 (which is useful in case some users are unfortunately stuck on this specific
320 version of *appA*). Specific messages are set to guide users toward selecting
321 another version for *appA*.
322
323 .. code-block:: console
324
325 $ module avail appA
326 -- ../example/hide-and-forbid-modules/modulefiles --
327 appA/1.0 appA/2.0
328 $ module load appA/1.0
329 Loading appA/1.0
330 WARNING: Access to module will be denied starting '2020-11-18'
331 appA/1.0 will be decommissioned, please use appA/2.0
332 Loading requirement: deplibA/1.0
333
334 Prior decommission date, module will still be visible and loadable. However
335 when loading the module a warning message will appear when the expiry date
336 will be close. The :envvar:`nearly_forbidden_days<MODULES_NEARLY_FORBIDDEN_DAYS>`
337 module configuration defines starting when such warning message should be
338 reported prior decommission date (14 days by default).
339
340 .. code-block:: console
341
342 $ module config nearly_forbidden_days
343 Modules Release 4.6.0 (2020-09-16)
344
345 - Config. name ---------.- Value (set by if default overridden) ---------------
346 nearly_forbidden_days 14
347
348 Once the decommissioned date is over, ``appA/1.0`` has disappeared unless for
349 users member of the ``eol`` Unix group.
350
351 .. code-block:: console
352
353 $ date
354 Thu 19 Nov 2020 02:46:27 PM CET
355 $ module avail appA
356 -- ../example/hide-and-forbid-modules/modulefiles --
357 appA/2.0
358 $ module load appA/1.0
359 ERROR: Access to module 'appA/1.0' is denied
360 appA/1.0 is decommissioned, please use appA/2.0
361
362 .. note::
363
364 Do not forget to protect the access to the directory where the software is
365 installed prior its general availability and after its decommissioned date.
366
367 .. vim:set tabstop=2 shiftwidth=2 expandtab autoindent: