Debian Codegraph @

doc/source/design/advanced-module-version-specifiers.rst

raw
1 .. _advanced-module-version-specifiers:
2
3 Advanced module version specifiers
4 ==================================
5
6 Configuration
7 -------------
8
9 - Introduce :mconfig:`advanced_version_spec` option name
10
11 - off by default in v4 as previously *soft@1* could be a module name
12 - on by default in v5
13 - in case :mconfig:`extended_default` is disabled
14
15 - means short-hand notation cannot be used
16
17 - for soft/1.1 query soft@1 returns nothing
18
19 - in case :mconfig:`implicit_default` is disabled
20
21 - means a default should be found over version range or list in selection context
22
23 Specification
24 -------------
25
26 - Following Spack spec
27
28 - see https://github.com/spack/spack/blob/develop/lib/spack/spack/spec.py
29 - or https://spack.readthedocs.io/en/latest/basic_usage.html#version-specifier
30 - this specs covers all needs to specify module versions finely
31 - Spack users are already familiar with it
32 - it copes very well with command-line typing, avoiding most problematic characters
33
34 - that are interpreted by shells (like < or >)
35
36 - specification for one module could
37
38 - be almost condensed into one word "[email protected]:"
39 - or be expanded into multiple "soft @1.8:"
40
41 - same grammar used whatever the context
42
43 - command-line or as argument to modulefile command (like command)
44
45 - versions are specified whether
46
47 - as specific words (separated by " ")
48 - or as suffix to module name
49
50 - change command specifications which were previously accepting list of modules
51
52 - like *module1 module2/vers module3*
53 - now these modules could express versions appended to their name with *@*
54
55 - like *[email protected] module2@vers module3*
56
57 - or these versions could be defined as words next to module name
58
59 - like *[email protected] module2 @vers module3*
60
61 - as a consequence, it denies use of *@* in module names
62 - such change requires an option to be enabled to avoid breaking compat
63
64 - single version could be specified with soft@vers
65
66 - which matches *soft/vers* modulefile
67
68 - version could be specified as range
69
70 - *soft@:vers* or *soft@vers:* or *soft@vers1:vers2*
71 - Tcl-dictionarily determine what is between specified range
72 - ``extended_default`` is always considered *on* when matching range
73
74 - as *2.10* is included in *@1:3* whatever the configuration
75
76 - to be specified in a range or compared to a range, version major element should match an hexadecimal number
77
78 - which also means be only composed by [0-9af] characters
79 - for instance *10a*, *1.2.3*, *1.foo*, *10.2.good* are versions valid for range comparison
80 - but *10g*, *default*, *foo.2*, *.1.3.4* are versions invalid for range comparison
81
82 - a version range using in its definition version invalid for range comparison raises error
83
84 - for instance *@bar:foo*
85
86 - existing module versions invalid for range comparison are ignored
87
88 - which means versions *10g*, *default*, *.1.13.4* or *new* are excluded from result for a *@1.10:* range query
89
90 - when range is defined as *@major:major.minor*, version matching *major* version but above *major.minor* are excluded
91
92 - for instance *@1:1.10* will matches *1.0* and *1.8* but not *1.12*
93
94 - version could be specified as list
95
96 - soft@vers,vers,vers
97 - version specified could be text, like if symbolic version names are used
98 - should benefit from extended default specification
99
100 - to just express version with their major release number for instance
101
102 - an empty string among list is considered as a specification error
103
104 - for instance *soft@vers,vers,* or *soft@vers,,vers*
105
106 - any version in list can be a single version or a version range
107
108 - like *[email protected],1.4:1.6,1.8*
109 - helps to designate all versions except a few ones
110
111 - when using extended default syntax
112
113 - version selection is performed same way for *@vers* than for */vers*
114 - described in extended default design
115
116 - when :mconfig:`icase` is enabled for selection context and multiple directories match module name
117
118 - for instance query is *[email protected],1.2,1.4* and following modules exist: *ICASE/1.1*, *icase/1.2*, *iCaSe/1.3* and *iCaSe/1.4*
119 - as no *ICase* directory exists, and a version in highest directory *icase* matches query (*1.2*), *icase/1.2* is returned
120 - if query is *[email protected],1.2,1.4*, *iCaSe/1.4* will be selected as *iCaSe* directory matches query module name
121 - if query is *[email protected],1.4* or *[email protected],1.4*, as no version match in highest directory *iCaSe/1.4* will be selected
122
123 - in case of deep modulefiles
124
125 - specified version is matched at the level directly under specified module name
126
127 - not below levels
128 - for instance soft@vers, will match *soft/vers*, not *soft/deep/vers*
129 - to specify version for deep modules:*"soft/deep@vers*
130 - to ease version comparison deep version cannot be specified after the *@* character like *soft@deep/vers*
131
132 - such specification will raise an error
133
134 - advanced version specifier cannot be used with full path modulefile
135
136 - when a full path modulefile is specified any advanced version set afterward is treated literally
137 - for instance */path/to/modulefiles/[email protected]* will lead to the access of file *[email protected]* in directory */path/to/modulefiles*
138
139 - in case version is specified multiple times
140
141 - lastly mentioned (read from left to right) value is retained (it overwrite previous values)
142 - like *[email protected] @2.0* or *[email protected]@2.0*
143 - beware of version specified over a fully qualified modulefile like in *soft/[email protected]" or "soft/1.8 @1.10*
144
145 - it resolves to *soft/1.8/1.10* as advanced version specified is treated as an additional directory level
146
147 - in case modulefile is named *module@vers* in filesystem
148
149 - it is not found when option ``advanced_version_spec`` is enabled
150 - as it is translated to *module/vers*
151
152 - when special characters like *?* or \* are used in version name or value
153
154 - they are evaluated as Tcl glob pattern on return all matching modules context
155 - they are treated literally on single module selection and compatibility check context, no wildcard meaning is applied
156 - like currently done when specifying module version on command-line
157
158 - which leads to errors as no corresponding module is found::
159
160 $ module load loc_dv6/*
161 ERROR: Unable to locate a modulefile for 'loc_dv6/*'
162
163 - if version range or list does not contain a defined default
164
165 - in a selection context
166
167 - highest version is returned if ``implicit_default`` is enabled
168 - error returned if ``implicit_default`` is disabled
169
170 - even if version range or list specifies non-existent modules and only one existent module
171
172 - in a compatibility expression context
173
174 - range or list is matched against loaded environment whether the ``implicit_default`` state
175 - when no match found and evaluation are triggered, selection context applies
176
177 - when version is specified over an alias
178
179 - should proceed like for real modulefile
180 - when alias equal to a bare module with no version
181
182 - foo is alias on bar modulefile, bar is a file (not a dir with version modulefiles)
183 - query alias@:2 should behave like query alias/2
184
185 - when alias equal to a module/version modulefile
186
187 - foo is alias on bar/3 modulefile
188 - query alias@:2 should behave like query alias/2
189
190 - Contexts where it could be used
191
192 - :ref:`module_version_specification_to_return_all_matching_modules`
193 - :ref:`module_version_specification_to_select_one_module`
194 - :ref:`module_version_specification_to_check_compatibility`
195
196 .. note:: Advanced version specifier does not apply for the moment to the
197 :ref:`module_identification_to_select_one_module` context. Adding support
198 for this context will require a significant rework on module alias and
199 symbolic version registering and resolving code.
200
201 - impact of advanced version specifier implementation over code
202
203 - question especially over :mconfig:`auto_handling` code like conflict and prereq handling
204 - it should not impact triggers and actions
205 - but consist in an overall change of procedures comparing queries against loaded environment
206
207 - procedures like ``doesModuleConflict``
208
209 - also adapting ``getModules`` to restrict version possibilities to what has been specified
210
211 - for instance with query *soft@1,2* should only return versions matching
212
213 - prereq/conflict persistency
214
215 - ``LMPREREQ`` and ``LMCONFLICT`` content should reflect specified version constraint
216 - it could be expressed in these variables somewhat like it is specified to the
217
218 - prereq/conflict modulefile commands
219 - for instance ``__MODULES_LMPREREQ=soft/1.10&[email protected],1.10&foo@<2|foo@3<4``
220 - delimiters characters are *:*, *&* and *|*
221
222 - so use of characters * *, *@*, *,* is not an issue
223 - but for *:* which express version ranges it should be substituted to *<*
224
225 - prereq/conflict specification
226
227 - could consolidate different version set for same module on the same prereq/conflict list
228
229 - to indicate a preferred order (if available)
230 - like ``prereq [email protected] [email protected]``
231 - or ``prereq foo @1.8 [email protected]``
232
233 - also to trigger alternative requirement resolution in case first one failed
234
235 - as each module version specification leads to one evaluation only
236
237 - even if multiple modulefiles correspond to this specification
238
239 - like ``prereq [email protected],1.9,1.10`` will lead to *soft/1.10* load
240
241 - best candidate is chosen from matches
242
243 - in case ``implicit_default`` is disabled an explicit default should be part of the list or range for the triggered evaluation to succeed
244
245 - whereas ``prereq [email protected] [email protected] [email protected]`` will lead to a tentative load
246
247 - of *soft/1.8*, then *soft/1.9* if it failed then *soft/1.8* if it also failed
248
249 - one module version specification may match multiple loaded modules
250
251 - like ``conflict [email protected],1.9,1.10`` matches loaded modules *soft/1.8* and *soft/1.10*
252 - similar to situations where requirement or conflict is expressed over module generic name, like *soft*, and multiple versions of module are loaded
253
254 Corner cases
255 ------------
256
257 - When ``icase`` is enabled on all contexts and multiple directories match same icase module name
258
259 - for instance following modules exist: *ICASE/1.1*, *icase/1.2*, *iCaSe/1.3* and *iCaSe/1.4*
260 - a ``module avail -i icase`` will sort *iCaSe/1.4* as the highest entry
261 - however a ``module load -i [email protected],1.2,1.4`` command will load *icase/1.2*
262
263 - as *icase* directory matches query and version *1.2* is found in *icase* directory
264
265 - but a ``module load -i [email protected],1.4`` command will load *iCaSe/1.4*
266
267 - as no version *1.1* nor *1.4* is found in *icase* directory
268
269 .. vim:set tabstop=2 shiftwidth=2 expandtab autoindent: