Debian Codegraph @

doc/source/design/advanced-module-version-specifiers.rst

raw
1 .. _advanced-module-version-specifiers:
2
3 Advanced module version specifiers
4 ==================================
5
6 Configuration
7 -------------
8
9 - Introduce :mconfig:`advanced_version_spec` option name
10
11 - off by default in v4 as previously *soft@1* could be a module name
12 - on by default in v5
13 - in case :mconfig:`extended_default` is disabled
14
15 - means short-hand notation cannot be used
16
17 - for soft/1.1 query soft@1 returns nothing
18
19 - in case :mconfig:`implicit_default` is disabled
20
21 - means a default should be found over version range or list in selection context
22
23 Specification
24 -------------
25
26 - Following Spack spec
27
28 - see https://github.com/spack/spack/blob/develop/lib/spack/spack/spec.py
29 - or https://spack.readthedocs.io/en/latest/basic_usage.html#version-specifier
30 - this specs covers all needs to specify module versions finely
31 - Spack users are already familiar with it
32 - it copes very well with command-line typing, avoiding most problematic characters
33
34 - that are interpreted by shells (like < or >)
35
36 - specification for one module could
37
38 - be almost condensed into one word "[email protected]:"
39 - or be expanded into multiple "soft @1.8:"
40
41 - same grammar used whatever the context
42
43 - command-line or as argument to modulefile command (like command)
44
45 - versions are specified whether
46
47 - as specific words (separated by " ")
48 - or as suffix to module name
49
50 - change command specifications which were previously accepting list of modules
51
52 - like *module1 module2/vers module3*
53 - now these modules could express versions appended to their name with *@*
54
55 - like *[email protected] module2@vers module3*
56
57 - or these versions could be defined as words next to module name
58
59 - like *[email protected] module2 @vers module3*
60
61 - as a consequence, it denies use of *@* in module names
62 - such change requires an option to be enabled to avoid breaking compat
63
64 - single version could be specified with soft@vers
65
66 - which matches *soft/vers* modulefile
67
68 - version could be specified as range
69
70 - *soft@:vers* or *soft@vers:* or *soft@vers1:vers2*
71 - Tcl-dictionarily determine what is between specified range
72 - ``extended_default`` is always considered *on* when matching range
73
74 - as *2.10* is included in *@1:3* whatever the configuration
75
76 - to be specified in a range or compared to a range, version major element should match an hexadecimal number
77
78 - which also means be only composed by [0-9af] characters
79 - for instance *10a*, *1.2.3*, *1.foo*, *10.2.good* are versions valid for range comparison
80 - but *10g*, *default*, *foo.2*, *.1.3.4* are versions invalid for range comparison
81
82 - a version range using in its definition version invalid for range comparison raises error
83
84 - for instance *@bar:foo*
85
86 - existing module versions invalid for range comparison are ignored
87
88 - which means versions *10g*, *default*, *.1.13.4* or *new* are excluded from result for a *@1.10:* range query
89
90 - when range is defined as *@major:major.minor*, version matching *major* version but above *major.minor* are excluded
91
92 - for instance *@1:1.10* will matches *1.0* and *1.8* but not *1.12*
93
94 - the upper bound may be less precise than the lower bound
95
96 - an abbreviated upper bound includes all versions that extend it,
97 provided they also satisfy the lower bound
98 - for instance *@1.2:1* matches *1.2* and *1.10* but not *1.1* or *2.0*
99 - a reversed range remains invalid unless the lower bound extends the
100 abbreviated upper bound, so *@2.0:1* is still rejected
101
102 - version could be specified as list
103
104 - soft@vers,vers,vers
105 - version specified could be text, like if symbolic version names are used
106 - should benefit from extended default specification
107
108 - to just express version with their major release number for instance
109
110 - an empty string among list is considered as a specification error
111
112 - for instance *soft@vers,vers,* or *soft@vers,,vers*
113
114 - any version in list can be a single version or a version range
115
116 - like *[email protected],1.4:1.6,1.8*
117 - helps to designate all versions except a few ones
118
119 - when using extended default syntax
120
121 - version selection is performed same way for *@vers* than for */vers*
122 - described in extended default design
123
124 - when :mconfig:`icase` is enabled for selection context and multiple directories match module name
125
126 - for instance query is *[email protected],1.2,1.4* and following modules exist: *ICASE/1.1*, *icase/1.2*, *iCaSe/1.3* and *iCaSe/1.4*
127 - as no *ICase* directory exists, and a version in highest directory *icase* matches query (*1.2*), *icase/1.2* is returned
128 - if query is *[email protected],1.2,1.4*, *iCaSe/1.4* will be selected as *iCaSe* directory matches query module name
129 - if query is *[email protected],1.4* or *[email protected],1.4*, as no version match in highest directory *iCaSe/1.4* will be selected
130
131 - in case of deep modulefiles
132
133 - specified version is matched at the level directly under specified module name
134
135 - not below levels
136 - for instance soft@vers, will match *soft/vers*, not *soft/deep/vers*
137 - to specify version for deep modules:*"soft/deep@vers*
138 - to ease version comparison deep version cannot be specified after the *@* character like *soft@deep/vers*
139
140 - such specification will raise an error
141
142 - advanced version specifier cannot be used with full path modulefile
143
144 - when a full path modulefile is specified any advanced version set afterward is treated literally
145 - for instance */path/to/modulefiles/[email protected]* will lead to the access of file *[email protected]* in directory */path/to/modulefiles*
146
147 - in case version is specified multiple times
148
149 - lastly mentioned (read from left to right) value is retained (it overwrite previous values)
150 - like *[email protected] @2.0* or *[email protected]@2.0*
151 - beware of version specified over a fully qualified modulefile like in *soft/[email protected]" or "soft/1.8 @1.10*
152
153 - it resolves to *soft/1.8/1.10* as advanced version specified is treated as an additional directory level
154
155 - in case modulefile is named *module@vers* in filesystem
156
157 - it is not found when option ``advanced_version_spec`` is enabled
158 - as it is translated to *module/vers*
159
160 - when special characters like *?* or \* are used in version name or value
161
162 - they are evaluated as Tcl glob pattern on return all matching modules context
163 - they are treated literally on single module selection and compatibility check context, no wildcard meaning is applied
164 - like currently done when specifying module version on command-line
165
166 - which leads to errors as no corresponding module is found::
167
168 $ module load loc_dv6/*
169 ERROR: Unable to locate a modulefile for 'loc_dv6/*'
170
171 - if version range or list does not contain a defined default
172
173 - in a selection context
174
175 - highest version is returned if ``implicit_default`` is enabled
176 - error returned if ``implicit_default`` is disabled
177
178 - even if version range or list specifies non-existent modules and only one existent module
179
180 - in a compatibility expression context
181
182 - range or list is matched against loaded environment whether the ``implicit_default`` state
183 - when no match found and evaluation are triggered, selection context applies
184
185 - when version is specified over an alias
186
187 - should proceed like for real modulefile
188 - when alias equal to a bare module with no version
189
190 - foo is alias on bar modulefile, bar is a file (not a dir with version modulefiles)
191 - query alias@:2 should behave like query alias/2
192
193 - when alias equal to a module/version modulefile
194
195 - foo is alias on bar/3 modulefile
196 - query alias@:2 should behave like query alias/2
197
198 - Contexts where it could be used
199
200 - :ref:`module_version_specification_to_return_all_matching_modules`
201 - :ref:`module_version_specification_to_select_one_module`
202 - :ref:`module_version_specification_to_check_compatibility`
203
204 .. note:: Advanced version specifier does not apply for the moment to the
205 :ref:`module_identification_to_select_one_module` context. Adding support
206 for this context will require a significant rework on module alias and
207 symbolic version registering and resolving code.
208
209 - impact of advanced version specifier implementation over code
210
211 - question especially over :mconfig:`auto_handling` code like conflict and prereq handling
212 - it should not impact triggers and actions
213 - but consist in an overall change of procedures comparing queries against loaded environment
214
215 - procedures like ``doesModuleConflict``
216
217 - also adapting ``getModules`` to restrict version possibilities to what has been specified
218
219 - for instance with query *soft@1,2* should only return versions matching
220
221 - prereq/conflict persistency
222
223 - ``LMPREREQ`` and ``LMCONFLICT`` content should reflect specified version constraint
224 - it could be expressed in these variables somewhat like it is specified to the
225
226 - prereq/conflict modulefile commands
227 - for instance ``__MODULES_LMPREREQ=soft/1.10&[email protected],1.10&foo@<2|foo@3<4``
228 - delimiters characters are *:*, *&* and *|*
229
230 - so use of characters * *, *@*, *,* is not an issue
231 - but for *:* which express version ranges it should be substituted to *<*
232
233 - prereq/conflict specification
234
235 - could consolidate different version set for same module on the same prereq/conflict list
236
237 - to indicate a preferred order (if available)
238 - like ``prereq [email protected] [email protected]``
239 - or ``prereq foo @1.8 [email protected]``
240
241 - also to trigger alternative requirement resolution in case first one failed
242
243 - as each module version specification leads to one evaluation only
244
245 - even if multiple modulefiles correspond to this specification
246
247 - like ``prereq [email protected],1.9,1.10`` will lead to *soft/1.10* load
248
249 - best candidate is chosen from matches
250
251 - in case ``implicit_default`` is disabled an explicit default should be part of the list or range for the triggered evaluation to succeed
252
253 - whereas ``prereq [email protected] [email protected] [email protected]`` will lead to a tentative load
254
255 - of *soft/1.8*, then *soft/1.9* if it failed then *soft/1.8* if it also failed
256
257 - one module version specification may match multiple loaded modules
258
259 - like ``conflict [email protected],1.9,1.10`` matches loaded modules *soft/1.8* and *soft/1.10*
260 - similar to situations where requirement or conflict is expressed over module generic name, like *soft*, and multiple versions of module are loaded
261
262 Corner cases
263 ------------
264
265 - When ``icase`` is enabled on all contexts and multiple directories match same icase module name
266
267 - for instance following modules exist: *ICASE/1.1*, *icase/1.2*, *iCaSe/1.3* and *iCaSe/1.4*
268 - a ``module avail -i icase`` will sort *iCaSe/1.4* as the highest entry
269 - however a ``module load -i [email protected],1.2,1.4`` command will load *icase/1.2*
270
271 - as *icase* directory matches query and version *1.2* is found in *icase* directory
272
273 - but a ``module load -i [email protected],1.4`` command will load *iCaSe/1.4*
274
275 - as no version *1.1* nor *1.4* is found in *icase* directory
276
277 .. vim:set tabstop=2 shiftwidth=2 expandtab autoindent: