| 1 |
.. _spider-sub-command: |
| 2 |
|
| 3 |
spider sub-command |
| 4 |
================== |
| 5 |
|
| 6 |
:subcmd:`spider` sub-command scans all the enabled modulepaths and the |
| 7 |
modulepaths enabled by modules in every modulepaths to return the modules |
| 8 |
matching the search query. |
| 9 |
|
| 10 |
Sub-command properties |
| 11 |
---------------------- |
| 12 |
|
| 13 |
General properties: |
| 14 |
|
| 15 |
* Shortcut name: ``spi`` |
| 16 |
* Accepted option: same as :subcmd:`avail` |
| 17 |
* Expected number of argument: 0 to N |
| 18 |
* Accept boolean variant specification: yes |
| 19 |
* Parse module version specification: yes |
| 20 |
* Accept extra match search: yes |
| 21 |
* Accept only extra match search: yes |
| 22 |
* Fully read modulefile when checking validity: no |
| 23 |
* Sub-command only called from top level: yes |
| 24 |
* Insensitive case match: ``search`` mode |
| 25 |
* Collect tags: yes |
| 26 |
|
| 27 |
:subcmd:`spider` is implemented as a two-step process: |
| 28 |
|
| 29 |
* find all modulepaths through a scan evaluation to fetch all ``use`` elements |
| 30 |
* perform an :subcmd:`avail` sub-command on all these modulepaths |
| 31 |
|
| 32 |
All kind of options and search queries supported by :subcmd:`avail` |
| 33 |
sub-command are supported by :subcmd:`spider`. |
| 34 |
|
| 35 |
The collect of modulepaths starts by scanning the global/user rc space and the |
| 36 |
currently enabled modulepaths in :envvar:`MODULEPATH`. Each modulefile in |
| 37 |
these modulepaths are evaluated to find if they enable other modulepaths with |
| 38 |
either ``module use``, ``append-path MODULEPATH`` or ``prepend-path |
| 39 |
MODULEPATH`` modulefile commands. Found modulepaths are added to the list of |
| 40 |
modulepaths to scan to evaluate in turn their modulefiles to find other |
| 41 |
modulepaths. |
| 42 |
|
| 43 |
Each modulepath is converted to its absolute path name to avoid multiple scan. |
| 44 |
Empty strings are discarded. Symbolic link modulepath are not converted into |
| 45 |
their target directory. Which means this symlink modulepath will appear on its |
| 46 |
own. |
| 47 |
|
| 48 |
Modulepath are reported in the following order. First each modulepath defined |
| 49 |
in the ``MODULEPATH`` environment variable. Then each modulepath enabled by |
| 50 |
modulefiles of the global/user rc space, then of the first modulepath |
| 51 |
evaluated, then of the second evaluated and so on. Being added in an *append* |
| 52 |
or a *prepend* mode does not change the report order. |
| 53 |
|
| 54 |
Configuration |
| 55 |
------------- |
| 56 |
|
| 57 |
Several configuration options are added to be able to set a different behavior |
| 58 |
for ``spider`` than for ``avail``: |
| 59 |
|
| 60 |
* :mconfig:`spider_output` and :mconfig:`spider_terse_output` configuration |
| 61 |
options define what content to report in ``spider`` output. See |
| 62 |
:ref:`output-configuration` document to learn the properties of these |
| 63 |
options. |
| 64 |
|
| 65 |
* :mconfig:`spider_indepth` configuration option to enable or disable indepth |
| 66 |
module search result. |
| 67 |
|
| 68 |
Reporting |
| 69 |
--------- |
| 70 |
|
| 71 |
With the introduction of the ``spider`` sub-command comes the *via* concept. |
| 72 |
When a modulepath is enabled by a given module, it is said that modulepath is |
| 73 |
available *via* this module. |
| 74 |
|
| 75 |
This information is reported on regular output next to the modulepath name. If |
| 76 |
a module enables a modulepath, the mention *(via module/version)* is added. |
| 77 |
Via information is reported if ``via`` element is set on the corresponding |
| 78 |
output configuration option. ``via`` is not supported on terse output mode, to |
| 79 |
avoid breaking parsing of such output. ``via`` is set by default on |
| 80 |
``spider_output`` configuration option. Via information is not reported if |
| 81 |
modulepath is not reported. |
| 82 |
|
| 83 |
During a ``spider`` processing the *via* information is collected from the |
| 84 |
extra specifier *use* employed by modules. First evaluated module that enable |
| 85 |
the modulepath will be the one reported. |
| 86 |
|
| 87 |
Modulepath usage record |
| 88 |
^^^^^^^^^^^^^^^^^^^^^^^ |
| 89 |
|
| 90 |
When loading a module that enable a modulepath, this information is stored in |
| 91 |
the loaded environment through the :envvar:`__MODULES_LMUSE` environment |
| 92 |
variable. This information is used to report the *via* information on |
| 93 |
``avail`` and ``spider`` output. With that, the *via* information is |
| 94 |
consistently available to user, whether the modulepath is already enabled or |
| 95 |
can be through the addition of another module. This *via* information stored |
| 96 |
in ``__MODULES_LMUSE`` is also used to establish a dependency link between the |
| 97 |
modulepath and the loaded module. |
| 98 |
|
| 99 |
Note that a path added with ``module use`` is converted in its absolute path |
| 100 |
form whereas with ``append-path`` and ``prepend-path`` the entry is set as-is. |
| 101 |
This is reflected in the path entry recorded in ``__MODULES_LMUSE``. Which |
| 102 |
means users should carefully define path in their absolute form when set to |
| 103 |
``MODULEPATH`` with ``append-path`` or ``prepend-path``. |
| 104 |
|
| 105 |
When path entry contains an environment variable reference, this reference is |
| 106 |
recorded as is in ``MODULEPATH`` and relative entry in ``__MODULES_LMUSE``. |
| 107 |
Path pointed by this entry evolves with changes made to referred environment |
| 108 |
variable. |
| 109 |
|
| 110 |
JSON output |
| 111 |
^^^^^^^^^^^ |
| 112 |
|
| 113 |
The *via* information is always reported on the JSON output. A ``via`` key is |
| 114 |
added on each module reported. If module is part of a modulepath that is |
| 115 |
enabled by another module, the value of the ``via`` key is set to the name |
| 116 |
and version of this module. As modulepaths are just keys in the JSON document |
| 117 |
produced, the via information is stored in each module JSON object rather once |
| 118 |
next to the modulepath information. |
| 119 |
|
| 120 |
Pre-enabled modulepath |
| 121 |
^^^^^^^^^^^^^^^^^^^^^^ |
| 122 |
|
| 123 |
When a modulepath is already enabled and a modulefile evaluation enables it |
| 124 |
again, the reference counter for this path is increased. This path reference |
| 125 |
counter increase is performed whatever the modulefile command used to enable |
| 126 |
the path (``module use``, ``append-path`` and ``prepend-path``). |
| 127 |
|
| 128 |
If the same path is then enabled again by another loading modulefile, the |
| 129 |
reference counter is increased again. Unload of these modulefiles decreases |
| 130 |
the reference counter. |
| 131 |
|
| 132 |
The reference counter information is taken into account to determine if |
| 133 |
modulepath is enabled via a loaded module. Which means that if a modulepath |
| 134 |
is pre-enabled then a modulefile is enabling it again, the reference counter |
| 135 |
will help to say that modulepath is not ``via`` this loaded module. There is |
| 136 |
no *via* if reference counter is greater than the number of loaded modules |
| 137 |
enabling the modulepath. |
| 138 |
|
| 139 |
Note that enabling an already enabled modulepath from the command line will |
| 140 |
not trigger any reference counter updates on Modules, even if a reference |
| 141 |
counter is specifically set for the given modulepath. As a result, the *via* |
| 142 |
information of an enabled modulepath is not changed if user enables it again |
| 143 |
from the command-line. |
| 144 |
|
| 145 |
Specific impact |
| 146 |
--------------- |
| 147 |
|
| 148 |
* As every ``.modulerc`` files are evaluated during the ``scan`` step (to |
| 149 |
collect modulepaths), tags, hiding rules & co applies to modulefiles from |
| 150 |
different modulepaths or those defined within a global rc file. |
| 151 |
|
| 152 |
* A bad environment state (like inconsistent loaded environment variables) |
| 153 |
produces an error exit code on ``spider`` |
| 154 |
|
| 155 |
Corner cases |
| 156 |
------------ |
| 157 |
|
| 158 |
* If a modulepath is used by a module but this module also unuse it thereafter |
| 159 |
this modulepath is kept recorded in the ``__MODULES_LMUSE`` tracking |
| 160 |
environment variable. |
| 161 |
|
| 162 |
* On :subcmd:`list` sub-command JSON output, ``via`` information is currently |
| 163 |
always empty. Could be improved in the future. |
| 164 |
|
| 165 |
.. vim:set tabstop=2 shiftwidth=2 expandtab autoindent: |