| 1 |
.. -*- mode: rst -*- |
| 2 |
.. vi: set ft=rst sts=4 ts=4 sw=4 et tw=79: |
| 3 |
|
| 4 |
.. _chap_design_result_records: |
| 5 |
|
| 6 |
************** |
| 7 |
Result records |
| 8 |
************** |
| 9 |
|
| 10 |
.. topic:: Specification scope and status |
| 11 |
|
| 12 |
This specification describes the current implementation. |
| 13 |
|
| 14 |
Result records are the standard return value format for all DataLad commands. |
| 15 |
Each command invocation yields one or more result records. Result records are |
| 16 |
routinely inspected throughout the code base, and are used to inform generic |
| 17 |
error handling, as well as particular calling commands on how to proceed with |
| 18 |
a specific operation. |
| 19 |
|
| 20 |
The technical implementation of a result record is a Python dictionary. This |
| 21 |
dictionary must contain a number of mandatory fields/keys (see below). However, |
| 22 |
an arbitrary number of additional fields may be added to a result record. |
| 23 |
|
| 24 |
The ``get_status_dict()`` function simplifies the creation of result records. |
| 25 |
|
| 26 |
.. note:: |
| 27 |
Developers *must* compose result records with care! DataLad supports custom |
| 28 |
user-provided hook configurations that use result record fields to |
| 29 |
decide when to trigger a custom post-result operation. Such custom hooks |
| 30 |
rely on a persistent naming and composition of result record fields. |
| 31 |
Changes to result records, including field name changes, field value changes, |
| 32 |
but also timing/order of record emitting potentially break user set ups! |
| 33 |
|
| 34 |
|
| 35 |
Mandatory fields |
| 36 |
================ |
| 37 |
|
| 38 |
The following keys *must* be present in any result record. If any of these |
| 39 |
keys is missing, DataLad's behavior is undefined. |
| 40 |
|
| 41 |
|
| 42 |
``action`` |
| 43 |
---------- |
| 44 |
|
| 45 |
A string label identifying which type of operation a result is associated with. |
| 46 |
Labels *must not* contain white space. They should be compact, and lower-cases, |
| 47 |
and use ``_`` (underscore) to separate words in compound labels. |
| 48 |
|
| 49 |
A result without an ``action`` label will not be processed and is discarded. |
| 50 |
|
| 51 |
|
| 52 |
``path`` |
| 53 |
-------- |
| 54 |
|
| 55 |
A string with an *absolute* path describing the local entity a result is |
| 56 |
associated with. Paths must be platform-specific (e.g., Windows paths on |
| 57 |
Windows, and POSIX paths on other operating systems). When a result is about an |
| 58 |
entity that has no meaningful relation to the local file system (e.g., a URL to |
| 59 |
be downloaded), to ``path`` value should be determined with respect to the |
| 60 |
potential impact of the result on any local entity (e.g., a URL downloaded |
| 61 |
to a local file path, a local dataset modified based on remote information). |
| 62 |
|
| 63 |
.. _target-result-status: |
| 64 |
|
| 65 |
``status`` |
| 66 |
---------- |
| 67 |
|
| 68 |
This field indicates the nature of a result in terms of four categories, identified |
| 69 |
by a string label. |
| 70 |
|
| 71 |
- ``ok``: a standard, to-be-expected result |
| 72 |
- ``notneeded``: an operation that was requested, but found to be unnecessary |
| 73 |
in order to achieve a desired goal |
| 74 |
- ``impossible``: a requested operation cannot be performed, possibly because |
| 75 |
its preconditions are not met |
| 76 |
- ``error``: an error occurred while performing an operation |
| 77 |
|
| 78 |
Based on the ``status`` field, a result is categorized into *success* (``ok``, |
| 79 |
``notneeded``) and *failure* (``impossible``, ``error``). Depending on the |
| 80 |
``on_failure`` parameterization of a command call, any failure-result emitted |
| 81 |
by a command can lead to an ``IncompleteResultsError`` being raised on command |
| 82 |
exit, or a non-zero exit code on the command line. With ``on_failure='stop'``, |
| 83 |
an operation is halted on the first failure and the command errors out |
| 84 |
immediately, with ``on_failure='continue'`` an operation will continue despite |
| 85 |
intermediate failures and the command only errors out at the very end, with |
| 86 |
``on_failure='ignore'`` the command will not error even when failures occurred. |
| 87 |
The latter mode can be used in cases where the initial status-characterization |
| 88 |
needs to be corrected for the particular context of an operation (e.g., to |
| 89 |
relabel expected and recoverable errors). |
| 90 |
|
| 91 |
|
| 92 |
Common optional fields |
| 93 |
====================== |
| 94 |
|
| 95 |
The following fields are not required, but can be used to enrich a result |
| 96 |
record with additional information that improves its interpretability, or |
| 97 |
triggers particular optional functionality in generic result processing. |
| 98 |
|
| 99 |
|
| 100 |
``type`` |
| 101 |
-------- |
| 102 |
|
| 103 |
This field indicates the type of entity a result is associated with. This may |
| 104 |
or may not be the type of the local entity identified by the ``path`` value. |
| 105 |
The following values are common, and should be used in matching cases, but |
| 106 |
arbitrary other values are supported too: |
| 107 |
|
| 108 |
- ``dataset``: a DataLad dataset |
| 109 |
- ``file``: a regular file |
| 110 |
- ``directory``: a directory |
| 111 |
- ``symlink``: a symbolic link |
| 112 |
- ``key``: a git-annex key |
| 113 |
- ``sibling``: a Dataset sibling or Git remote |
| 114 |
|
| 115 |
|
| 116 |
``message`` |
| 117 |
----------- |
| 118 |
|
| 119 |
A message providing additional human-readable information on the nature or |
| 120 |
provenance of a result. Any non-``ok`` results *should* have a message providing |
| 121 |
information on the rational of their status characterization. |
| 122 |
|
| 123 |
A message can be a string or a tuple. In case of a tuple, the second item can |
| 124 |
contain values for ``%``-expansion of the message string. Expansion is performed |
| 125 |
only immediately prior to actually outputting the message, hence string formatting |
| 126 |
runtime costs can be avoided this way, if a message is not actually shown. |
| 127 |
|
| 128 |
|
| 129 |
``logger`` |
| 130 |
---------- |
| 131 |
|
| 132 |
If a result record has a ``message`` field, then a given `Logger` instance |
| 133 |
(typically from ``logging.getLogger()``) will be used to automatically log |
| 134 |
this message. The log channel/level is determined based on |
| 135 |
``datalad.log.result-level`` configuration setting. By default, this is |
| 136 |
the ``debug`` level. When set to ``match-status`` the log level is determined |
| 137 |
based on the ``status`` field of a result record: |
| 138 |
|
| 139 |
- ``debug`` for ``'ok'``, and ``'notneeded'`` results |
| 140 |
- ``warning`` for ``'impossible'`` results |
| 141 |
- ``error`` for ``'error'`` results |
| 142 |
|
| 143 |
This feature should be used with care. Unconditional logging can lead to |
| 144 |
confusing double-reporting when results rendered and also visibly logged. |
| 145 |
|
| 146 |
|
| 147 |
``refds`` |
| 148 |
--------- |
| 149 |
|
| 150 |
This field can identify a path (using the same semantics and requirements as |
| 151 |
the ``path`` field) to a reference dataset that represents the larger context |
| 152 |
of an operation. For example, when recursively processing multiple files across |
| 153 |
a number of subdatasets, a ``refds`` value may point to the common superdataset. |
| 154 |
This value may influence, for example, how paths are rendered in user-output. |
| 155 |
|
| 156 |
|
| 157 |
``parentds`` |
| 158 |
------------ |
| 159 |
|
| 160 |
This field can identify a path (using the same semantics and requirements as |
| 161 |
the ``path`` field) to a dataset containing an entity. |
| 162 |
|
| 163 |
|
| 164 |
``state`` |
| 165 |
--------- |
| 166 |
|
| 167 |
A string label categorizing the state of an entity. Common values are: |
| 168 |
|
| 169 |
- ``clean`` |
| 170 |
- ``untracked`` |
| 171 |
- ``modified`` |
| 172 |
- ``deleted`` |
| 173 |
- ``absent`` |
| 174 |
- ``present`` |
| 175 |
|
| 176 |
|
| 177 |
``error_message`` |
| 178 |
----------------- |
| 179 |
|
| 180 |
An error message that was captured or produced while achieving a result. |
| 181 |
|
| 182 |
An error message can be a string or a tuple. In the case of a tuple, the |
| 183 |
second item can contain values for ``%``-expansion of the message string. |
| 184 |
|
| 185 |
|
| 186 |
``exception`` |
| 187 |
------------- |
| 188 |
|
| 189 |
An exception that occurred while achieving the reported result. |
| 190 |
|
| 191 |
|
| 192 |
``exception_traceback`` |
| 193 |
----------------------- |
| 194 |
|
| 195 |
A string with a traceback for the exception reported in ``exception``. |
| 196 |
|
| 197 |
|
| 198 |
Additional fields observed "in the wild" |
| 199 |
======================================== |
| 200 |
|
| 201 |
Given that arbitrary fields are supported in result records, it is impossible |
| 202 |
to compose a comprehensive list of field names (keys). However, in order to |
| 203 |
counteract needless proliferation, the following list describes fields that |
| 204 |
have been observed in implementations. Developers are encouraged to preferably |
| 205 |
use compatible names from this list, or extend the list for additional items. |
| 206 |
|
| 207 |
In alphabetical order: |
| 208 |
|
| 209 |
``bytesize`` |
| 210 |
The size of an entity in bytes (integer). |
| 211 |
|
| 212 |
``gitshasum`` |
| 213 |
SHA1 of an entity (string) |
| 214 |
|
| 215 |
``prev_gitshasum`` |
| 216 |
SHA1 of a previous state of an entity (string) |
| 217 |
|
| 218 |
``key`` |
| 219 |
The git-annex key associated with a ``type``-``file`` entity. |