Debian Codegraph @

docs/source/design/result_records.rst

raw
1 .. -*- mode: rst -*-
2 .. vi: set ft=rst sts=4 ts=4 sw=4 et tw=79:
3
4 .. _chap_design_result_records:
5
6 **************
7 Result records
8 **************
9
10 .. topic:: Specification scope and status
11
12 This specification describes the current implementation.
13
14 Result records are the standard return value format for all DataLad commands.
15 Each command invocation yields one or more result records. Result records are
16 routinely inspected throughout the code base, and are used to inform generic
17 error handling, as well as particular calling commands on how to proceed with
18 a specific operation.
19
20 The technical implementation of a result record is a Python dictionary. This
21 dictionary must contain a number of mandatory fields/keys (see below). However,
22 an arbitrary number of additional fields may be added to a result record.
23
24 The ``get_status_dict()`` function simplifies the creation of result records.
25
26 .. note::
27 Developers *must* compose result records with care! DataLad supports custom
28 user-provided hook configurations that use result record fields to
29 decide when to trigger a custom post-result operation. Such custom hooks
30 rely on a persistent naming and composition of result record fields.
31 Changes to result records, including field name changes, field value changes,
32 but also timing/order of record emitting potentially break user set ups!
33
34
35 Mandatory fields
36 ================
37
38 The following keys *must* be present in any result record. If any of these
39 keys is missing, DataLad's behavior is undefined.
40
41
42 ``action``
43 ----------
44
45 A string label identifying which type of operation a result is associated with.
46 Labels *must not* contain white space. They should be compact, and lower-cases,
47 and use ``_`` (underscore) to separate words in compound labels.
48
49 A result without an ``action`` label will not be processed and is discarded.
50
51
52 ``path``
53 --------
54
55 A string with an *absolute* path describing the local entity a result is
56 associated with. Paths must be platform-specific (e.g., Windows paths on
57 Windows, and POSIX paths on other operating systems). When a result is about an
58 entity that has no meaningful relation to the local file system (e.g., a URL to
59 be downloaded), to ``path`` value should be determined with respect to the
60 potential impact of the result on any local entity (e.g., a URL downloaded
61 to a local file path, a local dataset modified based on remote information).
62
63 .. _target-result-status:
64
65 ``status``
66 ----------
67
68 This field indicates the nature of a result in terms of four categories, identified
69 by a string label.
70
71 - ``ok``: a standard, to-be-expected result
72 - ``notneeded``: an operation that was requested, but found to be unnecessary
73 in order to achieve a desired goal
74 - ``impossible``: a requested operation cannot be performed, possibly because
75 its preconditions are not met
76 - ``error``: an error occurred while performing an operation
77
78 Based on the ``status`` field, a result is categorized into *success* (``ok``,
79 ``notneeded``) and *failure* (``impossible``, ``error``). Depending on the
80 ``on_failure`` parameterization of a command call, any failure-result emitted
81 by a command can lead to an ``IncompleteResultsError`` being raised on command
82 exit, or a non-zero exit code on the command line. With ``on_failure='stop'``,
83 an operation is halted on the first failure and the command errors out
84 immediately, with ``on_failure='continue'`` an operation will continue despite
85 intermediate failures and the command only errors out at the very end, with
86 ``on_failure='ignore'`` the command will not error even when failures occurred.
87 The latter mode can be used in cases where the initial status-characterization
88 needs to be corrected for the particular context of an operation (e.g., to
89 relabel expected and recoverable errors).
90
91
92 Common optional fields
93 ======================
94
95 The following fields are not required, but can be used to enrich a result
96 record with additional information that improves its interpretability, or
97 triggers particular optional functionality in generic result processing.
98
99
100 ``type``
101 --------
102
103 This field indicates the type of entity a result is associated with. This may
104 or may not be the type of the local entity identified by the ``path`` value.
105 The following values are common, and should be used in matching cases, but
106 arbitrary other values are supported too:
107
108 - ``dataset``: a DataLad dataset
109 - ``file``: a regular file
110 - ``directory``: a directory
111 - ``symlink``: a symbolic link
112 - ``key``: a git-annex key
113 - ``sibling``: a Dataset sibling or Git remote
114
115
116 ``message``
117 -----------
118
119 A message providing additional human-readable information on the nature or
120 provenance of a result. Any non-``ok`` results *should* have a message providing
121 information on the rational of their status characterization.
122
123 A message can be a string or a tuple. In case of a tuple, the second item can
124 contain values for ``%``-expansion of the message string. Expansion is performed
125 only immediately prior to actually outputting the message, hence string formatting
126 runtime costs can be avoided this way, if a message is not actually shown.
127
128
129 ``logger``
130 ----------
131
132 If a result record has a ``message`` field, then a given `Logger` instance
133 (typically from ``logging.getLogger()``) will be used to automatically log
134 this message. The log channel/level is determined based on
135 ``datalad.log.result-level`` configuration setting. By default, this is
136 the ``debug`` level. When set to ``match-status`` the log level is determined
137 based on the ``status`` field of a result record:
138
139 - ``debug`` for ``'ok'``, and ``'notneeded'`` results
140 - ``warning`` for ``'impossible'`` results
141 - ``error`` for ``'error'`` results
142
143 This feature should be used with care. Unconditional logging can lead to
144 confusing double-reporting when results rendered and also visibly logged.
145
146
147 ``refds``
148 ---------
149
150 This field can identify a path (using the same semantics and requirements as
151 the ``path`` field) to a reference dataset that represents the larger context
152 of an operation. For example, when recursively processing multiple files across
153 a number of subdatasets, a ``refds`` value may point to the common superdataset.
154 This value may influence, for example, how paths are rendered in user-output.
155
156
157 ``parentds``
158 ------------
159
160 This field can identify a path (using the same semantics and requirements as
161 the ``path`` field) to a dataset containing an entity.
162
163
164 ``state``
165 ---------
166
167 A string label categorizing the state of an entity. Common values are:
168
169 - ``clean``
170 - ``untracked``
171 - ``modified``
172 - ``deleted``
173 - ``absent``
174 - ``present``
175
176
177 ``error_message``
178 -----------------
179
180 An error message that was captured or produced while achieving a result.
181
182 An error message can be a string or a tuple. In the case of a tuple, the
183 second item can contain values for ``%``-expansion of the message string.
184
185
186 ``exception``
187 -------------
188
189 An exception that occurred while achieving the reported result.
190
191
192 ``exception_traceback``
193 -----------------------
194
195 A string with a traceback for the exception reported in ``exception``.
196
197
198 Additional fields observed "in the wild"
199 ========================================
200
201 Given that arbitrary fields are supported in result records, it is impossible
202 to compose a comprehensive list of field names (keys). However, in order to
203 counteract needless proliferation, the following list describes fields that
204 have been observed in implementations. Developers are encouraged to preferably
205 use compatible names from this list, or extend the list for additional items.
206
207 In alphabetical order:
208
209 ``bytesize``
210 The size of an entity in bytes (integer).
211
212 ``gitshasum``
213 SHA1 of an entity (string)
214
215 ``prev_gitshasum``
216 SHA1 of a previous state of an entity (string)
217
218 ``key``
219 The git-annex key associated with a ``type``-``file`` entity.