Debian Codegraph @

sites/docs/concepts/invoking-tasks.rst

raw
1 .. _invoking-tasks:
2
3 ==============
4 Invoking tasks
5 ==============
6
7 This page explains how to invoke your tasks on the CLI, both in terms of parser
8 mechanics (how your tasks' arguments are exposed as command-line options) and
9 execution strategies (which tasks actually get run, and in what order).
10
11 (For details on Invoke's core flags and options, see :doc:`/invoke`.)
12
13 .. contents::
14 :local:
15
16
17 .. _basic-cli-layout:
18
19 Basic command line layout
20 =========================
21
22 Invoke may be executed as ``invoke`` (or ``inv`` for short) and its command
23 line layout looks like this::
24
25 $ inv [--core-opts] task1 [--task1-opts] ... taskN [--taskN-opts]
26
27 Put plainly, Invoke's `CLI parser <.Parser>` splits your command line up into
28 multiple "`parser contexts <.ParserContext>`" which allows it to reason about
29 the args and options it will accept:
30
31 - Before any task names are given, the parser is in the "core" parse context,
32 and looks for core options and flags such as :option:`--echo`,
33 :option:`--list` or :option:`--help`.
34 - Any non-argument-like token (such as ``mytask``) causes a switch into a
35 per-task context (or an error, if no task matching that name seems to exist
36 in the :doc:`loaded collection </concepts/loading>`).
37 - At this point, argument-like tokens are expected to correspond to the
38 arguments for the previously named task (see :ref:`task-arguments`).
39 - Then this cycle repeats infinitely, allowing chained execution of arbitrary
40 numbers of tasks. (In practice, most users only execute one or two at a
41 time.)
42
43 For the core arguments and flags, see :doc:`/invoke`; for details on how your
44 tasks affect the CLI, read onwards.
45
46 .. note::
47 There is a minor convenience-minded exception to how parse contexts behave:
48 core options *may* also be given inside per-task contexts, *if and only if*
49 there is no conflict with similarly-named/prefixed arguments of the
50 being-parsed task.
51
52 For example, ``invoke mytask --echo`` will behave identically to ``invoke
53 --echo mytask``, *unless* ``mytask`` has its own ``echo`` flag (in which
54 case that flag is handed to the task context, as normal).
55
56 Similarly, ``invoke mytask -e`` will turn on command echoing too, unless
57 ``mytask`` has its own argument whose shortflag ends up set to ``-e`` (e.g.
58 ``def mytask(env)``).
59
60
61 .. _task-arguments:
62
63 Task command-line arguments
64 ===========================
65
66 The simplest task invocation, for a task requiring no parameterization::
67
68 $ inv mytask
69
70 Tasks may take parameters in the form of flag arguments::
71
72 $ inv build --format=html
73 $ inv build --format html
74 $ inv build -f pdf
75 $ inv build -f=pdf
76
77 Note that both long and short style flags are supported, and that equals signs
78 are optional in both cases.
79
80 Boolean options are simple flags with no arguments::
81
82 $ inv build --progress-bar
83
84 Naturally, more than one flag may be given at a time::
85
86 $ inv build --progress-bar -f pdf
87
88 Type casting
89 ------------
90
91 Natively, a command-line string is just that -- a string -- requiring some
92 leaps of logic to arrive at any non-string values on the Python end. Invoke has
93 a number of these tricks already at hand, and more will be implemented in the
94 future. Currently:
95
96 - Arguments with default values use those default values as a type hint, so
97 ``def mytask(c, count=1)`` will see ``inv mytask --count=5`` and result in
98 the Python integer value ``5`` instead of the string ``"5"``.
99
100 - Default values of ``None`` are effectively the same as having no default
101 value at all - no type casting occurs and you're left with a string.
102
103 - The primary exception to the previous rule is booleans: default values of
104 ``True`` or ``False`` cause those arguments to show up as actual
105 non-value-taking flags (``--argname`` to set the value to ``True`` if the
106 default was ``False``, or ``--no-argment`` in the opposite case). See
107 :ref:`boolean-flags` for more examples.
108 - List values (which you wouldn't want to set as an argument's default value
109 anyways -- it's a common Python misstep) are served by a special ``@task``
110 flag - see :ref:`iterable-flag-values` below.
111 - There's currently no way to set other compound values (such as dicts) on
112 the command-line; solving this more complex problem is left as an exercise to
113 the reader (though we may add helpers for such things in the future).
114
115 Per-task help / printing available flags
116 ----------------------------------------
117
118 To get help for a specific task, you can give the task name as an argument to
119 the core ``--help``/``-h`` option, or give ``--help``/``-h`` after the task
120 (which will trigger custom-to-``help`` behavior where the task name itself is
121 given to ``--help`` as its argument value).
122
123 When help is requested, you'll see the task's docstring (if any) and
124 per-argument/flag help output::
125
126 $ inv --help build # or invoke build --help
127
128 Docstring:
129 none
130
131 Options for 'build':
132 -f STRING, --format=STRING Which build format type to use
133 -p, --progress-bar Display progress bar
134
135 Globbed short flags
136 -------------------
137
138 Boolean short flags may be combined into one flag expression, so that e.g.::
139
140 $ inv build -qv
141
142 is equivalent to (and expanded into, during parsing)::
143
144 $ inv build -q -v
145
146 If the first flag in a globbed short flag token is not a boolean but takes a
147 value, the rest of the glob is taken to be the value instead. E.g.::
148
149 $ inv build -fpdf
150
151 is expanded into::
152
153 $ inv build -f pdf
154
155 and **not**::
156
157 $ inv build -f -p -d -f
158
159 .. _optional-values:
160
161 Optional flag values
162 --------------------
163
164 You saw a hint of this with ``--help`` specifically, but non-core options may
165 also take optional values, if declared as ``optional``. For example, say your
166 task has a ``--log`` flag that activates logging::
167
168 $ inv compile --log
169
170 but you also want it to be configurable regarding *where* to log::
171
172 $ inv compile --log=foo.log
173
174 You could implement this with an additional argument (e.g. ``--log`` and
175 ``--log-location``) but sometimes the concise API is the more useful one.
176
177 To enable this, specify which arguments are of this 'hybrid' optional-value
178 type inside ``@task``::
179
180 @task(optional=['log'])
181 def compile(c, log=None):
182 if log:
183 log_file = '/var/log/my.log'
184 # Value was given, vs just-True
185 if isinstance(log, str):
186 log_file = log
187 # Replace w/ your actual log setup...
188 set_log_destination(log_file)
189 # Do things that might log here...
190
191 When optional flag values are used, the values seen post-parse follow these
192 rules:
193
194 * If the flag is not given at all (``invoke compile``) the default value
195 is filled in as normal.
196 * If it is given with a value (``invoke compile --log=foo.log``) then the value
197 is stored normally.
198 * If the flag is given with no value (``invoke compile --log``), it is treated
199 as if it were a ``bool`` and set to ``True``.
200
201 Resolving ambiguity
202 ~~~~~~~~~~~~~~~~~~~
203
204 There are a number of situations where ambiguity could arise for a flag that
205 takes an optional value:
206
207 * When a task takes positional arguments and they haven't all been filled in by
208 the time the parser arrives at the optional-value flag;
209 * When the token following one of these flags looks like it is itself a flag;
210 or
211 * When that token has the same name as another task.
212
213 In most of these situations, Invoke's parser will `refuse the temptation to
214 guess
215 <http://zen-of-python.info/in-the-face-of-ambiguity-refuse-the-temptation-to-guess.html#12>`_
216 and raise an error.
217
218 However, in the case where the ambiguous token is flag-like, the current parse
219 context is checked to resolve the ambiguity:
220
221 - If the token is an otherwise legitimate argument, it is assumed that the user
222 meant to give that argument immediately after the current one, and no
223 optional value is set.
224
225 - E.g. in ``invoke compile --log --verbose`` (assuming ``--verbose`` is
226 another legit argument for ``compile``) the parser decides the user meant
227 to give ``--log`` without a value, and followed it up with the
228 ``--verbose`` flag.
229
230 - Otherwise, the token is interpreted literally and stored as the value for
231 the current flag.
232
233 - E.g. if ``--verbose`` is *not* a legitimate argument for ``compile``,
234 then ``invoke compile --log --verbose`` causes the parser to assign
235 ``"--verbose"`` as the value given to ``--log``. (This will probably
236 cause other problems in our contrived use case, but it illustrates our
237 point.)
238
239 .. _iterable-flag-values:
240
241 Iterable flag values
242 --------------------
243
244 A not-uncommon use case for CLI programs is the desire to build a list of
245 values for a given option, instead of a single value. While this *can* be done
246 via sub-string parsing -- e.g. having users invoke a command with ``--mylist
247 item1,item2,item3`` and splitting on the comma -- it's often preferable to
248 specify the option multiple times and store the values in a list (instead of
249 overwriting or erroring.)
250
251 In Invoke, this is enabled by hinting to the parser that one or more task
252 arguments are ``iterable`` in nature (similar to how one specifies ``optional``
253 or ``positional``)::
254
255 @task(iterable=['my_list'])
256 def mytask(c, my_list):
257 print(my_list)
258
259 When not given at all, the default value for ``my_list`` will be an empty list;
260 otherwise, the result is a list, appending each value seen, in order, without
261 any other manipulation (so no deduplication, etc)::
262
263 $ inv mytask
264 []
265 $ inv mytask --my-list foo
266 ['foo']
267 $ inv mytask --my-list foo --my-list bar
268 ['foo', 'bar']
269 $ inv mytask --my-list foo --my-list bar --my-list foo
270 ['foo', 'bar', 'foo']
271
272 .. _incrementable-flag-values:
273
274 Incrementable flag values
275 -------------------------
276
277 This is arguably a sub-case of :ref:`iterable flag values
278 <iterable-flag-values>` (seen above) - it has the same core interface of "give
279 a CLI argument multiple times, and have that do something other than error or
280 overwrite a single value." However, 'incrementables' (as you may have guessed)
281 increment an integer instead of building a list of strings. This is commonly
282 found in verbosity flags and similar functionality.
283
284 An example of exactly that::
285
286 @task(incrementable=['verbose'])
287 def mytask(c, verbose=0):
288 print(verbose)
289
290 And its use::
291
292 $ inv mytask
293 0
294 $ inv mytask --verbose
295 1
296 $ inv mytask -v
297 1
298 $inv mytask -vvv
299 3
300
301 Happily, because in Python ``0`` is 'falsey' and ``1`` (or any other number) is
302 'truthy', this functions a lot like a boolean flag as well, at least if one
303 defaults it to ``0``.
304
305 .. note::
306 You may supply any integer default value for such arguments (it simply
307 serves as the starting value), but take care that consumers of the argument
308 are written understanding that it is always going to appear 'truthy' unless
309 it's ``0``!
310
311 Dashes vs underscores in flag names
312 -----------------------------------
313
314 In Python, it's common to use ``underscored_names`` for keyword arguments,
315 e.g.::
316
317 @task
318 def mytask(c, my_option=False):
319 pass
320
321 However, the typical convention for command-line flags is dashes, which aren't
322 valid in Python identifiers::
323
324 $ inv mytask --my-option
325
326 Invoke works around this by automatically generating dashed versions of
327 underscored names, when it turns your task function signatures into
328 command-line parser flags.
329
330 Therefore, the two examples above actually work fine together -- ``my_option``
331 ends up mapping to ``--my-option``.
332
333 In addition, leading (``_myopt``) and trailing (``myopt_``) underscores are
334 ignored, since ``invoke ---myopt`` and ``invoke --myopt-`` don't make much
335 sense.
336
337 .. _boolean-flags:
338
339 Automatic Boolean inverse flags
340 -------------------------------
341
342 Boolean flags tend to work best when setting something that is normally
343 ``False``, to ``True``::
344
345 $ inv mytask --yes-please-do-x
346
347 However, in some cases, you want the opposite - a default of ``True``, which
348 can be easily disabled. For example, colored output::
349
350 @task
351 def run_tests(c, color=True):
352 # ...
353
354 Here, what we really want on the command line is a ``--no-color`` flag that
355 sets ``color=False``. Invoke handles this for you: when setting up CLI flags,
356 booleans which default to ``True`` generate a ``--no-<name>`` flag instead.
357
358
359 .. _how-tasks-run:
360
361 How tasks run
362 =============
363
364 Base case
365 ---------
366
367 In the simplest case, a task with no pre- or post-tasks runs one time.
368 Example::
369
370 @task
371 def hello(c):
372 print("Hello, world!")
373
374 Execution::
375
376 $ inv hello
377 Hello, world!
378
379 .. _pre-post-tasks:
380
381 Pre- and post-tasks
382 -------------------
383
384 Tasks that should always have another task executed before or after them, may
385 use the ``@task`` deocator's ``pre`` and/or ``post`` kwargs, like so::
386
387 @task
388 def clean(c):
389 print("Cleaning")
390
391 @task
392 def publish(c):
393 print("Publishing")
394
395 @task(pre=[clean], post=[publish])
396 def build(c):
397 print("Building")
398
399 Execution::
400
401 $ inv build
402 Cleaning
403 Building
404 Publishing
405
406 These keyword arguments always take iterables. As a convenience, pre-tasks (and
407 pre-tasks only) may be given as positional arguments, in a manner similar to
408 build systems like ``make``. E.g. we could present part of the above example
409 as::
410
411 @task
412 def clean(c):
413 print("Cleaning")
414
415 @task(clean)
416 def build(c):
417 print("Building")
418
419 As before, ``invoke build`` would cause ``clean`` to run, then ``build``.
420
421 Recursive/chained pre/post-tasks
422 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
423
424 Pre-tasks of pre-tasks will also be invoked (as will post-tasks of pre-tasks,
425 pre-tasks of post-tasks, etc) in a depth-first manner, recursively. Here's a
426 more complex (if slightly contrived) tasks file::
427
428 @task
429 def clean_html(c):
430 print("Cleaning HTML")
431
432 @task
433 def clean_tgz(c):
434 print("Cleaning .tar.gz files")
435
436 @task(clean_html, clean_tgz)
437 def clean(c):
438 print("Cleaned everything")
439
440 @task
441 def makedirs(c):
442 print("Making directories")
443
444 @task(clean, makedirs)
445 def build(c):
446 print("Building")
447
448 @task(build)
449 def deploy(c):
450 print("Deploying")
451
452 With a depth-first behavior, the below is hopefully intuitive to most users::
453
454 $ inv deploy
455 Cleaning HTML
456 Cleaning .tar.gz files
457 Cleaned everything
458 Making directories
459 Building
460 Deploying
461
462
463 .. _parameterizing-pre-post-tasks:
464
465 Parameterizing pre/post-tasks
466 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
467
468 By default, pre- and post-tasks are executed with no arguments, even if the
469 task triggering their execution was given some. When this is not suitable, you
470 can wrap the task objects with `~.tasks.call` objects which allow you to
471 specify a call signature::
472
473 @task
474 def clean(c, which=None):
475 which = which or 'pyc'
476 print("Cleaning {}".format(which))
477
478 @task(pre=[call(clean, which='all')]) # or call(clean, 'all')
479 def build(c):
480 print("Building")
481
482 Example output::
483
484 $ inv build
485 Cleaning all
486 Building
487
488
489 .. _deduping:
490
491 Task deduplication
492 ------------------
493
494 By default, any task that would run more than once during a session (due e.g.
495 to inclusion in pre/post tasks), will only be run once. Example task file::
496
497 @task
498 def clean(c):
499 print("Cleaning")
500
501 @task(clean)
502 def build(c):
503 print("Building")
504
505 @task(build)
506 def package(c):
507 print("Packaging")
508
509 With deduplication turned off (see below), the above would execute ``clean`` ->
510 ``build`` -> ``build`` again -> ``package``. With deduplication, the double
511 ``build`` does not occur::
512
513 $ inv build package
514 Cleaning
515 Building
516 Packaging
517
518 .. note::
519 Parameterized pre-tasks (using `~.tasks.call`) are deduped based on their
520 argument lists. For example, if ``clean`` was parameterized and hooked up
521 as a pre-task in two different ways - e.g. ``call(clean, 'html')`` and
522 ``call(clean, 'all')`` - they would not get deduped should both end up
523 running in the same session.
524
525 However, two separate references to ``call(clean, 'html')`` *would* become
526 deduplicated.
527
528 Disabling deduplication
529 ~~~~~~~~~~~~~~~~~~~~~~~
530
531 If you prefer your tasks to run every time no matter what, you can give the
532 ``--no-dedupe`` core CLI option at runtime, or set the ``tasks.dedupe``
533 :doc:`config setting </concepts/configuration>` to ``False``. While it
534 doesn't make a ton of real-world sense, let's imagine we wanted to apply
535 ``--no-dedupe`` to the above example; we'd see the following output::
536
537 $ inv --no-dedupe build package
538 Cleaning
539 Building
540 Building
541 Packaging
542
543 The build step is now running twice.