| 1 |
.. _invoking-tasks: |
| 2 |
|
| 3 |
============== |
| 4 |
Invoking tasks |
| 5 |
============== |
| 6 |
|
| 7 |
This page explains how to invoke your tasks on the CLI, both in terms of parser |
| 8 |
mechanics (how your tasks' arguments are exposed as command-line options) and |
| 9 |
execution strategies (which tasks actually get run, and in what order). |
| 10 |
|
| 11 |
(For details on Invoke's core flags and options, see :doc:`/invoke`.) |
| 12 |
|
| 13 |
.. contents:: |
| 14 |
:local: |
| 15 |
|
| 16 |
|
| 17 |
.. _basic-cli-layout: |
| 18 |
|
| 19 |
Basic command line layout |
| 20 |
========================= |
| 21 |
|
| 22 |
Invoke may be executed as ``invoke`` (or ``inv`` for short) and its command |
| 23 |
line layout looks like this:: |
| 24 |
|
| 25 |
$ inv [--core-opts] task1 [--task1-opts] ... taskN [--taskN-opts] |
| 26 |
|
| 27 |
Put plainly, Invoke's `CLI parser <.Parser>` splits your command line up into |
| 28 |
multiple "`parser contexts <.ParserContext>`" which allows it to reason about |
| 29 |
the args and options it will accept: |
| 30 |
|
| 31 |
- Before any task names are given, the parser is in the "core" parse context, |
| 32 |
and looks for core options and flags such as :option:`--echo`, |
| 33 |
:option:`--list` or :option:`--help`. |
| 34 |
- Any non-argument-like token (such as ``mytask``) causes a switch into a |
| 35 |
per-task context (or an error, if no task matching that name seems to exist |
| 36 |
in the :doc:`loaded collection </concepts/loading>`). |
| 37 |
- At this point, argument-like tokens are expected to correspond to the |
| 38 |
arguments for the previously named task (see :ref:`task-arguments`). |
| 39 |
- Then this cycle repeats infinitely, allowing chained execution of arbitrary |
| 40 |
numbers of tasks. (In practice, most users only execute one or two at a |
| 41 |
time.) |
| 42 |
|
| 43 |
For the core arguments and flags, see :doc:`/invoke`; for details on how your |
| 44 |
tasks affect the CLI, read onwards. |
| 45 |
|
| 46 |
.. note:: |
| 47 |
There is a minor convenience-minded exception to how parse contexts behave: |
| 48 |
core options *may* also be given inside per-task contexts, *if and only if* |
| 49 |
there is no conflict with similarly-named/prefixed arguments of the |
| 50 |
being-parsed task. |
| 51 |
|
| 52 |
For example, ``invoke mytask --echo`` will behave identically to ``invoke |
| 53 |
--echo mytask``, *unless* ``mytask`` has its own ``echo`` flag (in which |
| 54 |
case that flag is handed to the task context, as normal). |
| 55 |
|
| 56 |
Similarly, ``invoke mytask -e`` will turn on command echoing too, unless |
| 57 |
``mytask`` has its own argument whose shortflag ends up set to ``-e`` (e.g. |
| 58 |
``def mytask(env)``). |
| 59 |
|
| 60 |
|
| 61 |
.. _task-arguments: |
| 62 |
|
| 63 |
Task command-line arguments |
| 64 |
=========================== |
| 65 |
|
| 66 |
The simplest task invocation, for a task requiring no parameterization:: |
| 67 |
|
| 68 |
$ inv mytask |
| 69 |
|
| 70 |
Tasks may take parameters in the form of flag arguments:: |
| 71 |
|
| 72 |
$ inv build --format=html |
| 73 |
$ inv build --format html |
| 74 |
$ inv build -f pdf |
| 75 |
$ inv build -f=pdf |
| 76 |
|
| 77 |
Note that both long and short style flags are supported, and that equals signs |
| 78 |
are optional in both cases. |
| 79 |
|
| 80 |
Boolean options are simple flags with no arguments:: |
| 81 |
|
| 82 |
$ inv build --progress-bar |
| 83 |
|
| 84 |
Naturally, more than one flag may be given at a time:: |
| 85 |
|
| 86 |
$ inv build --progress-bar -f pdf |
| 87 |
|
| 88 |
Type casting |
| 89 |
------------ |
| 90 |
|
| 91 |
Natively, a command-line string is just that -- a string -- requiring some |
| 92 |
leaps of logic to arrive at any non-string values on the Python end. Invoke has |
| 93 |
a number of these tricks already at hand, and more will be implemented in the |
| 94 |
future. Currently: |
| 95 |
|
| 96 |
- Arguments with default values use those default values as a type hint, so |
| 97 |
``def mytask(c, count=1)`` will see ``inv mytask --count=5`` and result in |
| 98 |
the Python integer value ``5`` instead of the string ``"5"``. |
| 99 |
|
| 100 |
- Default values of ``None`` are effectively the same as having no default |
| 101 |
value at all - no type casting occurs and you're left with a string. |
| 102 |
|
| 103 |
- The primary exception to the previous rule is booleans: default values of |
| 104 |
``True`` or ``False`` cause those arguments to show up as actual |
| 105 |
non-value-taking flags (``--argname`` to set the value to ``True`` if the |
| 106 |
default was ``False``, or ``--no-argment`` in the opposite case). See |
| 107 |
:ref:`boolean-flags` for more examples. |
| 108 |
- List values (which you wouldn't want to set as an argument's default value |
| 109 |
anyways -- it's a common Python misstep) are served by a special ``@task`` |
| 110 |
flag - see :ref:`iterable-flag-values` below. |
| 111 |
- There's currently no way to set other compound values (such as dicts) on |
| 112 |
the command-line; solving this more complex problem is left as an exercise to |
| 113 |
the reader (though we may add helpers for such things in the future). |
| 114 |
|
| 115 |
Per-task help / printing available flags |
| 116 |
---------------------------------------- |
| 117 |
|
| 118 |
To get help for a specific task, you can give the task name as an argument to |
| 119 |
the core ``--help``/``-h`` option, or give ``--help``/``-h`` after the task |
| 120 |
(which will trigger custom-to-``help`` behavior where the task name itself is |
| 121 |
given to ``--help`` as its argument value). |
| 122 |
|
| 123 |
When help is requested, you'll see the task's docstring (if any) and |
| 124 |
per-argument/flag help output:: |
| 125 |
|
| 126 |
$ inv --help build # or invoke build --help |
| 127 |
|
| 128 |
Docstring: |
| 129 |
none |
| 130 |
|
| 131 |
Options for 'build': |
| 132 |
-f STRING, --format=STRING Which build format type to use |
| 133 |
-p, --progress-bar Display progress bar |
| 134 |
|
| 135 |
Globbed short flags |
| 136 |
------------------- |
| 137 |
|
| 138 |
Boolean short flags may be combined into one flag expression, so that e.g.:: |
| 139 |
|
| 140 |
$ inv build -qv |
| 141 |
|
| 142 |
is equivalent to (and expanded into, during parsing):: |
| 143 |
|
| 144 |
$ inv build -q -v |
| 145 |
|
| 146 |
If the first flag in a globbed short flag token is not a boolean but takes a |
| 147 |
value, the rest of the glob is taken to be the value instead. E.g.:: |
| 148 |
|
| 149 |
$ inv build -fpdf |
| 150 |
|
| 151 |
is expanded into:: |
| 152 |
|
| 153 |
$ inv build -f pdf |
| 154 |
|
| 155 |
and **not**:: |
| 156 |
|
| 157 |
$ inv build -f -p -d -f |
| 158 |
|
| 159 |
.. _optional-values: |
| 160 |
|
| 161 |
Optional flag values |
| 162 |
-------------------- |
| 163 |
|
| 164 |
You saw a hint of this with ``--help`` specifically, but non-core options may |
| 165 |
also take optional values, if declared as ``optional``. For example, say your |
| 166 |
task has a ``--log`` flag that activates logging:: |
| 167 |
|
| 168 |
$ inv compile --log |
| 169 |
|
| 170 |
but you also want it to be configurable regarding *where* to log:: |
| 171 |
|
| 172 |
$ inv compile --log=foo.log |
| 173 |
|
| 174 |
You could implement this with an additional argument (e.g. ``--log`` and |
| 175 |
``--log-location``) but sometimes the concise API is the more useful one. |
| 176 |
|
| 177 |
To enable this, specify which arguments are of this 'hybrid' optional-value |
| 178 |
type inside ``@task``:: |
| 179 |
|
| 180 |
@task(optional=['log']) |
| 181 |
def compile(c, log=None): |
| 182 |
if log: |
| 183 |
log_file = '/var/log/my.log' |
| 184 |
# Value was given, vs just-True |
| 185 |
if isinstance(log, str): |
| 186 |
log_file = log |
| 187 |
# Replace w/ your actual log setup... |
| 188 |
set_log_destination(log_file) |
| 189 |
# Do things that might log here... |
| 190 |
|
| 191 |
When optional flag values are used, the values seen post-parse follow these |
| 192 |
rules: |
| 193 |
|
| 194 |
* If the flag is not given at all (``invoke compile``) the default value |
| 195 |
is filled in as normal. |
| 196 |
* If it is given with a value (``invoke compile --log=foo.log``) then the value |
| 197 |
is stored normally. |
| 198 |
* If the flag is given with no value (``invoke compile --log``), it is treated |
| 199 |
as if it were a ``bool`` and set to ``True``. |
| 200 |
|
| 201 |
Resolving ambiguity |
| 202 |
~~~~~~~~~~~~~~~~~~~ |
| 203 |
|
| 204 |
There are a number of situations where ambiguity could arise for a flag that |
| 205 |
takes an optional value: |
| 206 |
|
| 207 |
* When a task takes positional arguments and they haven't all been filled in by |
| 208 |
the time the parser arrives at the optional-value flag; |
| 209 |
* When the token following one of these flags looks like it is itself a flag; |
| 210 |
or |
| 211 |
* When that token has the same name as another task. |
| 212 |
|
| 213 |
In most of these situations, Invoke's parser will `refuse the temptation to |
| 214 |
guess |
| 215 |
<http://zen-of-python.info/in-the-face-of-ambiguity-refuse-the-temptation-to-guess.html#12>`_ |
| 216 |
and raise an error. |
| 217 |
|
| 218 |
However, in the case where the ambiguous token is flag-like, the current parse |
| 219 |
context is checked to resolve the ambiguity: |
| 220 |
|
| 221 |
- If the token is an otherwise legitimate argument, it is assumed that the user |
| 222 |
meant to give that argument immediately after the current one, and no |
| 223 |
optional value is set. |
| 224 |
|
| 225 |
- E.g. in ``invoke compile --log --verbose`` (assuming ``--verbose`` is |
| 226 |
another legit argument for ``compile``) the parser decides the user meant |
| 227 |
to give ``--log`` without a value, and followed it up with the |
| 228 |
``--verbose`` flag. |
| 229 |
|
| 230 |
- Otherwise, the token is interpreted literally and stored as the value for |
| 231 |
the current flag. |
| 232 |
|
| 233 |
- E.g. if ``--verbose`` is *not* a legitimate argument for ``compile``, |
| 234 |
then ``invoke compile --log --verbose`` causes the parser to assign |
| 235 |
``"--verbose"`` as the value given to ``--log``. (This will probably |
| 236 |
cause other problems in our contrived use case, but it illustrates our |
| 237 |
point.) |
| 238 |
|
| 239 |
.. _iterable-flag-values: |
| 240 |
|
| 241 |
Iterable flag values |
| 242 |
-------------------- |
| 243 |
|
| 244 |
A not-uncommon use case for CLI programs is the desire to build a list of |
| 245 |
values for a given option, instead of a single value. While this *can* be done |
| 246 |
via sub-string parsing -- e.g. having users invoke a command with ``--mylist |
| 247 |
item1,item2,item3`` and splitting on the comma -- it's often preferable to |
| 248 |
specify the option multiple times and store the values in a list (instead of |
| 249 |
overwriting or erroring.) |
| 250 |
|
| 251 |
In Invoke, this is enabled by hinting to the parser that one or more task |
| 252 |
arguments are ``iterable`` in nature (similar to how one specifies ``optional`` |
| 253 |
or ``positional``):: |
| 254 |
|
| 255 |
@task(iterable=['my_list']) |
| 256 |
def mytask(c, my_list): |
| 257 |
print(my_list) |
| 258 |
|
| 259 |
When not given at all, the default value for ``my_list`` will be an empty list; |
| 260 |
otherwise, the result is a list, appending each value seen, in order, without |
| 261 |
any other manipulation (so no deduplication, etc):: |
| 262 |
|
| 263 |
$ inv mytask |
| 264 |
[] |
| 265 |
$ inv mytask --my-list foo |
| 266 |
['foo'] |
| 267 |
$ inv mytask --my-list foo --my-list bar |
| 268 |
['foo', 'bar'] |
| 269 |
$ inv mytask --my-list foo --my-list bar --my-list foo |
| 270 |
['foo', 'bar', 'foo'] |
| 271 |
|
| 272 |
.. _incrementable-flag-values: |
| 273 |
|
| 274 |
Incrementable flag values |
| 275 |
------------------------- |
| 276 |
|
| 277 |
This is arguably a sub-case of :ref:`iterable flag values |
| 278 |
<iterable-flag-values>` (seen above) - it has the same core interface of "give |
| 279 |
a CLI argument multiple times, and have that do something other than error or |
| 280 |
overwrite a single value." However, 'incrementables' (as you may have guessed) |
| 281 |
increment an integer instead of building a list of strings. This is commonly |
| 282 |
found in verbosity flags and similar functionality. |
| 283 |
|
| 284 |
An example of exactly that:: |
| 285 |
|
| 286 |
@task(incrementable=['verbose']) |
| 287 |
def mytask(c, verbose=0): |
| 288 |
print(verbose) |
| 289 |
|
| 290 |
And its use:: |
| 291 |
|
| 292 |
$ inv mytask |
| 293 |
0 |
| 294 |
$ inv mytask --verbose |
| 295 |
1 |
| 296 |
$ inv mytask -v |
| 297 |
1 |
| 298 |
$inv mytask -vvv |
| 299 |
3 |
| 300 |
|
| 301 |
Happily, because in Python ``0`` is 'falsey' and ``1`` (or any other number) is |
| 302 |
'truthy', this functions a lot like a boolean flag as well, at least if one |
| 303 |
defaults it to ``0``. |
| 304 |
|
| 305 |
.. note:: |
| 306 |
You may supply any integer default value for such arguments (it simply |
| 307 |
serves as the starting value), but take care that consumers of the argument |
| 308 |
are written understanding that it is always going to appear 'truthy' unless |
| 309 |
it's ``0``! |
| 310 |
|
| 311 |
Dashes vs underscores in flag names |
| 312 |
----------------------------------- |
| 313 |
|
| 314 |
In Python, it's common to use ``underscored_names`` for keyword arguments, |
| 315 |
e.g.:: |
| 316 |
|
| 317 |
@task |
| 318 |
def mytask(c, my_option=False): |
| 319 |
pass |
| 320 |
|
| 321 |
However, the typical convention for command-line flags is dashes, which aren't |
| 322 |
valid in Python identifiers:: |
| 323 |
|
| 324 |
$ inv mytask --my-option |
| 325 |
|
| 326 |
Invoke works around this by automatically generating dashed versions of |
| 327 |
underscored names, when it turns your task function signatures into |
| 328 |
command-line parser flags. |
| 329 |
|
| 330 |
Therefore, the two examples above actually work fine together -- ``my_option`` |
| 331 |
ends up mapping to ``--my-option``. |
| 332 |
|
| 333 |
In addition, leading (``_myopt``) and trailing (``myopt_``) underscores are |
| 334 |
ignored, since ``invoke ---myopt`` and ``invoke --myopt-`` don't make much |
| 335 |
sense. |
| 336 |
|
| 337 |
.. _boolean-flags: |
| 338 |
|
| 339 |
Automatic Boolean inverse flags |
| 340 |
------------------------------- |
| 341 |
|
| 342 |
Boolean flags tend to work best when setting something that is normally |
| 343 |
``False``, to ``True``:: |
| 344 |
|
| 345 |
$ inv mytask --yes-please-do-x |
| 346 |
|
| 347 |
However, in some cases, you want the opposite - a default of ``True``, which |
| 348 |
can be easily disabled. For example, colored output:: |
| 349 |
|
| 350 |
@task |
| 351 |
def run_tests(c, color=True): |
| 352 |
# ... |
| 353 |
|
| 354 |
Here, what we really want on the command line is a ``--no-color`` flag that |
| 355 |
sets ``color=False``. Invoke handles this for you: when setting up CLI flags, |
| 356 |
booleans which default to ``True`` generate a ``--no-<name>`` flag instead. |
| 357 |
|
| 358 |
|
| 359 |
.. _how-tasks-run: |
| 360 |
|
| 361 |
How tasks run |
| 362 |
============= |
| 363 |
|
| 364 |
Base case |
| 365 |
--------- |
| 366 |
|
| 367 |
In the simplest case, a task with no pre- or post-tasks runs one time. |
| 368 |
Example:: |
| 369 |
|
| 370 |
@task |
| 371 |
def hello(c): |
| 372 |
print("Hello, world!") |
| 373 |
|
| 374 |
Execution:: |
| 375 |
|
| 376 |
$ inv hello |
| 377 |
Hello, world! |
| 378 |
|
| 379 |
.. _pre-post-tasks: |
| 380 |
|
| 381 |
Pre- and post-tasks |
| 382 |
------------------- |
| 383 |
|
| 384 |
Tasks that should always have another task executed before or after them, may |
| 385 |
use the ``@task`` deocator's ``pre`` and/or ``post`` kwargs, like so:: |
| 386 |
|
| 387 |
@task |
| 388 |
def clean(c): |
| 389 |
print("Cleaning") |
| 390 |
|
| 391 |
@task |
| 392 |
def publish(c): |
| 393 |
print("Publishing") |
| 394 |
|
| 395 |
@task(pre=[clean], post=[publish]) |
| 396 |
def build(c): |
| 397 |
print("Building") |
| 398 |
|
| 399 |
Execution:: |
| 400 |
|
| 401 |
$ inv build |
| 402 |
Cleaning |
| 403 |
Building |
| 404 |
Publishing |
| 405 |
|
| 406 |
These keyword arguments always take iterables. As a convenience, pre-tasks (and |
| 407 |
pre-tasks only) may be given as positional arguments, in a manner similar to |
| 408 |
build systems like ``make``. E.g. we could present part of the above example |
| 409 |
as:: |
| 410 |
|
| 411 |
@task |
| 412 |
def clean(c): |
| 413 |
print("Cleaning") |
| 414 |
|
| 415 |
@task(clean) |
| 416 |
def build(c): |
| 417 |
print("Building") |
| 418 |
|
| 419 |
As before, ``invoke build`` would cause ``clean`` to run, then ``build``. |
| 420 |
|
| 421 |
Recursive/chained pre/post-tasks |
| 422 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 423 |
|
| 424 |
Pre-tasks of pre-tasks will also be invoked (as will post-tasks of pre-tasks, |
| 425 |
pre-tasks of post-tasks, etc) in a depth-first manner, recursively. Here's a |
| 426 |
more complex (if slightly contrived) tasks file:: |
| 427 |
|
| 428 |
@task |
| 429 |
def clean_html(c): |
| 430 |
print("Cleaning HTML") |
| 431 |
|
| 432 |
@task |
| 433 |
def clean_tgz(c): |
| 434 |
print("Cleaning .tar.gz files") |
| 435 |
|
| 436 |
@task(clean_html, clean_tgz) |
| 437 |
def clean(c): |
| 438 |
print("Cleaned everything") |
| 439 |
|
| 440 |
@task |
| 441 |
def makedirs(c): |
| 442 |
print("Making directories") |
| 443 |
|
| 444 |
@task(clean, makedirs) |
| 445 |
def build(c): |
| 446 |
print("Building") |
| 447 |
|
| 448 |
@task(build) |
| 449 |
def deploy(c): |
| 450 |
print("Deploying") |
| 451 |
|
| 452 |
With a depth-first behavior, the below is hopefully intuitive to most users:: |
| 453 |
|
| 454 |
$ inv deploy |
| 455 |
Cleaning HTML |
| 456 |
Cleaning .tar.gz files |
| 457 |
Cleaned everything |
| 458 |
Making directories |
| 459 |
Building |
| 460 |
Deploying |
| 461 |
|
| 462 |
|
| 463 |
.. _parameterizing-pre-post-tasks: |
| 464 |
|
| 465 |
Parameterizing pre/post-tasks |
| 466 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 467 |
|
| 468 |
By default, pre- and post-tasks are executed with no arguments, even if the |
| 469 |
task triggering their execution was given some. When this is not suitable, you |
| 470 |
can wrap the task objects with `~.tasks.call` objects which allow you to |
| 471 |
specify a call signature:: |
| 472 |
|
| 473 |
@task |
| 474 |
def clean(c, which=None): |
| 475 |
which = which or 'pyc' |
| 476 |
print("Cleaning {}".format(which)) |
| 477 |
|
| 478 |
@task(pre=[call(clean, which='all')]) # or call(clean, 'all') |
| 479 |
def build(c): |
| 480 |
print("Building") |
| 481 |
|
| 482 |
Example output:: |
| 483 |
|
| 484 |
$ inv build |
| 485 |
Cleaning all |
| 486 |
Building |
| 487 |
|
| 488 |
|
| 489 |
.. _deduping: |
| 490 |
|
| 491 |
Task deduplication |
| 492 |
------------------ |
| 493 |
|
| 494 |
By default, any task that would run more than once during a session (due e.g. |
| 495 |
to inclusion in pre/post tasks), will only be run once. Example task file:: |
| 496 |
|
| 497 |
@task |
| 498 |
def clean(c): |
| 499 |
print("Cleaning") |
| 500 |
|
| 501 |
@task(clean) |
| 502 |
def build(c): |
| 503 |
print("Building") |
| 504 |
|
| 505 |
@task(build) |
| 506 |
def package(c): |
| 507 |
print("Packaging") |
| 508 |
|
| 509 |
With deduplication turned off (see below), the above would execute ``clean`` -> |
| 510 |
``build`` -> ``build`` again -> ``package``. With deduplication, the double |
| 511 |
``build`` does not occur:: |
| 512 |
|
| 513 |
$ inv build package |
| 514 |
Cleaning |
| 515 |
Building |
| 516 |
Packaging |
| 517 |
|
| 518 |
.. note:: |
| 519 |
Parameterized pre-tasks (using `~.tasks.call`) are deduped based on their |
| 520 |
argument lists. For example, if ``clean`` was parameterized and hooked up |
| 521 |
as a pre-task in two different ways - e.g. ``call(clean, 'html')`` and |
| 522 |
``call(clean, 'all')`` - they would not get deduped should both end up |
| 523 |
running in the same session. |
| 524 |
|
| 525 |
However, two separate references to ``call(clean, 'html')`` *would* become |
| 526 |
deduplicated. |
| 527 |
|
| 528 |
Disabling deduplication |
| 529 |
~~~~~~~~~~~~~~~~~~~~~~~ |
| 530 |
|
| 531 |
If you prefer your tasks to run every time no matter what, you can give the |
| 532 |
``--no-dedupe`` core CLI option at runtime, or set the ``tasks.dedupe`` |
| 533 |
:doc:`config setting </concepts/configuration>` to ``False``. While it |
| 534 |
doesn't make a ton of real-world sense, let's imagine we wanted to apply |
| 535 |
``--no-dedupe`` to the above example; we'd see the following output:: |
| 536 |
|
| 537 |
$ inv --no-dedupe build package |
| 538 |
Cleaning |
| 539 |
Building |
| 540 |
Building |
| 541 |
Packaging |
| 542 |
|
| 543 |
The build step is now running twice. |