| 1 |
========================= |
| 2 |
Using Invoke as a library |
| 3 |
========================= |
| 4 |
|
| 5 |
While most of our documentation involves the user/CLI facing use cases of task |
| 6 |
management and command execution, Invoke was designed for its constituent parts |
| 7 |
to be usable independently by advanced users - either out of the box or with a |
| 8 |
minimum of extra work. CLI parsing, subprocess command execution, task |
| 9 |
organization, etc, are all written as broadly separated concerns. |
| 10 |
|
| 11 |
This document outlines use cases already known to work (because downstream |
| 12 |
tools like `Fabric <https://fabfile.org>`_ are already utilizing them). |
| 13 |
|
| 14 |
|
| 15 |
.. _reusing-as-a-binary: |
| 16 |
|
| 17 |
Reusing Invoke's CLI module as a distinct binary |
| 18 |
================================================ |
| 19 |
|
| 20 |
A major use case is distribution of your own program using Invoke under the |
| 21 |
hood, bound to a different binary name, and usually setting a specific task |
| 22 |
:doc:`namespace </concepts/namespaces>` as the default. (This maps somewhat |
| 23 |
closely to things like ``argparse`` from the standard library.) In some cases, |
| 24 |
removing, replacing and/or adding core CLI flags is also desired. |
| 25 |
|
| 26 |
Getting set up |
| 27 |
-------------- |
| 28 |
|
| 29 |
Say you want to distribute a test runner called ``tester`` offering two |
| 30 |
subcommands, ``unit`` and ``integration``, such that users could ``pip install |
| 31 |
tester`` and have access to commands like ``tester unit``, ``tester |
| 32 |
integration``, or ``tester integration --fail-fast``. |
| 33 |
|
| 34 |
First, as with any distinct Python package providing CLI |
| 35 |
'binaries', you'd inform your ``setup.py`` of your entrypoint:: |
| 36 |
|
| 37 |
setup( |
| 38 |
name='tester', |
| 39 |
version='0.1.0', |
| 40 |
packages=['tester'], |
| 41 |
install_requires=['invoke'], |
| 42 |
entry_points={ |
| 43 |
'console_scripts': ['tester = tester.main:program.run'] |
| 44 |
} |
| 45 |
) |
| 46 |
|
| 47 |
.. note:: |
| 48 |
This is just an example snippet and is not a fully valid ``setup.py``; if |
| 49 |
you don't know how Python packaging works, a good starting place is `the |
| 50 |
Python Packaging User's Guide |
| 51 |
<https://python-packaging-user-guide.readthedocs.io>`_. |
| 52 |
|
| 53 |
Nothing here is specific to Invoke - it's a standard way of telling Python to |
| 54 |
install a ``tester`` script that executes the ``run`` method of a ``program`` |
| 55 |
object defined inside the module ``tester.main``. |
| 56 |
|
| 57 |
Creating a ``Program`` |
| 58 |
---------------------- |
| 59 |
|
| 60 |
In our ``tester/main.py``, we start out importing Invoke's public CLI |
| 61 |
functionality:: |
| 62 |
|
| 63 |
from invoke import Program |
| 64 |
|
| 65 |
Then we define the ``program`` object we referenced in ``setup.py``, which is a |
| 66 |
simple `.Program` to do the heavy lifting, giving it our version number for |
| 67 |
starters:: |
| 68 |
|
| 69 |
program = Program(version='0.1.0') |
| 70 |
|
| 71 |
At this point, installing ``tester`` would give you the same functionality as |
| 72 |
Invoke's :doc:`built-in CLI tool </invoke>`, except named ``tester`` and |
| 73 |
exposing its own version number:: |
| 74 |
|
| 75 |
$ tester --version |
| 76 |
Tester 0.1.0 |
| 77 |
$ tester --help |
| 78 |
Usage: tester [--core-opts] task1 [--task1-opts] ... taskN [--taskN-opts] |
| 79 |
|
| 80 |
Core options: |
| 81 |
... core Invoke options here ... |
| 82 |
|
| 83 |
$ tester --list |
| 84 |
Can't find any collection named 'tasks'! |
| 85 |
|
| 86 |
This doesn't do us much good yet - there aren't any subcommands (and our users |
| 87 |
don't care about arbitrary 'tasks', so Invoke's own default ``--help`` and |
| 88 |
``--list`` output isn't a good fit). |
| 89 |
|
| 90 |
Specifying subcommands |
| 91 |
---------------------- |
| 92 |
|
| 93 |
For ``tester`` to expose ``unit`` and ``integration`` subcommands, we need to |
| 94 |
define them, in a regular Invoke tasks module or :doc:`namespace |
| 95 |
</concepts/namespaces>`. For our example, we'll just create ``tester/tasks.py`` |
| 96 |
(but as you'll see in a moment, this too is arbitrary and can be whatever you |
| 97 |
like):: |
| 98 |
|
| 99 |
from invoke import task |
| 100 |
|
| 101 |
@task |
| 102 |
def unit(c): |
| 103 |
print("Running unit tests!") |
| 104 |
|
| 105 |
@task |
| 106 |
def integration(c): |
| 107 |
print("Running integration tests!") |
| 108 |
|
| 109 |
As described in :doc:`/concepts/namespaces`, you can arrange this module |
| 110 |
however you want - the above snippet uses an implicit namespace for brevity's |
| 111 |
sake. |
| 112 |
|
| 113 |
.. note:: |
| 114 |
It's important to realize that there's nothing special about these |
| 115 |
"subcommands" - you could run them just as easily with vanilla Invoke, |
| 116 |
e.g. via ``invoke --collection=tester.tasks --list``. |
| 117 |
|
| 118 |
Now the useful part: telling our custom `.Program` that this namespace of tasks |
| 119 |
should be used as the subcommands for ``tester``, via the ``namespace`` kwarg:: |
| 120 |
|
| 121 |
from invoke import Collection, Program |
| 122 |
from tester import tasks |
| 123 |
|
| 124 |
program = Program(namespace=Collection.from_module(tasks), version='0.1.0') |
| 125 |
|
| 126 |
The result? |
| 127 |
|
| 128 |
:: |
| 129 |
|
| 130 |
$ tester --version |
| 131 |
Tester 0.1.0 |
| 132 |
$ tester --help |
| 133 |
Usage: tester [--core-opts] <subcommand> [--subcommand-opts] ... |
| 134 |
|
| 135 |
Core options: |
| 136 |
... core options here, minus task-related ones ... |
| 137 |
|
| 138 |
Subcommands: |
| 139 |
unit |
| 140 |
integration |
| 141 |
|
| 142 |
$ tester --list |
| 143 |
No idea what '--list' is! |
| 144 |
$ tester unit |
| 145 |
Running unit tests! |
| 146 |
|
| 147 |
Notice how the 'usage' line changed (to specify 'subcommands' instead of |
| 148 |
'tasks'); the list of specific subcommands is now printed as part of |
| 149 |
``--help``; and ``--list`` has been removed from the options. |
| 150 |
|
| 151 |
You can enable :ref:`tab-completion<tab-completion>` for your distinct |
| 152 |
binary and subcommands. |
| 153 |
|
| 154 |
|
| 155 |
Modifying core parser arguments |
| 156 |
------------------------------- |
| 157 |
|
| 158 |
A common need for this use case is tweaking the core parser arguments. |
| 159 |
`.Program` makes it easy: default core `Arguments <.Argument>` are returned by |
| 160 |
`.Program.core_args`. Extend this method's return value with ``super`` and |
| 161 |
you're done:: |
| 162 |
|
| 163 |
# Presumably, this is your setup.py-designated CLI module... |
| 164 |
|
| 165 |
from invoke import Program, Argument |
| 166 |
|
| 167 |
class MyProgram(Program): |
| 168 |
def core_args(self): |
| 169 |
core_args = super().core_args() |
| 170 |
extra_args = [ |
| 171 |
Argument(names=('foo', 'f'), help="Foo the bars"), |
| 172 |
# ... |
| 173 |
] |
| 174 |
return core_args + extra_args |
| 175 |
|
| 176 |
program = MyProgram() |
| 177 |
|
| 178 |
.. warning:: |
| 179 |
We don't recommend *omitting* any of the existing core arguments; a lot of |
| 180 |
basic functionality relies on their existence, even when left to default |
| 181 |
values. |
| 182 |
|
| 183 |
|
| 184 |
.. _customizing-config-defaults: |
| 185 |
|
| 186 |
Customizing the configuration system's defaults |
| 187 |
=============================================== |
| 188 |
|
| 189 |
Besides the CLI-oriented content of the previous section, another area of |
| 190 |
functionality that frequently needs updating when redistributing an Invoke |
| 191 |
codebase (CLI or no CLI) is configuration. There are typically two concerns |
| 192 |
here: |
| 193 |
|
| 194 |
- Configuration filenames and the env var prefix - crucial if you ever expect |
| 195 |
your users to use the configuration system; |
| 196 |
- Default configuration values - less critical (most defaults aren't labeled |
| 197 |
with anything Invoke-specific) but still sometimes desirable. |
| 198 |
|
| 199 |
.. note:: |
| 200 |
Both of these involve subclassing `.Config` (and, if using the CLI |
| 201 |
machinery, informing your `.Program` to use that subclass instead of the |
| 202 |
default one.) |
| 203 |
|
| 204 |
|
| 205 |
Changing filenames and/or env var prefix |
| 206 |
---------------------------------------- |
| 207 |
|
| 208 |
By default, Invoke's config system looks for files like ``/etc/invoke.yaml``, |
| 209 |
``~/.invoke.json``, etc. If you're distributing client code named something |
| 210 |
else, like the ``Tester`` example earlier, you might instead want the config |
| 211 |
system to load ``/etc/tester.json`` or ``$CWD/tester.py``. |
| 212 |
|
| 213 |
Similarly, the environment variable config level looks for env vars like |
| 214 |
``INVOKE_RUN_ECHO``; you might prefer ``TESTER_RUN_ECHO``. |
| 215 |
|
| 216 |
There are a few `.Config` attributes controlling these values: |
| 217 |
|
| 218 |
- ``prefix``: A generic, catchall prefix used directly as the file prefix, and |
| 219 |
used via all-caps as the env var prefix; |
| 220 |
- ``file_prefix``: For overriding just the filename prefix - otherwise, it |
| 221 |
defaults to the value of ``prefix``; |
| 222 |
- ``env_prefix``: For overriding just the env var prefix - as you might have |
| 223 |
guessed, it too defaults to the value of ``prefix``. |
| 224 |
|
| 225 |
Continuing our 'Tester' example, you'd do something like this:: |
| 226 |
|
| 227 |
from invoke import Config |
| 228 |
|
| 229 |
class TesterConfig(Config): |
| 230 |
prefix = 'tester' |
| 231 |
|
| 232 |
Or, to seek ``tester.yaml`` as before, but ``TEST_RUN_ECHO`` instead of |
| 233 |
``TESTER_RUN_ECHO``:: |
| 234 |
|
| 235 |
class TesterConfig(Config): |
| 236 |
prefix = 'tester' |
| 237 |
env_prefix = 'TEST' |
| 238 |
|
| 239 |
Modifying default config values |
| 240 |
------------------------------- |
| 241 |
|
| 242 |
Default config values are simple - they're just the return value of the |
| 243 |
staticmethod `.Config.global_defaults`, so override that and return whatever |
| 244 |
you like - ideally something based on the superclass' values, as many defaults |
| 245 |
are assumed to exist by the rest of the system. (The helper function |
| 246 |
`invoke.config.merge_dicts` can be useful here.) |
| 247 |
|
| 248 |
For example, say you want Tester to always echo shell commands by default when |
| 249 |
your codebase calls `.Context.run`:: |
| 250 |
|
| 251 |
from invoke import Program |
| 252 |
from invoke.config import Config, merge_dicts |
| 253 |
|
| 254 |
class TesterConfig(Config): |
| 255 |
@staticmethod |
| 256 |
def global_defaults(): |
| 257 |
their_defaults = Config.global_defaults() |
| 258 |
my_defaults = { |
| 259 |
'run': { |
| 260 |
'echo': True, |
| 261 |
}, |
| 262 |
} |
| 263 |
return merge_dicts(their_defaults, my_defaults) |
| 264 |
|
| 265 |
program = Program(config_class=TesterConfig, version='0.1.0') |
| 266 |
|
| 267 |
For reference, Invoke's own base defaults (the...default defaults, you could |
| 268 |
say) are documented at :ref:`default-values`. |