| 1 |
========================== |
| 2 |
Frequently asked questions |
| 3 |
========================== |
| 4 |
|
| 5 |
|
| 6 |
General project questions |
| 7 |
========================= |
| 8 |
|
| 9 |
.. _invoke-split-from-fabric: |
| 10 |
|
| 11 |
Why was Invoke split off from the `Fabric <https://fabfile.org>`_ project? |
| 12 |
-------------------------------------------------------------------------- |
| 13 |
|
| 14 |
Fabric (1.x and earlier) was a hybrid project implementing two feature sets: |
| 15 |
task execution (organization of task functions, execution of them via CLI, and |
| 16 |
local shell commands) and high level SSH actions (organization of |
| 17 |
servers/hosts, remote shell commands, and file transfer). |
| 18 |
|
| 19 |
For use cases requiring both feature sets, this arrangement worked well. |
| 20 |
However, over time it became clear many users only needed one or the other, |
| 21 |
with local-only users resenting heavy SSH/crypto install requirements, and |
| 22 |
remote-focused users struggling with API limitations caused by the hybrid |
| 23 |
codebase. |
| 24 |
|
| 25 |
When planning Fabric 2.x, having the "local" feature set as a standalone |
| 26 |
library made sense, and it seemed plausible to design the SSH component as a |
| 27 |
separate layer above. Thus, Invoke was created to focus exclusively on local |
| 28 |
and abstract concerns, leaving Fabric 2.x concerned only with servers and |
| 29 |
network commands. |
| 30 |
|
| 31 |
Fabric 2 leverages many parts of Invoke's API, and allows (but does not |
| 32 |
require!) use of Invoke's CLI features, allowing multiple use cases (build |
| 33 |
tool, high level SSH lib, hybrid build/orchestration tool) to coexist without |
| 34 |
negatively impacting each other. |
| 35 |
|
| 36 |
|
| 37 |
Defining/executing tasks |
| 38 |
======================== |
| 39 |
|
| 40 |
.. _bad-first-arg: |
| 41 |
|
| 42 |
My task's first argument isn't showing up in ``--help``! |
| 43 |
-------------------------------------------------------- |
| 44 |
|
| 45 |
This problem pops up if you forget to define an initial context argument for |
| 46 |
your task. |
| 47 |
|
| 48 |
For example, can you spot the problem in this sample task file? |
| 49 |
|
| 50 |
:: |
| 51 |
|
| 52 |
from invoke import task |
| 53 |
|
| 54 |
@task |
| 55 |
def build(c, where, clean=False): |
| 56 |
pass |
| 57 |
|
| 58 |
@task |
| 59 |
def clean(what): |
| 60 |
pass |
| 61 |
|
| 62 |
This task file doesn't cause obvious errors when sanity-checking it with |
| 63 |
``inv --list`` or ``inv --help``. However, ``clean`` forgot to set aside its |
| 64 |
first argument for the context - so Invoke is treating ``what`` as the context |
| 65 |
argument! This means it doesn't show up in help output or other command-line |
| 66 |
parsing stages. |
| 67 |
|
| 68 |
The command line says my task's first argument is invalid! |
| 69 |
---------------------------------------------------------- |
| 70 |
|
| 71 |
See :ref:`bad-first-arg` - it's probably the same issue. |
| 72 |
|
| 73 |
|
| 74 |
Running local shell commands (``run``) |
| 75 |
====================================== |
| 76 |
|
| 77 |
.. _program-behavior-ptys: |
| 78 |
|
| 79 |
Why is my command behaving differently under Invoke versus being run by hand? |
| 80 |
----------------------------------------------------------------------------- |
| 81 |
|
| 82 |
99% of the time, adding ``pty=True`` to your ``run`` call will make things work |
| 83 |
as you were expecting. Read on for why this is (and why ``pty=True`` is not the |
| 84 |
default). |
| 85 |
|
| 86 |
Command-line programs often change behavior depending on whether a controlling |
| 87 |
terminal is present; a common example is the use or disuse of colored output. |
| 88 |
When the recipient of your output is a human at a terminal, you may want to use |
| 89 |
color, tailor line length to match terminal width, etc. |
| 90 |
|
| 91 |
Conversely, when your output is being sent to another program (shell pipe, CI |
| 92 |
server, file, etc) color escape codes and other terminal-specific behaviors can |
| 93 |
result in unwanted garbage. |
| 94 |
|
| 95 |
Invoke's use cases span both of the above - sometimes you only want data |
| 96 |
displayed directly, sometimes you only want to capture it as a string; often |
| 97 |
you want both. Because of this, there is no "correct" default behavior re: use |
| 98 |
of a pseudo-terminal - some large chunk of use cases will be inconvenienced |
| 99 |
either way. |
| 100 |
|
| 101 |
For use cases which don't care, direct invocation without a pseudo-terminal is |
| 102 |
faster & cleaner, so it is the default. |
| 103 |
|
| 104 |
Calling Python or Python scripts prints all the output at the end of the run! |
| 105 |
----------------------------------------------------------------------------- |
| 106 |
|
| 107 |
The symptom is easy to spot - you're running a command that takes a few seconds |
| 108 |
or more to execute, it usually prints lines of text as it goes, but via |
| 109 |
`~invoke.run` nothing appears to happen at first, and then all the output |
| 110 |
prints once it's done executing. |
| 111 |
|
| 112 |
This is usually due to Python - the "inner" Python executable you're invoking, |
| 113 |
not the one Invoke is running under - performing unwanted buffering of its |
| 114 |
output streams. It does this when it thinks it's being called in a |
| 115 |
non-interactive fashion. |
| 116 |
|
| 117 |
The fix is to force Invoke to run the command in a pseudoterminal by |
| 118 |
saying ``pty=True`` (e.g. ``run("python foo", pty=True)``). |
| 119 |
|
| 120 |
Alternately, since both Invoke and the inner command are Python, you could try |
| 121 |
loading the inner Python module directly in your Invoke-using code, and call |
| 122 |
whichever methods its command-line stub is using - instead of using |
| 123 |
`~invoke.run`. This can often have other benefits too. |
| 124 |
|
| 125 |
.. _stdin-not-tty: |
| 126 |
|
| 127 |
Why do I sometimes see ``err: stdin: is not a tty``? |
| 128 |
---------------------------------------------------- |
| 129 |
|
| 130 |
See :ref:`program-behavior-ptys` - the same root cause (lack of a PTY by |
| 131 |
default) is probably what's going on. In some cases (such as via the Fabric |
| 132 |
library) it's happening because a shell's login files are calling programs that |
| 133 |
require a PTY (e.g. ``biff`` or ``mesg``) so make sure to look there if the |
| 134 |
actual foreground command doesn't seem at fault. |
| 135 |
|
| 136 |
Everything just exits silently after I run a command! |
| 137 |
----------------------------------------------------- |
| 138 |
|
| 139 |
Double check the command's exit code! By default, receiving nonzero exit codes |
| 140 |
at the end of a `~invoke.run` call will result in Invoke halting execution & |
| 141 |
exiting with that same code. Some programs (pylint, Nagios check scripts, |
| 142 |
etc) use exit codes to indicate non-fatal status, which can be confusing. |
| 143 |
|
| 144 |
The solution here is to add ``warn=True`` to your `~invoke.run` call, |
| 145 |
which disables the automatic exit behavior. Then you can check the result's |
| 146 |
``.exited`` attribute by hand to determine if it truly succeeded. |
| 147 |
|
| 148 |
The auto-responder functionality isn't working for my password prompts! |
| 149 |
----------------------------------------------------------------------- |
| 150 |
|
| 151 |
Some programs write password prompts or other output *directly* to the local |
| 152 |
terminal (the operating-system-level TTY device), bypassing the usual |
| 153 |
stdout/stderr streams. For example, this is exactly what `the stdlib's getpass |
| 154 |
module <getpass.getpass>` does, if you're calling a program that happens to be |
| 155 |
written in Python. |
| 156 |
|
| 157 |
When this happens, we're powerless, because all we get to see is the |
| 158 |
subprocess' regular output streams. Thankfully, the solution is usually easy: |
| 159 |
just add ``pty=True`` to your `~invoke.run` call. Forcing use of an explicit |
| 160 |
pseudo-terminal usually tricks these kinds of programs into writing prompts to |
| 161 |
stderr. |
| 162 |
|
| 163 |
I'm getting ``IOError: Inappropriate ioctl for device`` when I run commands! |
| 164 |
---------------------------------------------------------------------------- |
| 165 |
|
| 166 |
This error typically means some code in your project or its dependencies has |
| 167 |
replaced one of the process streams (``sys.stdin``, ``sys.stdout`` or |
| 168 |
``sys.stderr``) with an object that isn't actually hooked up to a terminal, but |
| 169 |
which pretends that it is. For example, test runners or build systems often do |
| 170 |
this. |
| 171 |
|
| 172 |
99% of the time, this pops up for stdin only, in which case you may be able to |
| 173 |
work around it by specifying ``in_stream=False`` to `~invoke.run` (note: |
| 174 |
``False``, **not** ``None``!) |
| 175 |
|
| 176 |
Gory details |
| 177 |
~~~~~~~~~~~~ |
| 178 |
|
| 179 |
Technically, what's happened is that the object handed to Invoke's command |
| 180 |
executor as e.g. ``run('command', in_stream=xxx)`` (or ``out_stream`` or etc; |
| 181 |
and these all default to the ``sys`` members listed above) implements a |
| 182 |
``fileno`` method that is not returning the ID of a real terminal file |
| 183 |
descriptor. Breaking the contract in this way is what's leading Invoke to do |
| 184 |
things the OS doesn't like. |
| 185 |
|
| 186 |
We're always trying to make this detection smarter; if upgrading to the latest |
| 187 |
version of Invoke doesn't fix the problem for you, please submit a bug report |
| 188 |
including details about the values and types of ``sys.stdin/stdout/stderr``. |
| 189 |
Hopefully we'll find another heuristic we can use! |