| 1 |
.. _testing-user-code: |
| 2 |
|
| 3 |
============================== |
| 4 |
Testing Invoke-using codebases |
| 5 |
============================== |
| 6 |
|
| 7 |
Strategies for testing codebases that use Invoke; some applicable to code |
| 8 |
focused on CLI tasks, and others applicable to more generic/refactored setups. |
| 9 |
|
| 10 |
|
| 11 |
Subclass & modify Invoke 'internals' |
| 12 |
==================================== |
| 13 |
|
| 14 |
A quick foreword: most users will find the subsequent approaches suitable, but |
| 15 |
advanced users should note that Invoke has been designed so it is itself easily |
| 16 |
testable. This means that in many cases, even Invoke's "internals" are exposed |
| 17 |
as low/no-shared-responsibility, publicly documented classes which can be |
| 18 |
subclassed and modified to inject test-friendly values or mocks. Be sure to |
| 19 |
look over the :ref:`API documentation <api>`! |
| 20 |
|
| 21 |
|
| 22 |
Use `.MockContext` |
| 23 |
================== |
| 24 |
|
| 25 |
An instance of subclassing Invoke's public API for test purposes is our own |
| 26 |
`.MockContext`. Codebases which revolve heavily around `.Context` objects and |
| 27 |
their methods (most task-oriented code) will find it easy to test by injecting |
| 28 |
`.MockContext` objects which have been instantiated to yield partial `.Result` |
| 29 |
objects. |
| 30 |
|
| 31 |
For example, take this task:: |
| 32 |
|
| 33 |
from invoke import task |
| 34 |
|
| 35 |
@task |
| 36 |
def get_platform(c): |
| 37 |
uname = c.run("uname -s").stdout.strip() |
| 38 |
if uname == 'Darwin': |
| 39 |
return "You paid the Apple tax!" |
| 40 |
elif uname == 'Linux': |
| 41 |
return "Year of Linux on the desktop!" |
| 42 |
|
| 43 |
An example of testing it with `.MockContext` could be the following:: |
| 44 |
|
| 45 |
from invoke import MockContext, Result |
| 46 |
from mytasks import get_platform |
| 47 |
|
| 48 |
def test_get_platform_on_mac(): |
| 49 |
c = MockContext(run=Result("Darwin\n")) |
| 50 |
assert "Apple" in get_platform(c) |
| 51 |
|
| 52 |
def test_get_platform_on_linux(): |
| 53 |
c = MockContext(run=Result("Linux\n")) |
| 54 |
assert "desktop" in get_platform(c) |
| 55 |
|
| 56 |
Putting the ``Mock`` in `.MockContext` |
| 57 |
-------------------------------------- |
| 58 |
|
| 59 |
Starting in Invoke 1.5, `.MockContext` will attempt to import the ``mock`` |
| 60 |
library at instantiation time and wrap its methods within ``Mock`` objects. |
| 61 |
This lets you not only present realistic return values to your code, but make |
| 62 |
test assertions about what commands your code is running. |
| 63 |
|
| 64 |
Here's another "platform sensitive" task:: |
| 65 |
|
| 66 |
from invoke import task |
| 67 |
|
| 68 |
@task |
| 69 |
def replace(c, path, search, replacement): |
| 70 |
# Assume systems have sed, and that some (eg macOS w/ Homebrew) may |
| 71 |
# have gsed, implying regular sed is BSD style. |
| 72 |
has_gsed = c.run("which gsed", warn=True, hide=True) |
| 73 |
# Set command to run accordingly |
| 74 |
binary = "gsed" if has_gsed else "sed" |
| 75 |
c.run(f"{binary} -e 's/{search}/{replacement}/g' {path}") |
| 76 |
|
| 77 |
The test code (again, which presumes that eg ``MockContext.run`` is now a |
| 78 |
``Mock`` wrapper) relies primarily on 'last call' assertions |
| 79 |
(``Mock.assert_called_with``) but you can of course use any ``Mock`` methods |
| 80 |
you need. It also shows how you can set the mock context to respond to multiple |
| 81 |
possible commands, using a dict value:: |
| 82 |
|
| 83 |
from invoke import MockContext, Result |
| 84 |
from mytasks import replace |
| 85 |
|
| 86 |
def test_regular_sed(): |
| 87 |
expected_sed = "sed -e s/foo/bar/g file.txt" |
| 88 |
c = MockContext(run={ |
| 89 |
"which gsed": Result(exited=1), |
| 90 |
expected_sed: Result(), |
| 91 |
}) |
| 92 |
replace(c, 'file.txt', 'foo', 'bar') |
| 93 |
c.run.assert_called_with(expected_sed) |
| 94 |
|
| 95 |
def test_homebrew_gsed(): |
| 96 |
expected_sed = "gsed -e s/foo/bar/g file.txt" |
| 97 |
c = MockContext(run={ |
| 98 |
"which gsed": Result(exited=0), |
| 99 |
expected_sed: Result(), |
| 100 |
}) |
| 101 |
replace(c, 'file.txt', 'foo', 'bar') |
| 102 |
c.run.assert_called_with(expected_sed) |
| 103 |
|
| 104 |
Boolean mock results |
| 105 |
-------------------- |
| 106 |
|
| 107 |
You may have noticed the above example uses a handful of 'empty' `.Result` |
| 108 |
objects; these stand in for "succeeded, but otherwise had no useful attributes" |
| 109 |
command executions (as `.Result` defaults to an exit code of ``0`` and empty |
| 110 |
strings for stdout/stderr). |
| 111 |
|
| 112 |
This is relatively common - think "interrogative" commands where the caller |
| 113 |
only cares for a boolean result, or times when a command is called purely for |
| 114 |
its side effects. To support this, there's a shorthand in `.MockContext`: |
| 115 |
passing ``True`` or ``False`` to stand in for otherwise blank Results with exit |
| 116 |
codes of ``0`` or ``1`` respectively. |
| 117 |
|
| 118 |
The example tests then look like this:: |
| 119 |
|
| 120 |
from invoke import MockContext, Result |
| 121 |
from mytasks import replace |
| 122 |
|
| 123 |
def test_regular_sed(): |
| 124 |
expected_sed = "sed -e s/foo/bar/g file.txt" |
| 125 |
c = MockContext(run={ |
| 126 |
"which gsed": False, |
| 127 |
expected_sed: True, |
| 128 |
}) |
| 129 |
replace(c, 'file.txt', 'foo', 'bar') |
| 130 |
c.run.assert_called_with(expected_sed) |
| 131 |
|
| 132 |
def test_homebrew_gsed(): |
| 133 |
expected_sed = "gsed -e s/foo/bar/g file.txt" |
| 134 |
c = MockContext(run={ |
| 135 |
"which gsed": True, |
| 136 |
expected_sed: True, |
| 137 |
}) |
| 138 |
replace(c, 'file.txt', 'foo', 'bar') |
| 139 |
c.run.assert_called_with(expected_sed) |
| 140 |
|
| 141 |
String mock results |
| 142 |
------------------- |
| 143 |
|
| 144 |
Another convenient shorthand is using string values, which are interpreted to |
| 145 |
be the stdout of the resulting `.Result`. This only really saves you from |
| 146 |
writing out the class itself (since ``stdout`` is the first positional arg of |
| 147 |
`.Result`!) but "command X results in stdout Y" is a common enough use case |
| 148 |
that we implemented it anyway. |
| 149 |
|
| 150 |
By example, let's modify an earlier example where we cared about stdout:: |
| 151 |
|
| 152 |
from invoke import MockContext |
| 153 |
from mytasks import get_platform |
| 154 |
|
| 155 |
def test_get_platform_on_mac(): |
| 156 |
c = MockContext(run="Darwin\n") |
| 157 |
assert "Apple" in get_platform(c) |
| 158 |
|
| 159 |
def test_get_platform_on_linux(): |
| 160 |
c = MockContext(run="Linux\n") |
| 161 |
assert "desktop" in get_platform(c) |
| 162 |
|
| 163 |
As with everything else in this document, this tactic can be applied to |
| 164 |
iterators or mappings as well as individual values. |
| 165 |
|
| 166 |
Regular expression command matching |
| 167 |
----------------------------------- |
| 168 |
|
| 169 |
The dict form of `.MockContext` kwarg can accept regular expression objects as |
| 170 |
keys, in addition to strings; ideal for situations where you either don't know |
| 171 |
the exact command being invoked, or simply don't need or want to write out the |
| 172 |
entire thing. |
| 173 |
|
| 174 |
Imagine you're writing a function to run package management commands on a few |
| 175 |
different Linux distros and you're trying to test its error handling. You might |
| 176 |
want to set up a context that pretends any arbitrary ``apt`` or ``yum`` command |
| 177 |
fails, and ensure the function returns stderr when it encounters a problem:: |
| 178 |
|
| 179 |
import re |
| 180 |
from invoke import MockContext |
| 181 |
from mypackage.tasks import install |
| 182 |
|
| 183 |
package_manager = re.compile(r"^(apt(-get)?|yum) .*") |
| 184 |
|
| 185 |
def test_package_success_returns_True(): |
| 186 |
c = MockContext(run={package_manager: True}) |
| 187 |
assert install(c, package="somepackage") is True |
| 188 |
|
| 189 |
def test_package_explosions_return_stderr(): |
| 190 |
c = MockContext(run={ |
| 191 |
package_manager: Result(stderr="oh no!", exited=1), |
| 192 |
}) |
| 193 |
assert install(c, package="otherpackage") == "oh no!" |
| 194 |
|
| 195 |
A bit contrived - there are a bunch of other ways to organize this exact test |
| 196 |
code so you don't truly need the regex - but hopefully it's clear that when you |
| 197 |
*do* need this flexibility, this is how you could go about it. |
| 198 |
|
| 199 |
Repeated results |
| 200 |
---------------- |
| 201 |
|
| 202 |
By default, the values in these mock structures are consumed, causing |
| 203 |
`.MockContext` to raise ``NotImplementedError`` afterwards (as it does for any |
| 204 |
unexpected command executions). This was designed with the assumption that |
| 205 |
most code under test will run a given command once. |
| 206 |
|
| 207 |
If your situation doesn't match this, give ``repeat=True`` to the constructor, |
| 208 |
and you'll see values repeat indefinitely instead (or in cycles, for |
| 209 |
iterables). |
| 210 |
|
| 211 |
|
| 212 |
Expect `Results <.Result>` |
| 213 |
========================== |
| 214 |
|
| 215 |
The core Invoke subprocess methods like `~.Context.run` all return `.Result` |
| 216 |
objects - which (as seen above) can be readily instantiated by themselves with |
| 217 |
only partial data (e.g. standard output, but no exit code or standard error). |
| 218 |
|
| 219 |
This means that well-organized code can be even easier to test and doesn't |
| 220 |
require as much use of `.MockContext`. |
| 221 |
|
| 222 |
An iteration on the initial `.MockContext`-using example above:: |
| 223 |
|
| 224 |
from invoke import task |
| 225 |
|
| 226 |
@task |
| 227 |
def get_platform(c): |
| 228 |
print(platform_response(c.run("uname -s"))) |
| 229 |
|
| 230 |
def platform_response(result): |
| 231 |
uname = result.stdout.strip() |
| 232 |
if uname == 'Darwin': |
| 233 |
return "You paid the Apple tax!" |
| 234 |
elif uname == 'Linux': |
| 235 |
return "Year of Linux on the desktop!" |
| 236 |
|
| 237 |
With the logic encapsulated in a subroutine, you can just unit test that |
| 238 |
function by itself, deferring testing of the task or its context:: |
| 239 |
|
| 240 |
from invoke import Result |
| 241 |
from mytasks import platform_response |
| 242 |
|
| 243 |
def test_platform_response_on_mac(): |
| 244 |
assert "Apple" in platform_response(Result("Darwin\n")) |
| 245 |
|
| 246 |
def test_platform_response_on_linux(): |
| 247 |
assert "desktop" in platform_response(Result("Linux\n")) |
| 248 |
|
| 249 |
|
| 250 |
Avoid mocking dependency code paths altogether |
| 251 |
============================================== |
| 252 |
|
| 253 |
This is more of a general software engineering tactic, but the natural endpoint |
| 254 |
of the above code examples would be where your primary logic doesn't care about |
| 255 |
Invoke at all -- only about basic Python (or locally defined) data types. This |
| 256 |
allows you to test logic in isolation and either ignore testing the Invoke side |
| 257 |
of things, or write targeted tests solely for where your code interfaces with |
| 258 |
Invoke. |
| 259 |
|
| 260 |
Another minor tweak to the task code:: |
| 261 |
|
| 262 |
from invoke import task |
| 263 |
|
| 264 |
@task |
| 265 |
def show_platform(c): |
| 266 |
uname = c.run("uname -s").stdout.strip() |
| 267 |
print(platform_response(uname)) |
| 268 |
|
| 269 |
def platform_response(uname): |
| 270 |
if uname == 'Darwin': |
| 271 |
return "You paid the Apple tax!" |
| 272 |
elif uname == 'Linux': |
| 273 |
return "Year of Linux on the desktop!" |
| 274 |
|
| 275 |
And the tests:: |
| 276 |
|
| 277 |
from mytasks import platform_response |
| 278 |
|
| 279 |
def test_platform_response_on_mac(): |
| 280 |
assert "Apple" in platform_response("Darwin\n") |
| 281 |
|
| 282 |
def test_platform_response_on_linux(): |
| 283 |
assert "desktop" in platform_response("Linux\n") |