Debian Codegraph @

sites/docs/concepts/testing.rst

raw
1 .. _testing-user-code:
2
3 ==============================
4 Testing Invoke-using codebases
5 ==============================
6
7 Strategies for testing codebases that use Invoke; some applicable to code
8 focused on CLI tasks, and others applicable to more generic/refactored setups.
9
10
11 Subclass & modify Invoke 'internals'
12 ====================================
13
14 A quick foreword: most users will find the subsequent approaches suitable, but
15 advanced users should note that Invoke has been designed so it is itself easily
16 testable. This means that in many cases, even Invoke's "internals" are exposed
17 as low/no-shared-responsibility, publicly documented classes which can be
18 subclassed and modified to inject test-friendly values or mocks. Be sure to
19 look over the :ref:`API documentation <api>`!
20
21
22 Use `.MockContext`
23 ==================
24
25 An instance of subclassing Invoke's public API for test purposes is our own
26 `.MockContext`. Codebases which revolve heavily around `.Context` objects and
27 their methods (most task-oriented code) will find it easy to test by injecting
28 `.MockContext` objects which have been instantiated to yield partial `.Result`
29 objects.
30
31 For example, take this task::
32
33 from invoke import task
34
35 @task
36 def get_platform(c):
37 uname = c.run("uname -s").stdout.strip()
38 if uname == 'Darwin':
39 return "You paid the Apple tax!"
40 elif uname == 'Linux':
41 return "Year of Linux on the desktop!"
42
43 An example of testing it with `.MockContext` could be the following::
44
45 from invoke import MockContext, Result
46 from mytasks import get_platform
47
48 def test_get_platform_on_mac():
49 c = MockContext(run=Result("Darwin\n"))
50 assert "Apple" in get_platform(c)
51
52 def test_get_platform_on_linux():
53 c = MockContext(run=Result("Linux\n"))
54 assert "desktop" in get_platform(c)
55
56 Putting the ``Mock`` in `.MockContext`
57 --------------------------------------
58
59 Starting in Invoke 1.5, `.MockContext` will attempt to import the ``mock``
60 library at instantiation time and wrap its methods within ``Mock`` objects.
61 This lets you not only present realistic return values to your code, but make
62 test assertions about what commands your code is running.
63
64 Here's another "platform sensitive" task::
65
66 from invoke import task
67
68 @task
69 def replace(c, path, search, replacement):
70 # Assume systems have sed, and that some (eg macOS w/ Homebrew) may
71 # have gsed, implying regular sed is BSD style.
72 has_gsed = c.run("which gsed", warn=True, hide=True)
73 # Set command to run accordingly
74 binary = "gsed" if has_gsed else "sed"
75 c.run(f"{binary} -e 's/{search}/{replacement}/g' {path}")
76
77 The test code (again, which presumes that eg ``MockContext.run`` is now a
78 ``Mock`` wrapper) relies primarily on 'last call' assertions
79 (``Mock.assert_called_with``) but you can of course use any ``Mock`` methods
80 you need. It also shows how you can set the mock context to respond to multiple
81 possible commands, using a dict value::
82
83 from invoke import MockContext, Result
84 from mytasks import replace
85
86 def test_regular_sed():
87 expected_sed = "sed -e s/foo/bar/g file.txt"
88 c = MockContext(run={
89 "which gsed": Result(exited=1),
90 expected_sed: Result(),
91 })
92 replace(c, 'file.txt', 'foo', 'bar')
93 c.run.assert_called_with(expected_sed)
94
95 def test_homebrew_gsed():
96 expected_sed = "gsed -e s/foo/bar/g file.txt"
97 c = MockContext(run={
98 "which gsed": Result(exited=0),
99 expected_sed: Result(),
100 })
101 replace(c, 'file.txt', 'foo', 'bar')
102 c.run.assert_called_with(expected_sed)
103
104 Boolean mock results
105 --------------------
106
107 You may have noticed the above example uses a handful of 'empty' `.Result`
108 objects; these stand in for "succeeded, but otherwise had no useful attributes"
109 command executions (as `.Result` defaults to an exit code of ``0`` and empty
110 strings for stdout/stderr).
111
112 This is relatively common - think "interrogative" commands where the caller
113 only cares for a boolean result, or times when a command is called purely for
114 its side effects. To support this, there's a shorthand in `.MockContext`:
115 passing ``True`` or ``False`` to stand in for otherwise blank Results with exit
116 codes of ``0`` or ``1`` respectively.
117
118 The example tests then look like this::
119
120 from invoke import MockContext, Result
121 from mytasks import replace
122
123 def test_regular_sed():
124 expected_sed = "sed -e s/foo/bar/g file.txt"
125 c = MockContext(run={
126 "which gsed": False,
127 expected_sed: True,
128 })
129 replace(c, 'file.txt', 'foo', 'bar')
130 c.run.assert_called_with(expected_sed)
131
132 def test_homebrew_gsed():
133 expected_sed = "gsed -e s/foo/bar/g file.txt"
134 c = MockContext(run={
135 "which gsed": True,
136 expected_sed: True,
137 })
138 replace(c, 'file.txt', 'foo', 'bar')
139 c.run.assert_called_with(expected_sed)
140
141 String mock results
142 -------------------
143
144 Another convenient shorthand is using string values, which are interpreted to
145 be the stdout of the resulting `.Result`. This only really saves you from
146 writing out the class itself (since ``stdout`` is the first positional arg of
147 `.Result`!) but "command X results in stdout Y" is a common enough use case
148 that we implemented it anyway.
149
150 By example, let's modify an earlier example where we cared about stdout::
151
152 from invoke import MockContext
153 from mytasks import get_platform
154
155 def test_get_platform_on_mac():
156 c = MockContext(run="Darwin\n")
157 assert "Apple" in get_platform(c)
158
159 def test_get_platform_on_linux():
160 c = MockContext(run="Linux\n")
161 assert "desktop" in get_platform(c)
162
163 As with everything else in this document, this tactic can be applied to
164 iterators or mappings as well as individual values.
165
166 Regular expression command matching
167 -----------------------------------
168
169 The dict form of `.MockContext` kwarg can accept regular expression objects as
170 keys, in addition to strings; ideal for situations where you either don't know
171 the exact command being invoked, or simply don't need or want to write out the
172 entire thing.
173
174 Imagine you're writing a function to run package management commands on a few
175 different Linux distros and you're trying to test its error handling. You might
176 want to set up a context that pretends any arbitrary ``apt`` or ``yum`` command
177 fails, and ensure the function returns stderr when it encounters a problem::
178
179 import re
180 from invoke import MockContext
181 from mypackage.tasks import install
182
183 package_manager = re.compile(r"^(apt(-get)?|yum) .*")
184
185 def test_package_success_returns_True():
186 c = MockContext(run={package_manager: True})
187 assert install(c, package="somepackage") is True
188
189 def test_package_explosions_return_stderr():
190 c = MockContext(run={
191 package_manager: Result(stderr="oh no!", exited=1),
192 })
193 assert install(c, package="otherpackage") == "oh no!"
194
195 A bit contrived - there are a bunch of other ways to organize this exact test
196 code so you don't truly need the regex - but hopefully it's clear that when you
197 *do* need this flexibility, this is how you could go about it.
198
199 Repeated results
200 ----------------
201
202 By default, the values in these mock structures are consumed, causing
203 `.MockContext` to raise ``NotImplementedError`` afterwards (as it does for any
204 unexpected command executions). This was designed with the assumption that
205 most code under test will run a given command once.
206
207 If your situation doesn't match this, give ``repeat=True`` to the constructor,
208 and you'll see values repeat indefinitely instead (or in cycles, for
209 iterables).
210
211
212 Expect `Results <.Result>`
213 ==========================
214
215 The core Invoke subprocess methods like `~.Context.run` all return `.Result`
216 objects - which (as seen above) can be readily instantiated by themselves with
217 only partial data (e.g. standard output, but no exit code or standard error).
218
219 This means that well-organized code can be even easier to test and doesn't
220 require as much use of `.MockContext`.
221
222 An iteration on the initial `.MockContext`-using example above::
223
224 from invoke import task
225
226 @task
227 def get_platform(c):
228 print(platform_response(c.run("uname -s")))
229
230 def platform_response(result):
231 uname = result.stdout.strip()
232 if uname == 'Darwin':
233 return "You paid the Apple tax!"
234 elif uname == 'Linux':
235 return "Year of Linux on the desktop!"
236
237 With the logic encapsulated in a subroutine, you can just unit test that
238 function by itself, deferring testing of the task or its context::
239
240 from invoke import Result
241 from mytasks import platform_response
242
243 def test_platform_response_on_mac():
244 assert "Apple" in platform_response(Result("Darwin\n"))
245
246 def test_platform_response_on_linux():
247 assert "desktop" in platform_response(Result("Linux\n"))
248
249
250 Avoid mocking dependency code paths altogether
251 ==============================================
252
253 This is more of a general software engineering tactic, but the natural endpoint
254 of the above code examples would be where your primary logic doesn't care about
255 Invoke at all -- only about basic Python (or locally defined) data types. This
256 allows you to test logic in isolation and either ignore testing the Invoke side
257 of things, or write targeted tests solely for where your code interfaces with
258 Invoke.
259
260 Another minor tweak to the task code::
261
262 from invoke import task
263
264 @task
265 def show_platform(c):
266 uname = c.run("uname -s").stdout.strip()
267 print(platform_response(uname))
268
269 def platform_response(uname):
270 if uname == 'Darwin':
271 return "You paid the Apple tax!"
272 elif uname == 'Linux':
273 return "Year of Linux on the desktop!"
274
275 And the tests::
276
277 from mytasks import platform_response
278
279 def test_platform_response_on_mac():
280 assert "Apple" in platform_response("Darwin\n")
281
282 def test_platform_response_on_linux():
283 assert "desktop" in platform_response("Linux\n")