Debian Codegraph @

features/runner.context_cleanup.feature

raw
1 Feature: Perform Context.cleanups at the end of a test-run, feature or scenario (scope guards)
2
3 As a test writer
4 I want to perform cleanups (tear-downs) of functionality at the end of a test-run, feature or scenario
5 So that I can easily implement the "scope guard" design pattern (Python: contextmanager)
6
7 . SPECIFICATION:
8 . * Context.add_cleanup(func) is provided to register cleanup functions
9 . * Cleanup functions are registered in the current layer of the context object stack.
10 . * Cleanup functions are executed before the current layer of the context object is removed/popped.
11 . * Cleanup functions are executed in reverse order of registration
12 . (LIFO: first registered cleanup function will be executed last, etc.)
13 . * Cleanup functions are executed in best-effort mode: If one fails, the others are still executed
14 . * Behaviour when the execution of cleanup function fails should be adjustable
15 .
16 . | Scope/Layer | Cleanup Registration Point | Cleanup Execution Point |
17 . | test-run | In before_all() hook | After after_all() hook is executed. |
18 . | feature | In before_feature() hook | After after_feature() hook is executed. |
19 . | feature | In before_tag() in Feature | After after_feature() hook is executed. |
20 . | scenario | In before_scenario() hook | After after_scenario() hook is executed. |
21 . | scenario | In before_tag() in Scenario | After after_scenario() hook is executed. |
22 . | scenario | In step implementation | After after_scenario() hook is executed. |
23 . | scenario | In step hooks | After after_scenario() hook is executed. |
24
25
26 @setup
27 Scenario: Test Setup
28 Given a new working directory
29 And a file named "features/environment.py" with:
30 """
31 from __future__ import print_function
32
33 # -- CLEANUP FUNCTIONS:
34 class CleanupFuntion(object):
35 def __init__(self, name=None):
36 self.name = name or ""
37
38 def __call__(self):
39 print("CALLED: CleanupFunction:%s" % self.name)
40
41 def cleanup_after_testrun():
42 print("CALLED: cleanup_after_testrun")
43
44 def cleanup_foo():
45 print("CALLED: cleanup_foo")
46
47 def cleanup_bar():
48 print("CALLED: cleanup_bar")
49
50 # -- HOOKS:
51 def before_all(context):
52 print("CALLED-HOOK: before_all")
53 userdata = context.config.userdata
54 use_cleanup = userdata.getbool("use_cleanup_after_testrun")
55 if use_cleanup:
56 print("REGISTER-CLEANUP: cleanup_after_testrun")
57 context.add_cleanup(cleanup_after_testrun)
58
59 def before_feature(context, feature):
60 print("CALLED-HOOK: before_feature:%s" % feature.name)
61 userdata = context.config.userdata
62 use_cleanup = userdata.getbool("use_cleanup_after_feature")
63 if use_cleanup and "cleanup.after_feature" in feature.tags:
64 print("REGISTER-CLEANUP: cleanup_foo")
65 context.add_cleanup(cleanup_foo)
66
67 def after_feature(context, feature):
68 print("CALLED-HOOK: after_feature: %s" % feature.name)
69
70 def after_all(context):
71 print("CALLED-HOOK: after_all")
72 """
73 And a file named "features/steps/use_steps.py" with:
74 """
75 import behave4cmd0.passing_steps
76 """
77 And a file named "features/alice.feature" with:
78 """
79 Feature: Alice
80 Scenario: A1
81 Given a step passes
82
83 Scenario: A2
84 When another step passes
85 """
86 And a file named "behave.ini" with:
87 """
88 [behave]
89 show_timings = false
90 stdout_capture = true
91 """
92
93 @cleanup.after_testrun
94 Scenario: Cleanup registered in before_all hook
95 When I run "behave -D use_cleanup_after_testrun -f plain features/alice.feature"
96 Then it should pass with:
97 """
98 CALLED-HOOK: before_all
99 REGISTER-CLEANUP: cleanup_after_testrun
100 CALLED-HOOK: before_feature:Alice
101 Feature: Alice
102 """
103 And the command output should contain:
104 """
105 Scenario: A2
106 When another step passes ... passed
107
108 CALLED-HOOK: after_feature: Alice
109 CALLED-HOOK: after_all
110 CALLED: cleanup_after_testrun
111 """
112
113 @cleanup.after_feature
114 Scenario: Cleanup registered in before_feature hook
115 Given a file named "features/environment.py" with:
116 """
117 from __future__ import print_function
118
119 # -- CLEANUP FUNCTIONS:
120 def cleanup_foo():
121 print("CALLED: cleanup_foo")
122
123 # -- HOOKS:
124 def before_feature(context, feature):
125 print("CALLED-HOOK: before_feature:%s" % feature.name)
126 if "cleanup.after_feature" in feature.tags:
127 print("REGISTER-CLEANUP: cleanup_foo")
128 context.add_cleanup(cleanup_foo)
129
130 def after_feature(context, feature):
131 print("CALLED-HOOK: after_feature:%s" % feature.name)
132
133 def after_all(context):
134 print("CALLED-HOOK: after_all")
135 """
136 And a file named "features/bob.feature" with:
137 """
138 @cleanup.after_feature
139 Feature: Bob
140 Scenario: B1
141 Given a step passes
142 """
143 When I run "behave -f plain features/bob.feature"
144 Then it should pass with:
145 """
146 CALLED-HOOK: before_feature:Bob
147 REGISTER-CLEANUP: cleanup_foo
148 Feature: Bob
149 """
150 And the command output should contain:
151 """
152 Scenario: B1
153 Given a step passes ... passed
154
155 CALLED-HOOK: after_feature:Bob
156 CALLED: cleanup_foo
157 CALLED-HOOK: after_all
158 """
159
160
161 @cleanup.after_scenario
162 Scenario: Cleanup registered in before_scenario hook
163 Given a file named "features/environment.py" with:
164 """
165 from __future__ import print_function
166
167 # -- CLEANUP FUNCTIONS:
168 def cleanup_foo():
169 print("CALLED: cleanup_foo")
170
171 # -- HOOKS:
172 def before_scenario(context, scenario):
173 print("CALLED-HOOK: before_scenario:%s" % scenario.name)
174 if "cleanup_foo" in scenario.tags:
175 print("REGISTER-CLEANUP: cleanup_foo")
176 context.add_cleanup(cleanup_foo)
177
178 def after_scenario(context, scenario):
179 print("CALLED-HOOK: after_scenario:%s" % scenario.name)
180 """
181 And a file named "features/charly.feature" with:
182 """
183 Feature: Charly
184 @cleanup_foo
185 Scenario: C1
186 Given a step passes
187
188 Scenario: C2
189 When a step passes
190 """
191 When I run "behave -f plain features/charly.feature"
192 Then it should pass with:
193 """
194 CALLED-HOOK: before_scenario:C1
195 REGISTER-CLEANUP: cleanup_foo
196 Scenario: C1
197 """
198 And the command output should contain:
199 """
200 Scenario: C1
201 Given a step passes ... passed
202
203 CALLED-HOOK: after_scenario:C1
204 CALLED: cleanup_foo
205 CALLED-HOOK: before_scenario:C2
206 """
207
208 @cleanup.after_scenario
209 Scenario: Cleanups are executed in reverse registration order
210
211 Given a file named "features/environment.py" with:
212 """
213 from __future__ import print_function
214
215 # -- CLEANUP FUNCTIONS:
216 def cleanup_foo():
217 print("CALLED: cleanup_foo")
218
219 def cleanup_bar():
220 print("CALLED: cleanup_bar")
221
222 # -- HOOKS:
223 def before_scenario(context, scenario):
224 print("CALLED-HOOK: before_scenario:%s" % scenario.name)
225 if "cleanup_foo" in scenario.tags:
226 print("REGISTER-CLEANUP: cleanup_foo")
227 context.add_cleanup(cleanup_foo)
228 if "cleanup_bar" in scenario.tags:
229 print("REGISTER-CLEANUP: cleanup_bar")
230 context.add_cleanup(cleanup_bar)
231
232 def after_scenario(context, scenario):
233 print("CALLED-HOOK: after_scenario:%s" % scenario.name)
234 """
235 And a file named "features/dodo.feature" with:
236 """
237 Feature: Dodo
238 @cleanup_foo
239 @cleanup_bar
240 Scenario: D1
241 Given a step passes
242
243 Scenario: D2
244 When a step passes
245 """
246 When I run "behave -f plain features/dodo.feature"
247 Then it should pass with:
248 """
249 CALLED-HOOK: before_scenario:D1
250 REGISTER-CLEANUP: cleanup_foo
251 REGISTER-CLEANUP: cleanup_bar
252 Scenario: D1
253 """
254 And the command output should contain:
255 """
256 Scenario: D1
257 Given a step passes ... passed
258
259 CALLED-HOOK: after_scenario:D1
260 CALLED: cleanup_bar
261 CALLED: cleanup_foo
262 CALLED-HOOK: before_scenario:D2
263 """
264 And the command output should contain 1 times:
265 """
266 CALLED: cleanup_bar
267 CALLED: cleanup_foo
268 """
269
270 @cleanup.after_scenario
271 Scenario: Cleanup registered in step implementation
272 Given a file named "features/environment.py" with:
273 """
274 from __future__ import print_function
275
276 # -- HOOKS:
277 def before_scenario(context, scenario):
278 print("CALLED-HOOK: before_scenario:%s" % scenario.name)
279
280 def after_scenario(context, scenario):
281 print("CALLED-HOOK: after_scenario:%s" % scenario.name)
282 """
283 And a file named "features/steps/cleanup_steps.py" with:
284 """
285 from behave import given
286
287 # -- CLEANUP FUNCTIONS:
288 def cleanup_foo():
289 print("CALLED: cleanup_foo")
290
291 # -- STEPS:
292 @given(u'I register a cleanup "{cleanup_name}"')
293 def step_register_cleanup(context, cleanup_name):
294 if cleanup_name == "cleanup_foo":
295 context.add_cleanup(cleanup_foo)
296 else:
297 raise KeyError("Unknown_cleanup:%s" % cleanup_name)
298 """
299 And a file named "features/emily.feature" with:
300 """
301 Feature: Emily
302 Scenario: E1
303 Given I register a cleanup "cleanup_foo"
304
305 Scenario: E2
306 When a step passes
307 """
308 When I run "behave -f plain features/emily.feature"
309 Then it should pass with:
310 """
311 Scenario: E1
312 Given I register a cleanup "cleanup_foo" ... passed
313
314 CALLED-HOOK: after_scenario:E1
315 CALLED: cleanup_foo
316 CALLED-HOOK: before_scenario:E2
317 """
318 And the command output should contain 1 times:
319 """
320 CALLED: cleanup_foo
321 """
322
323 @cleanup.after_scenario
324 Scenario: Registered cleanup function args are passed to cleanup
325 Given a file named "features/environment.py" with:
326 """
327 from __future__ import print_function
328
329 # -- CLEANUP FUNCTIONS:
330 def cleanup_foo(text):
331 print('CALLED: cleanup_foo("%s")' % text)
332
333 # -- HOOKS:
334 def before_scenario(context, scenario):
335 print("CALLED-HOOK: before_scenario:%s" % scenario.name)
336 if "cleanup_foo" in scenario.tags:
337 print('REGISTER-CLEANUP: cleanup_foo("Alice")')
338 context.add_cleanup(cleanup_foo, "Alice")
339 print('REGISTER-CLEANUP: cleanup_foo("Bob")')
340 context.add_cleanup(cleanup_foo, "Bob")
341
342 def after_scenario(context, scenario):
343 print("CALLED-HOOK: after_scenario:%s" % scenario.name)
344 """
345 And a file named "features/frank.feature" with:
346 """
347 Feature: Frank
348 @cleanup_foo
349 Scenario: F1
350 Given a step passes
351
352 Scenario: F2
353 When a step passes
354 """
355 When I run "behave -f plain features/frank.feature"
356 Then it should pass with:
357 """
358 CALLED-HOOK: before_scenario:F1
359 REGISTER-CLEANUP: cleanup_foo("Alice")
360 REGISTER-CLEANUP: cleanup_foo("Bob")
361 Scenario: F1
362 """
363 And the command output should contain:
364 """
365 Scenario: F1
366 Given a step passes ... passed
367
368 CALLED-HOOK: after_scenario:F1
369 CALLED: cleanup_foo("Bob")
370 CALLED: cleanup_foo("Alice")
371 CALLED-HOOK: before_scenario:F2
372 """