Debian Codegraph @

features/runner.abort_by_user.feature

raw
1 Feature: Test run can be aborted by the user
2
3 As a tester
4 I want sometimes to abort a test run (because it is anyway failing, etc.)
5 So that I am more productive.
6
7 . NOTES:
8 . * The test runner should fail gracefully (most of the times)
9 . * At least some cleanup hooks should be called (in general)
10
11
12 @setup
13 Scenario: Feature Setup
14 Given a new working directory
15 And a file named "features/steps/aborting_steps.py" with:
16 """
17 from behave import step
18
19 @step('{word:w} step passes')
20 def step_passes(context, word):
21 pass
22
23 @step('the user aborts the test run')
24 def step_user_aborts_testrun(context):
25 raise KeyboardInterrupt()
26 """
27 And a file named "features/scenarios_pass3.feature" with:
28 """
29 Feature:
30 Scenario:
31 Given a step passes
32 When another step passes
33
34 Scenario:
35 Given first step passes
36 When second step passes
37 Then third step passes
38
39 Scenario:
40 Then last step passes
41 """
42
43
44 Scenario: Abort test run in step definition
45 Given a file named "features/aborting_in_step.feature" with:
46 """
47 Feature: User aborts test run in a step definition
48 Scenario:
49 Given a step passes
50 When another step passes
51
52 Scenario: User aborts here
53 Given first step passes
54 When the user aborts the test run
55 Then third step passes
56
57 Scenario:
58 Then last step passes
59 """
60 And an empty file named "features/environment.py"
61 When I run "behave -f plain -T features/aborting_in_step.feature"
62 Then it should fail with:
63 """
64 0 features passed, 1 failed, 0 skipped
65 1 scenario passed, 1 failed, 0 skipped, 1 untested
66 3 steps passed, 1 failed, 1 skipped, 0 undefined, 1 untested
67 """
68 And the command output should contain:
69 """
70 Feature: User aborts test run in a step definition
71
72 Scenario:
73 Given a step passes ... passed
74 When another step passes ... passed
75
76 Scenario: User aborts here
77 Given first step passes ... passed
78 When the user aborts the test run ... failed
79 ABORTED: By user (KeyboardInterrupt).
80
81 ABORTED: By user.
82
83 Failing scenarios:
84 features/aborting_in_step.feature:6 User aborts here
85 """
86 But note that "the last scenario is untested (not-run) due to the user abort"
87
88
89 Scenario: Abort test run in before_scenario hook
90 Given a file named "features/aborting_in_before_scenario_hook.feature" with:
91 """
92 Feature: User aborts test run in before_scenario hook of S2
93 Scenario: S1
94 Given a step passes
95 When another step passes
96
97 @user.aborts.before_scenario
98 Scenario: S2 -- User aborts here
99 Given first step passes
100 When second step passes
101 Then third step passes
102
103 Scenario: S3
104 Then last step passes
105 """
106 And a file named "features/environment.py" with:
107 """
108 def before_scenario(context, scenario):
109 if "user.aborts.before_scenario" in scenario.tags:
110 user_aborts_testrun_here()
111
112 def user_aborts_testrun_here():
113 raise KeyboardInterrupt()
114 """
115 When I run "behave -f plain -T features/aborting_in_before_scenario_hook.feature"
116 Then it should fail with:
117 """
118 ABORTED: By user.
119 0 features passed, 1 failed, 0 skipped
120 1 scenario passed, 0 failed, 0 skipped, 2 untested
121 2 steps passed, 0 failed, 0 skipped, 0 undefined, 4 untested
122 """
123 And the command output should contain:
124 """
125 Feature: User aborts test run in before_scenario hook of S2
126
127 Scenario: S1
128 Given a step passes ... passed
129 When another step passes ... passed
130 """
131 But the command output should not contain "Scenario: S2 -- User aborts here"
132 But the command output should not contain "Scenario: S3"
133 And note that "the second and third scenario is not run"
134
135
136 Scenario: Abort test run in after_scenario hook
137 Given a file named "features/aborting_in_after_scenario_hook.feature" with:
138 """
139 Feature: User aborts test run in after_scenario hook
140 Scenario:
141 Given a step passes
142 When another step passes
143
144 @user.aborts.after_scenario
145 Scenario: User aborts here
146 Given first step passes
147 When second step passes
148 Then third step passes
149
150 Scenario:
151 Then last step passes
152 """
153 And a file named "features/environment.py" with:
154 """
155 def after_scenario(context, scenario):
156 if "user.aborts.after_scenario" in scenario.tags:
157 raise KeyboardInterrupt()
158 """
159 When I run "behave -f plain -T features/aborting_in_after_scenario_hook.feature"
160 Then it should fail with:
161 """
162 ABORTED: By user.
163 0 features passed, 1 failed, 0 skipped
164 2 scenarios passed, 0 failed, 0 skipped, 1 untested
165 5 steps passed, 0 failed, 0 skipped, 0 undefined, 1 untested
166 """
167 And the command output should contain:
168 """
169 Feature: User aborts test run in after_scenario hook
170
171 Scenario:
172 Given a step passes ... passed
173 When another step passes ... passed
174
175 Scenario: User aborts here
176 Given first step passes ... passed
177 When second step passes ... passed
178 Then third step passes ... passed
179 """
180 But the command output should not contain:
181 """
182 Scenario:
183 Then last step passes ... passed
184 """
185 And note that "the last scenario is not run"
186
187
188 Scenario: Abort test run in before_feature hook
189 Given a file named "features/aborting_in_before_feature_hook.feature" with:
190 """
191 @user.aborts.before_feature
192 Feature: User aborts test HERE
193 Scenario:
194 Given a step passes
195 When another step passes
196
197 Scenario:
198 Given first step passes
199 When second step passes
200 Then third step passes
201
202 Scenario:
203 Then last step passes
204 """
205 And a file named "features/environment.py" with:
206 """
207 from __future__ import print_function
208
209 def before_feature(context, feature):
210 if "user.aborts.before_feature" in feature.tags:
211 print("ABORTED in before_feature: %s" % feature.location)
212 raise KeyboardInterrupt()
213 """
214 When I run "behave -f plain -T features/aborting_in_before_feature_hook.feature"
215 Then it should fail with:
216 """
217 ABORTED in before_feature: features/aborting_in_before_feature_hook.feature:2
218
219 ABORTED: By user.
220 0 features passed, 0 failed, 0 skipped, 1 untested
221 0 scenarios passed, 0 failed, 0 skipped, 3 untested
222 0 steps passed, 0 failed, 0 skipped, 0 undefined, 6 untested
223 """
224 But note that "the feature is not run"
225 And note that "the formatters are not informed of this feature"
226
227
228 Scenario: Abort test run in after_feature hook
229 Given a file named "features/aborting_in_after_feature_hook.feature" with:
230 """
231 @user.aborts.after_feature
232 Feature: User aborts test after HERE
233 Scenario:
234 Given a step passes
235 When another step passes
236
237 Scenario:
238 Given first step passes
239 When second step passes
240 Then third step passes
241
242 Scenario:
243 Then last step passes
244 """
245 And a file named "features/environment.py" with:
246 """
247 def after_feature(context, feature):
248 if "user.aborts.after_feature" in feature.tags:
249 raise KeyboardInterrupt()
250 """
251 When I run "behave -f plain -T features/aborting_in_after_feature_hook.feature"
252 Then it should fail with:
253 """
254 ABORTED: By user.
255 1 feature passed, 0 failed, 0 skipped
256 3 scenarios passed, 0 failed, 0 skipped
257 6 steps passed, 0 failed, 0 skipped, 0 undefined
258 """
259 But note that "the behave command fails, but all features/scenarios passed"
260
261
262 Scenario: Abort test run in before_all hook
263
264 Note that this situation is not handled very gracefully (yet).
265
266 Given a file named "features/scenarios_pass3.feature" exists
267 And a file named "features/environment.py" with:
268 """
269 def before_all(context):
270 raise KeyboardInterrupt() #< ABORT-HERE
271 """
272 When I run "behave -f plain -T features/scenarios_pass3.feature"
273 Then it should fail with:
274 """
275 Traceback (most recent call last):
276 """
277 And the command output should contain:
278 """
279 File "features/environment.py", line 2, in before_all
280 raise KeyboardInterrupt() #< ABORT-HERE
281 """
282 And note that "no feature is not run"
283
284
285 Scenario: Abort test run in after_all hook
286
287 Note that this situation is not handled very gracefully (yet).
288
289 Given a file named "features/scenarios_pass3.feature" exists
290 And a file named "features/environment.py" with:
291 """
292 def after_all(context):
293 raise KeyboardInterrupt() #< ABORT-HERE
294 """
295 When I run "behave -f plain -T features/scenarios_pass3.feature"
296 Then it should fail with:
297 """
298 Traceback (most recent call last):
299 """
300 And the command output should contain:
301 """
302 File "features/environment.py", line 2, in after_all
303 raise KeyboardInterrupt() #< ABORT-HERE
304 """
305 And note that "all features are run"