Debian Codegraph @

features/runner.continue_after_failed_step.feature

raw
1 Feature: Runner should continue after a failed step
2
3 As a tester or test writer
4 I want that the remaining scenario steps are executed even when a step fails
5 So that I see all remaining failures.
6
7 . NOTES:
8 . This may cause a number of correlated failures in the remaining steps.
9 . Therefore the default behavior of the runner is to skip the
10 . remaining scenario steps after a failure has occurred.
11 .
12 . RELATED TO: CR #299 (duplicated: #314)
13
14 @setup
15 Scenario: Test Setup
16 Given a new working directory
17 And a file named "features/steps/steps.py" with:
18 """
19 from behave import step
20 import sys
21
22 @step('{word:w} step passes')
23 def step_passes(context, word):
24 pass
25
26 @step('{word:w} step fails')
27 def step_fails(context, word):
28 assert False, "XFAIL (in: %s step)" % word
29 """
30 And a file named "features/alice.feature" with:
31 """
32 Feature: Alice
33 Scenario: Passing
34 Given first step passes
35 When second step passes
36 Then third step passes
37
38 Scenario: Fails in second step
39 Given a step passes
40 When second step fails
41 Then another step passes
42
43 @runner.continue_after_failed_step
44 Scenario: Fails in first and third step
45 Given first step fails
46 When second step passes
47 Then third step fails
48 """
49
50
51 Scenario: Runner continues after failed step in all scenarios
52 Given a file named "features/environment.py" with:
53 """
54 from behave.model import Scenario
55
56 def before_all(context):
57 userdata = context.config.userdata
58 continue_after_failed = userdata.getbool("runner.continue_after_failed_step", False)
59 Scenario.continue_after_failed_step = continue_after_failed
60 """
61 And a file named "behave.ini" with:
62 """
63 [behave]
64 show_timings = false
65
66 [behave.userdata]
67 runner.continue_after_failed_step = true
68 """
69 When I run "behave -f plain features/alice.feature"
70 Then it should fail with:
71 """
72 Failing scenarios:
73 features/alice.feature:7 Fails in second step
74 features/alice.feature:13 Fails in first and third step
75
76 0 features passed, 1 failed, 0 skipped
77 1 scenario passed, 2 failed, 0 skipped
78 6 steps passed, 3 failed, 0 skipped, 0 undefined
79 """
80 And the command output should contain:
81 """
82 Scenario: Fails in second step
83 Given a step passes ... passed
84 When second step fails ... failed
85 Assertion Failed: XFAIL (in: second step)
86 Then another step passes ... passed
87
88 Scenario: Fails in first and third step
89 Given first step fails ... failed
90 Assertion Failed: XFAIL (in: first step)
91 When second step passes ... passed
92 Then third step fails ... failed
93 Assertion Failed: XFAIL (in: third step)
94 """
95 But note that "step execution continues after failed step(s)"
96 And note that "no steps are skipped"
97
98
99 Scenario: Runner continues after failed step in some scenarios
100
101 Enable this feature only on tagged scenarios (or features).
102
103 Given a file named "features/environment.py" with:
104 """
105 def before_scenario(context, scenario):
106 if "runner.continue_after_failed_step" in scenario.effective_tags:
107 scenario.continue_after_failed_step = True
108 """
109 When I run "behave -f plain -T features/alice.feature"
110 Then it should fail with:
111 """
112 Failing scenarios:
113 features/alice.feature:7 Fails in second step
114 features/alice.feature:13 Fails in first and third step
115
116 0 features passed, 1 failed, 0 skipped
117 1 scenario passed, 2 failed, 0 skipped
118 5 steps passed, 3 failed, 1 skipped, 0 undefined
119 """
120 And the command output should contain:
121 """
122 Scenario: Fails in second step
123 Given a step passes ... passed
124 When second step fails ... failed
125 Assertion Failed: XFAIL (in: second step)
126
127 Scenario: Fails in first and third step
128 Given first step fails ... failed
129 Assertion Failed: XFAIL (in: first step)
130 When second step passes ... passed
131 Then third step fails ... failed
132 Assertion Failed: XFAIL (in: third step)
133 """
134 But note that "step execution continues after failed step in tagged scenario only"
135 And note that "some steps are skipped (in 2nd, untagged scenario)"
136