Debian Codegraph @

features/runner.stop_after_failure.feature

raw
1 @sequential
2 Feature: Runner should stop after first failure if --stop option is used
3
4 As a tester
5 To abort testing early (sometimes)
6 When the first failure occurs.
7
8
9 @setup
10 Scenario: Test Setup
11 Given a new working directory
12 And a file named "features/steps/steps.py" with:
13 """
14 from behave import step
15 import sys
16
17 @step('a step passes')
18 def step_passes(context):
19 pass
20
21 @step('a step fails')
22 def step_fails(context):
23 assert False, "XFAIL"
24 """
25 And a file named "features/alice_fails.feature" with:
26 """
27 Feature: Alice
28 Scenario: A1
29 Given a step passes
30 When a step passes
31 Then a step passes
32
33 Scenario: A2 fails
34 Given a step passes
35 When a step fails
36 Then a step passes
37
38 Scenario: A3
39 Given a step passes
40
41 Scenario: A4 fails
42 Given a step fails
43 """
44 And a file named "features/bob_passes.feature" with:
45 """
46 Feature: Bob
47 Scenario: B1
48 Given a step passes
49 When a step passes
50 Then a step passes
51
52 Scenario: B2 fails
53 Given a step passes
54 When a step passes
55 Then a step passes
56 """
57
58
59 Scenario: Stop running after first failure with one feature
60 When I run "behave -f plain -T --stop features/alice_fails.feature"
61 Then it should fail with:
62 """
63 Failing scenarios:
64 features/alice_fails.feature:7 A2 fails
65
66 0 features passed, 1 failed, 0 skipped
67 1 scenario passed, 1 failed, 0 skipped, 2 untested
68 4 steps passed, 1 failed, 1 skipped, 0 undefined, 2 untested
69 """
70 And the command output should contain:
71 """
72 Feature: Alice
73 Scenario: A1
74 Given a step passes ... passed
75 When a step passes ... passed
76 Then a step passes ... passed
77
78 Scenario: A2 fails
79 Given a step passes ... passed
80 When a step fails ... failed
81 Assertion Failed: XFAIL
82 """
83 But the command output should not contain:
84 """
85 Scenario: A3
86 """
87
88 Scenario: Stop running after first failure with several features (CASE 1)
89 When I run "behave -f plain -T --stop features/alice_fails.feature features/bob_passes.feature "
90 Then it should fail with:
91 """
92 Failing scenarios:
93 features/alice_fails.feature:7 A2 fails
94
95 0 features passed, 1 failed, 0 skipped, 1 untested
96 1 scenario passed, 1 failed, 0 skipped, 4 untested
97 4 steps passed, 1 failed, 1 skipped, 0 undefined, 8 untested
98 """
99
100 Scenario: Stop running after first failure with several features (CASE 2: Different order)
101 When I run "behave -f plain -T --stop features/bob_passes.feature features/alice_fails.feature"
102 Then it should fail with:
103 """
104 Failing scenarios:
105 features/alice_fails.feature:7 A2 fails
106
107 1 feature passed, 1 failed, 0 skipped
108 3 scenarios passed, 1 failed, 0 skipped, 2 untested
109 10 steps passed, 1 failed, 1 skipped, 0 undefined, 2 untested
110 """
111
112 Scenario: Stop running after first failure with several features (CASE 3: Use directory)
113 When I run "behave -f plain -T --stop features/"
114 Then it should fail with:
115 """
116 Failing scenarios:
117 features/alice_fails.feature:7 A2 fails
118
119 0 features passed, 1 failed, 0 skipped, 1 untested
120 1 scenario passed, 1 failed, 0 skipped, 4 untested
121 4 steps passed, 1 failed, 1 skipped, 0 undefined, 8 untested
122 """