Debian Codegraph @

features/runner.dry_run.feature

raw
1 @sequential
2 Feature: Runner should support a --dry-run option
3
4 As a tester
5 I want to check if behave tests are syntactically correct
6 And all step definitions exist
7 Before I actually run the tests (by executing steps).
8
9 . SPECIFICATION: Dry-run mode
10 . * Undefined steps are detected
11 . * Marks steps as "untested" or "undefined"
12 . * Marks scenarios as "untested"
13 . * Marks features as "untested"
14 . * Causes no failed scenarios, features
15 . * Causes failed test-run when undefined steps are found.
16
17 @setup
18 Scenario: Feature Setup
19 Given a new working directory
20 And a file named "features/steps/steps.py" with:
21 """
22 from behave import step
23
24 @step('a step passes')
25 def step_passes(context):
26 pass
27
28 @step('a step fails')
29 def step_fails(context):
30 assert False, "XFAIL"
31 """
32 And a file named "features/alice.feature" with:
33 """
34 Feature: Alice
35
36 @selected
37 Scenario: A1
38 Given a step passes
39 When a step passes
40 Then a step passes
41
42 @other_selected
43 Scenario: A2
44 Given a step passes
45 When a step fails
46 Then a step passes
47
48 @selected
49 Scenario: A3
50 Given a step passes
51
52 @selected
53 Scenario: A4
54 Given a step fails
55 """
56 And a file named "features/bob.feature" with:
57 """
58 Feature: Bob
59 Scenario: B1
60 Given a step passes
61 When a step passes
62 Then a step passes
63
64 Scenario: B2
65 Given a step passes
66 When a step fails
67 Then a step passes
68 """
69 And a file named "features/undefined_steps.feature" with:
70 """
71 Feature: Undefined Steps
72
73 @selected
74 Scenario: U1
75 Given a step passes
76 When a step is undefined
77 Then a step fails
78
79 @other_selected
80 Scenario: U2 fails
81 Given a step is undefined
82 When a step passes
83 And a step fails
84 Then a step is undefined
85 """
86
87
88 Scenario: Dry-run one feature should mark feature/scenarios/steps as untested
89 When I run "behave -f plain --dry-run features/alice.feature"
90 Then it should pass with:
91 """
92 0 features passed, 0 failed, 0 skipped, 1 untested
93 0 scenarios passed, 0 failed, 0 skipped, 4 untested
94 0 steps passed, 0 failed, 0 skipped, 0 undefined, 8 untested
95 """
96 And the command output should contain
97 """
98 Scenario: A1
99 """
100 And the command output should contain:
101 """
102 Scenario: A2
103 """
104 And the command output should contain:
105 """
106 Scenario: A3
107 """
108 And the command output should contain:
109 """
110 Scenario: A4
111 """
112 And note that "all scenarios of this feature are contained"
113
114 Scenario: Dry-run one feature with tags should mark skipped scenario/steps as skipped
115 When I run "behave -f plain --dry-run --tags=@selected --no-skipped features/alice.feature"
116 Then it should pass with:
117 """
118 0 features passed, 0 failed, 0 skipped, 1 untested
119 0 scenarios passed, 0 failed, 1 skipped, 3 untested
120 0 steps passed, 0 failed, 3 skipped, 0 undefined, 5 untested
121 """
122 And the command output should contain:
123 """
124 Scenario: A1
125 """
126 And the command output should contain:
127 """
128 Scenario: A3
129 """
130 And the command output should contain:
131 """
132 Scenario: A4
133 """
134 But the command output should not contain:
135 """
136 Scenario: A2
137 """
138 And note that "only tagged scenarios of this feature are contained (3 of 4)"
139
140
141 Scenario: Dry-run two features
142 When I run "behave --dry-run features/alice.feature features/bob.feature"
143 Then it should pass with:
144 """
145 0 features passed, 0 failed, 0 skipped, 2 untested
146 0 scenarios passed, 0 failed, 0 skipped, 6 untested
147 0 steps passed, 0 failed, 0 skipped, 0 undefined, 14 untested
148 """
149
150 Scenario: Dry-run one feature with undefined steps
151 When I run "behave --dry-run features/undefined_steps.feature"
152 Then it should fail with:
153 """
154 0 features passed, 0 failed, 0 skipped, 1 untested
155 0 scenarios passed, 0 failed, 0 skipped, 2 untested
156 0 steps passed, 0 failed, 0 skipped, 3 undefined, 4 untested
157 """
158
159 Scenario: Dry-run two features, one with undefined steps
160 When I run "behave --dry-run features/alice.feature features/undefined_steps.feature"
161 Then it should fail with:
162 """
163 0 features passed, 0 failed, 0 skipped, 2 untested
164 0 scenarios passed, 0 failed, 0 skipped, 6 untested
165 0 steps passed, 0 failed, 0 skipped, 3 undefined, 12 untested
166 """
167
168 Scenario: Dry-run two features, one with undefined steps and use tags
169 When I run "behave --dry-run --tags=@selected features/alice.feature features/undefined_steps.feature"
170 Then it should fail with:
171 """
172 0 features passed, 0 failed, 0 skipped, 2 untested
173 0 scenarios passed, 0 failed, 2 skipped, 4 untested
174 0 steps passed, 0 failed, 7 skipped, 1 undefined, 7 untested
175 """
176
177 Scenario: Dry-run two features, one with undefined steps and use other tags
178 When I run "behave --dry-run --tags=@other_selected features/alice.feature features/undefined_steps.feature"
179 Then it should fail with:
180 """
181 0 features passed, 0 failed, 0 skipped, 2 untested
182 0 scenarios passed, 0 failed, 4 skipped, 2 untested
183 0 steps passed, 0 failed, 8 skipped, 2 undefined, 5 untested
184 """