Debian Codegraph @

features/formatter.progress3.feature

raw
1 @sequential
2 Feature: Progress3 Formatter
3
4 In order to effectively analyze output of many runs
5 As a tester
6 I want that behave generates to present results for each scenario
7
8 @setup
9 Scenario: Feature Setup
10 Given a new working directory
11 And a file named "features/steps/steps.py" with:
12 """
13 from behave import step
14
15 @step('{word:w} step passes')
16 def step_passes(context, word):
17 pass
18
19 @step('{word:w} step fails')
20 def step_fails(context, word):
21 assert False, "XFAIL-STEP"
22
23 @step(u'a step raises an error "{message}"')
24 def step_raises_exception(context, message):
25 raise RuntimeError(message)
26 """
27
28 Scenario: Use Progress3 formatter on simple feature
29 Given a file named "features/simple_feature_with_name.feature" with:
30 """
31 Feature: Simple, empty Feature
32 """
33 When I run "behave -f progress3 features/simple_feature_with_name.feature"
34 Then it should pass with:
35 """
36 0 features passed, 0 failed, 1 skipped
37 0 scenarios passed, 0 failed, 0 skipped
38 """
39 And the command output should contain:
40 """
41 Simple, empty Feature # features/simple_feature_with_name.feature
42 """
43
44 Scenario: Use Progress3 formatter with feature and one scenario without steps
45 Given a file named "features/simple_scenario.feature" with:
46 """
47 Feature: Simple feature with one scenario
48 Scenario: Simple scenario without steps
49 """
50 When I run "behave -f progress3 features/simple_scenario.feature"
51 Then it should pass with:
52 """
53 1 feature passed, 0 failed, 0 skipped
54 1 scenario passed, 0 failed, 0 skipped
55 """
56 And the command output should contain:
57 """
58 Simple feature with one scenario # features/simple_scenario.feature
59 Simple scenario without steps
60 """
61
62 Scenario: Use Progress3 formatter with feature and one scenario with all passing steps
63 Given a file named "features/scenario_with_steps.feature" with:
64 """
65 Feature: Feature with scenario
66 Scenario: Simple scenario with passing steps
67 Given a step passes
68 When another step passes
69 Then third step passes
70 And fourth step passes
71 But fifth step passes
72 """
73 When I run "behave -f progress3 features/scenario_with_steps.feature"
74 Then it should pass with:
75 """
76 1 feature passed, 0 failed, 0 skipped
77 1 scenario passed, 0 failed, 0 skipped
78 """
79 And the command output should contain:
80 """
81 Feature with scenario # features/scenario_with_steps.feature
82 Simple scenario with passing steps .....
83 """
84
85 Scenario: Use Progress3 formatter with feature and one scenario with a failing step
86 Given a file named "features/scenario_with_steps.feature" with:
87 """
88 Feature: Feature with scenario
89 Scenario: Simple scenario with last failing step
90 Given a step passes
91 When second step passes
92 Then third step passes
93 And another step passes
94 But last step fails
95 """
96 When I run "behave -f progress3 features/scenario_with_steps.feature"
97 Then it should fail with:
98 """
99 0 features passed, 1 failed, 0 skipped
100 0 scenarios passed, 1 failed, 0 skipped
101 """
102 And the command output should contain:
103 """
104 Feature with scenario # features/scenario_with_steps.feature
105 Simple scenario with last failing step ....F
106 --------------------------------------------------------------------------------
107 FAILURE in step 'last step fails' (features/scenario_with_steps.feature:7):
108 Assertion Failed: XFAIL-STEP
109 --------------------------------------------------------------------------------
110 """
111
112 Scenario: Use Progress3 formatter with feature and one scenario with an exception in the step
113 Given a file named "features/scenario_with_steps.feature" with:
114 """
115 Feature: Feature with scenario
116 Scenario: Simple scenario with error in the step
117 Given a step passes
118 When second step passes
119 Then third step passes
120 And another step passes
121 But a step raises an error "Error message here"
122 """
123 When I run "behave -f progress3 features/scenario_with_steps.feature"
124 Then it should fail with:
125 """
126 0 features passed, 1 failed, 0 skipped
127 0 scenarios passed, 1 failed, 0 skipped
128 """
129 And the command output should contain:
130 """
131 Feature with scenario # features/scenario_with_steps.feature
132 Simple scenario with error in the step ....E
133 --------------------------------------------------------------------------------
134 FAILURE in step 'a step raises an error "Error message here"' (features/scenario_with_steps.feature:7):
135 """
136 And the command output should contain:
137 """
138 RuntimeError: Error message here
139
140 --------------------------------------------------------------------------------
141 """
142
143 Scenario: Use Progress3 formatter with feature and three scenarios with all passing steps
144 Given a file named "features/scenario_with_steps.feature" with:
145 """
146 Feature: Feature with three scenarios
147 Scenario: First scenario with passing steps
148 Given a step passes
149 When second step passes
150 Then third step passes
151 And another step passes
152 But last step passes
153
154 Scenario: Second scenario with passing steps
155 Given a step passes
156 When second step passes
157 Then third step passes
158 And another step passes
159 But last step passes
160
161 Scenario: Third scenario with passing steps
162 Given a step passes
163 When second step passes
164 Then third step passes
165 And another step passes
166 But last step passes
167 """
168 When I run "behave -f progress3 features/scenario_with_steps.feature"
169 Then it should pass with:
170 """
171 1 feature passed, 0 failed, 0 skipped
172 3 scenarios passed, 0 failed, 0 skipped
173 15 steps passed, 0 failed, 0 skipped, 0 undefined
174 """
175 And the command output should contain:
176 """
177 Feature with three scenarios # features/scenario_with_steps.feature
178 First scenario with passing steps .....
179 Second scenario with passing steps .....
180 Third scenario with passing steps .....
181 """
182
183 Scenario: Use Progress3 formatter with feature and three scenarios with a failing step
184 Given a file named "features/scenario_with_steps.feature" with:
185 """
186 Feature: Feature with various results in scenarios
187 Scenario: First scenario with passing steps
188 Given a step passes
189 When second step passes
190 Then third step passes
191 And another step passes
192 But last step passes
193
194 Scenario: Second scenario with second failing step
195 Given a step passes
196 When second step fails
197 Then third step passes
198 And another step passes
199 But last step passes
200
201 Scenario: Third scenario with fourth failing step
202 Given a step passes
203 When second step passes
204 Then third step passes
205 And fourth step fails
206 But last step passes
207 """
208 When I run "behave -f progress3 features/scenario_with_steps.feature"
209 Then it should fail with:
210 """
211 0 features passed, 1 failed, 0 skipped
212 1 scenario passed, 2 failed, 0 skipped
213 9 steps passed, 2 failed, 4 skipped, 0 undefined
214 """
215 And the command output should contain:
216 """
217 Feature with various results in scenarios # features/scenario_with_steps.feature
218 First scenario with passing steps .....
219 Second scenario with second failing step .F
220 --------------------------------------------------------------------------------
221 FAILURE in step 'second step fails' (features/scenario_with_steps.feature:11):
222 Assertion Failed: XFAIL-STEP
223 --------------------------------------------------------------------------------
224
225 Third scenario with fourth failing step ...F
226 --------------------------------------------------------------------------------
227 FAILURE in step 'fourth step fails' (features/scenario_with_steps.feature:20):
228 Assertion Failed: XFAIL-STEP
229 --------------------------------------------------------------------------------
230
231
232 Failing scenarios:
233 features/scenario_with_steps.feature:9 Second scenario with second failing step
234 features/scenario_with_steps.feature:16 Third scenario with fourth failing step
235 """