Debian Codegraph @

features/tags.active_tags.feature

raw
1 Feature: Active Tags
2
3 As a test writer
4 I want that some tags are evaluated at runtime
5 So that some features/scenarios automatically excluded from the run-set.
6
7 . SPECIFICATION: Active tag
8 . * An active tag is evaluated at runtime.
9 . * An active tag is either enabled or disabled (decided by runtime decision logic).
10 . * A disabled active tag causes that its feature/scenario is excluded from the run-set.
11 . * If several active tags are used for a feature/scenario/scenario outline,
12 . the following tag logic is used:
13 .
14 . SIMPLIFIED ALGORITHM: For active-tag expression
15 .
16 . enabled := enabled(active-tag1) and enabled(active-tag2) and ...
17 .
18 . where all active-tags have a different category.
19 .
20 . REAL ALGORITHM:
21 . If multiple active-tags for the same catgory exist,
22 . then these active-tags need to be merged together into a tag_group.
23 .
24 . enabled := enabled(tag_group1) and enabled(tag_group2) and ...
25 . tag_group.enabled := enabled(tag1) or enabled(tag2) or ...
26 .
27 . where all the active-tags within a tag-group have the same category.
28 .
29 . ACTIVE TAG SCHEMA (dialect1):
30 . @use.with_{category}={value}
31 . @not.with_{category}={value}
32 .
33 . DEPRECATED: @only.with_{category}={value}
34 .
35 . ACTIVE TAG SCHEMA (dialect2):
36 . @active.with_{category}={value}
37 . @not_active.with_{category}={value}
38 .
39 . RATIONALE:
40 . Some aspects of the runtime environment are only known
41 . when the tests are running. Therefore, it many cases it is simpler
42 . to use such a mechanism as "active tags" instead of moving this decision
43 . to the build-script that runs the tests.
44 .
45 . This allows to automatically skip some tests (scenarios, features)
46 . that would otherwise fail anyway.
47 .
48 . NOTE:
49 . * DRY-RUN MODE: Hooks are not called in dry-run mode.
50
51
52 @setup
53 Scenario: Feature Setup
54 Given a new working directory
55 And a file named "features/steps/pass_steps.py" with:
56 """
57 from behave import step
58
59 @step('{word:w} step passes')
60 def step_passes(context, word):
61 pass
62
63 # -- REUSE: Step definitions.
64 from behave4cmd0 import note_steps
65 """
66 And a file named "features/environment.py" with:
67 """
68 from behave.tag_matcher import ActiveTagMatcher, setup_active_tag_values
69 import sys
70
71 # -- ACTIVE TAG SUPPORT: @use.with_{category}={value}, ...
72 active_tag_value_provider = {
73 "browser": "chrome",
74 "webserver": "apache",
75 }
76 active_tag_matcher = ActiveTagMatcher(active_tag_value_provider)
77
78 # -- BEHAVE-HOOKS:
79 def before_all(context):
80 setup_active_tag_values(active_tag_value_provider, context.config.userdata)
81
82 def before_scenario(context, scenario):
83 if active_tag_matcher.should_exclude_with(scenario.effective_tags):
84 sys.stdout.write("ACTIVE-TAG DISABLED: Scenario %s\n" % scenario.name)
85 scenario.skip(active_tag_matcher.exclude_reason)
86 """
87 And a file named "behave.ini" with:
88 """
89 [behave]
90 default_format = pretty
91 show_timings = no
92 show_skipped = no
93 color = no
94 """
95
96 Scenario: Use active-tag with Scenario and one category (tag schema dialect2)
97 Given a file named "features/e1.active_tags.feature" with:
98 """
99 Feature:
100
101 @use.with_browser=chrome
102 Scenario: Alice (Run only with Web-Browser Chrome)
103 Given a step passes
104 When another step passes
105
106 @only.with_browser=safari
107 Scenario: Bob (Run only with Web-Browser Safari)
108 Given some step passes
109 """
110 When I run "behave -f plain features/e1.active_tags.feature"
111 Then it should pass with:
112 """
113 1 scenario passed, 0 failed, 1 skipped
114 2 steps passed, 0 failed, 1 skipped, 0 undefined
115 """
116 And the command output should contain:
117 """
118 ACTIVE-TAG DISABLED: Scenario Bob
119 """
120 But the command output should not contain:
121 """
122 ACTIVE-TAG DISABLED: Scenario Alice
123 """
124
125 Scenario: Use tag schema dialect1 with several categories
126 Given a file named "features/e2.active_tags.feature" with:
127 """
128 Feature:
129
130 @active.with_webserver=apache
131 Scenario: Alice (Run only with Apache Web-Server)
132 Given a step passes
133 When another step passes
134
135 @active.with_browser=safari
136 Scenario: Bob (Run only with Safari Web-Browser)
137 Given another step passes
138 """
139 When I run "behave -f plain -D browser=firefox features/e2.active_tags.feature"
140 Then it should pass with:
141 """
142 1 scenario passed, 0 failed, 1 skipped
143 2 steps passed, 0 failed, 1 skipped, 0 undefined
144 """
145 And the command output should contain:
146 """
147 ACTIVE-TAG DISABLED: Scenario Bob
148 """
149 But the command output should not contain:
150 """
151 ACTIVE-TAG DISABLED: Scenario Alice
152 """
153
154 Scenario Outline: Use active tags with positive and negative logic: <browser>
155 Given a file named "features/e3.active_tags.feature" with:
156 """
157 Feature:
158
159 @active.with_browser=chrome
160 Scenario: Alice
161 Given a step passes
162 When another step passes
163
164 @not_active.with_browser=opera
165 Scenario: Bob
166 Given another step passes
167
168 @use.with_browser=safari
169 Scenario: Charly
170 Given some step passes
171 """
172 When I run "behave -f plain -D browser=<browser> features/e3.active_tags.feature"
173 Then it should pass with:
174 """
175 ACTIVE-TAG DISABLED: Scenario <disabled scenario>
176 """
177 And the command output should not contain:
178 """
179 ACTIVE-TAG DISABLED: Scenario <enabled scenario>
180 """
181 And note that "<comment>"
182
183 Examples:
184 | browser | enabled scenario | disabled scenario | comment |
185 | chrome | Alice | Charly | Bob is also enabled |
186 | firefox | Bob | Alice | Charly is also disabled |
187 | safari | Charly | Alice | Bob is also enabled |
188
189
190 Scenario: Tag logic with one active tag
191 Given I setup the current values for active tags with:
192 | category | value |
193 | foo | xxx |
194 Then the following active tag combinations are enabled:
195 | tags | enabled? |
196 | @active.with_foo=xxx | yes |
197 | @active.with_foo=other | no |
198 | @not_active.with_foo=xxx | no |
199 | @not_active.with_foo=other | yes |
200 And the following active tag combinations are enabled:
201 | tags | enabled? |
202 | @use.with_foo=xxx | yes |
203 | @use.with_foo=other | no |
204 | @only.with_foo=xxx | yes |
205 | @only.with_foo=other | no |
206 | @not.with_foo=xxx | no |
207 | @not.with_foo=other | yes |
208
209
210 Scenario: Tag logic with two active tags
211 Given I setup the current values for active tags with:
212 | category | value |
213 | foo | xxx |
214 | bar | zzz |
215 Then the following active tag combinations are enabled:
216 | tags | enabled? |
217 | @use.with_foo=xxx @use.with_bar=zzz | yes |
218 | @use.with_foo=xxx @use.with_bar=other | no |
219 | @use.with_foo=other @use.with_bar=zzz | no |
220 | @use.with_foo=other @use.with_bar=other | no |
221 | @not.with_foo=xxx @use.with_bar=zzz | no |
222 | @not.with_foo=xxx @use.with_bar=other | no |
223 | @not.with_foo=other @use.with_bar=zzz | yes |
224 | @not.with_foo=other @use.with_bar=other | no |
225 | @use.with_foo=xxx @not.with_bar=zzz | no |
226 | @use.with_foo=xxx @not.with_bar=other | yes |
227 | @use.with_foo=other @not.with_bar=zzz | no |
228 | @use.with_foo=other @not.with_bar=other | no |
229 | @not.with_foo=xxx @not.with_bar=zzz | no |
230 | @not.with_foo=xxx @not.with_bar=other | no |
231 | @not.with_foo=other @not.with_bar=zzz | no |
232 | @not.with_foo=other @not.with_bar=other | yes |
233
234
235 Scenario: Tag logic with two active tags of same category
236 Given I setup the current values for active tags with:
237 | category | value |
238 | foo | xxx |
239 Then the following active tag combinations are enabled:
240 | tags | enabled? | Comment |
241 | @use.with_foo=xxx @use.with_foo=other | yes | Enabled: tag1 |
242 | @use.with_foo=xxx @not.with_foo=other | yes | Enabled: tag1, tag2|
243 | @use.with_foo=xxx @not.with_foo=xxx | yes | Enabled: tag1 (BAD-SPEC) |
244 | @use.with_foo=other @not.with_foo=xxx | no | Enabled: none |
245 | @not.with_foo=other @not.with_foo=xxx | yes | Enabled: tag1 |
246
247
248 Scenario: Tag logic with unknown categories (case: ignored)
249
250 Unknown categories (where a value is missing) are ignored by default.
251 Therefore, the active tag that uses an unknown category acts
252 as if it is always enabled (even in the negated case).
253
254 Given I setup the current values for active tags with:
255 | category | value |
256 | foo | xxx |
257 When unknown categories are ignored in active tags
258 Then the following active tag combinations are enabled:
259 | tags | enabled? |
260 | @use.with_unknown=xxx | yes |
261 | @use.with_unknown=zzz | yes |
262 | @not.with_unknown=xxx | yes |
263 | @not.with_unknown=zzz | yes |
264 But note that "the active tag with the unknown category acts like a passive tag"
265
266
267 Scenario: Tag logic with unknown categories (case: not ignored)
268
269 If unknown categories are not ignored, then the active tag is disabled.
270
271 Given I setup the current values for active tags with:
272 | category | value |
273 | foo | xxx |
274 When unknown categories are not ignored in active tags
275 Then the following active tag combinations are enabled:
276 | tags | enabled? |
277 | @use.with_unknown=xxx | no |
278 | @use.with_unknown=zzz | no |
279 | @not.with_unknown=xxx | yes |
280 | @not.with_unknown=zzz | yes |
281 But note that "the active tag with the unknown category is disabled"
282
283
284 Scenario: ScenarioOutline with enabled active-tag is executed
285 Given a file named "features/outline1.active_tags.feature" with:
286 """
287 Feature:
288
289 @use.with_browser=chrome
290 Scenario Outline: Alice -- <name>, <language>
291 Given a step passes
292 But note that "<name> can speak <language>"
293
294 Examples:
295 | name | language |
296 | Anna | German |
297 | Arabella | English |
298 """
299 When I run "behave -D browser=chrome features/outline1.active_tags.feature"
300 Then it should pass with:
301 """
302 2 scenarios passed, 0 failed, 0 skipped
303 4 steps passed, 0 failed, 0 skipped, 0 undefined
304 """
305 And the command output should contain:
306 """
307 @use.with_browser=chrome
308 Scenario Outline: Alice -- Anna, German -- @1.1 # features/outline1.active_tags.feature:10
309 Given a step passes # features/steps/pass_steps.py:3
310 But note that "Anna can speak German" # ../behave4cmd0/note_steps.py:15
311
312 @use.with_browser=chrome
313 Scenario Outline: Alice -- Arabella, English -- @1.2 # features/outline1.active_tags.feature:11
314 Given a step passes # features/steps/pass_steps.py:3
315 But note that "Arabella can speak English" # ../behave4cmd0/note_steps.py:15
316 """
317 And the command output should not contain "ACTIVE-TAG DISABLED: Scenario Alice"
318 But note that "we check now that tags for example rows are generated correctly"
319 And the command output should not contain "@use.with_browserchrome"
320 But the command output should contain "@use.with_browser=chrome"
321
322
323 Scenario: ScenarioOutline with disabled active-tag is skipped
324 Given a file named "features/outline2.active_tags.feature" with:
325 """
326 Feature:
327
328 @use.with_browser=chrome
329 Scenario Outline: Bob -- <name>, <language>
330 Given some step passes
331 But note that "<name> can speak <language>"
332
333 Examples:
334 | name | language |
335 | Bernhard | French |
336 """
337 When I run "behave -D browser=other features/outline2.active_tags.feature"
338 Then it should pass with:
339 """
340 0 scenarios passed, 0 failed, 1 skipped
341 0 steps passed, 0 failed, 2 skipped, 0 undefined
342 """
343 And the command output should contain:
344 """
345 ACTIVE-TAG DISABLED: Scenario Bob -- Bernhard, French -- @1.1
346 """
347
348 Scenario: ScenarioOutline with generated active-tag
349 Given a file named "features/outline3.active_tags.feature" with:
350 """
351 Feature:
352
353 @use.with_browser=<browser>
354 Scenario Outline: Charly -- <name>, <language>, <browser>
355 Given a step passes
356 But note that "<name> can speak <language>"
357
358 Examples:
359 | name | language | browser | Comment |
360 | Anna | German | firefox | Should be skipped when browser=chrome |
361 | Arabella | English | chrome | Should be executed when browser=chrome |
362 """
363 When I run "behave -D browser=chrome features/outline3.active_tags.feature"
364 Then it should pass with:
365 """
366 1 scenario passed, 0 failed, 1 skipped
367 2 steps passed, 0 failed, 2 skipped, 0 undefined
368 """
369 And the command output should contain:
370 """
371 ACTIVE-TAG DISABLED: Scenario Charly -- Anna, German, firefox -- @1.1
372 """
373 And the command output should not contain:
374 """
375 ACTIVE-TAG DISABLED: Scenario Charly -- Arabella, English, chrome -- @1.2
376 """
377 And the command output should contain:
378 """
379 ACTIVE-TAG DISABLED: Scenario Charly -- Anna, German, firefox -- @1.1
380
381 @use.with_browser=chrome
382 Scenario Outline: Charly -- Arabella, English, chrome -- @1.2 # features/outline3.active_tags.feature:11
383 Given a step passes # features/steps/pass_steps.py:3
384 But note that "Arabella can speak English" # ../behave4cmd0/note_steps.py:15
385 """