Debian Codegraph @

issue.features/issue0064.feature

raw
1 @issue
2 Feature: Issue #64 Exit status not set to 1 even there are failures in certain cases
3
4 The behave exit status not always returns 1 when failure(s) occur.
5 The problem was associated with the Feature.run() logic implementation.
6
7 This problem was first discovered while verifying issue #52 (see comments).
8 See also similar test when tags select a subset of scenarios.
9
10 RELATED ISSUES:
11 * issue #52
12
13 Background: Test Setup
14 Given a new working directory
15 Given a file named "features/steps/steps.py" with:
16 """
17 from behave import given
18
19 @given(u'passing')
20 def step(context):
21 pass
22
23 @given(u'failing')
24 def step(context):
25 assert False, "failing"
26 """
27
28 Scenario: Failing in First Scenario
29 Given a file named "features/issue64_case1.feature" with:
30 """
31 Feature: Failing in First Scenario
32 Scenario:
33 Given failing
34
35 Scenario:
36 Given passing
37 """
38 When I run "behave --format=plain features/issue64_case1.feature"
39 Then it should fail with:
40 """
41 0 features passed, 1 failed, 0 skipped
42 1 scenario passed, 1 failed, 0 skipped
43 """
44
45 Scenario: Failing in Middle Scenario
46 Given a file named "features/issue64_case2.feature" with:
47 """
48 Feature: Failing in Middle Scenario
49 Scenario:
50 Given passing
51 Scenario:
52 Given failing
53 Scenario:
54 Given passing
55 """
56 When I run "behave --format=plain features/issue64_case2.feature"
57 Then it should fail with:
58 """
59 0 features passed, 1 failed, 0 skipped
60 2 scenarios passed, 1 failed, 0 skipped
61 """
62
63 Scenario: Failing in Last Scenario
64 Given a file named "features/issue64_case3.feature" with:
65 """
66 Feature: Failing in Last Scenario
67 Scenario:
68 Given passing
69 Scenario:
70 Given passing
71 Scenario:
72 Given failing
73 """
74 When I run "behave --format=plain features/issue64_case3.feature"
75 Then it should fail with:
76 """
77 0 features passed, 1 failed, 0 skipped
78 2 scenarios passed, 1 failed, 0 skipped
79 """
80
81 Scenario: Failing in First and Last Scenario
82 Given a file named "features/issue64_case4.feature" with:
83 """
84 Feature: Failing in First and Last Scenario
85 Scenario:
86 Given failing
87 Scenario:
88 Given passing
89 Scenario:
90 Given failing
91 """
92 When I run "behave --format=plain features/issue64_case4.feature"
93 Then it should fail with:
94 """
95 0 features passed, 1 failed, 0 skipped
96 1 scenario passed, 2 failed, 0 skipped
97 """