Debian Codegraph @

features/logcapture.feature

raw
1 @logging
2 @capture
3 Feature: Capture log output
4
5 As a tester
6 I want that log output is captured
7 But log-records are only shown when failures/errors occur
8 So that failure diagnostics are simplified
9
10 . SPECIFICATION:
11 . * log_capture mode is enabled per default
12 . * log_capture mode can be defined on command-line
13 . * log_capture mode can be defined in behave configuration file
14 . * In log_capture mode: Captured log-records are only shown if a scenario fails
15 .
16 . RELATED:
17 . * logcapture.*.feature
18 . * logging.*.feature
19
20 @setup
21 Scenario: Feature Setup
22 Given a new working directory
23 And a file named "features/steps/use_behave4cmd_steps.py" with:
24 """
25 import behave4cmd0.log.steps
26 import behave4cmd0.failing_steps
27 import behave4cmd0.passing_steps
28 """
29 And a file named "features/environment.py" with:
30 """
31 def before_all(context):
32 context.config.setup_logging()
33 # -- SAME-AS:
34 # import logging
35 # logging.basicConfig(level=context.config.logging_level)
36 """
37 And a file named "features/example.log_and_pass.feature" with:
38 """
39 Feature:
40 Scenario: Passing
41 Given I create log records with:
42 | category | level | message |
43 | root | FATAL | Hello Alice |
44 | foo | ERROR | Hello Bob |
45 | foo.bar | WARN | Hello Charly |
46 | bar | INFO | Hello Dora |
47 | baz | DEBUG | Hello Emily |
48 When another step passes
49 """
50 And a file named "features/example.log_and_fail.feature" with:
51 """
52 Feature:
53 Scenario: Failing
54 Given I create log records with:
55 | category | level | message |
56 | root | FATAL | Hello Alice |
57 | foo | ERROR | Hello Bob |
58 | foo.bar | WARN | Hello Charly |
59 | bar | INFO | Hello Dora |
60 | baz | DEBUG | Hello Emily |
61 When another step fails
62 """
63
64
65 Scenario: Captured log is suppressed if scenario passes
66 When I run "behave -f plain -T --logcapture features/example.log_and_pass.feature"
67 Then it should pass with:
68 """
69 1 scenario passed, 0 failed, 0 skipped
70 2 steps passed, 0 failed, 0 skipped, 0 undefined
71 """
72 And the command output should contain:
73 """
74 Scenario: Passing
75 Given I create log records with ... passed
76 | category | level | message |
77 | root | FATAL | Hello Alice |
78 | foo | ERROR | Hello Bob |
79 | foo.bar | WARN | Hello Charly |
80 | bar | INFO | Hello Dora |
81 | baz | DEBUG | Hello Emily |
82 When another step passes ... passed
83 """
84 And the command output should not contain:
85 """
86 Captured logging:
87 """
88 But the command output should not contain the following log records:
89 | category | level | message |
90 | root | FATAL | Hello Alice |
91 | foo | ERROR | Hello Bob |
92 | foo.bar | WARN | Hello Charly |
93 | bar | INFO | Hello Dora |
94 | baz | DEBUG | Hello Emily |
95
96
97 Scenario: Captured log is shown up to first failure if scenario fails
98 When I run "behave -f plain -T --logcapture features/example.log_and_fail.feature"
99 Then it should fail with:
100 """
101 0 scenarios passed, 1 failed, 0 skipped
102 1 step passed, 1 failed, 0 skipped, 0 undefined
103 """
104 And the command output should contain:
105 """
106 Feature:
107 Scenario: Failing
108 Given I create log records with ... passed
109 | category | level | message |
110 | root | FATAL | Hello Alice |
111 | foo | ERROR | Hello Bob |
112 | foo.bar | WARN | Hello Charly |
113 | bar | INFO | Hello Dora |
114 | baz | DEBUG | Hello Emily |
115 When another step fails ... failed
116 Assertion Failed: EXPECT: Failing step
117 Captured logging:
118 CRITICAL:root:Hello Alice
119 ERROR:foo:Hello Bob
120 WARNING:foo.bar:Hello Charly
121 INFO:bar:Hello Dora
122 """
123 And the command output should contain the following log records:
124 | category | level | message |
125 | root | FATAL | Hello Alice |
126 | foo | ERROR | Hello Bob |
127 | foo.bar | WARN | Hello Charly |
128 | bar | INFO | Hello Dora |
129 But the command output should not contain the following log records:
130 | category | level | message | Comment |
131 | baz | DEBUG | Hello Emily | Log-level too low: filtered-out |
132
133
134 Scenario: Captured log is shown up to first failure if log_record.level is not too low
135
136 Ensure that only log-records are shown that exceed the logging-level.
137
138 When I run "behave -f plain --logcapture --logging-level=ERROR features/example.log_and_fail.feature"
139 Then it should fail
140 And the command output should contain the following log records:
141 | category | level | message |
142 | root | FATAL | Hello Alice |
143 | foo | ERROR | Hello Bob |
144 But the command output should not contain the following log records:
145 | category | level | message |
146 | foo.bar | WARN | Hello Charly |
147 | bar | INFO | Hello Dora |
148 | baz | DEBUG | Hello Emily |
149
150
151 Scenario: Logcapture mode is enabled per default
152 When I run "behave -f plain features/example.log_and_fail.feature"
153 Then it should fail
154 And the command output should contain "Captured logging:"
155
156
157 Scenario: Logcapture mode can be enabled on command-line
158 When I run "behave -f plain --logcapture features/example.log_and_fail.feature"
159 Then it should fail
160 And the command output should contain "Captured logging:"
161
162
163 Scenario: Logcapture mode can be disabled on command-line
164 When I run "behave -f plain --no-logcapture features/example.log_and_fail.feature"
165 Then it should fail
166 And the command output should not contain "Captured logging:"
167
168
169 Scenario: Logcapture mode can be enabled in configfile
170 Given a file named "behave.ini" with:
171 """
172 [behave]
173 log_capture = true
174 """
175 When I run "behave -f plain features/example.log_and_fail.feature"
176 Then it should fail
177 And the command output should contain "Captured logging:"
178
179
180 Scenario: Logcapture mode can be disabled in configfile
181 Given a file named "behave.ini" with:
182 """
183 [behave]
184 log_capture = false
185 """
186 When I run "behave -f plain features/example.log_and_fail.feature"
187 Then it should fail
188 And the command output should not contain "Captured logging:"