Debian Codegraph @

CHANGES

raw
1 0.72 | 2022-03-22 09:21:34 +0100
2
3 * Release 0.72.
4
5 0.71-4 | 2022-03-22 09:19:53 +0100
6
7 * Make test `duplication-selection` independent of actual
8 scheduling. (Benjamin Bannier, Corelight)
9
10 0.71-2 | 2022-02-02 12:47:42 +0100
11
12 * Avoid assertion failure if same test required multiple times.
13 (Benjamin Bannier, Corelight)
14
15 0.71 | 2021-11-01 12:04:45 -0700
16
17 * Release 0.71.
18
19 0.70-6 | 2021-11-01 12:03:30 -0700
20
21 * Add PartInitializer option (Christian Kreibich, Corelight)
22
23 * Support for global and test-part teardowns (Christian Kreibich, Corelight)
24
25 * Remove Python 2.7 from manifest (Christian Kreibich, Corelight)
26
27 0.70 | 2021-10-28 12:14:34 -0700
28
29 * Release 0.70.
30
31 0.69-36 | 2021-10-28 12:12:47 -0700
32
33 * Prevent multiprocessing state leaks into toplevel .tmp directory (Benjmain
34 Bannier, Corelight)
35
36 * Check whether post-testing .tmp is empty (Benjamin Bannier and Christian
37 Kreibich, Corelight)
38
39 * Clean up registered temp files in btest-rst-cmd (Christian Kreibich,
40 Corelight)
41
42 * Add temp directory to the statefile-sorted test (Christian Kreibich,
43 Corelight)
44
45 0.69-31 | 2021-10-22 13:29:57 +0200
46
47 * Remove use of Python 3.7 feature `time.time_ns`. (Benjamin
48 Bannier, Corelight)
49
50 * Use appropiate Sphinx versions for different Python versions.
51 (Benjamin Bannier, Corelight)
52
53 * GH-66: Fix typo preventing use of configured Python version in CI.
54 (Benjamin Bannier, Corelight)
55
56 0.69-27 | 2021-10-19 13:33:43 -0700
57
58 * Remove uses of deprecated `distutils` package. (Benjamin Bannier, Corelight)
59
60 0.69-25 | 2021-10-08 11:55:13 +0200
61
62 * Improve backtick expansion in `btest.cfg`. Instead of naive
63 splitting of the command string into arguments, pass the command
64 as-is to a subshell for parsing. Note that this changes semantics
65 of backticks slightly. However, they were ill-defined to begin
66 with. (DJ Gregor, Corelight)
67
68 * Minor README tweaks. (DJ Gregor, Corelight)
69
70 0.69-22 | 2021-09-24 10:57:31 +0200
71
72 * Add pre-commit configuration. (Benjamin Bannier)
73
74 * Reformat Python scripts with yapf. (Benjamin Bannier)
75
76 * Reformat shell scripts with shellfmt. (Benjamin Bannier)
77
78 * Fix a couple issues tagged by shellcheck. (Benjamin Bannier)
79
80 * More precise environment filtering in
81 local-alternative-show-env.test. (Christian Kreibich, Corelight)
82
83 * Fix parallel/interleaved execution between the test wrapup
84 functions (such as testSucceeded) and the progress monitor.
85 (Christian Kreibich, Corelight)
86
87 * Switch Sphinx build test to use "text" output format. (Christian
88 Kreibich, Corelight)
89
90 * Add GitHub action exercising the test suite. (Benjamin Bannier,
91 Corelight)
92
93 * Make `btest-progress` more robust under concurrency. (Benjamin
94 Bannier, Corelight)
95
96 * Fix test `tests.sphinx.run-sphinx` for sphinx-3.5.4. (Benjamin
97 Bannier, Corelight)
98
99 * Show verbose failures when running test target. (Benjamin Bannier,
100 Corelight)
101
102 * Fix misspelled variable name in `btest-sphinx.py`. (Benjamin
103 Bannier, Corelight)
104
105 0.69-4 | 2021-08-10 08:25:39 +0200
106
107 * Add flag `-l` to print available tests. (Benjamin Bannier,
108 Corelight)
109
110 0.69-2 | 2021-08-04 18:12:33 +0200
111
112 * Always print a sorted list of failed tests to the state file.
113 (Benjamin Bannier, Corelight)
114
115 0.69 | 2021-06-07 09:33:51 +0200
116
117 * Release 0.69.
118
119 0.68-9 | 2021-06-07 09:31:03 +0200
120
121 * Documentation updates. (Christian Kreibich, Corelight)
122
123 - Add a section on running btest, to give some basic context.
124 - Add more detail on btest invocation and test selection.
125 - Update docs for download and installation, shifting focus to PyPI.
126 - Move TEST-START-NEXT description to its section.
127
128 0.68-4 | 2021-06-01 18:34:04 +0200
129
130 * Alternatives: Update `baselinedir` default after potentially
131 changing testbase. (Arne Welzel, Corelight)
132
133 0.68-2 | 2021-06-01 08:53:42 +0200
134
135 * Fix issue where global configuration object wasn't updated for
136 environment variables taken from alternatives. (Arne Welzel,
137 Corelight)
138
139 * Ignore environment variable when overridden by alternative. (Arne
140 Welzel, Corelight)
141
142 0.68 | 2021-04-16 16:21:36 -0700
143
144 * Release 0.68.
145
146 0.67-2 | 2021-04-16 16:21:14 -0700
147
148 * Preserve CRLF line-terminators in test files (Jon Siwek, Corelight)
149
150 Since the test itself may depend on their existence.
151
152 Related to https://github.com/zeek/zeek/issues/1497
153
154 0.67 | 2021-01-21 13:27:20 -0800
155
156 * Release 0.67.
157
158 0.66-2 | 2021-01-21 13:26:58 -0800
159
160 * Support BTest installation via CMake (Christian Kreibich, Corelight)
161
162 This copies the relevant files manually. Relying on setup.py is
163 feasible but requires careful separation of build and source
164 directories, so currently explicit file lists are easier to maintain.
165
166 0.66 | 2021-01-08 12:14:24 +0000
167
168 * Bug fix to apply TEST-REQUIRES to tests duplicated with TEST-START-NEXT.
169 (Robin Sommer, Corelight)
170
171 0.65 | 2020-12-07 09:32:03 +0000
172
173 * Release 0.65.
174
175 0.64-21 | 2020-12-07 09:01:13 +0000
176
177 * Default to `-j 1` to make CTRL-C work consistently. We now always use
178 multiprocessing with a single process, with one exception: interactive
179 baseline updates still run directly. (Christian Kreibich, Corelight)
180
181 * Avoid leaking processes due to lacking sync manager shutdown.
182 (Christian Kreibich, Corelight)
183
184 * Speed up the threading test since its built-in delays weren't actually
185 required for the test. (Christian Kreibich, Corelight)
186
187 0.64-18 | 2020-12-07 08:44:55 +0000
188
189 * Add "binary mode" to btest-diff. In binary mode, invoked with -b/--binary,
190 btest-diff compares test output and baselines for equality only, never
191 applies canonifiers, and doesn't apply our btest header when updating
192 baselines. (Christian Kreibich, Corelight)
193
194 0.64-16 | 2020-12-07 08:35:25 +0000
195
196 * GH-36: Fix --abort-on-failure for expected failures. (Robin Sommer,
197 Corelight)
198
199 0.64-14 | 2020-12-01 16:05:25 +0000
200
201 * Add support for multiple baseline directories. (Robin Sommer, Corelight)
202
203 This works be setting the environment variable BTEST_BASELINE_DIR to a
204 colon-separated list of directories. They will be searched in order when
205 looking a baseline file to compare against. Updating a baseline will always
206 put the new content into the first directory. One can now also generally
207 set a different baseline directory through BTEST_BASELINE_DIR. See the
208 README for more.
209
210 * Require Python >= 3.5. (Jon Siwek, Corelight)
211
212 0.64-1 | 2020-11-11 12:57:12 +0000
213
214 * Fix for Python 3. (Robin Sommer, Corelight)
215
216 0.64 | 2020-10-20 14:02:01 +0000
217
218 * Fix canonification problem. (Robin Sommer, Corelight)
219
220 0.63 | 2020-10-20 08:32:28 +0000
221
222 * Add support for minimum BTest version requirement in the config
223 file. If an entry called "MinVersion" is present in the "btest"
224 section, it spells out a minimum version that BTest must have in
225 order to run. If that condition isn't fulfilled, we exit with
226 error code 1 and a corresponding error message on stderr.
227 (Christian Kreibich, Corelight)
228
229 * GH-29: Add new options -F/--abort-on-failure that will abort once
230 at least one test has failed. (Robin Sommer, Corelight)
231
232 * Documentation and code cleanup. (Christian Kreibich, Corelight)
233
234 0.62-13 | 2020-10-09 07:33:49 +0000
235
236 * Canonify outputs when updating baselines via btest-diff.
237 (Christian Kreibich, Corelight)
238
239 This leverages the same canonicalization btest-diff already
240 applies during test output comparison against baselines also when
241 updating those baselines (usually via btest -U/-u). This removes a
242 bunch of noise from the baselines, including personal home
243 directories, timestamps, etc.
244
245 Since btest-diff doesn't know whether the baseline has undergone
246 this canonicalization, it continues to canonicalize the baseline
247 prior to diffing. To track whether it has canonicalized a baseline
248 when updating, btest-diff now also prepends a header to the
249 generated baseline that warns users about the fact that it is
250 auto-generated. The presence of this header doubles as a marker
251 for canonicalization.
252
253 * Clean up btest-diff's shell code. (Christian Kreibich, Corelight)
254
255 0.62-7 | 2020-09-24 07:47:26 -0700
256
257 * GH-26: Explicitly use multiprocessing "fork" start-method (Jon Siwek, Corelight)
258
259 The default start-method on macOS in Python 3.8+ is "spawn", but that
260 emits a RuntimeError with current btest structuring.
261
262 0.62-5 | 2020-08-25 07:57:23 +0000
263
264 * GH-11: Fix %DIR not being set correctly in cloned tests. (Jon
265 Siwek, Corelight)
266
267 * Sort XML attributes for more output stability across Python
268 versions. (Jon Siwek, Corelight)
269
270 * Use Sphinx logging API directly for compatibility with Sphinx
271 2.0+. (Jon Siwek, Corelight)
272
273 0.62 | 2020-05-08 15:07:11 +0000
274
275 * Release 0.62.
276
277 0.61-10 | 2020-05-08 15:05:46 +0000
278
279 * Catch keyboard interrupt to abort orderly and immediately. The
280 existing workers will still run to completion in the background.
281 (Benjamin Bannier, Corelight)
282
283 0.61-8 | 2020-04-28 10:00:41 +0000
284
285 * Add `--trace-file` option for recording Chrome trace file.
286 (Benjamin Bannier, Corelight)
287
288 0.61-2 | 2020-02-13 18:53:41 +0000
289
290 * Provide a more apt description for btest. (Christopher M. Hobbs,
291 Corelight)
292
293 0.61 | 2020-02-07 10:19:25 +0000
294
295 * Release 0.61.
296
297 0.6-4 | 2020-02-07 10:18:14 +0000
298
299 * Change --retries option to not increment test name on retries.
300 (Jon Siwek, Corelight)
301
302 * Fix retrying tests that use additional files. (Jon Siwek,
303 Corelight)
304
305 0.60 | 2020-01-17 09:13:56 +0000
306
307 * Show diagnostics for expected failures when --diagnostics-all is
308 used. (Robin Sommer, Corelight)
309
310 * Fix btest exit code to indicate success if only tests are failing
311 that are expected to. (Robin Sommer, Corelight)
312
313 * Extend XML test case to cover -j flag. (Jon Siwek, Corelight)
314
315 * Fix XML output option -x to work with -j. (Jon Siwek, Corelight)
316
317 0.59-12 | 2019-09-09 11:30:26 +0000
318
319 * Add timestamps to "btest-progress" output. They are added to
320 stderr only, so that they will get recorded but not displayed
321 during execution. The new option -T suppresses the timestamp. (DJ
322 Gregor, Corelight)
323
324 0.59-8 | 2019-08-16 16:15:41 +0000
325
326 * New option -z <RETRIES> to retry any failed tests a few times to
327 see if they might just be unstable. (Dev Bali, Corelight)
328
329 0.59-1 | 2019-08-09 09:11:32 -0700
330
331 * Update username in `make upload` target (Jon Siwek, Corelight)
332
333 0.59 | 2019-08-01 12:04:06 -0700
334
335 * Release 0.59.
336
337 0.58-12 | 2019-08-01 12:02:01 -0700
338
339 * Drop use of Python 2.6 for Travis CI tests (Jon Siwek, Corelight)
340
341 0.58-11 | 2019-06-27 16:54:17 +0000
342
343 * Adding clarity to difficult to understand error message. (Sam
344 Zaydel, Corelight)
345
346 0.58-9 | 2019-06-17 20:17:32 -0700
347
348 * Update Travis config for bro to zeek renaming (Daniel Thayer)
349
350 0.58-7 | 2019-05-24 16:16:07 +0000
351
352 * Use more portable platform.system() for determining platform name.
353 (woot4moo)
354
355 0.58-5 | 2018-12-07 16:32:52 -0600
356
357 * Update github/download link (Jon Siwek, Corelight)
358
359 0.58-4 | 2018-11-29 16:55:33 -0600
360
361 * Add TEST-PORT command and PortRange option (Jon Siwek, Corelight)
362
363 These control assignment of TCP ports to environment variables for
364 use during test execution. Helps in writing unit tests that can be
365 run concurrently or just in reducing the risk of a unit test failing
366 due to a port already being used by some external process.
367
368 0.58 | 2018-05-21 22:32:21 +0000
369
370 * Release 0.58.
371
372 0.57-32 | 2018-05-21 22:31:42 +0000
373
374 * Show number of skipped tests even when none fail. (Daniel Thayer)
375
376 * Delete a test's temporary directory if skipped, unless the user
377 wants these. (Corelight)
378
379 0.57-28 | 2018-05-08 10:03:20 -0500
380
381 * BIT-1735: open btest files as utf-8 if locale has no default encoding
382 (Corelight)
383
384 * Normalize output of a sphinx-related unit test (Corelight)
385
386 * Improve a unicode decode error message (Daniel Thayer)
387
388 0.57-24 | 2018-04-18 14:56:56 -0700
389
390 * Improving console output. (Robin Sommer)
391
392 - When showing a progress message, always clear to end of line
393 to delete any content that a previous, longer message may
394 have left.
395
396 - Ensure to turn the cursor back on at exit.
397
398 0.57-23 | 2018-03-15 14:58:20 -0700
399
400 * Configure Travis CI email recipients and build branches. (Daniel
401 Thayer)
402
403 0.57-21 | 2018-02-05 15:05:43 -0800
404
405 * Add a .travis.yml file. (Daniel Thayer)
406
407 0.57-19 | 2018-01-19 15:14:02 -0800
408
409 * Fix a bug when setting base dir using a relative path. Addresses
410 BIT-1892. (Daniel Thayer)
411
412 * Improve testing of setting a non-default base directory. (Daniel
413 Thayer)
414
415 0.57-16 | 2017-11-17 15:03:16 -0800
416
417 * Fix "btest -R" to preserve sorted output ordering. (Daniel Thayer)
418
419 * Add more tests to "doc.test". (Daniel Thayer)
420
421 0.57-13 | 2017-10-23 15:35:21 -0700
422
423 * Tweak -A|--show-all to use only coloring, not cursor navigation.
424 (Christian Kreibich)
425
426 * Fix the doc.test. (Daniel Thayer)
427
428 * Improve the console.test and document "--show-all" option. (Daniel
429 Thayer)
430
431 * Added documentation of the "--show-all" option. (Daniel Thayer)
432
433 * Allow multiple TEST-DOC keywords in a test file. (Daniel Thayer)
434
435 0.57-7 | 2017-10-16 12:18:28 -0700
436
437 * Fix the console.test to work on FreeBSD and macOS. (Christian
438 Kreibich/Daniel Thayer)
439
440 0.57-5 | 2017-10-06 15:01:23 -0700
441
442 * Additional control over TTY-based output handling This adds
443 -A|--show-all, which makes console output preserve output lines
444 for passing/skipped tests. (Christian Kreibich)
445
446 * Fix btest-rst-cmd script to remove tmp files. (Daniel Thayer)
447
448 * Added TMPDIR to btest.cfg so that temporary files are stored in a
449 local directory instead of a system-wide tmp directory. (Daniel
450 Thayer)
451
452 0.57 | 2017-05-15 16:13:33 -0700
453
454 * Release 0.57.
455
456 0.56-22 | 2017-05-15 16:13:23 -0700
457
458 * Fixing broken version numbers. (Robin Sommer)
459
460 0.56-21 | 2017-05-15 16:05:18 -0700
461
462 * Catching CTRL-C and cleaning up. (Robin Sommer)
463
464 0.56-20 | 2017-03-21 17:56:10 -0700
465
466 * Catching exception that wasn't caught. (Robin Sommer)
467
468 0.56-19 | 2017-03-03 12:50:42 -0800
469
470 * Fix btest-progress output to stderr when run from btest. (Daniel
471 Thayer)
472
473 0.56-17 | 2017-03-02 16:24:31 -0800
474
475 * Cosmectics for progress output: Delete it before asking for
476 baseline updates. (Robin Sommer)
477
478 * Fixing missing output for back-to-back btest-progress calls.
479 Addresses BIT-1800. (Robin Sommer)
480
481 * Fix for augmented output to console. (Robin Sommer)
482
483 * Send btest-progress output to stderr as well. (Robin Sommer)
484
485 0.56-13 | 2017-02-23 10:14:56 -0800
486
487 * Prevent socket path length from exceeding system limits. Addresses
488 BIT-862. (Daniel Thayer)
489
490 0.56-11 | 2017-02-03 12:38:01 -0800
491
492 * Adding btest-progress to setup.py. (Robin Sommer)
493
494 0.56-10 | 2017-01-25 13:04:07 -0800
495
496 * Fix a failing test on FreeBSD. (Daniel Thayer)
497
498 * Fix a bug in btest-progress when using the "-q" option. (Daniel
499 Thayer)
500
501 * Fix some trivial errors in documentation and Makefile. (Daniel
502 Thayer)
503
504 * Add 'upload' Makefile target to upload to PyPi. (Jon Siwek)
505
506 0.56-5 | 2017-01-24 08:45:29 -0800
507
508 * Bugfix for recent btest-progress changes. (Robin Sommer)
509
510 0.56-4 | 2017-01-23 19:59:59 -0800
511
512 * New utility btest-progress to display progress messages while a
513 test is executing. These messages appear in real-time while the
514 rest is still running. When stdout is a tty, the progress messages
515 are incorporated into the colored one-line status message. By
516 default, btest-progress also prints the message to a test's
517 standard output. That can be suppressed by giving it an option -q.
518 (Robin Sommer)
519
520 * Experimental automatic generation of test reference documentation.
521 The new command-line option "-R <format>" prints out a list of all
522 tests in either Markdown (format 'md') or reStructuredText (format
523 'rst'). The list includes a documentation string with each test
524 that gets defined through a new "@TEST-DOC: <docstring>"
525 directive. This is experimental. (Robin Sommer)
526
527 * Fix pylint warnings. (Robin Sommer)
528
529 0.56 | 2016-10-31 14:23:57 -0700
530
531 * Release 0.56.
532
533 0.55-6 | 2016-10-31 14:23:24 -0700
534
535 * Python 3 compatibility fixes for btest-sphinx.py. (Daniel Thayer)
536
537 0.55-4 | 2016-10-25 09:31:25 -0700
538
539 * Fix diff-max-lines.test to work on openbsd. (Daniel Thayer)
540
541 0.55-2 | 2016-10-10 08:18:54 -0700
542
543 * Fix the btest-rst-cmd script to work with Python 3. (Daniel
544 Thayer)
545
546 0.55 | 2016-02-23 14:02:35 -0800
547
548 * Release 0.55.
549
550 0.54-65 | 2016-02-23 14:00:10 -0800
551
552 * Fine-tuning diagnostic output. It needlessly stripped leading
553 whitespace. (Robin Sommer)
554
555 0.54-63 | 2016-02-07 19:39:54 -0800
556
557 * Extending --groups to allow running everything *except* a set of
558 groups. (Robin Sommer)
559
560 * Fix portability issue with use of mktemp. (Daniel Thayer)
561
562 0.54-60 | 2015-11-16 07:30:38 -0800
563
564 * Updates for Python 3. (Fabian Affolter)
565
566 0.54-58 | 2015-10-01 16:04:51 -0700
567
568 * Improved test of TEST_DIFF_FILE_MAX_LINES. (Daniel Thayer)
569
570 * Added ability for a user to override the default number of lines
571 to show for diffs by setting the environment variable
572 TEST_DIFF_FILE_MAX_LINES. Reduced the default to 100. (Daniel
573 Thayer)
574
575 * When no baseline exists, changed btest-diff to always just show
576 the entire file. (Daniel Thayer)
577
578 0.54-55 | 2015-08-25 07:47:22 -0700
579
580 * Port to Python 3. (Daniel Thayer)
581
582 * Various cleanup, bug fix, simplifications, and smaller
583 improvements. (Daniel Thayer)
584
585 * Improve and extend test suite substantially. (Daniel Thayer)
586
587 0.54-9 | 2015-07-03 18:21:52 -0700
588
589 * Make sure IgnoreDirs works with toplevel globbing. (Robin Sommer)
590
591 0.54-8 | 2015-07-03 16:31:24 -0700
592
593 * Expanding globs in TestDirs, relative to TestBase. (Robin Sommer)
594
595 0.54-7 | 2015-06-22 13:07:42 -0700
596
597 * Allow BTEST_TEST_BASE overriding in alternative configuration.
598 (Vlad Grigorescu)
599
600 * Create README symlink for GitHub rendering. (Vlad Grigorescu)
601
602 0.54-1 | 2015-06-18 09:08:34 -0700
603
604 * Add support for BTEST_TEST_BASE environment variable for
605 overriding the test base directory. (Robin Sommer)
606
607 0.54 | 2015-03-02 17:22:22 -0800
608
609 * Release 0.54.
610
611 0.53-6 | 2015-03-02 17:21:26 -0800
612
613 * Improve documentation of timing functionality. (Daniel Thayer)
614
615 * Add a new section to documentation that lists the BTest
616 prerequisites. (Daniel Thayer)
617
618 * Add warning when btest cannot create timing baseline. (Daniel
619 Thayer)
620
621 0.53-3 | 2015-01-22 07:25:01 -0800
622
623 * Fix some typos in the README. (Daniel Thayer)
624
625 0.53-1 | 2014-11-11 13:21:10 -0800
626
627 * In diagnostics, do not show verbose output for tests known to
628 fail. (Robin Sommer)
629
630 0.53 | 2014-07-22 17:36:24 -0700
631
632 * Release 0.53.
633
634 0.52-2 | 2014-07-22 17:36:15 -0700
635
636 * Update MANIFEST.in and setup.py to fix packaging. (Jon Siwek)
637
638 0.52 | 2014-03-13 14:05:44 -0700
639
640 * Release 0.52.
641
642 0.51-14 | 2014-03-13 14:05:36 -0700
643
644 * Fix a link in the README. (Jon Siwek)
645
646 0.51-12 | 2014-02-11 16:12:44 -0800
647
648 * Work-around for systems reporting that a socket path is too long.
649 Addresses BIT-862. (Robin Sommer)
650
651 0.51-11 | 2014-02-11 15:37:40 -0800
652
653 * Fix for Linux systems that have the perf tool but don't support
654 measuring instructions. (Robin Sommer)
655
656 * No longer tracking tests that are expected to fail in state file.
657 (Robin Sommer)
658
659 * Refactoring the timing code to no longer execute at all when not
660 needed.(Robin Sommer)
661
662 0.51-7 | 2014-02-06 21:06:40 -0800
663
664 * Fix for platforms that don't support timing measurements yet.
665 (Robin Sommer)
666
667 0.51-6 | 2014-02-06 18:19:08 -0800
668
669 * Adding a timing mode that records test execution times per host.
670 This is for catching regressions (or improvements :) that lets
671 execution times divert significantly. Linux only for now. See the
672 README for more information. (Robin Sommer)
673
674 * Adding color to test status when writing to console. (Robin Sommer)
675
676 * A bit of refactoring to define the status messages ("ok", "failed")
677 only at a single location.
678
679 Also added a note when a test declared as expecting failure in fact
680 succeeds. (Robin Sommer)
681
682
683 0.51-2 | 2013-11-17 20:21:08 -0800
684
685 * New keyword ``TEST-KNOWN-FAILURE`` to mark tests that are
686 currently known to fail. (Robin Sommer)
687
688 0.51-1 | 2013-11-11 13:36:36 -0800
689
690 * Fixing bug with tests potentially being ignored when using
691 alternatives. (Robin Sommer)
692
693 0.51 | 2013-10-07 17:29:50 -0700
694
695 * Updating copyright notice. (Robin Sommer)
696
697 0.5-1 | 2013-10-07 17:26:30 -0700
698
699 * Polishing how included commands and files are shown. (Robin Sommer)
700
701 - Enabling CSS styling to command lines and shown file names
702 via the new "btest-include" and "btest-cmd" classes.
703
704 - Fix to enable showing line numbers in btest-sphinx generated
705 output.
706
707 - Fix to enable Pygments coloring in output.
708
709 0.5 | 2013-09-20 14:48:01 -0700
710
711 * Fix the btest-rst-pipe script. (Daniel Thayer)
712
713 * A set of of documentation fixes, clarifications, and extensions.
714 (Daniel Thayer)
715
716 * A set of changes to Sphinx commands and directives. (Robin Sommer)
717
718 btest-rst-*:
719 - Always show line numbers.
720
721 - Highlight the command executed.
722
723 - rst-cmd-include gets an option -n <i> to include only upto i lines.
724
725 - rst-cmd-include prefixes output with "<file>" to show what we're
726 including.
727
728 btest-include:
729 - Set Pygments language automatically if we show a file with an
730 extension we know (in particular ".bro").
731
732 - Prefix output with "<file>" to show what we're including.
733
734 0.4-63 | 2013-08-28 21:10:39 -0700
735
736 * btest-sphinx now provides a new directive btest-include. This
737 works like literalinclude (with all its options) but it also saves
738 a version of the included text as a test to detect changes. (Robin
739 Sommer)
740
741 0.4-60 | 2013-08-28 18:54:51 -0700
742
743 * Fix typos and reST formatting in README (Daniel Thayer)
744
745 * Fix a couple of error messages. (Daniel Thayer)
746
747 * Fixed a reference to a non-existent variable which was causing the
748 "-w" option to have no effect. (Daniel Thayer)
749
750 * Test portability fix. (Robin Sommer)
751
752 0.4-55 | 2013-08-22 16:09:21 -0700
753
754 * New "Sphinx-mode" for BTest, activated with -S. This allows to
755 capture a test's diagnostic output when running from inside
756 Sphinx; the output will now be inserted into the generated
757 document. (Robin Sommer)
758
759 * Adding an option -n to btest-rst-cmd that truncates output longer
760 than N lines. (Robin Sommer)
761
762 * Adding a PartFinalizer that runs a commmand at the completion of
763 each test part. (Robin Sommer)
764
765 0.4-51 | 2013-08-22 10:36:34 -0700
766
767 * Improve cleanup of processes that don't terminate with
768 btest-bg-wait. (Jon Siwek)
769
770 0.4-49 | 2013-08-13 18:43:03 -0700
771
772 * Fixing test portability problems. (Daniel Thayer)
773
774 * Adding TEST_BASE environment variable. The existing TESTBASE isn't
775 always behaving as expected and wasn't documented to begin with.
776 (Robin Sommer)
777
778 0.4-43 | 2013-08-12 16:04:53 -0700
779
780 * Bugfix for ignored tests. (Robin Sommer)
781
782 0.4-42 | 2013-07-31 20:46:30 -0700
783
784 * Adding support for "parts": One can split a single test across
785 multiple files by adding a numerical ``#<n>`` postfix to their
786 names, where each ``<n>`` represents a separate part of the test.
787 ``btests`` will combine all of a test's parts in numerical order
788 and execute them subsequently within the same sandbox. Example in
789 the README. (Robin Sommer)
790
791 * When running a command, TEST_PART contains the current part
792 number. (Robin Sommer)
793
794 * Extending Sphinx support. (Robin Sommer)
795
796 * Adding tests for Sphinx functionality.
797
798 * Support for parts in Sphinx directives. If multiple btest
799 directives reference the same test name, each will turn into
800 a part of a single test.
801
802 * Internal change restructuring the btest Sphinx directive. We
803 now process it in two passes: one to save the test at parse
804 time, and one later to execute once everything has been
805 parsed.
806
807 * Adding Sphinx sandbox for testing.
808
809 * Fix for tests returning no output to render at all. (Robin Sommer)
810
811 0.4-28 | 2013-07-17 21:56:18 -0700
812
813 * btest-diff now passes the name of the file under consideration on to
814 canonifiers. (Robin Sommer)
815
816 0.4-27 | 2013-07-14 21:19:59 -0700
817
818 * When searching for tests, BTest now ignores a directories if it finds
819 a file ".btest-ignore" in there. (Robin Sommer)
820
821 0.4-26 | 2013-07-08 20:46:22 -0700
822
823 * Fixing bug with @TEST-START-NEXT naming. (Robin Sommer)
824
825 0.4-25 | 2013-07-08 13:25:50 -0700
826
827 * A test-suite for btest. Using, of course, btest. "make test" will
828 test most of btest's features. The main missing piece is testing
829 the Sphinx support, we will add that next. (Robin Sommer)
830
831 * When creating directories, we know also create intermediaries.
832 That in particular means that "@TEST-START-FILE a/b/c" now creates
833 a directory "a/b" automatically and puts the file in there. (Robin
834 Sommer)
835
836 * IgnoreDirs now also works for sub directories. (Robin Sommer)
837
838 * Documentation updates. (Robin Sommer)
839
840 * Adding "Initializer" option, which runs a command before each
841 test. (Robin Sommer)
842
843 * Adding "CommandPrefix" option that changes the naming of all btest
844 commands by replacing the "@TEST-" prefix with a custom string.
845 (Robin Sommer)
846
847 * Default configuration file can be overriden via BTEST_CFG
848 environment variable. (Robin Sommer)
849
850 * s/bro-ids.org/bro.org/g (Robin Sommer)
851
852 * Bugfix for -j without number. (Robin Sommer)
853
854 * New @TEST-ALTERNATIVE that activates tests only for the given
855 alternative. Renamed @TEST-NO-ALTERNATIVE to
856 @TEST-NOT-ALTERNATIVE, and allowing "default" for both
857 @TEST-ALTERNATIVE and @TEST-NOTALTERNATIVE to specify the case
858 that BTest runs without any alternative given. (Robin Sommer)
859
860 * Fix for alternative names containing white spaces. (Robin Sommer)
861
862 0.4-14 | 2013-01-23 18:11:22 -0800
863
864 * Fixing links in README and removing TODOs. (Robin Sommer)
865
866 0.4-13 | 2013-01-23 14:33:23 -0800
867
868 * Allowing use of -j without a value. BTest then uses the number of
869 CPU cores as reported by the OS. (Robin Sommer)
870
871 0.4-11 | 2013-01-21 17:50:40 -0800
872
873 * Adding a new "alternative" concept that combines filters and
874 substitutions, and adds per-alternative environment variables.
875 (Robin Sommer)
876
877 Instead of defining filters and substitutions separately, one now
878 specifies an alternative configuration to run with "-A <name>" and
879 that then checks for both "[substitutions-<name>]" and
880 "[filter-<name>]" section. In addition, "[environment-<name>]"
881 allows to define alternative-specific environment variables.
882
883 The old filter/substitutions options -F and -s are gone. The
884 sections for substitutions are renamed to "[substitutions-<name>]"
885 from "[subst-<name>]".
886
887 0.4-10 | 2013-01-07 09:45:35 -0800
888
889 * btest now sets a new environment variable TEST_VERBOSE, giving the
890 path of a file where a test can record further information about
891 its execution that will be included with btest's ``--verbose``
892 output. (Robin Sommer)
893
894 0.4-9 | 2012-12-20 12:20:44 -0800
895
896 * Documentation fixes/clarifications. (Daniel Thayer)
897
898 * Fix the btest "-c" option, which didn't work when the specified
899 config file was not in the current working directory. (Daniel
900 Thayer)
901
902 0.4-6 | 2012-11-08 16:33:51 -0800
903
904 * Putting a limit on how many input line btest-diff shows. (Robin
905 Sommer)
906
907 0.4-5 | 2012-11-01 16:14:29 -0700
908
909 * Making Sphinx module tolerant against docutils version change.
910 (Robin Sommer)
911
912 0.4-4 | 2012-09-25 06:24:59 -0700
913
914 * Fix a couple of reST formatting problems. (Daniel Thayer)
915
916 0.4-2 | 2012-09-24 11:41:06 -0700
917
918 * Add option -x to output test results in an XML (JUnit-like)
919 format. (Jon Siwek)
920
921 0.4 | 2012-06-15 15:15:13 -0700
922
923 * Remove code to expand environment variables on command line. (Not
924 needed because the command line is just passed to the shell.)
925 (Daniel Thayer)
926
927 * Clarify explanation about expansion of environment variables.
928 (Daniel Thayer)
929
930 * Fix errors in README and btest help output; added documentation
931 for the -q option. (Daniel Thayer)
932
933 * Fixed a bug in btest where it was looking for "filters-" (instead
934 of "filter-") in the btest config file. (Daniel Thayer)
935
936 0.31-45 | 2012-05-24 16:43:14 -0700
937
938 * Correct typos in documentation. (Daniel Thayer)
939
940 * Failed tests are now only recorded into the state file when we're
941 not updating. That allows to run "btest -r" repeatedly while
942 updating baselines in between. (Robin Sommer)
943
944 * Experimentation Sphinx directive to write a btest with a Sphinx
945 document. See README for more information.
946
947 * Fixing typos, plus an console output tweak. (Robin Sommer)
948
949 * Option -q now implies -b as well. (Robin Sommer)
950
951 0.31-33 | 2012-05-13 17:08:15 -0700
952
953 * New command to copy a file into a test's directory.
954
955 ``@TEST-COPY-FILE: <file>``
956 Copy the given file into the test's directory before the test is
957 run. If ``<file>`` is a relative path, it's interpreted relative
958 to the BTest's base directory. Environment variables in ``<file>``
959 will be replaced if enclosed in ``$
960 {
961 ..
962 }``. This command can be
963 given multiple times. (Robin Sommer)
964
965 * Suppressing error messages when btest-diff can't remove diag file.
966 (Robin Sommer)
967
968 * Adding option -q/--quiet to suppress informational non-error
969 output. (Robin Sommer)
970
971 * Option -F also takes a comma-separated list to specify multiple
972 filters , rather than having to give -F multiple times. (Robin
973 Sommer)
974
975 0.31-28 | 2012-05-06 21:27:15 -0700
976
977 * Separating semantics of groups and thread serialization into
978 separate options. -g still specifices @TEST-GROUPs that are to be
979 executed, but these groups don't any longer control which tests
980 get serialized in a parallel execution. For that, there's a new
981 "@TEST-SERIALIZE: <tag>" command that takes a tag and then makes
982 sure all tests with the same tag are run within the same thread.
983 (Robin Sommer)
984
985 * TEST-GROUPS can now be given multiple times now to assign a test
986 to a set of groups. (Robin Sommer)
987
988 * Extended -g to accept a comma-separated list of groups names to
989 run more than one test group. (Robin Sommer)
990
991 * New output handler for console output. This output is now the
992 default when stdout is a terminal. It prints out a compressed
993 output that updates as btest goes through; it also indicates the
994 progress so far. If btest's output is redirected to a
995 non-terminal, is switches back to the old style. (Robin Sommer)
996
997 * New test command @TEST-NO-FILTER: <filter>
998
999 This allows to ignore a test when running a specific filter. (Robin Sommer)
1000
1001 * Changing the way filters are activated.
1002
1003 -F <filter> now activates only the given filter, but doesn't run
1004 the standard tests in addition. But one can now give -F a
1005 command-separated list of filters to activate them all, and refer
1006 to the standard tests without filter as ``-``. (Robin Sommer)
1007
1008 * Fix to allow numbered test to be given individually on the command
1009 line. (E.g., integer.geq-3 for a file that contains three tests).
1010 (Robin Sommer)
1011
1012
1013 0.31-23 | 2012-04-16 18:10:02 -0700
1014
1015 * A number of smaller fixes for bugs, plus polishing, caused by the
1016 recent restructuring. (Robin Sommer)
1017
1018 * Removing the error given when using -r with tests on the command
1019 line. It's unnessary and confusing compared to when listing tests
1020 in btest.cfg. (Robin Sommer)
1021
1022 * Adding a new "finalizer" option.
1023
1024 ``Finalizer``
1025 An executable that will be executed each time any test has
1026 succesfully run. It runs in the same directory as the test itself
1027 and receives the name of the test as its parameter. The return
1028 value indicates whether the test should indeed be considered
1029 succeeded. By default, there's no finalizer set. (Robin Sommer)
1030
1031 * btest is now again overwriting old diag files instead of appending
1032 (i.e., back to as it used to be). (Robin Sommer)
1033
1034 * Diag output is now line-buffered. (Daniel Thayer)
1035
1036
1037 0.31-13 | 2012-03-13 15:59:51 -0700
1038
1039 * Adding new option -r that reruns all tests that failed last time.
1040 btest now always records all failed tests in a file called. (Robin
1041 Sommer)
1042
1043 * Internal restructuring to factor output out into sublcasses.
1044 (Robin Sommer)
1045
1046 * Adding parallel test execution to btest. (Robin Sommer)
1047
1048 - A new option "-j <n>" allows to run up to <n> tests in
1049 parallel.
1050
1051 - A new @TEST-GROUP directive allows to group tests that can't
1052 be parallelized. All tests of the same group will be
1053 executed sequentially.
1054
1055 - A new option "-g <group>" allows to run only tests of a
1056 certain group, or with "-g -" all tests that don't have a
1057 group.
1058
1059
1060 0.31-2 | 2012-01-25 16:58:29 -0800
1061
1062 * Don't add btest's path to PATH anymore. (Jon Siwek)
1063
1064 0.31 | 2011-11-29 12:11:49 -0600
1065
1066 * Submodule README conformity changes. (Jon Siwek)
1067
1068 0.3 | 2011-10-25 19:58:26 -0700
1069
1070 * More graceful error handling at startup if btest.cfg not found.
1071 (Robin Sommer)
1072
1073 * Python 2.4 compat changes. (Jon Siwek)
1074
1075 * When in brief mode, btest-diff now shows full output if we don't
1076 have a baseline yet. (Robin Sommer)
1077
1078 * Adding executable permission back to script. (Robin Sommer)
1079
1080 * Cleaning up distribution. (Robin Sommer)
1081
1082 0.22-28 | 2011-09-15 15:18:11 -0700
1083
1084 * New environment variable TEST_DIFF_BRIEF. If set btest-diff no
1085 longer includes a mismatching file's full content it the
1086 diagnostic output. This can be useful if the file being compared
1087 is very large. (Robin Sommer)
1088
1089 0.22-27 | 2011-08-12 22:56:12 -0700
1090
1091 * Fix btest-bg-wait's kill trap and -k option. (Jon Siwek)
1092
1093 0.22-18 | 2011-07-23 11:54:07 -0700
1094
1095 * A new option -u for interactively updating baselines.
1096
1097 * Teach btest's TEST-START-FILE to make subdirectories (Jon Siwek)
1098
1099 * Output polishing. (Robin Sommer)
1100
1101 * Have distutils install 'btest-setsid' script. (Jon Siwek)
1102
1103 * A portable setsid. (Robin Sommer)
1104
1105 * Fixes for background execution of processes.
1106
1107 * Fixing exit codes. (Robin Sommer)
1108
1109 0.22-6 | 2011-07-19 17:38:03 -0700
1110
1111 * Teach btest's TEST-START-FILE to make subdirectories (Jon Siwek)
1112
1113 0.22-5 | 2011-05-02 08:41:34 -0700
1114
1115 * A number of bug fixes, and output polishing. (Robin Sommer)
1116
1117 * More robust background execution by btest-bg-*. (Robin Sommer)
1118
1119 0.22-4 | 2011-03-29 21:38:13 -0700
1120
1121 * A test command can now signal to btest that even if it fails
1122 subsequent test commands should still run by returning exit code 100.
1123 btest-diff uses this to continue in the case that no baseline has
1124 yet been established.
1125
1126 * New test option @TEST-REQUIRES for running a test conditionally.
1127 See the README for more information.
1128
1129 0.22-2 | 2011-03-03 21:44:18 -0800
1130
1131 * Two new helper scripts for spawning processes in the background.
1132 See README for more information.
1133
1134 * btest-diff can now deal with files specificied with paths.
1135
1136 0.22 | 2011-02-08 14:06:13 -0800
1137
1138 * BTest is now hosted along with the other Bro repositories on
1139 git.bro-ids.org.
1140
1141 0.21 | 2011-01-09 21:29:18 -0800
1142
1143 * In btest.cfg, option values can now include commands to execute in
1144 backticks.
1145
1146 Example:
1147
1148 [environment]
1149 CC=clang -emit-llvm -g `hilti-config --cflags`
1150
1151 * Limiting substitutions to replacing whole words.
1152
1153 * Adding "substitutions". Substitutions are similar to filters, yet
1154 they do not adapt the input but the command line being exectued.
1155 See README for more information.
1156
1157 * Instead of giving a test's file name on the command line, one can
1158 now also use its "dotted" name as it's printed out when btest is
1159 running (e.g., "foo.bar"). That allows for easier copy/paste.
1160
1161 * Starting CHANGES.