Debian Codegraph @

LspCpp/third_party/uri/deps/docs/advanced.md

raw
1 # Advanced googletest Topics
2
3 ## Introduction
4
5 Now that you have read the [googletest Primer](primer.md) and learned how to
6 write tests using googletest, it's time to learn some new tricks. This document
7 will show you more assertions as well as how to construct complex failure
8 messages, propagate fatal failures, reuse and speed up your test fixtures, and
9 use various flags with your tests.
10
11 ## More Assertions
12
13 This section covers some less frequently used, but still significant,
14 assertions.
15
16 ### Explicit Success and Failure
17
18 See [Explicit Success and Failure](reference/assertions.md#success-failure) in
19 the Assertions Reference.
20
21 ### Exception Assertions
22
23 See [Exception Assertions](reference/assertions.md#exceptions) in the Assertions
24 Reference.
25
26 ### Predicate Assertions for Better Error Messages
27
28 Even though googletest has a rich set of assertions, they can never be complete,
29 as it's impossible (nor a good idea) to anticipate all scenarios a user might
30 run into. Therefore, sometimes a user has to use `EXPECT_TRUE()` to check a
31 complex expression, for lack of a better macro. This has the problem of not
32 showing you the values of the parts of the expression, making it hard to
33 understand what went wrong. As a workaround, some users choose to construct the
34 failure message by themselves, streaming it into `EXPECT_TRUE()`. However, this
35 is awkward especially when the expression has side-effects or is expensive to
36 evaluate.
37
38 googletest gives you three different options to solve this problem:
39
40 #### Using an Existing Boolean Function
41
42 If you already have a function or functor that returns `bool` (or a type that
43 can be implicitly converted to `bool`), you can use it in a *predicate
44 assertion* to get the function arguments printed for free. See
45 [`EXPECT_PRED*`](reference/assertions.md#EXPECT_PRED) in the Assertions
46 Reference for details.
47
48 #### Using a Function That Returns an AssertionResult
49
50 While `EXPECT_PRED*()` and friends are handy for a quick job, the syntax is not
51 satisfactory: you have to use different macros for different arities, and it
52 feels more like Lisp than C++. The `::testing::AssertionResult` class solves
53 this problem.
54
55 An `AssertionResult` object represents the result of an assertion (whether it's
56 a success or a failure, and an associated message). You can create an
57 `AssertionResult` using one of these factory functions:
58
59 ```c++
60 namespace testing {
61
62 // Returns an AssertionResult object to indicate that an assertion has
63 // succeeded.
64 AssertionResult AssertionSuccess();
65
66 // Returns an AssertionResult object to indicate that an assertion has
67 // failed.
68 AssertionResult AssertionFailure();
69
70 }
71 ```
72
73 You can then use the `<<` operator to stream messages to the `AssertionResult`
74 object.
75
76 To provide more readable messages in Boolean assertions (e.g. `EXPECT_TRUE()`),
77 write a predicate function that returns `AssertionResult` instead of `bool`. For
78 example, if you define `IsEven()` as:
79
80 ```c++
81 testing::AssertionResult IsEven(int n) {
82 if ((n % 2) == 0)
83 return testing::AssertionSuccess();
84 else
85 return testing::AssertionFailure() << n << " is odd";
86 }
87 ```
88
89 instead of:
90
91 ```c++
92 bool IsEven(int n) {
93 return (n % 2) == 0;
94 }
95 ```
96
97 the failed assertion `EXPECT_TRUE(IsEven(Fib(4)))` will print:
98
99 ```none
100 Value of: IsEven(Fib(4))
101 Actual: false (3 is odd)
102 Expected: true
103 ```
104
105 instead of a more opaque
106
107 ```none
108 Value of: IsEven(Fib(4))
109 Actual: false
110 Expected: true
111 ```
112
113 If you want informative messages in `EXPECT_FALSE` and `ASSERT_FALSE` as well
114 (one third of Boolean assertions in the Google code base are negative ones), and
115 are fine with making the predicate slower in the success case, you can supply a
116 success message:
117
118 ```c++
119 testing::AssertionResult IsEven(int n) {
120 if ((n % 2) == 0)
121 return testing::AssertionSuccess() << n << " is even";
122 else
123 return testing::AssertionFailure() << n << " is odd";
124 }
125 ```
126
127 Then the statement `EXPECT_FALSE(IsEven(Fib(6)))` will print
128
129 ```none
130 Value of: IsEven(Fib(6))
131 Actual: true (8 is even)
132 Expected: false
133 ```
134
135 #### Using a Predicate-Formatter
136
137 If you find the default message generated by
138 [`EXPECT_PRED*`](reference/assertions.md#EXPECT_PRED) and
139 [`EXPECT_TRUE`](reference/assertions.md#EXPECT_TRUE) unsatisfactory, or some
140 arguments to your predicate do not support streaming to `ostream`, you can
141 instead use *predicate-formatter assertions* to *fully* customize how the
142 message is formatted. See
143 [`EXPECT_PRED_FORMAT*`](reference/assertions.md#EXPECT_PRED_FORMAT) in the
144 Assertions Reference for details.
145
146 ### Floating-Point Comparison
147
148 See [Floating-Point Comparison](reference/assertions.md#floating-point) in the
149 Assertions Reference.
150
151 #### Floating-Point Predicate-Format Functions
152
153 Some floating-point operations are useful, but not that often used. In order to
154 avoid an explosion of new macros, we provide them as predicate-format functions
155 that can be used in the predicate assertion macro
156 [`EXPECT_PRED_FORMAT2`](reference/assertions.md#EXPECT_PRED_FORMAT), for
157 example:
158
159 ```c++
160 using ::testing::FloatLE;
161 using ::testing::DoubleLE;
162 ...
163 EXPECT_PRED_FORMAT2(FloatLE, val1, val2);
164 EXPECT_PRED_FORMAT2(DoubleLE, val1, val2);
165 ```
166
167 The above code verifies that `val1` is less than, or approximately equal to,
168 `val2`.
169
170 ### Asserting Using gMock Matchers
171
172 See [`EXPECT_THAT`](reference/assertions.md#EXPECT_THAT) in the Assertions
173 Reference.
174
175 ### More String Assertions
176
177 (Please read the [previous](#asserting-using-gmock-matchers) section first if
178 you haven't.)
179
180 You can use the gMock [string matchers](reference/matchers.md#string-matchers)
181 with [`EXPECT_THAT`](reference/assertions.md#EXPECT_THAT) to do more string
182 comparison tricks (sub-string, prefix, suffix, regular expression, and etc). For
183 example,
184
185 ```c++
186 using ::testing::HasSubstr;
187 using ::testing::MatchesRegex;
188 ...
189 ASSERT_THAT(foo_string, HasSubstr("needle"));
190 EXPECT_THAT(bar_string, MatchesRegex("\\w*\\d+"));
191 ```
192
193 ### Windows HRESULT assertions
194
195 See [Windows HRESULT Assertions](reference/assertions.md#HRESULT) in the
196 Assertions Reference.
197
198 ### Type Assertions
199
200 You can call the function
201
202 ```c++
203 ::testing::StaticAssertTypeEq<T1, T2>();
204 ```
205
206 to assert that types `T1` and `T2` are the same. The function does nothing if
207 the assertion is satisfied. If the types are different, the function call will
208 fail to compile, the compiler error message will say that `T1 and T2 are not the
209 same type` and most likely (depending on the compiler) show you the actual
210 values of `T1` and `T2`. This is mainly useful inside template code.
211
212 **Caveat**: When used inside a member function of a class template or a function
213 template, `StaticAssertTypeEq<T1, T2>()` is effective only if the function is
214 instantiated. For example, given:
215
216 ```c++
217 template <typename T> class Foo {
218 public:
219 void Bar() { testing::StaticAssertTypeEq<int, T>(); }
220 };
221 ```
222
223 the code:
224
225 ```c++
226 void Test1() { Foo<bool> foo; }
227 ```
228
229 will not generate a compiler error, as `Foo<bool>::Bar()` is never actually
230 instantiated. Instead, you need:
231
232 ```c++
233 void Test2() { Foo<bool> foo; foo.Bar(); }
234 ```
235
236 to cause a compiler error.
237
238 ### Assertion Placement
239
240 You can use assertions in any C++ function. In particular, it doesn't have to be
241 a method of the test fixture class. The one constraint is that assertions that
242 generate a fatal failure (`FAIL*` and `ASSERT_*`) can only be used in
243 void-returning functions. This is a consequence of Google's not using
244 exceptions. By placing it in a non-void function you'll get a confusing compile
245 error like `"error: void value not ignored as it ought to be"` or `"cannot
246 initialize return object of type 'bool' with an rvalue of type 'void'"` or
247 `"error: no viable conversion from 'void' to 'string'"`.
248
249 If you need to use fatal assertions in a function that returns non-void, one
250 option is to make the function return the value in an out parameter instead. For
251 example, you can rewrite `T2 Foo(T1 x)` to `void Foo(T1 x, T2* result)`. You
252 need to make sure that `*result` contains some sensible value even when the
253 function returns prematurely. As the function now returns `void`, you can use
254 any assertion inside of it.
255
256 If changing the function's type is not an option, you should just use assertions
257 that generate non-fatal failures, such as `ADD_FAILURE*` and `EXPECT_*`.
258
259 {: .callout .note}
260 NOTE: Constructors and destructors are not considered void-returning functions,
261 according to the C++ language specification, and so you may not use fatal
262 assertions in them; you'll get a compilation error if you try. Instead, either
263 call `abort` and crash the entire test executable, or put the fatal assertion in
264 a `SetUp`/`TearDown` function; see
265 [constructor/destructor vs. `SetUp`/`TearDown`](faq.md#CtorVsSetUp)
266
267 {: .callout .warning}
268 WARNING: A fatal assertion in a helper function (private void-returning method)
269 called from a constructor or destructor does not terminate the current test, as
270 your intuition might suggest: it merely returns from the constructor or
271 destructor early, possibly leaving your object in a partially-constructed or
272 partially-destructed state! You almost certainly want to `abort` or use
273 `SetUp`/`TearDown` instead.
274
275 ## Skipping test execution
276
277 Related to the assertions `SUCCEED()` and `FAIL()`, you can prevent further test
278 execution at runtime with the `GTEST_SKIP()` macro. This is useful when you need
279 to check for preconditions of the system under test during runtime and skip
280 tests in a meaningful way.
281
282 `GTEST_SKIP()` can be used in individual test cases or in the `SetUp()` methods
283 of classes derived from either `::testing::Environment` or `::testing::Test`.
284 For example:
285
286 ```c++
287 TEST(SkipTest, DoesSkip) {
288 GTEST_SKIP() << "Skipping single test";
289 EXPECT_EQ(0, 1); // Won't fail; it won't be executed
290 }
291
292 class SkipFixture : public ::testing::Test {
293 protected:
294 void SetUp() override {
295 GTEST_SKIP() << "Skipping all tests for this fixture";
296 }
297 };
298
299 // Tests for SkipFixture won't be executed.
300 TEST_F(SkipFixture, SkipsOneTest) {
301 EXPECT_EQ(5, 7); // Won't fail
302 }
303 ```
304
305 As with assertion macros, you can stream a custom message into `GTEST_SKIP()`.
306
307 ## Teaching googletest How to Print Your Values
308
309 When a test assertion such as `EXPECT_EQ` fails, googletest prints the argument
310 values to help you debug. It does this using a user-extensible value printer.
311
312 This printer knows how to print built-in C++ types, native arrays, STL
313 containers, and any type that supports the `<<` operator. For other types, it
314 prints the raw bytes in the value and hopes that you the user can figure it out.
315
316 As mentioned earlier, the printer is *extensible*. That means you can teach it
317 to do a better job at printing your particular type than to dump the bytes. To
318 do that, define `<<` for your type:
319
320 ```c++
321 #include <ostream>
322
323 namespace foo {
324
325 class Bar { // We want googletest to be able to print instances of this.
326 ...
327 // Create a free inline friend function.
328 friend std::ostream& operator<<(std::ostream& os, const Bar& bar) {
329 return os << bar.DebugString(); // whatever needed to print bar to os
330 }
331 };
332
333 // If you can't declare the function in the class it's important that the
334 // << operator is defined in the SAME namespace that defines Bar. C++'s look-up
335 // rules rely on that.
336 std::ostream& operator<<(std::ostream& os, const Bar& bar) {
337 return os << bar.DebugString(); // whatever needed to print bar to os
338 }
339
340 } // namespace foo
341 ```
342
343 Sometimes, this might not be an option: your team may consider it bad style to
344 have a `<<` operator for `Bar`, or `Bar` may already have a `<<` operator that
345 doesn't do what you want (and you cannot change it). If so, you can instead
346 define a `PrintTo()` function like this:
347
348 ```c++
349 #include <ostream>
350
351 namespace foo {
352
353 class Bar {
354 ...
355 friend void PrintTo(const Bar& bar, std::ostream* os) {
356 *os << bar.DebugString(); // whatever needed to print bar to os
357 }
358 };
359
360 // If you can't declare the function in the class it's important that PrintTo()
361 // is defined in the SAME namespace that defines Bar. C++'s look-up rules rely
362 // on that.
363 void PrintTo(const Bar& bar, std::ostream* os) {
364 *os << bar.DebugString(); // whatever needed to print bar to os
365 }
366
367 } // namespace foo
368 ```
369
370 If you have defined both `<<` and `PrintTo()`, the latter will be used when
371 googletest is concerned. This allows you to customize how the value appears in
372 googletest's output without affecting code that relies on the behavior of its
373 `<<` operator.
374
375 If you want to print a value `x` using googletest's value printer yourself, just
376 call `::testing::PrintToString(x)`, which returns an `std::string`:
377
378 ```c++
379 vector<pair<Bar, int> > bar_ints = GetBarIntVector();
380
381 EXPECT_TRUE(IsCorrectBarIntVector(bar_ints))
382 << "bar_ints = " << testing::PrintToString(bar_ints);
383 ```
384
385 ## Death Tests
386
387 In many applications, there are assertions that can cause application failure if
388 a condition is not met. These consistency checks, which ensure that the program
389 is in a known good state, are there to fail at the earliest possible time after
390 some program state is corrupted. If the assertion checks the wrong condition,
391 then the program may proceed in an erroneous state, which could lead to memory
392 corruption, security holes, or worse. Hence it is vitally important to test that
393 such assertion statements work as expected.
394
395 Since these precondition checks cause the processes to die, we call such tests
396 _death tests_. More generally, any test that checks that a program terminates
397 (except by throwing an exception) in an expected fashion is also a death test.
398
399 Note that if a piece of code throws an exception, we don't consider it "death"
400 for the purpose of death tests, as the caller of the code could catch the
401 exception and avoid the crash. If you want to verify exceptions thrown by your
402 code, see [Exception Assertions](#ExceptionAssertions).
403
404 If you want to test `EXPECT_*()/ASSERT_*()` failures in your test code, see
405 ["Catching" Failures](#catching-failures).
406
407 ### How to Write a Death Test
408
409 GoogleTest provides assertion macros to support death tests. See
410 [Death Assertions](reference/assertions.md#death) in the Assertions Reference
411 for details.
412
413 To write a death test, simply use one of the macros inside your test function.
414 For example,
415
416 ```c++
417 TEST(MyDeathTest, Foo) {
418 // This death test uses a compound statement.
419 ASSERT_DEATH({
420 int n = 5;
421 Foo(&n);
422 }, "Error on line .* of Foo()");
423 }
424
425 TEST(MyDeathTest, NormalExit) {
426 EXPECT_EXIT(NormalExit(), testing::ExitedWithCode(0), "Success");
427 }
428
429 TEST(MyDeathTest, KillProcess) {
430 EXPECT_EXIT(KillProcess(), testing::KilledBySignal(SIGKILL),
431 "Sending myself unblockable signal");
432 }
433 ```
434
435 verifies that:
436
437 * calling `Foo(5)` causes the process to die with the given error message,
438 * calling `NormalExit()` causes the process to print `"Success"` to stderr and
439 exit with exit code 0, and
440 * calling `KillProcess()` kills the process with signal `SIGKILL`.
441
442 The test function body may contain other assertions and statements as well, if
443 necessary.
444
445 Note that a death test only cares about three things:
446
447 1. does `statement` abort or exit the process?
448 2. (in the case of `ASSERT_EXIT` and `EXPECT_EXIT`) does the exit status
449 satisfy `predicate`? Or (in the case of `ASSERT_DEATH` and `EXPECT_DEATH`)
450 is the exit status non-zero? And
451 3. does the stderr output match `matcher`?
452
453 In particular, if `statement` generates an `ASSERT_*` or `EXPECT_*` failure, it
454 will **not** cause the death test to fail, as googletest assertions don't abort
455 the process.
456
457 ### Death Test Naming
458
459 {: .callout .important}
460 IMPORTANT: We strongly recommend you to follow the convention of naming your
461 **test suite** (not test) `*DeathTest` when it contains a death test, as
462 demonstrated in the above example. The
463 [Death Tests And Threads](#death-tests-and-threads) section below explains why.
464
465 If a test fixture class is shared by normal tests and death tests, you can use
466 `using` or `typedef` to introduce an alias for the fixture class and avoid
467 duplicating its code:
468
469 ```c++
470 class FooTest : public testing::Test { ... };
471
472 using FooDeathTest = FooTest;
473
474 TEST_F(FooTest, DoesThis) {
475 // normal test
476 }
477
478 TEST_F(FooDeathTest, DoesThat) {
479 // death test
480 }
481 ```
482
483 ### Regular Expression Syntax
484
485 On POSIX systems (e.g. Linux, Cygwin, and Mac), googletest uses the
486 [POSIX extended regular expression](http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap09.html#tag_09_04)
487 syntax. To learn about this syntax, you may want to read this
488 [Wikipedia entry](http://en.wikipedia.org/wiki/Regular_expression#POSIX_Extended_Regular_Expressions).
489
490 On Windows, googletest uses its own simple regular expression implementation. It
491 lacks many features. For example, we don't support union (`"x|y"`), grouping
492 (`"(xy)"`), brackets (`"[xy]"`), and repetition count (`"x{5,7}"`), among
493 others. Below is what we do support (`A` denotes a literal character, period
494 (`.`), or a single `\\ ` escape sequence; `x` and `y` denote regular
495 expressions.):
496
497 Expression | Meaning
498 ---------- | --------------------------------------------------------------
499 `c` | matches any literal character `c`
500 `\\d` | matches any decimal digit
501 `\\D` | matches any character that's not a decimal digit
502 `\\f` | matches `\f`
503 `\\n` | matches `\n`
504 `\\r` | matches `\r`
505 `\\s` | matches any ASCII whitespace, including `\n`
506 `\\S` | matches any character that's not a whitespace
507 `\\t` | matches `\t`
508 `\\v` | matches `\v`
509 `\\w` | matches any letter, `_`, or decimal digit
510 `\\W` | matches any character that `\\w` doesn't match
511 `\\c` | matches any literal character `c`, which must be a punctuation
512 `.` | matches any single character except `\n`
513 `A?` | matches 0 or 1 occurrences of `A`
514 `A*` | matches 0 or many occurrences of `A`
515 `A+` | matches 1 or many occurrences of `A`
516 `^` | matches the beginning of a string (not that of each line)
517 `$` | matches the end of a string (not that of each line)
518 `xy` | matches `x` followed by `y`
519
520 To help you determine which capability is available on your system, googletest
521 defines macros to govern which regular expression it is using. The macros are:
522 `GTEST_USES_SIMPLE_RE=1` or `GTEST_USES_POSIX_RE=1`. If you want your death
523 tests to work in all cases, you can either `#if` on these macros or use the more
524 limited syntax only.
525
526 ### How It Works
527
528 See [Death Assertions](reference/assertions.md#death) in the Assertions
529 Reference.
530
531 ### Death Tests And Threads
532
533 The reason for the two death test styles has to do with thread safety. Due to
534 well-known problems with forking in the presence of threads, death tests should
535 be run in a single-threaded context. Sometimes, however, it isn't feasible to
536 arrange that kind of environment. For example, statically-initialized modules
537 may start threads before main is ever reached. Once threads have been created,
538 it may be difficult or impossible to clean them up.
539
540 googletest has three features intended to raise awareness of threading issues.
541
542 1. A warning is emitted if multiple threads are running when a death test is
543 encountered.
544 2. Test suites with a name ending in "DeathTest" are run before all other
545 tests.
546 3. It uses `clone()` instead of `fork()` to spawn the child process on Linux
547 (`clone()` is not available on Cygwin and Mac), as `fork()` is more likely
548 to cause the child to hang when the parent process has multiple threads.
549
550 It's perfectly fine to create threads inside a death test statement; they are
551 executed in a separate process and cannot affect the parent.
552
553 ### Death Test Styles
554
555 The "threadsafe" death test style was introduced in order to help mitigate the
556 risks of testing in a possibly multithreaded environment. It trades increased
557 test execution time (potentially dramatically so) for improved thread safety.
558
559 The automated testing framework does not set the style flag. You can choose a
560 particular style of death tests by setting the flag programmatically:
561
562 ```c++
563 GTEST_FLAG_SET(death_test_style, "threadsafe")
564 ```
565
566 You can do this in `main()` to set the style for all death tests in the binary,
567 or in individual tests. Recall that flags are saved before running each test and
568 restored afterwards, so you need not do that yourself. For example:
569
570 ```c++
571 int main(int argc, char** argv) {
572 testing::InitGoogleTest(&argc, argv);
573 GTEST_FLAG_SET(death_test_style, "fast");
574 return RUN_ALL_TESTS();
575 }
576
577 TEST(MyDeathTest, TestOne) {
578 GTEST_FLAG_SET(death_test_style, "threadsafe");
579 // This test is run in the "threadsafe" style:
580 ASSERT_DEATH(ThisShouldDie(), "");
581 }
582
583 TEST(MyDeathTest, TestTwo) {
584 // This test is run in the "fast" style:
585 ASSERT_DEATH(ThisShouldDie(), "");
586 }
587 ```
588
589 ### Caveats
590
591 The `statement` argument of `ASSERT_EXIT()` can be any valid C++ statement. If
592 it leaves the current function via a `return` statement or by throwing an
593 exception, the death test is considered to have failed. Some googletest macros
594 may return from the current function (e.g. `ASSERT_TRUE()`), so be sure to avoid
595 them in `statement`.
596
597 Since `statement` runs in the child process, any in-memory side effect (e.g.
598 modifying a variable, releasing memory, etc) it causes will *not* be observable
599 in the parent process. In particular, if you release memory in a death test,
600 your program will fail the heap check as the parent process will never see the
601 memory reclaimed. To solve this problem, you can
602
603 1. try not to free memory in a death test;
604 2. free the memory again in the parent process; or
605 3. do not use the heap checker in your program.
606
607 Due to an implementation detail, you cannot place multiple death test assertions
608 on the same line; otherwise, compilation will fail with an unobvious error
609 message.
610
611 Despite the improved thread safety afforded by the "threadsafe" style of death
612 test, thread problems such as deadlock are still possible in the presence of
613 handlers registered with `pthread_atfork(3)`.
614
615 ## Using Assertions in Sub-routines
616
617 {: .callout .note}
618 Note: If you want to put a series of test assertions in a subroutine to check
619 for a complex condition, consider using
620 [a custom GMock matcher](gmock_cook_book.md#NewMatchers) instead. This lets you
621 provide a more readable error message in case of failure and avoid all of the
622 issues described below.
623
624 ### Adding Traces to Assertions
625
626 If a test sub-routine is called from several places, when an assertion inside it
627 fails, it can be hard to tell which invocation of the sub-routine the failure is
628 from. You can alleviate this problem using extra logging or custom failure
629 messages, but that usually clutters up your tests. A better solution is to use
630 the `SCOPED_TRACE` macro or the `ScopedTrace` utility:
631
632 ```c++
633 SCOPED_TRACE(message);
634 ```
635
636 ```c++
637 ScopedTrace trace("file_path", line_number, message);
638 ```
639
640 where `message` can be anything streamable to `std::ostream`. `SCOPED_TRACE`
641 macro will cause the current file name, line number, and the given message to be
642 added in every failure message. `ScopedTrace` accepts explicit file name and
643 line number in arguments, which is useful for writing test helpers. The effect
644 will be undone when the control leaves the current lexical scope.
645
646 For example,
647
648 ```c++
649 10: void Sub1(int n) {
650 11: EXPECT_EQ(Bar(n), 1);
651 12: EXPECT_EQ(Bar(n + 1), 2);
652 13: }
653 14:
654 15: TEST(FooTest, Bar) {
655 16: {
656 17: SCOPED_TRACE("A"); // This trace point will be included in
657 18: // every failure in this scope.
658 19: Sub1(1);
659 20: }
660 21: // Now it won't.
661 22: Sub1(9);
662 23: }
663 ```
664
665 could result in messages like these:
666
667 ```none
668 path/to/foo_test.cc:11: Failure
669 Value of: Bar(n)
670 Expected: 1
671 Actual: 2
672 Google Test trace:
673 path/to/foo_test.cc:17: A
674
675 path/to/foo_test.cc:12: Failure
676 Value of: Bar(n + 1)
677 Expected: 2
678 Actual: 3
679 ```
680
681 Without the trace, it would've been difficult to know which invocation of
682 `Sub1()` the two failures come from respectively. (You could add an extra
683 message to each assertion in `Sub1()` to indicate the value of `n`, but that's
684 tedious.)
685
686 Some tips on using `SCOPED_TRACE`:
687
688 1. With a suitable message, it's often enough to use `SCOPED_TRACE` at the
689 beginning of a sub-routine, instead of at each call site.
690 2. When calling sub-routines inside a loop, make the loop iterator part of the
691 message in `SCOPED_TRACE` such that you can know which iteration the failure
692 is from.
693 3. Sometimes the line number of the trace point is enough for identifying the
694 particular invocation of a sub-routine. In this case, you don't have to
695 choose a unique message for `SCOPED_TRACE`. You can simply use `""`.
696 4. You can use `SCOPED_TRACE` in an inner scope when there is one in the outer
697 scope. In this case, all active trace points will be included in the failure
698 messages, in reverse order they are encountered.
699 5. The trace dump is clickable in Emacs - hit `return` on a line number and
700 you'll be taken to that line in the source file!
701
702 ### Propagating Fatal Failures
703
704 A common pitfall when using `ASSERT_*` and `FAIL*` is not understanding that
705 when they fail they only abort the _current function_, not the entire test. For
706 example, the following test will segfault:
707
708 ```c++
709 void Subroutine() {
710 // Generates a fatal failure and aborts the current function.
711 ASSERT_EQ(1, 2);
712
713 // The following won't be executed.
714 ...
715 }
716
717 TEST(FooTest, Bar) {
718 Subroutine(); // The intended behavior is for the fatal failure
719 // in Subroutine() to abort the entire test.
720
721 // The actual behavior: the function goes on after Subroutine() returns.
722 int* p = nullptr;
723 *p = 3; // Segfault!
724 }
725 ```
726
727 To alleviate this, googletest provides three different solutions. You could use
728 either exceptions, the `(ASSERT|EXPECT)_NO_FATAL_FAILURE` assertions or the
729 `HasFatalFailure()` function. They are described in the following two
730 subsections.
731
732 #### Asserting on Subroutines with an exception
733
734 The following code can turn ASSERT-failure into an exception:
735
736 ```c++
737 class ThrowListener : public testing::EmptyTestEventListener {
738 void OnTestPartResult(const testing::TestPartResult& result) override {
739 if (result.type() == testing::TestPartResult::kFatalFailure) {
740 throw testing::AssertionException(result);
741 }
742 }
743 };
744 int main(int argc, char** argv) {
745 ...
746 testing::UnitTest::GetInstance()->listeners().Append(new ThrowListener);
747 return RUN_ALL_TESTS();
748 }
749 ```
750
751 This listener should be added after other listeners if you have any, otherwise
752 they won't see failed `OnTestPartResult`.
753
754 #### Asserting on Subroutines
755
756 As shown above, if your test calls a subroutine that has an `ASSERT_*` failure
757 in it, the test will continue after the subroutine returns. This may not be what
758 you want.
759
760 Often people want fatal failures to propagate like exceptions. For that
761 googletest offers the following macros:
762
763 Fatal assertion | Nonfatal assertion | Verifies
764 ------------------------------------- | ------------------------------------- | --------
765 `ASSERT_NO_FATAL_FAILURE(statement);` | `EXPECT_NO_FATAL_FAILURE(statement);` | `statement` doesn't generate any new fatal failures in the current thread.
766
767 Only failures in the thread that executes the assertion are checked to determine
768 the result of this type of assertions. If `statement` creates new threads,
769 failures in these threads are ignored.
770
771 Examples:
772
773 ```c++
774 ASSERT_NO_FATAL_FAILURE(Foo());
775
776 int i;
777 EXPECT_NO_FATAL_FAILURE({
778 i = Bar();
779 });
780 ```
781
782 Assertions from multiple threads are currently not supported on Windows.
783
784 #### Checking for Failures in the Current Test
785
786 `HasFatalFailure()` in the `::testing::Test` class returns `true` if an
787 assertion in the current test has suffered a fatal failure. This allows
788 functions to catch fatal failures in a sub-routine and return early.
789
790 ```c++
791 class Test {
792 public:
793 ...
794 static bool HasFatalFailure();
795 };
796 ```
797
798 The typical usage, which basically simulates the behavior of a thrown exception,
799 is:
800
801 ```c++
802 TEST(FooTest, Bar) {
803 Subroutine();
804 // Aborts if Subroutine() had a fatal failure.
805 if (HasFatalFailure()) return;
806
807 // The following won't be executed.
808 ...
809 }
810 ```
811
812 If `HasFatalFailure()` is used outside of `TEST()` , `TEST_F()` , or a test
813 fixture, you must add the `::testing::Test::` prefix, as in:
814
815 ```c++
816 if (testing::Test::HasFatalFailure()) return;
817 ```
818
819 Similarly, `HasNonfatalFailure()` returns `true` if the current test has at
820 least one non-fatal failure, and `HasFailure()` returns `true` if the current
821 test has at least one failure of either kind.
822
823 ## Logging Additional Information
824
825 In your test code, you can call `RecordProperty("key", value)` to log additional
826 information, where `value` can be either a string or an `int`. The *last* value
827 recorded for a key will be emitted to the
828 [XML output](#generating-an-xml-report) if you specify one. For example, the
829 test
830
831 ```c++
832 TEST_F(WidgetUsageTest, MinAndMaxWidgets) {
833 RecordProperty("MaximumWidgets", ComputeMaxUsage());
834 RecordProperty("MinimumWidgets", ComputeMinUsage());
835 }
836 ```
837
838 will output XML like this:
839
840 ```xml
841 ...
842 <testcase name="MinAndMaxWidgets" status="run" time="0.006" classname="WidgetUsageTest" MaximumWidgets="12" MinimumWidgets="9" />
843 ...
844 ```
845
846 {: .callout .note}
847 > NOTE:
848 >
849 > * `RecordProperty()` is a static member of the `Test` class. Therefore it
850 > needs to be prefixed with `::testing::Test::` if used outside of the
851 > `TEST` body and the test fixture class.
852 > * *`key`* must be a valid XML attribute name, and cannot conflict with the
853 > ones already used by googletest (`name`, `status`, `time`, `classname`,
854 > `type_param`, and `value_param`).
855 > * Calling `RecordProperty()` outside of the lifespan of a test is allowed.
856 > If it's called outside of a test but between a test suite's
857 > `SetUpTestSuite()` and `TearDownTestSuite()` methods, it will be
858 > attributed to the XML element for the test suite. If it's called outside
859 > of all test suites (e.g. in a test environment), it will be attributed to
860 > the top-level XML element.
861
862 ## Sharing Resources Between Tests in the Same Test Suite
863
864 googletest creates a new test fixture object for each test in order to make
865 tests independent and easier to debug. However, sometimes tests use resources
866 that are expensive to set up, making the one-copy-per-test model prohibitively
867 expensive.
868
869 If the tests don't change the resource, there's no harm in their sharing a
870 single resource copy. So, in addition to per-test set-up/tear-down, googletest
871 also supports per-test-suite set-up/tear-down. To use it:
872
873 1. In your test fixture class (say `FooTest` ), declare as `static` some member
874 variables to hold the shared resources.
875 2. Outside your test fixture class (typically just below it), define those
876 member variables, optionally giving them initial values.
877 3. In the same test fixture class, define a `static void SetUpTestSuite()`
878 function (remember not to spell it as **`SetupTestSuite`** with a small
879 `u`!) to set up the shared resources and a `static void TearDownTestSuite()`
880 function to tear them down.
881
882 That's it! googletest automatically calls `SetUpTestSuite()` before running the
883 *first test* in the `FooTest` test suite (i.e. before creating the first
884 `FooTest` object), and calls `TearDownTestSuite()` after running the *last test*
885 in it (i.e. after deleting the last `FooTest` object). In between, the tests can
886 use the shared resources.
887
888 Remember that the test order is undefined, so your code can't depend on a test
889 preceding or following another. Also, the tests must either not modify the state
890 of any shared resource, or, if they do modify the state, they must restore the
891 state to its original value before passing control to the next test.
892
893 Note that `SetUpTestSuite()` may be called multiple times for a test fixture
894 class that has derived classes, so you should not expect code in the function
895 body to be run only once. Also, derived classes still have access to shared
896 resources defined as static members, so careful consideration is needed when
897 managing shared resources to avoid memory leaks.
898
899 Here's an example of per-test-suite set-up and tear-down:
900
901 ```c++
902 class FooTest : public testing::Test {
903 protected:
904 // Per-test-suite set-up.
905 // Called before the first test in this test suite.
906 // Can be omitted if not needed.
907 static void SetUpTestSuite() {
908 // Avoid reallocating static objects if called in subclasses of FooTest.
909 if (shared_resource_ == nullptr) {
910 shared_resource_ = new ...;
911 }
912 }
913
914 // Per-test-suite tear-down.
915 // Called after the last test in this test suite.
916 // Can be omitted if not needed.
917 static void TearDownTestSuite() {
918 delete shared_resource_;
919 shared_resource_ = nullptr;
920 }
921
922 // You can define per-test set-up logic as usual.
923 void SetUp() override { ... }
924
925 // You can define per-test tear-down logic as usual.
926 void TearDown() override { ... }
927
928 // Some expensive resource shared by all tests.
929 static T* shared_resource_;
930 };
931
932 T* FooTest::shared_resource_ = nullptr;
933
934 TEST_F(FooTest, Test1) {
935 ... you can refer to shared_resource_ here ...
936 }
937
938 TEST_F(FooTest, Test2) {
939 ... you can refer to shared_resource_ here ...
940 }
941 ```
942
943 {: .callout .note}
944 NOTE: Though the above code declares `SetUpTestSuite()` protected, it may
945 sometimes be necessary to declare it public, such as when using it with
946 `TEST_P`.
947
948 ## Global Set-Up and Tear-Down
949
950 Just as you can do set-up and tear-down at the test level and the test suite
951 level, you can also do it at the test program level. Here's how.
952
953 First, you subclass the `::testing::Environment` class to define a test
954 environment, which knows how to set-up and tear-down:
955
956 ```c++
957 class Environment : public ::testing::Environment {
958 public:
959 ~Environment() override {}
960
961 // Override this to define how to set up the environment.
962 void SetUp() override {}
963
964 // Override this to define how to tear down the environment.
965 void TearDown() override {}
966 };
967 ```
968
969 Then, you register an instance of your environment class with googletest by
970 calling the `::testing::AddGlobalTestEnvironment()` function:
971
972 ```c++
973 Environment* AddGlobalTestEnvironment(Environment* env);
974 ```
975
976 Now, when `RUN_ALL_TESTS()` is called, it first calls the `SetUp()` method of
977 each environment object, then runs the tests if none of the environments
978 reported fatal failures and `GTEST_SKIP()` was not called. `RUN_ALL_TESTS()`
979 always calls `TearDown()` with each environment object, regardless of whether or
980 not the tests were run.
981
982 It's OK to register multiple environment objects. In this suite, their `SetUp()`
983 will be called in the order they are registered, and their `TearDown()` will be
984 called in the reverse order.
985
986 Note that googletest takes ownership of the registered environment objects.
987 Therefore **do not delete them** by yourself.
988
989 You should call `AddGlobalTestEnvironment()` before `RUN_ALL_TESTS()` is called,
990 probably in `main()`. If you use `gtest_main`, you need to call this before
991 `main()` starts for it to take effect. One way to do this is to define a global
992 variable like this:
993
994 ```c++
995 testing::Environment* const foo_env =
996 testing::AddGlobalTestEnvironment(new FooEnvironment);
997 ```
998
999 However, we strongly recommend you to write your own `main()` and call
1000 `AddGlobalTestEnvironment()` there, as relying on initialization of global
1001 variables makes the code harder to read and may cause problems when you register
1002 multiple environments from different translation units and the environments have
1003 dependencies among them (remember that the compiler doesn't guarantee the order
1004 in which global variables from different translation units are initialized).
1005
1006 ## Value-Parameterized Tests
1007
1008 *Value-parameterized tests* allow you to test your code with different
1009 parameters without writing multiple copies of the same test. This is useful in a
1010 number of situations, for example:
1011
1012 * You have a piece of code whose behavior is affected by one or more
1013 command-line flags. You want to make sure your code performs correctly for
1014 various values of those flags.
1015 * You want to test different implementations of an OO interface.
1016 * You want to test your code over various inputs (a.k.a. data-driven testing).
1017 This feature is easy to abuse, so please exercise your good sense when doing
1018 it!
1019
1020 ### How to Write Value-Parameterized Tests
1021
1022 To write value-parameterized tests, first you should define a fixture class. It
1023 must be derived from both `testing::Test` and `testing::WithParamInterface<T>`
1024 (the latter is a pure interface), where `T` is the type of your parameter
1025 values. For convenience, you can just derive the fixture class from
1026 `testing::TestWithParam<T>`, which itself is derived from both `testing::Test`
1027 and `testing::WithParamInterface<T>`. `T` can be any copyable type. If it's a
1028 raw pointer, you are responsible for managing the lifespan of the pointed
1029 values.
1030
1031 {: .callout .note}
1032 NOTE: If your test fixture defines `SetUpTestSuite()` or `TearDownTestSuite()`
1033 they must be declared **public** rather than **protected** in order to use
1034 `TEST_P`.
1035
1036 ```c++
1037 class FooTest :
1038 public testing::TestWithParam<const char*> {
1039 // You can implement all the usual fixture class members here.
1040 // To access the test parameter, call GetParam() from class
1041 // TestWithParam<T>.
1042 };
1043
1044 // Or, when you want to add parameters to a pre-existing fixture class:
1045 class BaseTest : public testing::Test {
1046 ...
1047 };
1048 class BarTest : public BaseTest,
1049 public testing::WithParamInterface<const char*> {
1050 ...
1051 };
1052 ```
1053
1054 Then, use the `TEST_P` macro to define as many test patterns using this fixture
1055 as you want. The `_P` suffix is for "parameterized" or "pattern", whichever you
1056 prefer to think.
1057
1058 ```c++
1059 TEST_P(FooTest, DoesBlah) {
1060 // Inside a test, access the test parameter with the GetParam() method
1061 // of the TestWithParam<T> class:
1062 EXPECT_TRUE(foo.Blah(GetParam()));
1063 ...
1064 }
1065
1066 TEST_P(FooTest, HasBlahBlah) {
1067 ...
1068 }
1069 ```
1070
1071 Finally, you can use the `INSTANTIATE_TEST_SUITE_P` macro to instantiate the
1072 test suite with any set of parameters you want. GoogleTest defines a number of
1073 functions for generating test parameters—see details at
1074 [`INSTANTIATE_TEST_SUITE_P`](reference/testing.md#INSTANTIATE_TEST_SUITE_P) in
1075 the Testing Reference.
1076
1077 For example, the following statement will instantiate tests from the `FooTest`
1078 test suite each with parameter values `"meeny"`, `"miny"`, and `"moe"` using the
1079 [`Values`](reference/testing.md#param-generators) parameter generator:
1080
1081 ```c++
1082 INSTANTIATE_TEST_SUITE_P(MeenyMinyMoe,
1083 FooTest,
1084 testing::Values("meeny", "miny", "moe"));
1085 ```
1086
1087 {: .callout .note}
1088 NOTE: The code above must be placed at global or namespace scope, not at
1089 function scope.
1090
1091 The first argument to `INSTANTIATE_TEST_SUITE_P` is a unique name for the
1092 instantiation of the test suite. The next argument is the name of the test
1093 pattern, and the last is the
1094 [parameter generator](reference/testing.md#param-generators).
1095
1096 You can instantiate a test pattern more than once, so to distinguish different
1097 instances of the pattern, the instantiation name is added as a prefix to the
1098 actual test suite name. Remember to pick unique prefixes for different
1099 instantiations. The tests from the instantiation above will have these names:
1100
1101 * `MeenyMinyMoe/FooTest.DoesBlah/0` for `"meeny"`
1102 * `MeenyMinyMoe/FooTest.DoesBlah/1` for `"miny"`
1103 * `MeenyMinyMoe/FooTest.DoesBlah/2` for `"moe"`
1104 * `MeenyMinyMoe/FooTest.HasBlahBlah/0` for `"meeny"`
1105 * `MeenyMinyMoe/FooTest.HasBlahBlah/1` for `"miny"`
1106 * `MeenyMinyMoe/FooTest.HasBlahBlah/2` for `"moe"`
1107
1108 You can use these names in [`--gtest_filter`](#running-a-subset-of-the-tests).
1109
1110 The following statement will instantiate all tests from `FooTest` again, each
1111 with parameter values `"cat"` and `"dog"` using the
1112 [`ValuesIn`](reference/testing.md#param-generators) parameter generator:
1113
1114 ```c++
1115 const char* pets[] = {"cat", "dog"};
1116 INSTANTIATE_TEST_SUITE_P(Pets, FooTest, testing::ValuesIn(pets));
1117 ```
1118
1119 The tests from the instantiation above will have these names:
1120
1121 * `Pets/FooTest.DoesBlah/0` for `"cat"`
1122 * `Pets/FooTest.DoesBlah/1` for `"dog"`
1123 * `Pets/FooTest.HasBlahBlah/0` for `"cat"`
1124 * `Pets/FooTest.HasBlahBlah/1` for `"dog"`
1125
1126 Please note that `INSTANTIATE_TEST_SUITE_P` will instantiate *all* tests in the
1127 given test suite, whether their definitions come before or *after* the
1128 `INSTANTIATE_TEST_SUITE_P` statement.
1129
1130 Additionally, by default, every `TEST_P` without a corresponding
1131 `INSTANTIATE_TEST_SUITE_P` causes a failing test in test suite
1132 `GoogleTestVerification`. If you have a test suite where that omission is not an
1133 error, for example it is in a library that may be linked in for other reasons or
1134 where the list of test cases is dynamic and may be empty, then this check can be
1135 suppressed by tagging the test suite:
1136
1137 ```c++
1138 GTEST_ALLOW_UNINSTANTIATED_PARAMETERIZED_TEST(FooTest);
1139 ```
1140
1141 You can see [sample7_unittest.cc] and [sample8_unittest.cc] for more examples.
1142
1143 [sample7_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample7_unittest.cc "Parameterized Test example"
1144 [sample8_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample8_unittest.cc "Parameterized Test example with multiple parameters"
1145
1146 ### Creating Value-Parameterized Abstract Tests
1147
1148 In the above, we define and instantiate `FooTest` in the *same* source file.
1149 Sometimes you may want to define value-parameterized tests in a library and let
1150 other people instantiate them later. This pattern is known as *abstract tests*.
1151 As an example of its application, when you are designing an interface you can
1152 write a standard suite of abstract tests (perhaps using a factory function as
1153 the test parameter) that all implementations of the interface are expected to
1154 pass. When someone implements the interface, they can instantiate your suite to
1155 get all the interface-conformance tests for free.
1156
1157 To define abstract tests, you should organize your code like this:
1158
1159 1. Put the definition of the parameterized test fixture class (e.g. `FooTest`)
1160 in a header file, say `foo_param_test.h`. Think of this as *declaring* your
1161 abstract tests.
1162 2. Put the `TEST_P` definitions in `foo_param_test.cc`, which includes
1163 `foo_param_test.h`. Think of this as *implementing* your abstract tests.
1164
1165 Once they are defined, you can instantiate them by including `foo_param_test.h`,
1166 invoking `INSTANTIATE_TEST_SUITE_P()`, and depending on the library target that
1167 contains `foo_param_test.cc`. You can instantiate the same abstract test suite
1168 multiple times, possibly in different source files.
1169
1170 ### Specifying Names for Value-Parameterized Test Parameters
1171
1172 The optional last argument to `INSTANTIATE_TEST_SUITE_P()` allows the user to
1173 specify a function or functor that generates custom test name suffixes based on
1174 the test parameters. The function should accept one argument of type
1175 `testing::TestParamInfo<class ParamType>`, and return `std::string`.
1176
1177 `testing::PrintToStringParamName` is a builtin test suffix generator that
1178 returns the value of `testing::PrintToString(GetParam())`. It does not work for
1179 `std::string` or C strings.
1180
1181 {: .callout .note}
1182 NOTE: test names must be non-empty, unique, and may only contain ASCII
1183 alphanumeric characters. In particular, they
1184 [should not contain underscores](faq.md#why-should-test-suite-names-and-test-names-not-contain-underscore)
1185
1186 ```c++
1187 class MyTestSuite : public testing::TestWithParam<int> {};
1188
1189 TEST_P(MyTestSuite, MyTest)
1190 {
1191 std::cout << "Example Test Param: " << GetParam() << std::endl;
1192 }
1193
1194 INSTANTIATE_TEST_SUITE_P(MyGroup, MyTestSuite, testing::Range(0, 10),
1195 testing::PrintToStringParamName());
1196 ```
1197
1198 Providing a custom functor allows for more control over test parameter name
1199 generation, especially for types where the automatic conversion does not
1200 generate helpful parameter names (e.g. strings as demonstrated above). The
1201 following example illustrates this for multiple parameters, an enumeration type
1202 and a string, and also demonstrates how to combine generators. It uses a lambda
1203 for conciseness:
1204
1205 ```c++
1206 enum class MyType { MY_FOO = 0, MY_BAR = 1 };
1207
1208 class MyTestSuite : public testing::TestWithParam<std::tuple<MyType, std::string>> {
1209 };
1210
1211 INSTANTIATE_TEST_SUITE_P(
1212 MyGroup, MyTestSuite,
1213 testing::Combine(
1214 testing::Values(MyType::MY_FOO, MyType::MY_BAR),
1215 testing::Values("A", "B")),
1216 [](const testing::TestParamInfo<MyTestSuite::ParamType>& info) {
1217 std::string name = absl::StrCat(
1218 std::get<0>(info.param) == MyType::MY_FOO ? "Foo" : "Bar",
1219 std::get<1>(info.param));
1220 absl::c_replace_if(name, [](char c) { return !std::isalnum(c); }, '_');
1221 return name;
1222 });
1223 ```
1224
1225 ## Typed Tests
1226
1227 Suppose you have multiple implementations of the same interface and want to make
1228 sure that all of them satisfy some common requirements. Or, you may have defined
1229 several types that are supposed to conform to the same "concept" and you want to
1230 verify it. In both cases, you want the same test logic repeated for different
1231 types.
1232
1233 While you can write one `TEST` or `TEST_F` for each type you want to test (and
1234 you may even factor the test logic into a function template that you invoke from
1235 the `TEST`), it's tedious and doesn't scale: if you want `m` tests over `n`
1236 types, you'll end up writing `m*n` `TEST`s.
1237
1238 *Typed tests* allow you to repeat the same test logic over a list of types. You
1239 only need to write the test logic once, although you must know the type list
1240 when writing typed tests. Here's how you do it:
1241
1242 First, define a fixture class template. It should be parameterized by a type.
1243 Remember to derive it from `::testing::Test`:
1244
1245 ```c++
1246 template <typename T>
1247 class FooTest : public testing::Test {
1248 public:
1249 ...
1250 using List = std::list<T>;
1251 static T shared_;
1252 T value_;
1253 };
1254 ```
1255
1256 Next, associate a list of types with the test suite, which will be repeated for
1257 each type in the list:
1258
1259 ```c++
1260 using MyTypes = ::testing::Types<char, int, unsigned int>;
1261 TYPED_TEST_SUITE(FooTest, MyTypes);
1262 ```
1263
1264 The type alias (`using` or `typedef`) is necessary for the `TYPED_TEST_SUITE`
1265 macro to parse correctly. Otherwise the compiler will think that each comma in
1266 the type list introduces a new macro argument.
1267
1268 Then, use `TYPED_TEST()` instead of `TEST_F()` to define a typed test for this
1269 test suite. You can repeat this as many times as you want:
1270
1271 ```c++
1272 TYPED_TEST(FooTest, DoesBlah) {
1273 // Inside a test, refer to the special name TypeParam to get the type
1274 // parameter. Since we are inside a derived class template, C++ requires
1275 // us to visit the members of FooTest via 'this'.
1276 TypeParam n = this->value_;
1277
1278 // To visit static members of the fixture, add the 'TestFixture::'
1279 // prefix.
1280 n += TestFixture::shared_;
1281
1282 // To refer to typedefs in the fixture, add the 'typename TestFixture::'
1283 // prefix. The 'typename' is required to satisfy the compiler.
1284 typename TestFixture::List values;
1285
1286 values.push_back(n);
1287 ...
1288 }
1289
1290 TYPED_TEST(FooTest, HasPropertyA) { ... }
1291 ```
1292
1293 You can see [sample6_unittest.cc] for a complete example.
1294
1295 [sample6_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample6_unittest.cc "Typed Test example"
1296
1297 ## Type-Parameterized Tests
1298
1299 *Type-parameterized tests* are like typed tests, except that they don't require
1300 you to know the list of types ahead of time. Instead, you can define the test
1301 logic first and instantiate it with different type lists later. You can even
1302 instantiate it more than once in the same program.
1303
1304 If you are designing an interface or concept, you can define a suite of
1305 type-parameterized tests to verify properties that any valid implementation of
1306 the interface/concept should have. Then, the author of each implementation can
1307 just instantiate the test suite with their type to verify that it conforms to
1308 the requirements, without having to write similar tests repeatedly. Here's an
1309 example:
1310
1311 First, define a fixture class template, as we did with typed tests:
1312
1313 ```c++
1314 template <typename T>
1315 class FooTest : public testing::Test {
1316 ...
1317 };
1318 ```
1319
1320 Next, declare that you will define a type-parameterized test suite:
1321
1322 ```c++
1323 TYPED_TEST_SUITE_P(FooTest);
1324 ```
1325
1326 Then, use `TYPED_TEST_P()` to define a type-parameterized test. You can repeat
1327 this as many times as you want:
1328
1329 ```c++
1330 TYPED_TEST_P(FooTest, DoesBlah) {
1331 // Inside a test, refer to TypeParam to get the type parameter.
1332 TypeParam n = 0;
1333 ...
1334 }
1335
1336 TYPED_TEST_P(FooTest, HasPropertyA) { ... }
1337 ```
1338
1339 Now the tricky part: you need to register all test patterns using the
1340 `REGISTER_TYPED_TEST_SUITE_P` macro before you can instantiate them. The first
1341 argument of the macro is the test suite name; the rest are the names of the
1342 tests in this test suite:
1343
1344 ```c++
1345 REGISTER_TYPED_TEST_SUITE_P(FooTest,
1346 DoesBlah, HasPropertyA);
1347 ```
1348
1349 Finally, you are free to instantiate the pattern with the types you want. If you
1350 put the above code in a header file, you can `#include` it in multiple C++
1351 source files and instantiate it multiple times.
1352
1353 ```c++
1354 using MyTypes = ::testing::Types<char, int, unsigned int>;
1355 INSTANTIATE_TYPED_TEST_SUITE_P(My, FooTest, MyTypes);
1356 ```
1357
1358 To distinguish different instances of the pattern, the first argument to the
1359 `INSTANTIATE_TYPED_TEST_SUITE_P` macro is a prefix that will be added to the
1360 actual test suite name. Remember to pick unique prefixes for different
1361 instances.
1362
1363 In the special case where the type list contains only one type, you can write
1364 that type directly without `::testing::Types<...>`, like this:
1365
1366 ```c++
1367 INSTANTIATE_TYPED_TEST_SUITE_P(My, FooTest, int);
1368 ```
1369
1370 You can see [sample6_unittest.cc] for a complete example.
1371
1372 ## Testing Private Code
1373
1374 If you change your software's internal implementation, your tests should not
1375 break as long as the change is not observable by users. Therefore, **per the
1376 black-box testing principle, most of the time you should test your code through
1377 its public interfaces.**
1378
1379 **If you still find yourself needing to test internal implementation code,
1380 consider if there's a better design.** The desire to test internal
1381 implementation is often a sign that the class is doing too much. Consider
1382 extracting an implementation class, and testing it. Then use that implementation
1383 class in the original class.
1384
1385 If you absolutely have to test non-public interface code though, you can. There
1386 are two cases to consider:
1387
1388 * Static functions ( *not* the same as static member functions!) or unnamed
1389 namespaces, and
1390 * Private or protected class members
1391
1392 To test them, we use the following special techniques:
1393
1394 * Both static functions and definitions/declarations in an unnamed namespace
1395 are only visible within the same translation unit. To test them, you can
1396 `#include` the entire `.cc` file being tested in your `*_test.cc` file.
1397 (#including `.cc` files is not a good way to reuse code - you should not do
1398 this in production code!)
1399
1400 However, a better approach is to move the private code into the
1401 `foo::internal` namespace, where `foo` is the namespace your project
1402 normally uses, and put the private declarations in a `*-internal.h` file.
1403 Your production `.cc` files and your tests are allowed to include this
1404 internal header, but your clients are not. This way, you can fully test your
1405 internal implementation without leaking it to your clients.
1406
1407 * Private class members are only accessible from within the class or by
1408 friends. To access a class' private members, you can declare your test
1409 fixture as a friend to the class and define accessors in your fixture. Tests
1410 using the fixture can then access the private members of your production
1411 class via the accessors in the fixture. Note that even though your fixture
1412 is a friend to your production class, your tests are not automatically
1413 friends to it, as they are technically defined in sub-classes of the
1414 fixture.
1415
1416 Another way to test private members is to refactor them into an
1417 implementation class, which is then declared in a `*-internal.h` file. Your
1418 clients aren't allowed to include this header but your tests can. Such is
1419 called the
1420 [Pimpl](https://www.gamedev.net/articles/programming/general-and-gameplay-programming/the-c-pimpl-r1794/)
1421 (Private Implementation) idiom.
1422
1423 Or, you can declare an individual test as a friend of your class by adding
1424 this line in the class body:
1425
1426 ```c++
1427 FRIEND_TEST(TestSuiteName, TestName);
1428 ```
1429
1430 For example,
1431
1432 ```c++
1433 // foo.h
1434 class Foo {
1435 ...
1436 private:
1437 FRIEND_TEST(FooTest, BarReturnsZeroOnNull);
1438
1439 int Bar(void* x);
1440 };
1441
1442 // foo_test.cc
1443 ...
1444 TEST(FooTest, BarReturnsZeroOnNull) {
1445 Foo foo;
1446 EXPECT_EQ(foo.Bar(NULL), 0); // Uses Foo's private member Bar().
1447 }
1448 ```
1449
1450 Pay special attention when your class is defined in a namespace. If you want
1451 your test fixtures and tests to be friends of your class, then they must be
1452 defined in the exact same namespace (no anonymous or inline namespaces).
1453
1454 For example, if the code to be tested looks like:
1455
1456 ```c++
1457 namespace my_namespace {
1458
1459 class Foo {
1460 friend class FooTest;
1461 FRIEND_TEST(FooTest, Bar);
1462 FRIEND_TEST(FooTest, Baz);
1463 ... definition of the class Foo ...
1464 };
1465
1466 } // namespace my_namespace
1467 ```
1468
1469 Your test code should be something like:
1470
1471 ```c++
1472 namespace my_namespace {
1473
1474 class FooTest : public testing::Test {
1475 protected:
1476 ...
1477 };
1478
1479 TEST_F(FooTest, Bar) { ... }
1480 TEST_F(FooTest, Baz) { ... }
1481
1482 } // namespace my_namespace
1483 ```
1484
1485 ## "Catching" Failures
1486
1487 If you are building a testing utility on top of googletest, you'll want to test
1488 your utility. What framework would you use to test it? googletest, of course.
1489
1490 The challenge is to verify that your testing utility reports failures correctly.
1491 In frameworks that report a failure by throwing an exception, you could catch
1492 the exception and assert on it. But googletest doesn't use exceptions, so how do
1493 we test that a piece of code generates an expected failure?
1494
1495 `"gtest/gtest-spi.h"` contains some constructs to do this.
1496 After #including this header, you can use
1497
1498 ```c++
1499 EXPECT_FATAL_FAILURE(statement, substring);
1500 ```
1501
1502 to assert that `statement` generates a fatal (e.g. `ASSERT_*`) failure in the
1503 current thread whose message contains the given `substring`, or use
1504
1505 ```c++
1506 EXPECT_NONFATAL_FAILURE(statement, substring);
1507 ```
1508
1509 if you are expecting a non-fatal (e.g. `EXPECT_*`) failure.
1510
1511 Only failures in the current thread are checked to determine the result of this
1512 type of expectations. If `statement` creates new threads, failures in these
1513 threads are also ignored. If you want to catch failures in other threads as
1514 well, use one of the following macros instead:
1515
1516 ```c++
1517 EXPECT_FATAL_FAILURE_ON_ALL_THREADS(statement, substring);
1518 EXPECT_NONFATAL_FAILURE_ON_ALL_THREADS(statement, substring);
1519 ```
1520
1521 {: .callout .note}
1522 NOTE: Assertions from multiple threads are currently not supported on Windows.
1523
1524 For technical reasons, there are some caveats:
1525
1526 1. You cannot stream a failure message to either macro.
1527
1528 2. `statement` in `EXPECT_FATAL_FAILURE{_ON_ALL_THREADS}()` cannot reference
1529 local non-static variables or non-static members of `this` object.
1530
1531 3. `statement` in `EXPECT_FATAL_FAILURE{_ON_ALL_THREADS}()` cannot return a
1532 value.
1533
1534 ## Registering tests programmatically
1535
1536 The `TEST` macros handle the vast majority of all use cases, but there are few
1537 where runtime registration logic is required. For those cases, the framework
1538 provides the `::testing::RegisterTest` that allows callers to register arbitrary
1539 tests dynamically.
1540
1541 This is an advanced API only to be used when the `TEST` macros are insufficient.
1542 The macros should be preferred when possible, as they avoid most of the
1543 complexity of calling this function.
1544
1545 It provides the following signature:
1546
1547 ```c++
1548 template <typename Factory>
1549 TestInfo* RegisterTest(const char* test_suite_name, const char* test_name,
1550 const char* type_param, const char* value_param,
1551 const char* file, int line, Factory factory);
1552 ```
1553
1554 The `factory` argument is a factory callable (move-constructible) object or
1555 function pointer that creates a new instance of the Test object. It handles
1556 ownership to the caller. The signature of the callable is `Fixture*()`, where
1557 `Fixture` is the test fixture class for the test. All tests registered with the
1558 same `test_suite_name` must return the same fixture type. This is checked at
1559 runtime.
1560
1561 The framework will infer the fixture class from the factory and will call the
1562 `SetUpTestSuite` and `TearDownTestSuite` for it.
1563
1564 Must be called before `RUN_ALL_TESTS()` is invoked, otherwise behavior is
1565 undefined.
1566
1567 Use case example:
1568
1569 ```c++
1570 class MyFixture : public testing::Test {
1571 public:
1572 // All of these optional, just like in regular macro usage.
1573 static void SetUpTestSuite() { ... }
1574 static void TearDownTestSuite() { ... }
1575 void SetUp() override { ... }
1576 void TearDown() override { ... }
1577 };
1578
1579 class MyTest : public MyFixture {
1580 public:
1581 explicit MyTest(int data) : data_(data) {}
1582 void TestBody() override { ... }
1583
1584 private:
1585 int data_;
1586 };
1587
1588 void RegisterMyTests(const std::vector<int>& values) {
1589 for (int v : values) {
1590 testing::RegisterTest(
1591 "MyFixture", ("Test" + std::to_string(v)).c_str(), nullptr,
1592 std::to_string(v).c_str(),
1593 __FILE__, __LINE__,
1594 // Important to use the fixture type as the return type here.
1595 [=]() -> MyFixture* { return new MyTest(v); });
1596 }
1597 }
1598 ...
1599 int main(int argc, char** argv) {
1600 testing::InitGoogleTest(&argc, argv);
1601 std::vector<int> values_to_test = LoadValuesFromConfig();
1602 RegisterMyTests(values_to_test);
1603 ...
1604 return RUN_ALL_TESTS();
1605 }
1606 ```
1607
1608 ## Getting the Current Test's Name
1609
1610 Sometimes a function may need to know the name of the currently running test.
1611 For example, you may be using the `SetUp()` method of your test fixture to set
1612 the golden file name based on which test is running. The
1613 [`TestInfo`](reference/testing.md#TestInfo) class has this information.
1614
1615 To obtain a `TestInfo` object for the currently running test, call
1616 `current_test_info()` on the [`UnitTest`](reference/testing.md#UnitTest)
1617 singleton object:
1618
1619 ```c++
1620 // Gets information about the currently running test.
1621 // Do NOT delete the returned object - it's managed by the UnitTest class.
1622 const testing::TestInfo* const test_info =
1623 testing::UnitTest::GetInstance()->current_test_info();
1624
1625 printf("We are in test %s of test suite %s.\n",
1626 test_info->name(),
1627 test_info->test_suite_name());
1628 ```
1629
1630 `current_test_info()` returns a null pointer if no test is running. In
1631 particular, you cannot find the test suite name in `SetUpTestSuite()`,
1632 `TearDownTestSuite()` (where you know the test suite name implicitly), or
1633 functions called from them.
1634
1635 ## Extending googletest by Handling Test Events
1636
1637 googletest provides an **event listener API** to let you receive notifications
1638 about the progress of a test program and test failures. The events you can
1639 listen to include the start and end of the test program, a test suite, or a test
1640 method, among others. You may use this API to augment or replace the standard
1641 console output, replace the XML output, or provide a completely different form
1642 of output, such as a GUI or a database. You can also use test events as
1643 checkpoints to implement a resource leak checker, for example.
1644
1645 ### Defining Event Listeners
1646
1647 To define a event listener, you subclass either
1648 [`testing::TestEventListener`](reference/testing.md#TestEventListener) or
1649 [`testing::EmptyTestEventListener`](reference/testing.md#EmptyTestEventListener)
1650 The former is an (abstract) interface, where *each pure virtual method can be
1651 overridden to handle a test event* (For example, when a test starts, the
1652 `OnTestStart()` method will be called.). The latter provides an empty
1653 implementation of all methods in the interface, such that a subclass only needs
1654 to override the methods it cares about.
1655
1656 When an event is fired, its context is passed to the handler function as an
1657 argument. The following argument types are used:
1658
1659 * UnitTest reflects the state of the entire test program,
1660 * TestSuite has information about a test suite, which can contain one or more
1661 tests,
1662 * TestInfo contains the state of a test, and
1663 * TestPartResult represents the result of a test assertion.
1664
1665 An event handler function can examine the argument it receives to find out
1666 interesting information about the event and the test program's state.
1667
1668 Here's an example:
1669
1670 ```c++
1671 class MinimalistPrinter : public testing::EmptyTestEventListener {
1672 // Called before a test starts.
1673 void OnTestStart(const testing::TestInfo& test_info) override {
1674 printf("*** Test %s.%s starting.\n",
1675 test_info.test_suite_name(), test_info.name());
1676 }
1677
1678 // Called after a failed assertion or a SUCCESS().
1679 void OnTestPartResult(const testing::TestPartResult& test_part_result) override {
1680 printf("%s in %s:%d\n%s\n",
1681 test_part_result.failed() ? "*** Failure" : "Success",
1682 test_part_result.file_name(),
1683 test_part_result.line_number(),
1684 test_part_result.summary());
1685 }
1686
1687 // Called after a test ends.
1688 void OnTestEnd(const testing::TestInfo& test_info) override {
1689 printf("*** Test %s.%s ending.\n",
1690 test_info.test_suite_name(), test_info.name());
1691 }
1692 };
1693 ```
1694
1695 ### Using Event Listeners
1696
1697 To use the event listener you have defined, add an instance of it to the
1698 googletest event listener list (represented by class
1699 [`TestEventListeners`](reference/testing.md#TestEventListeners) - note the "s"
1700 at the end of the name) in your `main()` function, before calling
1701 `RUN_ALL_TESTS()`:
1702
1703 ```c++
1704 int main(int argc, char** argv) {
1705 testing::InitGoogleTest(&argc, argv);
1706 // Gets hold of the event listener list.
1707 testing::TestEventListeners& listeners =
1708 testing::UnitTest::GetInstance()->listeners();
1709 // Adds a listener to the end. googletest takes the ownership.
1710 listeners.Append(new MinimalistPrinter);
1711 return RUN_ALL_TESTS();
1712 }
1713 ```
1714
1715 There's only one problem: the default test result printer is still in effect, so
1716 its output will mingle with the output from your minimalist printer. To suppress
1717 the default printer, just release it from the event listener list and delete it.
1718 You can do so by adding one line:
1719
1720 ```c++
1721 ...
1722 delete listeners.Release(listeners.default_result_printer());
1723 listeners.Append(new MinimalistPrinter);
1724 return RUN_ALL_TESTS();
1725 ```
1726
1727 Now, sit back and enjoy a completely different output from your tests. For more
1728 details, see [sample9_unittest.cc].
1729
1730 [sample9_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample9_unittest.cc "Event listener example"
1731
1732 You may append more than one listener to the list. When an `On*Start()` or
1733 `OnTestPartResult()` event is fired, the listeners will receive it in the order
1734 they appear in the list (since new listeners are added to the end of the list,
1735 the default text printer and the default XML generator will receive the event
1736 first). An `On*End()` event will be received by the listeners in the *reverse*
1737 order. This allows output by listeners added later to be framed by output from
1738 listeners added earlier.
1739
1740 ### Generating Failures in Listeners
1741
1742 You may use failure-raising macros (`EXPECT_*()`, `ASSERT_*()`, `FAIL()`, etc)
1743 when processing an event. There are some restrictions:
1744
1745 1. You cannot generate any failure in `OnTestPartResult()` (otherwise it will
1746 cause `OnTestPartResult()` to be called recursively).
1747 2. A listener that handles `OnTestPartResult()` is not allowed to generate any
1748 failure.
1749
1750 When you add listeners to the listener list, you should put listeners that
1751 handle `OnTestPartResult()` *before* listeners that can generate failures. This
1752 ensures that failures generated by the latter are attributed to the right test
1753 by the former.
1754
1755 See [sample10_unittest.cc] for an example of a failure-raising listener.
1756
1757 [sample10_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample10_unittest.cc "Failure-raising listener example"
1758
1759 ## Running Test Programs: Advanced Options
1760
1761 googletest test programs are ordinary executables. Once built, you can run them
1762 directly and affect their behavior via the following environment variables
1763 and/or command line flags. For the flags to work, your programs must call
1764 `::testing::InitGoogleTest()` before calling `RUN_ALL_TESTS()`.
1765
1766 To see a list of supported flags and their usage, please run your test program
1767 with the `--help` flag. You can also use `-h`, `-?`, or `/?` for short.
1768
1769 If an option is specified both by an environment variable and by a flag, the
1770 latter takes precedence.
1771
1772 ### Selecting Tests
1773
1774 #### Listing Test Names
1775
1776 Sometimes it is necessary to list the available tests in a program before
1777 running them so that a filter may be applied if needed. Including the flag
1778 `--gtest_list_tests` overrides all other flags and lists tests in the following
1779 format:
1780
1781 ```none
1782 TestSuite1.
1783 TestName1
1784 TestName2
1785 TestSuite2.
1786 TestName
1787 ```
1788
1789 None of the tests listed are actually run if the flag is provided. There is no
1790 corresponding environment variable for this flag.
1791
1792 #### Running a Subset of the Tests
1793
1794 By default, a googletest program runs all tests the user has defined. Sometimes,
1795 you want to run only a subset of the tests (e.g. for debugging or quickly
1796 verifying a change). If you set the `GTEST_FILTER` environment variable or the
1797 `--gtest_filter` flag to a filter string, googletest will only run the tests
1798 whose full names (in the form of `TestSuiteName.TestName`) match the filter.
1799
1800 The format of a filter is a '`:`'-separated list of wildcard patterns (called
1801 the *positive patterns*) optionally followed by a '`-`' and another
1802 '`:`'-separated pattern list (called the *negative patterns*). A test matches
1803 the filter if and only if it matches any of the positive patterns but does not
1804 match any of the negative patterns.
1805
1806 A pattern may contain `'*'` (matches any string) or `'?'` (matches any single
1807 character). For convenience, the filter `'*-NegativePatterns'` can be also
1808 written as `'-NegativePatterns'`.
1809
1810 For example:
1811
1812 * `./foo_test` Has no flag, and thus runs all its tests.
1813 * `./foo_test --gtest_filter=*` Also runs everything, due to the single
1814 match-everything `*` value.
1815 * `./foo_test --gtest_filter=FooTest.*` Runs everything in test suite
1816 `FooTest` .
1817 * `./foo_test --gtest_filter=*Null*:*Constructor*` Runs any test whose full
1818 name contains either `"Null"` or `"Constructor"` .
1819 * `./foo_test --gtest_filter=-*DeathTest.*` Runs all non-death tests.
1820 * `./foo_test --gtest_filter=FooTest.*-FooTest.Bar` Runs everything in test
1821 suite `FooTest` except `FooTest.Bar`.
1822 * `./foo_test --gtest_filter=FooTest.*:BarTest.*-FooTest.Bar:BarTest.Foo` Runs
1823 everything in test suite `FooTest` except `FooTest.Bar` and everything in
1824 test suite `BarTest` except `BarTest.Foo`.
1825
1826 #### Stop test execution upon first failure
1827
1828 By default, a googletest program runs all tests the user has defined. In some
1829 cases (e.g. iterative test development & execution) it may be desirable stop
1830 test execution upon first failure (trading improved latency for completeness).
1831 If `GTEST_FAIL_FAST` environment variable or `--gtest_fail_fast` flag is set,
1832 the test runner will stop execution as soon as the first test failure is found.
1833
1834 #### Temporarily Disabling Tests
1835
1836 If you have a broken test that you cannot fix right away, you can add the
1837 `DISABLED_` prefix to its name. This will exclude it from execution. This is
1838 better than commenting out the code or using `#if 0`, as disabled tests are
1839 still compiled (and thus won't rot).
1840
1841 If you need to disable all tests in a test suite, you can either add `DISABLED_`
1842 to the front of the name of each test, or alternatively add it to the front of
1843 the test suite name.
1844
1845 For example, the following tests won't be run by googletest, even though they
1846 will still be compiled:
1847
1848 ```c++
1849 // Tests that Foo does Abc.
1850 TEST(FooTest, DISABLED_DoesAbc) { ... }
1851
1852 class DISABLED_BarTest : public testing::Test { ... };
1853
1854 // Tests that Bar does Xyz.
1855 TEST_F(DISABLED_BarTest, DoesXyz) { ... }
1856 ```
1857
1858 {: .callout .note}
1859 NOTE: This feature should only be used for temporary pain-relief. You still have
1860 to fix the disabled tests at a later date. As a reminder, googletest will print
1861 a banner warning you if a test program contains any disabled tests.
1862
1863 {: .callout .tip}
1864 TIP: You can easily count the number of disabled tests you have using
1865 `grep`. This number can be used as a metric for
1866 improving your test quality.
1867
1868 #### Temporarily Enabling Disabled Tests
1869
1870 To include disabled tests in test execution, just invoke the test program with
1871 the `--gtest_also_run_disabled_tests` flag or set the
1872 `GTEST_ALSO_RUN_DISABLED_TESTS` environment variable to a value other than `0`.
1873 You can combine this with the `--gtest_filter` flag to further select which
1874 disabled tests to run.
1875
1876 ### Repeating the Tests
1877
1878 Once in a while you'll run into a test whose result is hit-or-miss. Perhaps it
1879 will fail only 1% of the time, making it rather hard to reproduce the bug under
1880 a debugger. This can be a major source of frustration.
1881
1882 The `--gtest_repeat` flag allows you to repeat all (or selected) test methods in
1883 a program many times. Hopefully, a flaky test will eventually fail and give you
1884 a chance to debug. Here's how to use it:
1885
1886 ```none
1887 $ foo_test --gtest_repeat=1000
1888 Repeat foo_test 1000 times and don't stop at failures.
1889
1890 $ foo_test --gtest_repeat=-1
1891 A negative count means repeating forever.
1892
1893 $ foo_test --gtest_repeat=1000 --gtest_break_on_failure
1894 Repeat foo_test 1000 times, stopping at the first failure. This
1895 is especially useful when running under a debugger: when the test
1896 fails, it will drop into the debugger and you can then inspect
1897 variables and stacks.
1898
1899 $ foo_test --gtest_repeat=1000 --gtest_filter=FooBar.*
1900 Repeat the tests whose name matches the filter 1000 times.
1901 ```
1902
1903 If your test program contains
1904 [global set-up/tear-down](#global-set-up-and-tear-down) code, it will be
1905 repeated in each iteration as well, as the flakiness may be in it. You can also
1906 specify the repeat count by setting the `GTEST_REPEAT` environment variable.
1907
1908 ### Shuffling the Tests
1909
1910 You can specify the `--gtest_shuffle` flag (or set the `GTEST_SHUFFLE`
1911 environment variable to `1`) to run the tests in a program in a random order.
1912 This helps to reveal bad dependencies between tests.
1913
1914 By default, googletest uses a random seed calculated from the current time.
1915 Therefore you'll get a different order every time. The console output includes
1916 the random seed value, such that you can reproduce an order-related test failure
1917 later. To specify the random seed explicitly, use the `--gtest_random_seed=SEED`
1918 flag (or set the `GTEST_RANDOM_SEED` environment variable), where `SEED` is an
1919 integer in the range [0, 99999]. The seed value 0 is special: it tells
1920 googletest to do the default behavior of calculating the seed from the current
1921 time.
1922
1923 If you combine this with `--gtest_repeat=N`, googletest will pick a different
1924 random seed and re-shuffle the tests in each iteration.
1925
1926 ### Distributing Test Functions to Multiple Machines
1927
1928 If you have more than one machine you can use to run a test program, you might
1929 want to run the test functions in parallel and get the result faster. We call
1930 this technique *sharding*, where each machine is called a *shard*.
1931
1932 GoogleTest is compatible with test sharding. To take advantage of this feature,
1933 your test runner (not part of GoogleTest) needs to do the following:
1934
1935 1. Allocate a number of machines (shards) to run the tests.
1936 1. On each shard, set the `GTEST_TOTAL_SHARDS` environment variable to the total
1937 number of shards. It must be the same for all shards.
1938 1. On each shard, set the `GTEST_SHARD_INDEX` environment variable to the index
1939 of the shard. Different shards must be assigned different indices, which
1940 must be in the range `[0, GTEST_TOTAL_SHARDS - 1]`.
1941 1. Run the same test program on all shards. When GoogleTest sees the above two
1942 environment variables, it will select a subset of the test functions to run.
1943 Across all shards, each test function in the program will be run exactly
1944 once.
1945 1. Wait for all shards to finish, then collect and report the results.
1946
1947 Your project may have tests that were written without GoogleTest and thus don't
1948 understand this protocol. In order for your test runner to figure out which test
1949 supports sharding, it can set the environment variable `GTEST_SHARD_STATUS_FILE`
1950 to a non-existent file path. If a test program supports sharding, it will create
1951 this file to acknowledge that fact; otherwise it will not create it. The actual
1952 contents of the file are not important at this time, although we may put some
1953 useful information in it in the future.
1954
1955 Here's an example to make it clear. Suppose you have a test program `foo_test`
1956 that contains the following 5 test functions:
1957
1958 ```
1959 TEST(A, V)
1960 TEST(A, W)
1961 TEST(B, X)
1962 TEST(B, Y)
1963 TEST(B, Z)
1964 ```
1965
1966 Suppose you have 3 machines at your disposal. To run the test functions in
1967 parallel, you would set `GTEST_TOTAL_SHARDS` to 3 on all machines, and set
1968 `GTEST_SHARD_INDEX` to 0, 1, and 2 on the machines respectively. Then you would
1969 run the same `foo_test` on each machine.
1970
1971 GoogleTest reserves the right to change how the work is distributed across the
1972 shards, but here's one possible scenario:
1973
1974 * Machine #0 runs `A.V` and `B.X`.
1975 * Machine #1 runs `A.W` and `B.Y`.
1976 * Machine #2 runs `B.Z`.
1977
1978 ### Controlling Test Output
1979
1980 #### Colored Terminal Output
1981
1982 googletest can use colors in its terminal output to make it easier to spot the
1983 important information:
1984
1985 <pre>...
1986 <font color="green">[----------]</font> 1 test from FooTest
1987 <font color="green">[ RUN ]</font> FooTest.DoesAbc
1988 <font color="green">[ OK ]</font> FooTest.DoesAbc
1989 <font color="green">[----------]</font> 2 tests from BarTest
1990 <font color="green">[ RUN ]</font> BarTest.HasXyzProperty
1991 <font color="green">[ OK ]</font> BarTest.HasXyzProperty
1992 <font color="green">[ RUN ]</font> BarTest.ReturnsTrueOnSuccess
1993 ... some error messages ...
1994 <font color="red">[ FAILED ]</font> BarTest.ReturnsTrueOnSuccess
1995 ...
1996 <font color="green">[==========]</font> 30 tests from 14 test suites ran.
1997 <font color="green">[ PASSED ]</font> 28 tests.
1998 <font color="red">[ FAILED ]</font> 2 tests, listed below:
1999 <font color="red">[ FAILED ]</font> BarTest.ReturnsTrueOnSuccess
2000 <font color="red">[ FAILED ]</font> AnotherTest.DoesXyz
2001
2002 2 FAILED TESTS
2003 </pre>
2004
2005 You can set the `GTEST_COLOR` environment variable or the `--gtest_color`
2006 command line flag to `yes`, `no`, or `auto` (the default) to enable colors,
2007 disable colors, or let googletest decide. When the value is `auto`, googletest
2008 will use colors if and only if the output goes to a terminal and (on non-Windows
2009 platforms) the `TERM` environment variable is set to `xterm` or `xterm-color`.
2010
2011 #### Suppressing test passes
2012
2013 By default, googletest prints 1 line of output for each test, indicating if it
2014 passed or failed. To show only test failures, run the test program with
2015 `--gtest_brief=1`, or set the GTEST_BRIEF environment variable to `1`.
2016
2017 #### Suppressing the Elapsed Time
2018
2019 By default, googletest prints the time it takes to run each test. To disable
2020 that, run the test program with the `--gtest_print_time=0` command line flag, or
2021 set the GTEST_PRINT_TIME environment variable to `0`.
2022
2023 #### Suppressing UTF-8 Text Output
2024
2025 In case of assertion failures, googletest prints expected and actual values of
2026 type `string` both as hex-encoded strings as well as in readable UTF-8 text if
2027 they contain valid non-ASCII UTF-8 characters. If you want to suppress the UTF-8
2028 text because, for example, you don't have an UTF-8 compatible output medium, run
2029 the test program with `--gtest_print_utf8=0` or set the `GTEST_PRINT_UTF8`
2030 environment variable to `0`.
2031
2032 #### Generating an XML Report
2033
2034 googletest can emit a detailed XML report to a file in addition to its normal
2035 textual output. The report contains the duration of each test, and thus can help
2036 you identify slow tests.
2037
2038 To generate the XML report, set the `GTEST_OUTPUT` environment variable or the
2039 `--gtest_output` flag to the string `"xml:path_to_output_file"`, which will
2040 create the file at the given location. You can also just use the string `"xml"`,
2041 in which case the output can be found in the `test_detail.xml` file in the
2042 current directory.
2043
2044 If you specify a directory (for example, `"xml:output/directory/"` on Linux or
2045 `"xml:output\directory\"` on Windows), googletest will create the XML file in
2046 that directory, named after the test executable (e.g. `foo_test.xml` for test
2047 program `foo_test` or `foo_test.exe`). If the file already exists (perhaps left
2048 over from a previous run), googletest will pick a different name (e.g.
2049 `foo_test_1.xml`) to avoid overwriting it.
2050
2051 The report is based on the `junitreport` Ant task. Since that format was
2052 originally intended for Java, a little interpretation is required to make it
2053 apply to googletest tests, as shown here:
2054
2055 ```xml
2056 <testsuites name="AllTests" ...>
2057 <testsuite name="test_case_name" ...>
2058 <testcase name="test_name" ...>
2059 <failure message="..."/>
2060 <failure message="..."/>
2061 <failure message="..."/>
2062 </testcase>
2063 </testsuite>
2064 </testsuites>
2065 ```
2066
2067 * The root `<testsuites>` element corresponds to the entire test program.
2068 * `<testsuite>` elements correspond to googletest test suites.
2069 * `<testcase>` elements correspond to googletest test functions.
2070
2071 For instance, the following program
2072
2073 ```c++
2074 TEST(MathTest, Addition) { ... }
2075 TEST(MathTest, Subtraction) { ... }
2076 TEST(LogicTest, NonContradiction) { ... }
2077 ```
2078
2079 could generate this report:
2080
2081 ```xml
2082 <?xml version="1.0" encoding="UTF-8"?>
2083 <testsuites tests="3" failures="1" errors="0" time="0.035" timestamp="2011-10-31T18:52:42" name="AllTests">
2084 <testsuite name="MathTest" tests="2" failures="1" errors="0" time="0.015">
2085 <testcase name="Addition" status="run" time="0.007" classname="">
2086 <failure message="Value of: add(1, 1)&#x0A; Actual: 3&#x0A;Expected: 2" type="">...</failure>
2087 <failure message="Value of: add(1, -1)&#x0A; Actual: 1&#x0A;Expected: 0" type="">...</failure>
2088 </testcase>
2089 <testcase name="Subtraction" status="run" time="0.005" classname="">
2090 </testcase>
2091 </testsuite>
2092 <testsuite name="LogicTest" tests="1" failures="0" errors="0" time="0.005">
2093 <testcase name="NonContradiction" status="run" time="0.005" classname="">
2094 </testcase>
2095 </testsuite>
2096 </testsuites>
2097 ```
2098
2099 Things to note:
2100
2101 * The `tests` attribute of a `<testsuites>` or `<testsuite>` element tells how
2102 many test functions the googletest program or test suite contains, while the
2103 `failures` attribute tells how many of them failed.
2104
2105 * The `time` attribute expresses the duration of the test, test suite, or
2106 entire test program in seconds.
2107
2108 * The `timestamp` attribute records the local date and time of the test
2109 execution.
2110
2111 * Each `<failure>` element corresponds to a single failed googletest
2112 assertion.
2113
2114 #### Generating a JSON Report
2115
2116 googletest can also emit a JSON report as an alternative format to XML. To
2117 generate the JSON report, set the `GTEST_OUTPUT` environment variable or the
2118 `--gtest_output` flag to the string `"json:path_to_output_file"`, which will
2119 create the file at the given location. You can also just use the string
2120 `"json"`, in which case the output can be found in the `test_detail.json` file
2121 in the current directory.
2122
2123 The report format conforms to the following JSON Schema:
2124
2125 ```json
2126 {
2127 "$schema": "http://json-schema.org/schema#",
2128 "type": "object",
2129 "definitions": {
2130 "TestCase": {
2131 "type": "object",
2132 "properties": {
2133 "name": { "type": "string" },
2134 "tests": { "type": "integer" },
2135 "failures": { "type": "integer" },
2136 "disabled": { "type": "integer" },
2137 "time": { "type": "string" },
2138 "testsuite": {
2139 "type": "array",
2140 "items": {
2141 "$ref": "#/definitions/TestInfo"
2142 }
2143 }
2144 }
2145 },
2146 "TestInfo": {
2147 "type": "object",
2148 "properties": {
2149 "name": { "type": "string" },
2150 "status": {
2151 "type": "string",
2152 "enum": ["RUN", "NOTRUN"]
2153 },
2154 "time": { "type": "string" },
2155 "classname": { "type": "string" },
2156 "failures": {
2157 "type": "array",
2158 "items": {
2159 "$ref": "#/definitions/Failure"
2160 }
2161 }
2162 }
2163 },
2164 "Failure": {
2165 "type": "object",
2166 "properties": {
2167 "failures": { "type": "string" },
2168 "type": { "type": "string" }
2169 }
2170 }
2171 },
2172 "properties": {
2173 "tests": { "type": "integer" },
2174 "failures": { "type": "integer" },
2175 "disabled": { "type": "integer" },
2176 "errors": { "type": "integer" },
2177 "timestamp": {
2178 "type": "string",
2179 "format": "date-time"
2180 },
2181 "time": { "type": "string" },
2182 "name": { "type": "string" },
2183 "testsuites": {
2184 "type": "array",
2185 "items": {
2186 "$ref": "#/definitions/TestCase"
2187 }
2188 }
2189 }
2190 }
2191 ```
2192
2193 The report uses the format that conforms to the following Proto3 using the
2194 [JSON encoding](https://developers.google.com/protocol-buffers/docs/proto3#json):
2195
2196 ```proto
2197 syntax = "proto3";
2198
2199 package googletest;
2200
2201 import "google/protobuf/timestamp.proto";
2202 import "google/protobuf/duration.proto";
2203
2204 message UnitTest {
2205 int32 tests = 1;
2206 int32 failures = 2;
2207 int32 disabled = 3;
2208 int32 errors = 4;
2209 google.protobuf.Timestamp timestamp = 5;
2210 google.protobuf.Duration time = 6;
2211 string name = 7;
2212 repeated TestCase testsuites = 8;
2213 }
2214
2215 message TestCase {
2216 string name = 1;
2217 int32 tests = 2;
2218 int32 failures = 3;
2219 int32 disabled = 4;
2220 int32 errors = 5;
2221 google.protobuf.Duration time = 6;
2222 repeated TestInfo testsuite = 7;
2223 }
2224
2225 message TestInfo {
2226 string name = 1;
2227 enum Status {
2228 RUN = 0;
2229 NOTRUN = 1;
2230 }
2231 Status status = 2;
2232 google.protobuf.Duration time = 3;
2233 string classname = 4;
2234 message Failure {
2235 string failures = 1;
2236 string type = 2;
2237 }
2238 repeated Failure failures = 5;
2239 }
2240 ```
2241
2242 For instance, the following program
2243
2244 ```c++
2245 TEST(MathTest, Addition) { ... }
2246 TEST(MathTest, Subtraction) { ... }
2247 TEST(LogicTest, NonContradiction) { ... }
2248 ```
2249
2250 could generate this report:
2251
2252 ```json
2253 {
2254 "tests": 3,
2255 "failures": 1,
2256 "errors": 0,
2257 "time": "0.035s",
2258 "timestamp": "2011-10-31T18:52:42Z",
2259 "name": "AllTests",
2260 "testsuites": [
2261 {
2262 "name": "MathTest",
2263 "tests": 2,
2264 "failures": 1,
2265 "errors": 0,
2266 "time": "0.015s",
2267 "testsuite": [
2268 {
2269 "name": "Addition",
2270 "status": "RUN",
2271 "time": "0.007s",
2272 "classname": "",
2273 "failures": [
2274 {
2275 "message": "Value of: add(1, 1)\n Actual: 3\nExpected: 2",
2276 "type": ""
2277 },
2278 {
2279 "message": "Value of: add(1, -1)\n Actual: 1\nExpected: 0",
2280 "type": ""
2281 }
2282 ]
2283 },
2284 {
2285 "name": "Subtraction",
2286 "status": "RUN",
2287 "time": "0.005s",
2288 "classname": ""
2289 }
2290 ]
2291 },
2292 {
2293 "name": "LogicTest",
2294 "tests": 1,
2295 "failures": 0,
2296 "errors": 0,
2297 "time": "0.005s",
2298 "testsuite": [
2299 {
2300 "name": "NonContradiction",
2301 "status": "RUN",
2302 "time": "0.005s",
2303 "classname": ""
2304 }
2305 ]
2306 }
2307 ]
2308 }
2309 ```
2310
2311 {: .callout .important}
2312 IMPORTANT: The exact format of the JSON document is subject to change.
2313
2314 ### Controlling How Failures Are Reported
2315
2316 #### Detecting Test Premature Exit
2317
2318 Google Test implements the _premature-exit-file_ protocol for test runners to
2319 catch any kind of unexpected exits of test programs. Upon start, Google Test
2320 creates the file which will be automatically deleted after all work has been
2321 finished. Then, the test runner can check if this file exists. In case the file
2322 remains undeleted, the inspected test has exited prematurely.
2323
2324 This feature is enabled only if the `TEST_PREMATURE_EXIT_FILE` environment
2325 variable has been set.
2326
2327 #### Turning Assertion Failures into Break-Points
2328
2329 When running test programs under a debugger, it's very convenient if the
2330 debugger can catch an assertion failure and automatically drop into interactive
2331 mode. googletest's *break-on-failure* mode supports this behavior.
2332
2333 To enable it, set the `GTEST_BREAK_ON_FAILURE` environment variable to a value
2334 other than `0`. Alternatively, you can use the `--gtest_break_on_failure`
2335 command line flag.
2336
2337 #### Disabling Catching Test-Thrown Exceptions
2338
2339 googletest can be used either with or without exceptions enabled. If a test
2340 throws a C++ exception or (on Windows) a structured exception (SEH), by default
2341 googletest catches it, reports it as a test failure, and continues with the next
2342 test method. This maximizes the coverage of a test run. Also, on Windows an
2343 uncaught exception will cause a pop-up window, so catching the exceptions allows
2344 you to run the tests automatically.
2345
2346 When debugging the test failures, however, you may instead want the exceptions
2347 to be handled by the debugger, such that you can examine the call stack when an
2348 exception is thrown. To achieve that, set the `GTEST_CATCH_EXCEPTIONS`
2349 environment variable to `0`, or use the `--gtest_catch_exceptions=0` flag when
2350 running the tests.
2351
2352 ### Sanitizer Integration
2353
2354 The
2355 [Undefined Behavior Sanitizer](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html),
2356 [Address Sanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizer),
2357 and
2358 [Thread Sanitizer](https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual)
2359 all provide weak functions that you can override to trigger explicit failures
2360 when they detect sanitizer errors, such as creating a reference from `nullptr`.
2361 To override these functions, place definitions for them in a source file that
2362 you compile as part of your main binary:
2363
2364 ```
2365 extern "C" {
2366 void __ubsan_on_report() {
2367 FAIL() << "Encountered an undefined behavior sanitizer error";
2368 }
2369 void __asan_on_error() {
2370 FAIL() << "Encountered an address sanitizer error";
2371 }
2372 void __tsan_on_report() {
2373 FAIL() << "Encountered a thread sanitizer error";
2374 }
2375 } // extern "C"
2376 ```
2377
2378 After compiling your project with one of the sanitizers enabled, if a particular
2379 test triggers a sanitizer error, googletest will report that it failed.