| 1 |
# Advanced googletest Topics |
| 2 |
|
| 3 |
## Introduction |
| 4 |
|
| 5 |
Now that you have read the [googletest Primer](primer.md) and learned how to |
| 6 |
write tests using googletest, it's time to learn some new tricks. This document |
| 7 |
will show you more assertions as well as how to construct complex failure |
| 8 |
messages, propagate fatal failures, reuse and speed up your test fixtures, and |
| 9 |
use various flags with your tests. |
| 10 |
|
| 11 |
## More Assertions |
| 12 |
|
| 13 |
This section covers some less frequently used, but still significant, |
| 14 |
assertions. |
| 15 |
|
| 16 |
### Explicit Success and Failure |
| 17 |
|
| 18 |
See [Explicit Success and Failure](reference/assertions.md#success-failure) in |
| 19 |
the Assertions Reference. |
| 20 |
|
| 21 |
### Exception Assertions |
| 22 |
|
| 23 |
See [Exception Assertions](reference/assertions.md#exceptions) in the Assertions |
| 24 |
Reference. |
| 25 |
|
| 26 |
### Predicate Assertions for Better Error Messages |
| 27 |
|
| 28 |
Even though googletest has a rich set of assertions, they can never be complete, |
| 29 |
as it's impossible (nor a good idea) to anticipate all scenarios a user might |
| 30 |
run into. Therefore, sometimes a user has to use `EXPECT_TRUE()` to check a |
| 31 |
complex expression, for lack of a better macro. This has the problem of not |
| 32 |
showing you the values of the parts of the expression, making it hard to |
| 33 |
understand what went wrong. As a workaround, some users choose to construct the |
| 34 |
failure message by themselves, streaming it into `EXPECT_TRUE()`. However, this |
| 35 |
is awkward especially when the expression has side-effects or is expensive to |
| 36 |
evaluate. |
| 37 |
|
| 38 |
googletest gives you three different options to solve this problem: |
| 39 |
|
| 40 |
#### Using an Existing Boolean Function |
| 41 |
|
| 42 |
If you already have a function or functor that returns `bool` (or a type that |
| 43 |
can be implicitly converted to `bool`), you can use it in a *predicate |
| 44 |
assertion* to get the function arguments printed for free. See |
| 45 |
[`EXPECT_PRED*`](reference/assertions.md#EXPECT_PRED) in the Assertions |
| 46 |
Reference for details. |
| 47 |
|
| 48 |
#### Using a Function That Returns an AssertionResult |
| 49 |
|
| 50 |
While `EXPECT_PRED*()` and friends are handy for a quick job, the syntax is not |
| 51 |
satisfactory: you have to use different macros for different arities, and it |
| 52 |
feels more like Lisp than C++. The `::testing::AssertionResult` class solves |
| 53 |
this problem. |
| 54 |
|
| 55 |
An `AssertionResult` object represents the result of an assertion (whether it's |
| 56 |
a success or a failure, and an associated message). You can create an |
| 57 |
`AssertionResult` using one of these factory functions: |
| 58 |
|
| 59 |
```c++ |
| 60 |
namespace testing { |
| 61 |
|
| 62 |
// Returns an AssertionResult object to indicate that an assertion has |
| 63 |
// succeeded. |
| 64 |
AssertionResult AssertionSuccess(); |
| 65 |
|
| 66 |
// Returns an AssertionResult object to indicate that an assertion has |
| 67 |
// failed. |
| 68 |
AssertionResult AssertionFailure(); |
| 69 |
|
| 70 |
} |
| 71 |
``` |
| 72 |
|
| 73 |
You can then use the `<<` operator to stream messages to the `AssertionResult` |
| 74 |
object. |
| 75 |
|
| 76 |
To provide more readable messages in Boolean assertions (e.g. `EXPECT_TRUE()`), |
| 77 |
write a predicate function that returns `AssertionResult` instead of `bool`. For |
| 78 |
example, if you define `IsEven()` as: |
| 79 |
|
| 80 |
```c++ |
| 81 |
testing::AssertionResult IsEven(int n) { |
| 82 |
if ((n % 2) == 0) |
| 83 |
return testing::AssertionSuccess(); |
| 84 |
else |
| 85 |
return testing::AssertionFailure() << n << " is odd"; |
| 86 |
} |
| 87 |
``` |
| 88 |
|
| 89 |
instead of: |
| 90 |
|
| 91 |
```c++ |
| 92 |
bool IsEven(int n) { |
| 93 |
return (n % 2) == 0; |
| 94 |
} |
| 95 |
``` |
| 96 |
|
| 97 |
the failed assertion `EXPECT_TRUE(IsEven(Fib(4)))` will print: |
| 98 |
|
| 99 |
```none |
| 100 |
Value of: IsEven(Fib(4)) |
| 101 |
Actual: false (3 is odd) |
| 102 |
Expected: true |
| 103 |
``` |
| 104 |
|
| 105 |
instead of a more opaque |
| 106 |
|
| 107 |
```none |
| 108 |
Value of: IsEven(Fib(4)) |
| 109 |
Actual: false |
| 110 |
Expected: true |
| 111 |
``` |
| 112 |
|
| 113 |
If you want informative messages in `EXPECT_FALSE` and `ASSERT_FALSE` as well |
| 114 |
(one third of Boolean assertions in the Google code base are negative ones), and |
| 115 |
are fine with making the predicate slower in the success case, you can supply a |
| 116 |
success message: |
| 117 |
|
| 118 |
```c++ |
| 119 |
testing::AssertionResult IsEven(int n) { |
| 120 |
if ((n % 2) == 0) |
| 121 |
return testing::AssertionSuccess() << n << " is even"; |
| 122 |
else |
| 123 |
return testing::AssertionFailure() << n << " is odd"; |
| 124 |
} |
| 125 |
``` |
| 126 |
|
| 127 |
Then the statement `EXPECT_FALSE(IsEven(Fib(6)))` will print |
| 128 |
|
| 129 |
```none |
| 130 |
Value of: IsEven(Fib(6)) |
| 131 |
Actual: true (8 is even) |
| 132 |
Expected: false |
| 133 |
``` |
| 134 |
|
| 135 |
#### Using a Predicate-Formatter |
| 136 |
|
| 137 |
If you find the default message generated by |
| 138 |
[`EXPECT_PRED*`](reference/assertions.md#EXPECT_PRED) and |
| 139 |
[`EXPECT_TRUE`](reference/assertions.md#EXPECT_TRUE) unsatisfactory, or some |
| 140 |
arguments to your predicate do not support streaming to `ostream`, you can |
| 141 |
instead use *predicate-formatter assertions* to *fully* customize how the |
| 142 |
message is formatted. See |
| 143 |
[`EXPECT_PRED_FORMAT*`](reference/assertions.md#EXPECT_PRED_FORMAT) in the |
| 144 |
Assertions Reference for details. |
| 145 |
|
| 146 |
### Floating-Point Comparison |
| 147 |
|
| 148 |
See [Floating-Point Comparison](reference/assertions.md#floating-point) in the |
| 149 |
Assertions Reference. |
| 150 |
|
| 151 |
#### Floating-Point Predicate-Format Functions |
| 152 |
|
| 153 |
Some floating-point operations are useful, but not that often used. In order to |
| 154 |
avoid an explosion of new macros, we provide them as predicate-format functions |
| 155 |
that can be used in the predicate assertion macro |
| 156 |
[`EXPECT_PRED_FORMAT2`](reference/assertions.md#EXPECT_PRED_FORMAT), for |
| 157 |
example: |
| 158 |
|
| 159 |
```c++ |
| 160 |
using ::testing::FloatLE; |
| 161 |
using ::testing::DoubleLE; |
| 162 |
... |
| 163 |
EXPECT_PRED_FORMAT2(FloatLE, val1, val2); |
| 164 |
EXPECT_PRED_FORMAT2(DoubleLE, val1, val2); |
| 165 |
``` |
| 166 |
|
| 167 |
The above code verifies that `val1` is less than, or approximately equal to, |
| 168 |
`val2`. |
| 169 |
|
| 170 |
### Asserting Using gMock Matchers |
| 171 |
|
| 172 |
See [`EXPECT_THAT`](reference/assertions.md#EXPECT_THAT) in the Assertions |
| 173 |
Reference. |
| 174 |
|
| 175 |
### More String Assertions |
| 176 |
|
| 177 |
(Please read the [previous](#asserting-using-gmock-matchers) section first if |
| 178 |
you haven't.) |
| 179 |
|
| 180 |
You can use the gMock [string matchers](reference/matchers.md#string-matchers) |
| 181 |
with [`EXPECT_THAT`](reference/assertions.md#EXPECT_THAT) to do more string |
| 182 |
comparison tricks (sub-string, prefix, suffix, regular expression, and etc). For |
| 183 |
example, |
| 184 |
|
| 185 |
```c++ |
| 186 |
using ::testing::HasSubstr; |
| 187 |
using ::testing::MatchesRegex; |
| 188 |
... |
| 189 |
ASSERT_THAT(foo_string, HasSubstr("needle")); |
| 190 |
EXPECT_THAT(bar_string, MatchesRegex("\\w*\\d+")); |
| 191 |
``` |
| 192 |
|
| 193 |
### Windows HRESULT assertions |
| 194 |
|
| 195 |
See [Windows HRESULT Assertions](reference/assertions.md#HRESULT) in the |
| 196 |
Assertions Reference. |
| 197 |
|
| 198 |
### Type Assertions |
| 199 |
|
| 200 |
You can call the function |
| 201 |
|
| 202 |
```c++ |
| 203 |
::testing::StaticAssertTypeEq<T1, T2>(); |
| 204 |
``` |
| 205 |
|
| 206 |
to assert that types `T1` and `T2` are the same. The function does nothing if |
| 207 |
the assertion is satisfied. If the types are different, the function call will |
| 208 |
fail to compile, the compiler error message will say that `T1 and T2 are not the |
| 209 |
same type` and most likely (depending on the compiler) show you the actual |
| 210 |
values of `T1` and `T2`. This is mainly useful inside template code. |
| 211 |
|
| 212 |
**Caveat**: When used inside a member function of a class template or a function |
| 213 |
template, `StaticAssertTypeEq<T1, T2>()` is effective only if the function is |
| 214 |
instantiated. For example, given: |
| 215 |
|
| 216 |
```c++ |
| 217 |
template <typename T> class Foo { |
| 218 |
public: |
| 219 |
void Bar() { testing::StaticAssertTypeEq<int, T>(); } |
| 220 |
}; |
| 221 |
``` |
| 222 |
|
| 223 |
the code: |
| 224 |
|
| 225 |
```c++ |
| 226 |
void Test1() { Foo<bool> foo; } |
| 227 |
``` |
| 228 |
|
| 229 |
will not generate a compiler error, as `Foo<bool>::Bar()` is never actually |
| 230 |
instantiated. Instead, you need: |
| 231 |
|
| 232 |
```c++ |
| 233 |
void Test2() { Foo<bool> foo; foo.Bar(); } |
| 234 |
``` |
| 235 |
|
| 236 |
to cause a compiler error. |
| 237 |
|
| 238 |
### Assertion Placement |
| 239 |
|
| 240 |
You can use assertions in any C++ function. In particular, it doesn't have to be |
| 241 |
a method of the test fixture class. The one constraint is that assertions that |
| 242 |
generate a fatal failure (`FAIL*` and `ASSERT_*`) can only be used in |
| 243 |
void-returning functions. This is a consequence of Google's not using |
| 244 |
exceptions. By placing it in a non-void function you'll get a confusing compile |
| 245 |
error like `"error: void value not ignored as it ought to be"` or `"cannot |
| 246 |
initialize return object of type 'bool' with an rvalue of type 'void'"` or |
| 247 |
`"error: no viable conversion from 'void' to 'string'"`. |
| 248 |
|
| 249 |
If you need to use fatal assertions in a function that returns non-void, one |
| 250 |
option is to make the function return the value in an out parameter instead. For |
| 251 |
example, you can rewrite `T2 Foo(T1 x)` to `void Foo(T1 x, T2* result)`. You |
| 252 |
need to make sure that `*result` contains some sensible value even when the |
| 253 |
function returns prematurely. As the function now returns `void`, you can use |
| 254 |
any assertion inside of it. |
| 255 |
|
| 256 |
If changing the function's type is not an option, you should just use assertions |
| 257 |
that generate non-fatal failures, such as `ADD_FAILURE*` and `EXPECT_*`. |
| 258 |
|
| 259 |
{: .callout .note} |
| 260 |
NOTE: Constructors and destructors are not considered void-returning functions, |
| 261 |
according to the C++ language specification, and so you may not use fatal |
| 262 |
assertions in them; you'll get a compilation error if you try. Instead, either |
| 263 |
call `abort` and crash the entire test executable, or put the fatal assertion in |
| 264 |
a `SetUp`/`TearDown` function; see |
| 265 |
[constructor/destructor vs. `SetUp`/`TearDown`](faq.md#CtorVsSetUp) |
| 266 |
|
| 267 |
{: .callout .warning} |
| 268 |
WARNING: A fatal assertion in a helper function (private void-returning method) |
| 269 |
called from a constructor or destructor does not terminate the current test, as |
| 270 |
your intuition might suggest: it merely returns from the constructor or |
| 271 |
destructor early, possibly leaving your object in a partially-constructed or |
| 272 |
partially-destructed state! You almost certainly want to `abort` or use |
| 273 |
`SetUp`/`TearDown` instead. |
| 274 |
|
| 275 |
## Skipping test execution |
| 276 |
|
| 277 |
Related to the assertions `SUCCEED()` and `FAIL()`, you can prevent further test |
| 278 |
execution at runtime with the `GTEST_SKIP()` macro. This is useful when you need |
| 279 |
to check for preconditions of the system under test during runtime and skip |
| 280 |
tests in a meaningful way. |
| 281 |
|
| 282 |
`GTEST_SKIP()` can be used in individual test cases or in the `SetUp()` methods |
| 283 |
of classes derived from either `::testing::Environment` or `::testing::Test`. |
| 284 |
For example: |
| 285 |
|
| 286 |
```c++ |
| 287 |
TEST(SkipTest, DoesSkip) { |
| 288 |
GTEST_SKIP() << "Skipping single test"; |
| 289 |
EXPECT_EQ(0, 1); // Won't fail; it won't be executed |
| 290 |
} |
| 291 |
|
| 292 |
class SkipFixture : public ::testing::Test { |
| 293 |
protected: |
| 294 |
void SetUp() override { |
| 295 |
GTEST_SKIP() << "Skipping all tests for this fixture"; |
| 296 |
} |
| 297 |
}; |
| 298 |
|
| 299 |
// Tests for SkipFixture won't be executed. |
| 300 |
TEST_F(SkipFixture, SkipsOneTest) { |
| 301 |
EXPECT_EQ(5, 7); // Won't fail |
| 302 |
} |
| 303 |
``` |
| 304 |
|
| 305 |
As with assertion macros, you can stream a custom message into `GTEST_SKIP()`. |
| 306 |
|
| 307 |
## Teaching googletest How to Print Your Values |
| 308 |
|
| 309 |
When a test assertion such as `EXPECT_EQ` fails, googletest prints the argument |
| 310 |
values to help you debug. It does this using a user-extensible value printer. |
| 311 |
|
| 312 |
This printer knows how to print built-in C++ types, native arrays, STL |
| 313 |
containers, and any type that supports the `<<` operator. For other types, it |
| 314 |
prints the raw bytes in the value and hopes that you the user can figure it out. |
| 315 |
|
| 316 |
As mentioned earlier, the printer is *extensible*. That means you can teach it |
| 317 |
to do a better job at printing your particular type than to dump the bytes. To |
| 318 |
do that, define `<<` for your type: |
| 319 |
|
| 320 |
```c++ |
| 321 |
#include <ostream> |
| 322 |
|
| 323 |
namespace foo { |
| 324 |
|
| 325 |
class Bar { // We want googletest to be able to print instances of this. |
| 326 |
... |
| 327 |
// Create a free inline friend function. |
| 328 |
friend std::ostream& operator<<(std::ostream& os, const Bar& bar) { |
| 329 |
return os << bar.DebugString(); // whatever needed to print bar to os |
| 330 |
} |
| 331 |
}; |
| 332 |
|
| 333 |
// If you can't declare the function in the class it's important that the |
| 334 |
// << operator is defined in the SAME namespace that defines Bar. C++'s look-up |
| 335 |
// rules rely on that. |
| 336 |
std::ostream& operator<<(std::ostream& os, const Bar& bar) { |
| 337 |
return os << bar.DebugString(); // whatever needed to print bar to os |
| 338 |
} |
| 339 |
|
| 340 |
} // namespace foo |
| 341 |
``` |
| 342 |
|
| 343 |
Sometimes, this might not be an option: your team may consider it bad style to |
| 344 |
have a `<<` operator for `Bar`, or `Bar` may already have a `<<` operator that |
| 345 |
doesn't do what you want (and you cannot change it). If so, you can instead |
| 346 |
define a `PrintTo()` function like this: |
| 347 |
|
| 348 |
```c++ |
| 349 |
#include <ostream> |
| 350 |
|
| 351 |
namespace foo { |
| 352 |
|
| 353 |
class Bar { |
| 354 |
... |
| 355 |
friend void PrintTo(const Bar& bar, std::ostream* os) { |
| 356 |
*os << bar.DebugString(); // whatever needed to print bar to os |
| 357 |
} |
| 358 |
}; |
| 359 |
|
| 360 |
// If you can't declare the function in the class it's important that PrintTo() |
| 361 |
// is defined in the SAME namespace that defines Bar. C++'s look-up rules rely |
| 362 |
// on that. |
| 363 |
void PrintTo(const Bar& bar, std::ostream* os) { |
| 364 |
*os << bar.DebugString(); // whatever needed to print bar to os |
| 365 |
} |
| 366 |
|
| 367 |
} // namespace foo |
| 368 |
``` |
| 369 |
|
| 370 |
If you have defined both `<<` and `PrintTo()`, the latter will be used when |
| 371 |
googletest is concerned. This allows you to customize how the value appears in |
| 372 |
googletest's output without affecting code that relies on the behavior of its |
| 373 |
`<<` operator. |
| 374 |
|
| 375 |
If you want to print a value `x` using googletest's value printer yourself, just |
| 376 |
call `::testing::PrintToString(x)`, which returns an `std::string`: |
| 377 |
|
| 378 |
```c++ |
| 379 |
vector<pair<Bar, int> > bar_ints = GetBarIntVector(); |
| 380 |
|
| 381 |
EXPECT_TRUE(IsCorrectBarIntVector(bar_ints)) |
| 382 |
<< "bar_ints = " << testing::PrintToString(bar_ints); |
| 383 |
``` |
| 384 |
|
| 385 |
## Death Tests |
| 386 |
|
| 387 |
In many applications, there are assertions that can cause application failure if |
| 388 |
a condition is not met. These consistency checks, which ensure that the program |
| 389 |
is in a known good state, are there to fail at the earliest possible time after |
| 390 |
some program state is corrupted. If the assertion checks the wrong condition, |
| 391 |
then the program may proceed in an erroneous state, which could lead to memory |
| 392 |
corruption, security holes, or worse. Hence it is vitally important to test that |
| 393 |
such assertion statements work as expected. |
| 394 |
|
| 395 |
Since these precondition checks cause the processes to die, we call such tests |
| 396 |
_death tests_. More generally, any test that checks that a program terminates |
| 397 |
(except by throwing an exception) in an expected fashion is also a death test. |
| 398 |
|
| 399 |
Note that if a piece of code throws an exception, we don't consider it "death" |
| 400 |
for the purpose of death tests, as the caller of the code could catch the |
| 401 |
exception and avoid the crash. If you want to verify exceptions thrown by your |
| 402 |
code, see [Exception Assertions](#ExceptionAssertions). |
| 403 |
|
| 404 |
If you want to test `EXPECT_*()/ASSERT_*()` failures in your test code, see |
| 405 |
["Catching" Failures](#catching-failures). |
| 406 |
|
| 407 |
### How to Write a Death Test |
| 408 |
|
| 409 |
GoogleTest provides assertion macros to support death tests. See |
| 410 |
[Death Assertions](reference/assertions.md#death) in the Assertions Reference |
| 411 |
for details. |
| 412 |
|
| 413 |
To write a death test, simply use one of the macros inside your test function. |
| 414 |
For example, |
| 415 |
|
| 416 |
```c++ |
| 417 |
TEST(MyDeathTest, Foo) { |
| 418 |
// This death test uses a compound statement. |
| 419 |
ASSERT_DEATH({ |
| 420 |
int n = 5; |
| 421 |
Foo(&n); |
| 422 |
}, "Error on line .* of Foo()"); |
| 423 |
} |
| 424 |
|
| 425 |
TEST(MyDeathTest, NormalExit) { |
| 426 |
EXPECT_EXIT(NormalExit(), testing::ExitedWithCode(0), "Success"); |
| 427 |
} |
| 428 |
|
| 429 |
TEST(MyDeathTest, KillProcess) { |
| 430 |
EXPECT_EXIT(KillProcess(), testing::KilledBySignal(SIGKILL), |
| 431 |
"Sending myself unblockable signal"); |
| 432 |
} |
| 433 |
``` |
| 434 |
|
| 435 |
verifies that: |
| 436 |
|
| 437 |
* calling `Foo(5)` causes the process to die with the given error message, |
| 438 |
* calling `NormalExit()` causes the process to print `"Success"` to stderr and |
| 439 |
exit with exit code 0, and |
| 440 |
* calling `KillProcess()` kills the process with signal `SIGKILL`. |
| 441 |
|
| 442 |
The test function body may contain other assertions and statements as well, if |
| 443 |
necessary. |
| 444 |
|
| 445 |
Note that a death test only cares about three things: |
| 446 |
|
| 447 |
1. does `statement` abort or exit the process? |
| 448 |
2. (in the case of `ASSERT_EXIT` and `EXPECT_EXIT`) does the exit status |
| 449 |
satisfy `predicate`? Or (in the case of `ASSERT_DEATH` and `EXPECT_DEATH`) |
| 450 |
is the exit status non-zero? And |
| 451 |
3. does the stderr output match `matcher`? |
| 452 |
|
| 453 |
In particular, if `statement` generates an `ASSERT_*` or `EXPECT_*` failure, it |
| 454 |
will **not** cause the death test to fail, as googletest assertions don't abort |
| 455 |
the process. |
| 456 |
|
| 457 |
### Death Test Naming |
| 458 |
|
| 459 |
{: .callout .important} |
| 460 |
IMPORTANT: We strongly recommend you to follow the convention of naming your |
| 461 |
**test suite** (not test) `*DeathTest` when it contains a death test, as |
| 462 |
demonstrated in the above example. The |
| 463 |
[Death Tests And Threads](#death-tests-and-threads) section below explains why. |
| 464 |
|
| 465 |
If a test fixture class is shared by normal tests and death tests, you can use |
| 466 |
`using` or `typedef` to introduce an alias for the fixture class and avoid |
| 467 |
duplicating its code: |
| 468 |
|
| 469 |
```c++ |
| 470 |
class FooTest : public testing::Test { ... }; |
| 471 |
|
| 472 |
using FooDeathTest = FooTest; |
| 473 |
|
| 474 |
TEST_F(FooTest, DoesThis) { |
| 475 |
// normal test |
| 476 |
} |
| 477 |
|
| 478 |
TEST_F(FooDeathTest, DoesThat) { |
| 479 |
// death test |
| 480 |
} |
| 481 |
``` |
| 482 |
|
| 483 |
### Regular Expression Syntax |
| 484 |
|
| 485 |
On POSIX systems (e.g. Linux, Cygwin, and Mac), googletest uses the |
| 486 |
[POSIX extended regular expression](http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap09.html#tag_09_04) |
| 487 |
syntax. To learn about this syntax, you may want to read this |
| 488 |
[Wikipedia entry](http://en.wikipedia.org/wiki/Regular_expression#POSIX_Extended_Regular_Expressions). |
| 489 |
|
| 490 |
On Windows, googletest uses its own simple regular expression implementation. It |
| 491 |
lacks many features. For example, we don't support union (`"x|y"`), grouping |
| 492 |
(`"(xy)"`), brackets (`"[xy]"`), and repetition count (`"x{5,7}"`), among |
| 493 |
others. Below is what we do support (`A` denotes a literal character, period |
| 494 |
(`.`), or a single `\\ ` escape sequence; `x` and `y` denote regular |
| 495 |
expressions.): |
| 496 |
|
| 497 |
Expression | Meaning |
| 498 |
---------- | -------------------------------------------------------------- |
| 499 |
`c` | matches any literal character `c` |
| 500 |
`\\d` | matches any decimal digit |
| 501 |
`\\D` | matches any character that's not a decimal digit |
| 502 |
`\\f` | matches `\f` |
| 503 |
`\\n` | matches `\n` |
| 504 |
`\\r` | matches `\r` |
| 505 |
`\\s` | matches any ASCII whitespace, including `\n` |
| 506 |
`\\S` | matches any character that's not a whitespace |
| 507 |
`\\t` | matches `\t` |
| 508 |
`\\v` | matches `\v` |
| 509 |
`\\w` | matches any letter, `_`, or decimal digit |
| 510 |
`\\W` | matches any character that `\\w` doesn't match |
| 511 |
`\\c` | matches any literal character `c`, which must be a punctuation |
| 512 |
`.` | matches any single character except `\n` |
| 513 |
`A?` | matches 0 or 1 occurrences of `A` |
| 514 |
`A*` | matches 0 or many occurrences of `A` |
| 515 |
`A+` | matches 1 or many occurrences of `A` |
| 516 |
`^` | matches the beginning of a string (not that of each line) |
| 517 |
`$` | matches the end of a string (not that of each line) |
| 518 |
`xy` | matches `x` followed by `y` |
| 519 |
|
| 520 |
To help you determine which capability is available on your system, googletest |
| 521 |
defines macros to govern which regular expression it is using. The macros are: |
| 522 |
`GTEST_USES_SIMPLE_RE=1` or `GTEST_USES_POSIX_RE=1`. If you want your death |
| 523 |
tests to work in all cases, you can either `#if` on these macros or use the more |
| 524 |
limited syntax only. |
| 525 |
|
| 526 |
### How It Works |
| 527 |
|
| 528 |
See [Death Assertions](reference/assertions.md#death) in the Assertions |
| 529 |
Reference. |
| 530 |
|
| 531 |
### Death Tests And Threads |
| 532 |
|
| 533 |
The reason for the two death test styles has to do with thread safety. Due to |
| 534 |
well-known problems with forking in the presence of threads, death tests should |
| 535 |
be run in a single-threaded context. Sometimes, however, it isn't feasible to |
| 536 |
arrange that kind of environment. For example, statically-initialized modules |
| 537 |
may start threads before main is ever reached. Once threads have been created, |
| 538 |
it may be difficult or impossible to clean them up. |
| 539 |
|
| 540 |
googletest has three features intended to raise awareness of threading issues. |
| 541 |
|
| 542 |
1. A warning is emitted if multiple threads are running when a death test is |
| 543 |
encountered. |
| 544 |
2. Test suites with a name ending in "DeathTest" are run before all other |
| 545 |
tests. |
| 546 |
3. It uses `clone()` instead of `fork()` to spawn the child process on Linux |
| 547 |
(`clone()` is not available on Cygwin and Mac), as `fork()` is more likely |
| 548 |
to cause the child to hang when the parent process has multiple threads. |
| 549 |
|
| 550 |
It's perfectly fine to create threads inside a death test statement; they are |
| 551 |
executed in a separate process and cannot affect the parent. |
| 552 |
|
| 553 |
### Death Test Styles |
| 554 |
|
| 555 |
The "threadsafe" death test style was introduced in order to help mitigate the |
| 556 |
risks of testing in a possibly multithreaded environment. It trades increased |
| 557 |
test execution time (potentially dramatically so) for improved thread safety. |
| 558 |
|
| 559 |
The automated testing framework does not set the style flag. You can choose a |
| 560 |
particular style of death tests by setting the flag programmatically: |
| 561 |
|
| 562 |
```c++ |
| 563 |
GTEST_FLAG_SET(death_test_style, "threadsafe") |
| 564 |
``` |
| 565 |
|
| 566 |
You can do this in `main()` to set the style for all death tests in the binary, |
| 567 |
or in individual tests. Recall that flags are saved before running each test and |
| 568 |
restored afterwards, so you need not do that yourself. For example: |
| 569 |
|
| 570 |
```c++ |
| 571 |
int main(int argc, char** argv) { |
| 572 |
testing::InitGoogleTest(&argc, argv); |
| 573 |
GTEST_FLAG_SET(death_test_style, "fast"); |
| 574 |
return RUN_ALL_TESTS(); |
| 575 |
} |
| 576 |
|
| 577 |
TEST(MyDeathTest, TestOne) { |
| 578 |
GTEST_FLAG_SET(death_test_style, "threadsafe"); |
| 579 |
// This test is run in the "threadsafe" style: |
| 580 |
ASSERT_DEATH(ThisShouldDie(), ""); |
| 581 |
} |
| 582 |
|
| 583 |
TEST(MyDeathTest, TestTwo) { |
| 584 |
// This test is run in the "fast" style: |
| 585 |
ASSERT_DEATH(ThisShouldDie(), ""); |
| 586 |
} |
| 587 |
``` |
| 588 |
|
| 589 |
### Caveats |
| 590 |
|
| 591 |
The `statement` argument of `ASSERT_EXIT()` can be any valid C++ statement. If |
| 592 |
it leaves the current function via a `return` statement or by throwing an |
| 593 |
exception, the death test is considered to have failed. Some googletest macros |
| 594 |
may return from the current function (e.g. `ASSERT_TRUE()`), so be sure to avoid |
| 595 |
them in `statement`. |
| 596 |
|
| 597 |
Since `statement` runs in the child process, any in-memory side effect (e.g. |
| 598 |
modifying a variable, releasing memory, etc) it causes will *not* be observable |
| 599 |
in the parent process. In particular, if you release memory in a death test, |
| 600 |
your program will fail the heap check as the parent process will never see the |
| 601 |
memory reclaimed. To solve this problem, you can |
| 602 |
|
| 603 |
1. try not to free memory in a death test; |
| 604 |
2. free the memory again in the parent process; or |
| 605 |
3. do not use the heap checker in your program. |
| 606 |
|
| 607 |
Due to an implementation detail, you cannot place multiple death test assertions |
| 608 |
on the same line; otherwise, compilation will fail with an unobvious error |
| 609 |
message. |
| 610 |
|
| 611 |
Despite the improved thread safety afforded by the "threadsafe" style of death |
| 612 |
test, thread problems such as deadlock are still possible in the presence of |
| 613 |
handlers registered with `pthread_atfork(3)`. |
| 614 |
|
| 615 |
## Using Assertions in Sub-routines |
| 616 |
|
| 617 |
{: .callout .note} |
| 618 |
Note: If you want to put a series of test assertions in a subroutine to check |
| 619 |
for a complex condition, consider using |
| 620 |
[a custom GMock matcher](gmock_cook_book.md#NewMatchers) instead. This lets you |
| 621 |
provide a more readable error message in case of failure and avoid all of the |
| 622 |
issues described below. |
| 623 |
|
| 624 |
### Adding Traces to Assertions |
| 625 |
|
| 626 |
If a test sub-routine is called from several places, when an assertion inside it |
| 627 |
fails, it can be hard to tell which invocation of the sub-routine the failure is |
| 628 |
from. You can alleviate this problem using extra logging or custom failure |
| 629 |
messages, but that usually clutters up your tests. A better solution is to use |
| 630 |
the `SCOPED_TRACE` macro or the `ScopedTrace` utility: |
| 631 |
|
| 632 |
```c++ |
| 633 |
SCOPED_TRACE(message); |
| 634 |
``` |
| 635 |
|
| 636 |
```c++ |
| 637 |
ScopedTrace trace("file_path", line_number, message); |
| 638 |
``` |
| 639 |
|
| 640 |
where `message` can be anything streamable to `std::ostream`. `SCOPED_TRACE` |
| 641 |
macro will cause the current file name, line number, and the given message to be |
| 642 |
added in every failure message. `ScopedTrace` accepts explicit file name and |
| 643 |
line number in arguments, which is useful for writing test helpers. The effect |
| 644 |
will be undone when the control leaves the current lexical scope. |
| 645 |
|
| 646 |
For example, |
| 647 |
|
| 648 |
```c++ |
| 649 |
10: void Sub1(int n) { |
| 650 |
11: EXPECT_EQ(Bar(n), 1); |
| 651 |
12: EXPECT_EQ(Bar(n + 1), 2); |
| 652 |
13: } |
| 653 |
14: |
| 654 |
15: TEST(FooTest, Bar) { |
| 655 |
16: { |
| 656 |
17: SCOPED_TRACE("A"); // This trace point will be included in |
| 657 |
18: // every failure in this scope. |
| 658 |
19: Sub1(1); |
| 659 |
20: } |
| 660 |
21: // Now it won't. |
| 661 |
22: Sub1(9); |
| 662 |
23: } |
| 663 |
``` |
| 664 |
|
| 665 |
could result in messages like these: |
| 666 |
|
| 667 |
```none |
| 668 |
path/to/foo_test.cc:11: Failure |
| 669 |
Value of: Bar(n) |
| 670 |
Expected: 1 |
| 671 |
Actual: 2 |
| 672 |
Google Test trace: |
| 673 |
path/to/foo_test.cc:17: A |
| 674 |
|
| 675 |
path/to/foo_test.cc:12: Failure |
| 676 |
Value of: Bar(n + 1) |
| 677 |
Expected: 2 |
| 678 |
Actual: 3 |
| 679 |
``` |
| 680 |
|
| 681 |
Without the trace, it would've been difficult to know which invocation of |
| 682 |
`Sub1()` the two failures come from respectively. (You could add an extra |
| 683 |
message to each assertion in `Sub1()` to indicate the value of `n`, but that's |
| 684 |
tedious.) |
| 685 |
|
| 686 |
Some tips on using `SCOPED_TRACE`: |
| 687 |
|
| 688 |
1. With a suitable message, it's often enough to use `SCOPED_TRACE` at the |
| 689 |
beginning of a sub-routine, instead of at each call site. |
| 690 |
2. When calling sub-routines inside a loop, make the loop iterator part of the |
| 691 |
message in `SCOPED_TRACE` such that you can know which iteration the failure |
| 692 |
is from. |
| 693 |
3. Sometimes the line number of the trace point is enough for identifying the |
| 694 |
particular invocation of a sub-routine. In this case, you don't have to |
| 695 |
choose a unique message for `SCOPED_TRACE`. You can simply use `""`. |
| 696 |
4. You can use `SCOPED_TRACE` in an inner scope when there is one in the outer |
| 697 |
scope. In this case, all active trace points will be included in the failure |
| 698 |
messages, in reverse order they are encountered. |
| 699 |
5. The trace dump is clickable in Emacs - hit `return` on a line number and |
| 700 |
you'll be taken to that line in the source file! |
| 701 |
|
| 702 |
### Propagating Fatal Failures |
| 703 |
|
| 704 |
A common pitfall when using `ASSERT_*` and `FAIL*` is not understanding that |
| 705 |
when they fail they only abort the _current function_, not the entire test. For |
| 706 |
example, the following test will segfault: |
| 707 |
|
| 708 |
```c++ |
| 709 |
void Subroutine() { |
| 710 |
// Generates a fatal failure and aborts the current function. |
| 711 |
ASSERT_EQ(1, 2); |
| 712 |
|
| 713 |
// The following won't be executed. |
| 714 |
... |
| 715 |
} |
| 716 |
|
| 717 |
TEST(FooTest, Bar) { |
| 718 |
Subroutine(); // The intended behavior is for the fatal failure |
| 719 |
// in Subroutine() to abort the entire test. |
| 720 |
|
| 721 |
// The actual behavior: the function goes on after Subroutine() returns. |
| 722 |
int* p = nullptr; |
| 723 |
*p = 3; // Segfault! |
| 724 |
} |
| 725 |
``` |
| 726 |
|
| 727 |
To alleviate this, googletest provides three different solutions. You could use |
| 728 |
either exceptions, the `(ASSERT|EXPECT)_NO_FATAL_FAILURE` assertions or the |
| 729 |
`HasFatalFailure()` function. They are described in the following two |
| 730 |
subsections. |
| 731 |
|
| 732 |
#### Asserting on Subroutines with an exception |
| 733 |
|
| 734 |
The following code can turn ASSERT-failure into an exception: |
| 735 |
|
| 736 |
```c++ |
| 737 |
class ThrowListener : public testing::EmptyTestEventListener { |
| 738 |
void OnTestPartResult(const testing::TestPartResult& result) override { |
| 739 |
if (result.type() == testing::TestPartResult::kFatalFailure) { |
| 740 |
throw testing::AssertionException(result); |
| 741 |
} |
| 742 |
} |
| 743 |
}; |
| 744 |
int main(int argc, char** argv) { |
| 745 |
... |
| 746 |
testing::UnitTest::GetInstance()->listeners().Append(new ThrowListener); |
| 747 |
return RUN_ALL_TESTS(); |
| 748 |
} |
| 749 |
``` |
| 750 |
|
| 751 |
This listener should be added after other listeners if you have any, otherwise |
| 752 |
they won't see failed `OnTestPartResult`. |
| 753 |
|
| 754 |
#### Asserting on Subroutines |
| 755 |
|
| 756 |
As shown above, if your test calls a subroutine that has an `ASSERT_*` failure |
| 757 |
in it, the test will continue after the subroutine returns. This may not be what |
| 758 |
you want. |
| 759 |
|
| 760 |
Often people want fatal failures to propagate like exceptions. For that |
| 761 |
googletest offers the following macros: |
| 762 |
|
| 763 |
Fatal assertion | Nonfatal assertion | Verifies |
| 764 |
------------------------------------- | ------------------------------------- | -------- |
| 765 |
`ASSERT_NO_FATAL_FAILURE(statement);` | `EXPECT_NO_FATAL_FAILURE(statement);` | `statement` doesn't generate any new fatal failures in the current thread. |
| 766 |
|
| 767 |
Only failures in the thread that executes the assertion are checked to determine |
| 768 |
the result of this type of assertions. If `statement` creates new threads, |
| 769 |
failures in these threads are ignored. |
| 770 |
|
| 771 |
Examples: |
| 772 |
|
| 773 |
```c++ |
| 774 |
ASSERT_NO_FATAL_FAILURE(Foo()); |
| 775 |
|
| 776 |
int i; |
| 777 |
EXPECT_NO_FATAL_FAILURE({ |
| 778 |
i = Bar(); |
| 779 |
}); |
| 780 |
``` |
| 781 |
|
| 782 |
Assertions from multiple threads are currently not supported on Windows. |
| 783 |
|
| 784 |
#### Checking for Failures in the Current Test |
| 785 |
|
| 786 |
`HasFatalFailure()` in the `::testing::Test` class returns `true` if an |
| 787 |
assertion in the current test has suffered a fatal failure. This allows |
| 788 |
functions to catch fatal failures in a sub-routine and return early. |
| 789 |
|
| 790 |
```c++ |
| 791 |
class Test { |
| 792 |
public: |
| 793 |
... |
| 794 |
static bool HasFatalFailure(); |
| 795 |
}; |
| 796 |
``` |
| 797 |
|
| 798 |
The typical usage, which basically simulates the behavior of a thrown exception, |
| 799 |
is: |
| 800 |
|
| 801 |
```c++ |
| 802 |
TEST(FooTest, Bar) { |
| 803 |
Subroutine(); |
| 804 |
// Aborts if Subroutine() had a fatal failure. |
| 805 |
if (HasFatalFailure()) return; |
| 806 |
|
| 807 |
// The following won't be executed. |
| 808 |
... |
| 809 |
} |
| 810 |
``` |
| 811 |
|
| 812 |
If `HasFatalFailure()` is used outside of `TEST()` , `TEST_F()` , or a test |
| 813 |
fixture, you must add the `::testing::Test::` prefix, as in: |
| 814 |
|
| 815 |
```c++ |
| 816 |
if (testing::Test::HasFatalFailure()) return; |
| 817 |
``` |
| 818 |
|
| 819 |
Similarly, `HasNonfatalFailure()` returns `true` if the current test has at |
| 820 |
least one non-fatal failure, and `HasFailure()` returns `true` if the current |
| 821 |
test has at least one failure of either kind. |
| 822 |
|
| 823 |
## Logging Additional Information |
| 824 |
|
| 825 |
In your test code, you can call `RecordProperty("key", value)` to log additional |
| 826 |
information, where `value` can be either a string or an `int`. The *last* value |
| 827 |
recorded for a key will be emitted to the |
| 828 |
[XML output](#generating-an-xml-report) if you specify one. For example, the |
| 829 |
test |
| 830 |
|
| 831 |
```c++ |
| 832 |
TEST_F(WidgetUsageTest, MinAndMaxWidgets) { |
| 833 |
RecordProperty("MaximumWidgets", ComputeMaxUsage()); |
| 834 |
RecordProperty("MinimumWidgets", ComputeMinUsage()); |
| 835 |
} |
| 836 |
``` |
| 837 |
|
| 838 |
will output XML like this: |
| 839 |
|
| 840 |
```xml |
| 841 |
... |
| 842 |
<testcase name="MinAndMaxWidgets" status="run" time="0.006" classname="WidgetUsageTest" MaximumWidgets="12" MinimumWidgets="9" /> |
| 843 |
... |
| 844 |
``` |
| 845 |
|
| 846 |
{: .callout .note} |
| 847 |
> NOTE: |
| 848 |
> |
| 849 |
> * `RecordProperty()` is a static member of the `Test` class. Therefore it |
| 850 |
> needs to be prefixed with `::testing::Test::` if used outside of the |
| 851 |
> `TEST` body and the test fixture class. |
| 852 |
> * *`key`* must be a valid XML attribute name, and cannot conflict with the |
| 853 |
> ones already used by googletest (`name`, `status`, `time`, `classname`, |
| 854 |
> `type_param`, and `value_param`). |
| 855 |
> * Calling `RecordProperty()` outside of the lifespan of a test is allowed. |
| 856 |
> If it's called outside of a test but between a test suite's |
| 857 |
> `SetUpTestSuite()` and `TearDownTestSuite()` methods, it will be |
| 858 |
> attributed to the XML element for the test suite. If it's called outside |
| 859 |
> of all test suites (e.g. in a test environment), it will be attributed to |
| 860 |
> the top-level XML element. |
| 861 |
|
| 862 |
## Sharing Resources Between Tests in the Same Test Suite |
| 863 |
|
| 864 |
googletest creates a new test fixture object for each test in order to make |
| 865 |
tests independent and easier to debug. However, sometimes tests use resources |
| 866 |
that are expensive to set up, making the one-copy-per-test model prohibitively |
| 867 |
expensive. |
| 868 |
|
| 869 |
If the tests don't change the resource, there's no harm in their sharing a |
| 870 |
single resource copy. So, in addition to per-test set-up/tear-down, googletest |
| 871 |
also supports per-test-suite set-up/tear-down. To use it: |
| 872 |
|
| 873 |
1. In your test fixture class (say `FooTest` ), declare as `static` some member |
| 874 |
variables to hold the shared resources. |
| 875 |
2. Outside your test fixture class (typically just below it), define those |
| 876 |
member variables, optionally giving them initial values. |
| 877 |
3. In the same test fixture class, define a `static void SetUpTestSuite()` |
| 878 |
function (remember not to spell it as **`SetupTestSuite`** with a small |
| 879 |
`u`!) to set up the shared resources and a `static void TearDownTestSuite()` |
| 880 |
function to tear them down. |
| 881 |
|
| 882 |
That's it! googletest automatically calls `SetUpTestSuite()` before running the |
| 883 |
*first test* in the `FooTest` test suite (i.e. before creating the first |
| 884 |
`FooTest` object), and calls `TearDownTestSuite()` after running the *last test* |
| 885 |
in it (i.e. after deleting the last `FooTest` object). In between, the tests can |
| 886 |
use the shared resources. |
| 887 |
|
| 888 |
Remember that the test order is undefined, so your code can't depend on a test |
| 889 |
preceding or following another. Also, the tests must either not modify the state |
| 890 |
of any shared resource, or, if they do modify the state, they must restore the |
| 891 |
state to its original value before passing control to the next test. |
| 892 |
|
| 893 |
Note that `SetUpTestSuite()` may be called multiple times for a test fixture |
| 894 |
class that has derived classes, so you should not expect code in the function |
| 895 |
body to be run only once. Also, derived classes still have access to shared |
| 896 |
resources defined as static members, so careful consideration is needed when |
| 897 |
managing shared resources to avoid memory leaks. |
| 898 |
|
| 899 |
Here's an example of per-test-suite set-up and tear-down: |
| 900 |
|
| 901 |
```c++ |
| 902 |
class FooTest : public testing::Test { |
| 903 |
protected: |
| 904 |
// Per-test-suite set-up. |
| 905 |
// Called before the first test in this test suite. |
| 906 |
// Can be omitted if not needed. |
| 907 |
static void SetUpTestSuite() { |
| 908 |
// Avoid reallocating static objects if called in subclasses of FooTest. |
| 909 |
if (shared_resource_ == nullptr) { |
| 910 |
shared_resource_ = new ...; |
| 911 |
} |
| 912 |
} |
| 913 |
|
| 914 |
// Per-test-suite tear-down. |
| 915 |
// Called after the last test in this test suite. |
| 916 |
// Can be omitted if not needed. |
| 917 |
static void TearDownTestSuite() { |
| 918 |
delete shared_resource_; |
| 919 |
shared_resource_ = nullptr; |
| 920 |
} |
| 921 |
|
| 922 |
// You can define per-test set-up logic as usual. |
| 923 |
void SetUp() override { ... } |
| 924 |
|
| 925 |
// You can define per-test tear-down logic as usual. |
| 926 |
void TearDown() override { ... } |
| 927 |
|
| 928 |
// Some expensive resource shared by all tests. |
| 929 |
static T* shared_resource_; |
| 930 |
}; |
| 931 |
|
| 932 |
T* FooTest::shared_resource_ = nullptr; |
| 933 |
|
| 934 |
TEST_F(FooTest, Test1) { |
| 935 |
... you can refer to shared_resource_ here ... |
| 936 |
} |
| 937 |
|
| 938 |
TEST_F(FooTest, Test2) { |
| 939 |
... you can refer to shared_resource_ here ... |
| 940 |
} |
| 941 |
``` |
| 942 |
|
| 943 |
{: .callout .note} |
| 944 |
NOTE: Though the above code declares `SetUpTestSuite()` protected, it may |
| 945 |
sometimes be necessary to declare it public, such as when using it with |
| 946 |
`TEST_P`. |
| 947 |
|
| 948 |
## Global Set-Up and Tear-Down |
| 949 |
|
| 950 |
Just as you can do set-up and tear-down at the test level and the test suite |
| 951 |
level, you can also do it at the test program level. Here's how. |
| 952 |
|
| 953 |
First, you subclass the `::testing::Environment` class to define a test |
| 954 |
environment, which knows how to set-up and tear-down: |
| 955 |
|
| 956 |
```c++ |
| 957 |
class Environment : public ::testing::Environment { |
| 958 |
public: |
| 959 |
~Environment() override {} |
| 960 |
|
| 961 |
// Override this to define how to set up the environment. |
| 962 |
void SetUp() override {} |
| 963 |
|
| 964 |
// Override this to define how to tear down the environment. |
| 965 |
void TearDown() override {} |
| 966 |
}; |
| 967 |
``` |
| 968 |
|
| 969 |
Then, you register an instance of your environment class with googletest by |
| 970 |
calling the `::testing::AddGlobalTestEnvironment()` function: |
| 971 |
|
| 972 |
```c++ |
| 973 |
Environment* AddGlobalTestEnvironment(Environment* env); |
| 974 |
``` |
| 975 |
|
| 976 |
Now, when `RUN_ALL_TESTS()` is called, it first calls the `SetUp()` method of |
| 977 |
each environment object, then runs the tests if none of the environments |
| 978 |
reported fatal failures and `GTEST_SKIP()` was not called. `RUN_ALL_TESTS()` |
| 979 |
always calls `TearDown()` with each environment object, regardless of whether or |
| 980 |
not the tests were run. |
| 981 |
|
| 982 |
It's OK to register multiple environment objects. In this suite, their `SetUp()` |
| 983 |
will be called in the order they are registered, and their `TearDown()` will be |
| 984 |
called in the reverse order. |
| 985 |
|
| 986 |
Note that googletest takes ownership of the registered environment objects. |
| 987 |
Therefore **do not delete them** by yourself. |
| 988 |
|
| 989 |
You should call `AddGlobalTestEnvironment()` before `RUN_ALL_TESTS()` is called, |
| 990 |
probably in `main()`. If you use `gtest_main`, you need to call this before |
| 991 |
`main()` starts for it to take effect. One way to do this is to define a global |
| 992 |
variable like this: |
| 993 |
|
| 994 |
```c++ |
| 995 |
testing::Environment* const foo_env = |
| 996 |
testing::AddGlobalTestEnvironment(new FooEnvironment); |
| 997 |
``` |
| 998 |
|
| 999 |
However, we strongly recommend you to write your own `main()` and call |
| 1000 |
`AddGlobalTestEnvironment()` there, as relying on initialization of global |
| 1001 |
variables makes the code harder to read and may cause problems when you register |
| 1002 |
multiple environments from different translation units and the environments have |
| 1003 |
dependencies among them (remember that the compiler doesn't guarantee the order |
| 1004 |
in which global variables from different translation units are initialized). |
| 1005 |
|
| 1006 |
## Value-Parameterized Tests |
| 1007 |
|
| 1008 |
*Value-parameterized tests* allow you to test your code with different |
| 1009 |
parameters without writing multiple copies of the same test. This is useful in a |
| 1010 |
number of situations, for example: |
| 1011 |
|
| 1012 |
* You have a piece of code whose behavior is affected by one or more |
| 1013 |
command-line flags. You want to make sure your code performs correctly for |
| 1014 |
various values of those flags. |
| 1015 |
* You want to test different implementations of an OO interface. |
| 1016 |
* You want to test your code over various inputs (a.k.a. data-driven testing). |
| 1017 |
This feature is easy to abuse, so please exercise your good sense when doing |
| 1018 |
it! |
| 1019 |
|
| 1020 |
### How to Write Value-Parameterized Tests |
| 1021 |
|
| 1022 |
To write value-parameterized tests, first you should define a fixture class. It |
| 1023 |
must be derived from both `testing::Test` and `testing::WithParamInterface<T>` |
| 1024 |
(the latter is a pure interface), where `T` is the type of your parameter |
| 1025 |
values. For convenience, you can just derive the fixture class from |
| 1026 |
`testing::TestWithParam<T>`, which itself is derived from both `testing::Test` |
| 1027 |
and `testing::WithParamInterface<T>`. `T` can be any copyable type. If it's a |
| 1028 |
raw pointer, you are responsible for managing the lifespan of the pointed |
| 1029 |
values. |
| 1030 |
|
| 1031 |
{: .callout .note} |
| 1032 |
NOTE: If your test fixture defines `SetUpTestSuite()` or `TearDownTestSuite()` |
| 1033 |
they must be declared **public** rather than **protected** in order to use |
| 1034 |
`TEST_P`. |
| 1035 |
|
| 1036 |
```c++ |
| 1037 |
class FooTest : |
| 1038 |
public testing::TestWithParam<const char*> { |
| 1039 |
// You can implement all the usual fixture class members here. |
| 1040 |
// To access the test parameter, call GetParam() from class |
| 1041 |
// TestWithParam<T>. |
| 1042 |
}; |
| 1043 |
|
| 1044 |
// Or, when you want to add parameters to a pre-existing fixture class: |
| 1045 |
class BaseTest : public testing::Test { |
| 1046 |
... |
| 1047 |
}; |
| 1048 |
class BarTest : public BaseTest, |
| 1049 |
public testing::WithParamInterface<const char*> { |
| 1050 |
... |
| 1051 |
}; |
| 1052 |
``` |
| 1053 |
|
| 1054 |
Then, use the `TEST_P` macro to define as many test patterns using this fixture |
| 1055 |
as you want. The `_P` suffix is for "parameterized" or "pattern", whichever you |
| 1056 |
prefer to think. |
| 1057 |
|
| 1058 |
```c++ |
| 1059 |
TEST_P(FooTest, DoesBlah) { |
| 1060 |
// Inside a test, access the test parameter with the GetParam() method |
| 1061 |
// of the TestWithParam<T> class: |
| 1062 |
EXPECT_TRUE(foo.Blah(GetParam())); |
| 1063 |
... |
| 1064 |
} |
| 1065 |
|
| 1066 |
TEST_P(FooTest, HasBlahBlah) { |
| 1067 |
... |
| 1068 |
} |
| 1069 |
``` |
| 1070 |
|
| 1071 |
Finally, you can use the `INSTANTIATE_TEST_SUITE_P` macro to instantiate the |
| 1072 |
test suite with any set of parameters you want. GoogleTest defines a number of |
| 1073 |
functions for generating test parameters—see details at |
| 1074 |
[`INSTANTIATE_TEST_SUITE_P`](reference/testing.md#INSTANTIATE_TEST_SUITE_P) in |
| 1075 |
the Testing Reference. |
| 1076 |
|
| 1077 |
For example, the following statement will instantiate tests from the `FooTest` |
| 1078 |
test suite each with parameter values `"meeny"`, `"miny"`, and `"moe"` using the |
| 1079 |
[`Values`](reference/testing.md#param-generators) parameter generator: |
| 1080 |
|
| 1081 |
```c++ |
| 1082 |
INSTANTIATE_TEST_SUITE_P(MeenyMinyMoe, |
| 1083 |
FooTest, |
| 1084 |
testing::Values("meeny", "miny", "moe")); |
| 1085 |
``` |
| 1086 |
|
| 1087 |
{: .callout .note} |
| 1088 |
NOTE: The code above must be placed at global or namespace scope, not at |
| 1089 |
function scope. |
| 1090 |
|
| 1091 |
The first argument to `INSTANTIATE_TEST_SUITE_P` is a unique name for the |
| 1092 |
instantiation of the test suite. The next argument is the name of the test |
| 1093 |
pattern, and the last is the |
| 1094 |
[parameter generator](reference/testing.md#param-generators). |
| 1095 |
|
| 1096 |
You can instantiate a test pattern more than once, so to distinguish different |
| 1097 |
instances of the pattern, the instantiation name is added as a prefix to the |
| 1098 |
actual test suite name. Remember to pick unique prefixes for different |
| 1099 |
instantiations. The tests from the instantiation above will have these names: |
| 1100 |
|
| 1101 |
* `MeenyMinyMoe/FooTest.DoesBlah/0` for `"meeny"` |
| 1102 |
* `MeenyMinyMoe/FooTest.DoesBlah/1` for `"miny"` |
| 1103 |
* `MeenyMinyMoe/FooTest.DoesBlah/2` for `"moe"` |
| 1104 |
* `MeenyMinyMoe/FooTest.HasBlahBlah/0` for `"meeny"` |
| 1105 |
* `MeenyMinyMoe/FooTest.HasBlahBlah/1` for `"miny"` |
| 1106 |
* `MeenyMinyMoe/FooTest.HasBlahBlah/2` for `"moe"` |
| 1107 |
|
| 1108 |
You can use these names in [`--gtest_filter`](#running-a-subset-of-the-tests). |
| 1109 |
|
| 1110 |
The following statement will instantiate all tests from `FooTest` again, each |
| 1111 |
with parameter values `"cat"` and `"dog"` using the |
| 1112 |
[`ValuesIn`](reference/testing.md#param-generators) parameter generator: |
| 1113 |
|
| 1114 |
```c++ |
| 1115 |
const char* pets[] = {"cat", "dog"}; |
| 1116 |
INSTANTIATE_TEST_SUITE_P(Pets, FooTest, testing::ValuesIn(pets)); |
| 1117 |
``` |
| 1118 |
|
| 1119 |
The tests from the instantiation above will have these names: |
| 1120 |
|
| 1121 |
* `Pets/FooTest.DoesBlah/0` for `"cat"` |
| 1122 |
* `Pets/FooTest.DoesBlah/1` for `"dog"` |
| 1123 |
* `Pets/FooTest.HasBlahBlah/0` for `"cat"` |
| 1124 |
* `Pets/FooTest.HasBlahBlah/1` for `"dog"` |
| 1125 |
|
| 1126 |
Please note that `INSTANTIATE_TEST_SUITE_P` will instantiate *all* tests in the |
| 1127 |
given test suite, whether their definitions come before or *after* the |
| 1128 |
`INSTANTIATE_TEST_SUITE_P` statement. |
| 1129 |
|
| 1130 |
Additionally, by default, every `TEST_P` without a corresponding |
| 1131 |
`INSTANTIATE_TEST_SUITE_P` causes a failing test in test suite |
| 1132 |
`GoogleTestVerification`. If you have a test suite where that omission is not an |
| 1133 |
error, for example it is in a library that may be linked in for other reasons or |
| 1134 |
where the list of test cases is dynamic and may be empty, then this check can be |
| 1135 |
suppressed by tagging the test suite: |
| 1136 |
|
| 1137 |
```c++ |
| 1138 |
GTEST_ALLOW_UNINSTANTIATED_PARAMETERIZED_TEST(FooTest); |
| 1139 |
``` |
| 1140 |
|
| 1141 |
You can see [sample7_unittest.cc] and [sample8_unittest.cc] for more examples. |
| 1142 |
|
| 1143 |
[sample7_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample7_unittest.cc "Parameterized Test example" |
| 1144 |
[sample8_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample8_unittest.cc "Parameterized Test example with multiple parameters" |
| 1145 |
|
| 1146 |
### Creating Value-Parameterized Abstract Tests |
| 1147 |
|
| 1148 |
In the above, we define and instantiate `FooTest` in the *same* source file. |
| 1149 |
Sometimes you may want to define value-parameterized tests in a library and let |
| 1150 |
other people instantiate them later. This pattern is known as *abstract tests*. |
| 1151 |
As an example of its application, when you are designing an interface you can |
| 1152 |
write a standard suite of abstract tests (perhaps using a factory function as |
| 1153 |
the test parameter) that all implementations of the interface are expected to |
| 1154 |
pass. When someone implements the interface, they can instantiate your suite to |
| 1155 |
get all the interface-conformance tests for free. |
| 1156 |
|
| 1157 |
To define abstract tests, you should organize your code like this: |
| 1158 |
|
| 1159 |
1. Put the definition of the parameterized test fixture class (e.g. `FooTest`) |
| 1160 |
in a header file, say `foo_param_test.h`. Think of this as *declaring* your |
| 1161 |
abstract tests. |
| 1162 |
2. Put the `TEST_P` definitions in `foo_param_test.cc`, which includes |
| 1163 |
`foo_param_test.h`. Think of this as *implementing* your abstract tests. |
| 1164 |
|
| 1165 |
Once they are defined, you can instantiate them by including `foo_param_test.h`, |
| 1166 |
invoking `INSTANTIATE_TEST_SUITE_P()`, and depending on the library target that |
| 1167 |
contains `foo_param_test.cc`. You can instantiate the same abstract test suite |
| 1168 |
multiple times, possibly in different source files. |
| 1169 |
|
| 1170 |
### Specifying Names for Value-Parameterized Test Parameters |
| 1171 |
|
| 1172 |
The optional last argument to `INSTANTIATE_TEST_SUITE_P()` allows the user to |
| 1173 |
specify a function or functor that generates custom test name suffixes based on |
| 1174 |
the test parameters. The function should accept one argument of type |
| 1175 |
`testing::TestParamInfo<class ParamType>`, and return `std::string`. |
| 1176 |
|
| 1177 |
`testing::PrintToStringParamName` is a builtin test suffix generator that |
| 1178 |
returns the value of `testing::PrintToString(GetParam())`. It does not work for |
| 1179 |
`std::string` or C strings. |
| 1180 |
|
| 1181 |
{: .callout .note} |
| 1182 |
NOTE: test names must be non-empty, unique, and may only contain ASCII |
| 1183 |
alphanumeric characters. In particular, they |
| 1184 |
[should not contain underscores](faq.md#why-should-test-suite-names-and-test-names-not-contain-underscore) |
| 1185 |
|
| 1186 |
```c++ |
| 1187 |
class MyTestSuite : public testing::TestWithParam<int> {}; |
| 1188 |
|
| 1189 |
TEST_P(MyTestSuite, MyTest) |
| 1190 |
{ |
| 1191 |
std::cout << "Example Test Param: " << GetParam() << std::endl; |
| 1192 |
} |
| 1193 |
|
| 1194 |
INSTANTIATE_TEST_SUITE_P(MyGroup, MyTestSuite, testing::Range(0, 10), |
| 1195 |
testing::PrintToStringParamName()); |
| 1196 |
``` |
| 1197 |
|
| 1198 |
Providing a custom functor allows for more control over test parameter name |
| 1199 |
generation, especially for types where the automatic conversion does not |
| 1200 |
generate helpful parameter names (e.g. strings as demonstrated above). The |
| 1201 |
following example illustrates this for multiple parameters, an enumeration type |
| 1202 |
and a string, and also demonstrates how to combine generators. It uses a lambda |
| 1203 |
for conciseness: |
| 1204 |
|
| 1205 |
```c++ |
| 1206 |
enum class MyType { MY_FOO = 0, MY_BAR = 1 }; |
| 1207 |
|
| 1208 |
class MyTestSuite : public testing::TestWithParam<std::tuple<MyType, std::string>> { |
| 1209 |
}; |
| 1210 |
|
| 1211 |
INSTANTIATE_TEST_SUITE_P( |
| 1212 |
MyGroup, MyTestSuite, |
| 1213 |
testing::Combine( |
| 1214 |
testing::Values(MyType::MY_FOO, MyType::MY_BAR), |
| 1215 |
testing::Values("A", "B")), |
| 1216 |
[](const testing::TestParamInfo<MyTestSuite::ParamType>& info) { |
| 1217 |
std::string name = absl::StrCat( |
| 1218 |
std::get<0>(info.param) == MyType::MY_FOO ? "Foo" : "Bar", |
| 1219 |
std::get<1>(info.param)); |
| 1220 |
absl::c_replace_if(name, [](char c) { return !std::isalnum(c); }, '_'); |
| 1221 |
return name; |
| 1222 |
}); |
| 1223 |
``` |
| 1224 |
|
| 1225 |
## Typed Tests |
| 1226 |
|
| 1227 |
Suppose you have multiple implementations of the same interface and want to make |
| 1228 |
sure that all of them satisfy some common requirements. Or, you may have defined |
| 1229 |
several types that are supposed to conform to the same "concept" and you want to |
| 1230 |
verify it. In both cases, you want the same test logic repeated for different |
| 1231 |
types. |
| 1232 |
|
| 1233 |
While you can write one `TEST` or `TEST_F` for each type you want to test (and |
| 1234 |
you may even factor the test logic into a function template that you invoke from |
| 1235 |
the `TEST`), it's tedious and doesn't scale: if you want `m` tests over `n` |
| 1236 |
types, you'll end up writing `m*n` `TEST`s. |
| 1237 |
|
| 1238 |
*Typed tests* allow you to repeat the same test logic over a list of types. You |
| 1239 |
only need to write the test logic once, although you must know the type list |
| 1240 |
when writing typed tests. Here's how you do it: |
| 1241 |
|
| 1242 |
First, define a fixture class template. It should be parameterized by a type. |
| 1243 |
Remember to derive it from `::testing::Test`: |
| 1244 |
|
| 1245 |
```c++ |
| 1246 |
template <typename T> |
| 1247 |
class FooTest : public testing::Test { |
| 1248 |
public: |
| 1249 |
... |
| 1250 |
using List = std::list<T>; |
| 1251 |
static T shared_; |
| 1252 |
T value_; |
| 1253 |
}; |
| 1254 |
``` |
| 1255 |
|
| 1256 |
Next, associate a list of types with the test suite, which will be repeated for |
| 1257 |
each type in the list: |
| 1258 |
|
| 1259 |
```c++ |
| 1260 |
using MyTypes = ::testing::Types<char, int, unsigned int>; |
| 1261 |
TYPED_TEST_SUITE(FooTest, MyTypes); |
| 1262 |
``` |
| 1263 |
|
| 1264 |
The type alias (`using` or `typedef`) is necessary for the `TYPED_TEST_SUITE` |
| 1265 |
macro to parse correctly. Otherwise the compiler will think that each comma in |
| 1266 |
the type list introduces a new macro argument. |
| 1267 |
|
| 1268 |
Then, use `TYPED_TEST()` instead of `TEST_F()` to define a typed test for this |
| 1269 |
test suite. You can repeat this as many times as you want: |
| 1270 |
|
| 1271 |
```c++ |
| 1272 |
TYPED_TEST(FooTest, DoesBlah) { |
| 1273 |
// Inside a test, refer to the special name TypeParam to get the type |
| 1274 |
// parameter. Since we are inside a derived class template, C++ requires |
| 1275 |
// us to visit the members of FooTest via 'this'. |
| 1276 |
TypeParam n = this->value_; |
| 1277 |
|
| 1278 |
// To visit static members of the fixture, add the 'TestFixture::' |
| 1279 |
// prefix. |
| 1280 |
n += TestFixture::shared_; |
| 1281 |
|
| 1282 |
// To refer to typedefs in the fixture, add the 'typename TestFixture::' |
| 1283 |
// prefix. The 'typename' is required to satisfy the compiler. |
| 1284 |
typename TestFixture::List values; |
| 1285 |
|
| 1286 |
values.push_back(n); |
| 1287 |
... |
| 1288 |
} |
| 1289 |
|
| 1290 |
TYPED_TEST(FooTest, HasPropertyA) { ... } |
| 1291 |
``` |
| 1292 |
|
| 1293 |
You can see [sample6_unittest.cc] for a complete example. |
| 1294 |
|
| 1295 |
[sample6_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample6_unittest.cc "Typed Test example" |
| 1296 |
|
| 1297 |
## Type-Parameterized Tests |
| 1298 |
|
| 1299 |
*Type-parameterized tests* are like typed tests, except that they don't require |
| 1300 |
you to know the list of types ahead of time. Instead, you can define the test |
| 1301 |
logic first and instantiate it with different type lists later. You can even |
| 1302 |
instantiate it more than once in the same program. |
| 1303 |
|
| 1304 |
If you are designing an interface or concept, you can define a suite of |
| 1305 |
type-parameterized tests to verify properties that any valid implementation of |
| 1306 |
the interface/concept should have. Then, the author of each implementation can |
| 1307 |
just instantiate the test suite with their type to verify that it conforms to |
| 1308 |
the requirements, without having to write similar tests repeatedly. Here's an |
| 1309 |
example: |
| 1310 |
|
| 1311 |
First, define a fixture class template, as we did with typed tests: |
| 1312 |
|
| 1313 |
```c++ |
| 1314 |
template <typename T> |
| 1315 |
class FooTest : public testing::Test { |
| 1316 |
... |
| 1317 |
}; |
| 1318 |
``` |
| 1319 |
|
| 1320 |
Next, declare that you will define a type-parameterized test suite: |
| 1321 |
|
| 1322 |
```c++ |
| 1323 |
TYPED_TEST_SUITE_P(FooTest); |
| 1324 |
``` |
| 1325 |
|
| 1326 |
Then, use `TYPED_TEST_P()` to define a type-parameterized test. You can repeat |
| 1327 |
this as many times as you want: |
| 1328 |
|
| 1329 |
```c++ |
| 1330 |
TYPED_TEST_P(FooTest, DoesBlah) { |
| 1331 |
// Inside a test, refer to TypeParam to get the type parameter. |
| 1332 |
TypeParam n = 0; |
| 1333 |
... |
| 1334 |
} |
| 1335 |
|
| 1336 |
TYPED_TEST_P(FooTest, HasPropertyA) { ... } |
| 1337 |
``` |
| 1338 |
|
| 1339 |
Now the tricky part: you need to register all test patterns using the |
| 1340 |
`REGISTER_TYPED_TEST_SUITE_P` macro before you can instantiate them. The first |
| 1341 |
argument of the macro is the test suite name; the rest are the names of the |
| 1342 |
tests in this test suite: |
| 1343 |
|
| 1344 |
```c++ |
| 1345 |
REGISTER_TYPED_TEST_SUITE_P(FooTest, |
| 1346 |
DoesBlah, HasPropertyA); |
| 1347 |
``` |
| 1348 |
|
| 1349 |
Finally, you are free to instantiate the pattern with the types you want. If you |
| 1350 |
put the above code in a header file, you can `#include` it in multiple C++ |
| 1351 |
source files and instantiate it multiple times. |
| 1352 |
|
| 1353 |
```c++ |
| 1354 |
using MyTypes = ::testing::Types<char, int, unsigned int>; |
| 1355 |
INSTANTIATE_TYPED_TEST_SUITE_P(My, FooTest, MyTypes); |
| 1356 |
``` |
| 1357 |
|
| 1358 |
To distinguish different instances of the pattern, the first argument to the |
| 1359 |
`INSTANTIATE_TYPED_TEST_SUITE_P` macro is a prefix that will be added to the |
| 1360 |
actual test suite name. Remember to pick unique prefixes for different |
| 1361 |
instances. |
| 1362 |
|
| 1363 |
In the special case where the type list contains only one type, you can write |
| 1364 |
that type directly without `::testing::Types<...>`, like this: |
| 1365 |
|
| 1366 |
```c++ |
| 1367 |
INSTANTIATE_TYPED_TEST_SUITE_P(My, FooTest, int); |
| 1368 |
``` |
| 1369 |
|
| 1370 |
You can see [sample6_unittest.cc] for a complete example. |
| 1371 |
|
| 1372 |
## Testing Private Code |
| 1373 |
|
| 1374 |
If you change your software's internal implementation, your tests should not |
| 1375 |
break as long as the change is not observable by users. Therefore, **per the |
| 1376 |
black-box testing principle, most of the time you should test your code through |
| 1377 |
its public interfaces.** |
| 1378 |
|
| 1379 |
**If you still find yourself needing to test internal implementation code, |
| 1380 |
consider if there's a better design.** The desire to test internal |
| 1381 |
implementation is often a sign that the class is doing too much. Consider |
| 1382 |
extracting an implementation class, and testing it. Then use that implementation |
| 1383 |
class in the original class. |
| 1384 |
|
| 1385 |
If you absolutely have to test non-public interface code though, you can. There |
| 1386 |
are two cases to consider: |
| 1387 |
|
| 1388 |
* Static functions ( *not* the same as static member functions!) or unnamed |
| 1389 |
namespaces, and |
| 1390 |
* Private or protected class members |
| 1391 |
|
| 1392 |
To test them, we use the following special techniques: |
| 1393 |
|
| 1394 |
* Both static functions and definitions/declarations in an unnamed namespace |
| 1395 |
are only visible within the same translation unit. To test them, you can |
| 1396 |
`#include` the entire `.cc` file being tested in your `*_test.cc` file. |
| 1397 |
(#including `.cc` files is not a good way to reuse code - you should not do |
| 1398 |
this in production code!) |
| 1399 |
|
| 1400 |
However, a better approach is to move the private code into the |
| 1401 |
`foo::internal` namespace, where `foo` is the namespace your project |
| 1402 |
normally uses, and put the private declarations in a `*-internal.h` file. |
| 1403 |
Your production `.cc` files and your tests are allowed to include this |
| 1404 |
internal header, but your clients are not. This way, you can fully test your |
| 1405 |
internal implementation without leaking it to your clients. |
| 1406 |
|
| 1407 |
* Private class members are only accessible from within the class or by |
| 1408 |
friends. To access a class' private members, you can declare your test |
| 1409 |
fixture as a friend to the class and define accessors in your fixture. Tests |
| 1410 |
using the fixture can then access the private members of your production |
| 1411 |
class via the accessors in the fixture. Note that even though your fixture |
| 1412 |
is a friend to your production class, your tests are not automatically |
| 1413 |
friends to it, as they are technically defined in sub-classes of the |
| 1414 |
fixture. |
| 1415 |
|
| 1416 |
Another way to test private members is to refactor them into an |
| 1417 |
implementation class, which is then declared in a `*-internal.h` file. Your |
| 1418 |
clients aren't allowed to include this header but your tests can. Such is |
| 1419 |
called the |
| 1420 |
[Pimpl](https://www.gamedev.net/articles/programming/general-and-gameplay-programming/the-c-pimpl-r1794/) |
| 1421 |
(Private Implementation) idiom. |
| 1422 |
|
| 1423 |
Or, you can declare an individual test as a friend of your class by adding |
| 1424 |
this line in the class body: |
| 1425 |
|
| 1426 |
```c++ |
| 1427 |
FRIEND_TEST(TestSuiteName, TestName); |
| 1428 |
``` |
| 1429 |
|
| 1430 |
For example, |
| 1431 |
|
| 1432 |
```c++ |
| 1433 |
// foo.h |
| 1434 |
class Foo { |
| 1435 |
... |
| 1436 |
private: |
| 1437 |
FRIEND_TEST(FooTest, BarReturnsZeroOnNull); |
| 1438 |
|
| 1439 |
int Bar(void* x); |
| 1440 |
}; |
| 1441 |
|
| 1442 |
// foo_test.cc |
| 1443 |
... |
| 1444 |
TEST(FooTest, BarReturnsZeroOnNull) { |
| 1445 |
Foo foo; |
| 1446 |
EXPECT_EQ(foo.Bar(NULL), 0); // Uses Foo's private member Bar(). |
| 1447 |
} |
| 1448 |
``` |
| 1449 |
|
| 1450 |
Pay special attention when your class is defined in a namespace. If you want |
| 1451 |
your test fixtures and tests to be friends of your class, then they must be |
| 1452 |
defined in the exact same namespace (no anonymous or inline namespaces). |
| 1453 |
|
| 1454 |
For example, if the code to be tested looks like: |
| 1455 |
|
| 1456 |
```c++ |
| 1457 |
namespace my_namespace { |
| 1458 |
|
| 1459 |
class Foo { |
| 1460 |
friend class FooTest; |
| 1461 |
FRIEND_TEST(FooTest, Bar); |
| 1462 |
FRIEND_TEST(FooTest, Baz); |
| 1463 |
... definition of the class Foo ... |
| 1464 |
}; |
| 1465 |
|
| 1466 |
} // namespace my_namespace |
| 1467 |
``` |
| 1468 |
|
| 1469 |
Your test code should be something like: |
| 1470 |
|
| 1471 |
```c++ |
| 1472 |
namespace my_namespace { |
| 1473 |
|
| 1474 |
class FooTest : public testing::Test { |
| 1475 |
protected: |
| 1476 |
... |
| 1477 |
}; |
| 1478 |
|
| 1479 |
TEST_F(FooTest, Bar) { ... } |
| 1480 |
TEST_F(FooTest, Baz) { ... } |
| 1481 |
|
| 1482 |
} // namespace my_namespace |
| 1483 |
``` |
| 1484 |
|
| 1485 |
## "Catching" Failures |
| 1486 |
|
| 1487 |
If you are building a testing utility on top of googletest, you'll want to test |
| 1488 |
your utility. What framework would you use to test it? googletest, of course. |
| 1489 |
|
| 1490 |
The challenge is to verify that your testing utility reports failures correctly. |
| 1491 |
In frameworks that report a failure by throwing an exception, you could catch |
| 1492 |
the exception and assert on it. But googletest doesn't use exceptions, so how do |
| 1493 |
we test that a piece of code generates an expected failure? |
| 1494 |
|
| 1495 |
`"gtest/gtest-spi.h"` contains some constructs to do this. |
| 1496 |
After #including this header, you can use |
| 1497 |
|
| 1498 |
```c++ |
| 1499 |
EXPECT_FATAL_FAILURE(statement, substring); |
| 1500 |
``` |
| 1501 |
|
| 1502 |
to assert that `statement` generates a fatal (e.g. `ASSERT_*`) failure in the |
| 1503 |
current thread whose message contains the given `substring`, or use |
| 1504 |
|
| 1505 |
```c++ |
| 1506 |
EXPECT_NONFATAL_FAILURE(statement, substring); |
| 1507 |
``` |
| 1508 |
|
| 1509 |
if you are expecting a non-fatal (e.g. `EXPECT_*`) failure. |
| 1510 |
|
| 1511 |
Only failures in the current thread are checked to determine the result of this |
| 1512 |
type of expectations. If `statement` creates new threads, failures in these |
| 1513 |
threads are also ignored. If you want to catch failures in other threads as |
| 1514 |
well, use one of the following macros instead: |
| 1515 |
|
| 1516 |
```c++ |
| 1517 |
EXPECT_FATAL_FAILURE_ON_ALL_THREADS(statement, substring); |
| 1518 |
EXPECT_NONFATAL_FAILURE_ON_ALL_THREADS(statement, substring); |
| 1519 |
``` |
| 1520 |
|
| 1521 |
{: .callout .note} |
| 1522 |
NOTE: Assertions from multiple threads are currently not supported on Windows. |
| 1523 |
|
| 1524 |
For technical reasons, there are some caveats: |
| 1525 |
|
| 1526 |
1. You cannot stream a failure message to either macro. |
| 1527 |
|
| 1528 |
2. `statement` in `EXPECT_FATAL_FAILURE{_ON_ALL_THREADS}()` cannot reference |
| 1529 |
local non-static variables or non-static members of `this` object. |
| 1530 |
|
| 1531 |
3. `statement` in `EXPECT_FATAL_FAILURE{_ON_ALL_THREADS}()` cannot return a |
| 1532 |
value. |
| 1533 |
|
| 1534 |
## Registering tests programmatically |
| 1535 |
|
| 1536 |
The `TEST` macros handle the vast majority of all use cases, but there are few |
| 1537 |
where runtime registration logic is required. For those cases, the framework |
| 1538 |
provides the `::testing::RegisterTest` that allows callers to register arbitrary |
| 1539 |
tests dynamically. |
| 1540 |
|
| 1541 |
This is an advanced API only to be used when the `TEST` macros are insufficient. |
| 1542 |
The macros should be preferred when possible, as they avoid most of the |
| 1543 |
complexity of calling this function. |
| 1544 |
|
| 1545 |
It provides the following signature: |
| 1546 |
|
| 1547 |
```c++ |
| 1548 |
template <typename Factory> |
| 1549 |
TestInfo* RegisterTest(const char* test_suite_name, const char* test_name, |
| 1550 |
const char* type_param, const char* value_param, |
| 1551 |
const char* file, int line, Factory factory); |
| 1552 |
``` |
| 1553 |
|
| 1554 |
The `factory` argument is a factory callable (move-constructible) object or |
| 1555 |
function pointer that creates a new instance of the Test object. It handles |
| 1556 |
ownership to the caller. The signature of the callable is `Fixture*()`, where |
| 1557 |
`Fixture` is the test fixture class for the test. All tests registered with the |
| 1558 |
same `test_suite_name` must return the same fixture type. This is checked at |
| 1559 |
runtime. |
| 1560 |
|
| 1561 |
The framework will infer the fixture class from the factory and will call the |
| 1562 |
`SetUpTestSuite` and `TearDownTestSuite` for it. |
| 1563 |
|
| 1564 |
Must be called before `RUN_ALL_TESTS()` is invoked, otherwise behavior is |
| 1565 |
undefined. |
| 1566 |
|
| 1567 |
Use case example: |
| 1568 |
|
| 1569 |
```c++ |
| 1570 |
class MyFixture : public testing::Test { |
| 1571 |
public: |
| 1572 |
// All of these optional, just like in regular macro usage. |
| 1573 |
static void SetUpTestSuite() { ... } |
| 1574 |
static void TearDownTestSuite() { ... } |
| 1575 |
void SetUp() override { ... } |
| 1576 |
void TearDown() override { ... } |
| 1577 |
}; |
| 1578 |
|
| 1579 |
class MyTest : public MyFixture { |
| 1580 |
public: |
| 1581 |
explicit MyTest(int data) : data_(data) {} |
| 1582 |
void TestBody() override { ... } |
| 1583 |
|
| 1584 |
private: |
| 1585 |
int data_; |
| 1586 |
}; |
| 1587 |
|
| 1588 |
void RegisterMyTests(const std::vector<int>& values) { |
| 1589 |
for (int v : values) { |
| 1590 |
testing::RegisterTest( |
| 1591 |
"MyFixture", ("Test" + std::to_string(v)).c_str(), nullptr, |
| 1592 |
std::to_string(v).c_str(), |
| 1593 |
__FILE__, __LINE__, |
| 1594 |
// Important to use the fixture type as the return type here. |
| 1595 |
[=]() -> MyFixture* { return new MyTest(v); }); |
| 1596 |
} |
| 1597 |
} |
| 1598 |
... |
| 1599 |
int main(int argc, char** argv) { |
| 1600 |
testing::InitGoogleTest(&argc, argv); |
| 1601 |
std::vector<int> values_to_test = LoadValuesFromConfig(); |
| 1602 |
RegisterMyTests(values_to_test); |
| 1603 |
... |
| 1604 |
return RUN_ALL_TESTS(); |
| 1605 |
} |
| 1606 |
``` |
| 1607 |
|
| 1608 |
## Getting the Current Test's Name |
| 1609 |
|
| 1610 |
Sometimes a function may need to know the name of the currently running test. |
| 1611 |
For example, you may be using the `SetUp()` method of your test fixture to set |
| 1612 |
the golden file name based on which test is running. The |
| 1613 |
[`TestInfo`](reference/testing.md#TestInfo) class has this information. |
| 1614 |
|
| 1615 |
To obtain a `TestInfo` object for the currently running test, call |
| 1616 |
`current_test_info()` on the [`UnitTest`](reference/testing.md#UnitTest) |
| 1617 |
singleton object: |
| 1618 |
|
| 1619 |
```c++ |
| 1620 |
// Gets information about the currently running test. |
| 1621 |
// Do NOT delete the returned object - it's managed by the UnitTest class. |
| 1622 |
const testing::TestInfo* const test_info = |
| 1623 |
testing::UnitTest::GetInstance()->current_test_info(); |
| 1624 |
|
| 1625 |
printf("We are in test %s of test suite %s.\n", |
| 1626 |
test_info->name(), |
| 1627 |
test_info->test_suite_name()); |
| 1628 |
``` |
| 1629 |
|
| 1630 |
`current_test_info()` returns a null pointer if no test is running. In |
| 1631 |
particular, you cannot find the test suite name in `SetUpTestSuite()`, |
| 1632 |
`TearDownTestSuite()` (where you know the test suite name implicitly), or |
| 1633 |
functions called from them. |
| 1634 |
|
| 1635 |
## Extending googletest by Handling Test Events |
| 1636 |
|
| 1637 |
googletest provides an **event listener API** to let you receive notifications |
| 1638 |
about the progress of a test program and test failures. The events you can |
| 1639 |
listen to include the start and end of the test program, a test suite, or a test |
| 1640 |
method, among others. You may use this API to augment or replace the standard |
| 1641 |
console output, replace the XML output, or provide a completely different form |
| 1642 |
of output, such as a GUI or a database. You can also use test events as |
| 1643 |
checkpoints to implement a resource leak checker, for example. |
| 1644 |
|
| 1645 |
### Defining Event Listeners |
| 1646 |
|
| 1647 |
To define a event listener, you subclass either |
| 1648 |
[`testing::TestEventListener`](reference/testing.md#TestEventListener) or |
| 1649 |
[`testing::EmptyTestEventListener`](reference/testing.md#EmptyTestEventListener) |
| 1650 |
The former is an (abstract) interface, where *each pure virtual method can be |
| 1651 |
overridden to handle a test event* (For example, when a test starts, the |
| 1652 |
`OnTestStart()` method will be called.). The latter provides an empty |
| 1653 |
implementation of all methods in the interface, such that a subclass only needs |
| 1654 |
to override the methods it cares about. |
| 1655 |
|
| 1656 |
When an event is fired, its context is passed to the handler function as an |
| 1657 |
argument. The following argument types are used: |
| 1658 |
|
| 1659 |
* UnitTest reflects the state of the entire test program, |
| 1660 |
* TestSuite has information about a test suite, which can contain one or more |
| 1661 |
tests, |
| 1662 |
* TestInfo contains the state of a test, and |
| 1663 |
* TestPartResult represents the result of a test assertion. |
| 1664 |
|
| 1665 |
An event handler function can examine the argument it receives to find out |
| 1666 |
interesting information about the event and the test program's state. |
| 1667 |
|
| 1668 |
Here's an example: |
| 1669 |
|
| 1670 |
```c++ |
| 1671 |
class MinimalistPrinter : public testing::EmptyTestEventListener { |
| 1672 |
// Called before a test starts. |
| 1673 |
void OnTestStart(const testing::TestInfo& test_info) override { |
| 1674 |
printf("*** Test %s.%s starting.\n", |
| 1675 |
test_info.test_suite_name(), test_info.name()); |
| 1676 |
} |
| 1677 |
|
| 1678 |
// Called after a failed assertion or a SUCCESS(). |
| 1679 |
void OnTestPartResult(const testing::TestPartResult& test_part_result) override { |
| 1680 |
printf("%s in %s:%d\n%s\n", |
| 1681 |
test_part_result.failed() ? "*** Failure" : "Success", |
| 1682 |
test_part_result.file_name(), |
| 1683 |
test_part_result.line_number(), |
| 1684 |
test_part_result.summary()); |
| 1685 |
} |
| 1686 |
|
| 1687 |
// Called after a test ends. |
| 1688 |
void OnTestEnd(const testing::TestInfo& test_info) override { |
| 1689 |
printf("*** Test %s.%s ending.\n", |
| 1690 |
test_info.test_suite_name(), test_info.name()); |
| 1691 |
} |
| 1692 |
}; |
| 1693 |
``` |
| 1694 |
|
| 1695 |
### Using Event Listeners |
| 1696 |
|
| 1697 |
To use the event listener you have defined, add an instance of it to the |
| 1698 |
googletest event listener list (represented by class |
| 1699 |
[`TestEventListeners`](reference/testing.md#TestEventListeners) - note the "s" |
| 1700 |
at the end of the name) in your `main()` function, before calling |
| 1701 |
`RUN_ALL_TESTS()`: |
| 1702 |
|
| 1703 |
```c++ |
| 1704 |
int main(int argc, char** argv) { |
| 1705 |
testing::InitGoogleTest(&argc, argv); |
| 1706 |
// Gets hold of the event listener list. |
| 1707 |
testing::TestEventListeners& listeners = |
| 1708 |
testing::UnitTest::GetInstance()->listeners(); |
| 1709 |
// Adds a listener to the end. googletest takes the ownership. |
| 1710 |
listeners.Append(new MinimalistPrinter); |
| 1711 |
return RUN_ALL_TESTS(); |
| 1712 |
} |
| 1713 |
``` |
| 1714 |
|
| 1715 |
There's only one problem: the default test result printer is still in effect, so |
| 1716 |
its output will mingle with the output from your minimalist printer. To suppress |
| 1717 |
the default printer, just release it from the event listener list and delete it. |
| 1718 |
You can do so by adding one line: |
| 1719 |
|
| 1720 |
```c++ |
| 1721 |
... |
| 1722 |
delete listeners.Release(listeners.default_result_printer()); |
| 1723 |
listeners.Append(new MinimalistPrinter); |
| 1724 |
return RUN_ALL_TESTS(); |
| 1725 |
``` |
| 1726 |
|
| 1727 |
Now, sit back and enjoy a completely different output from your tests. For more |
| 1728 |
details, see [sample9_unittest.cc]. |
| 1729 |
|
| 1730 |
[sample9_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample9_unittest.cc "Event listener example" |
| 1731 |
|
| 1732 |
You may append more than one listener to the list. When an `On*Start()` or |
| 1733 |
`OnTestPartResult()` event is fired, the listeners will receive it in the order |
| 1734 |
they appear in the list (since new listeners are added to the end of the list, |
| 1735 |
the default text printer and the default XML generator will receive the event |
| 1736 |
first). An `On*End()` event will be received by the listeners in the *reverse* |
| 1737 |
order. This allows output by listeners added later to be framed by output from |
| 1738 |
listeners added earlier. |
| 1739 |
|
| 1740 |
### Generating Failures in Listeners |
| 1741 |
|
| 1742 |
You may use failure-raising macros (`EXPECT_*()`, `ASSERT_*()`, `FAIL()`, etc) |
| 1743 |
when processing an event. There are some restrictions: |
| 1744 |
|
| 1745 |
1. You cannot generate any failure in `OnTestPartResult()` (otherwise it will |
| 1746 |
cause `OnTestPartResult()` to be called recursively). |
| 1747 |
2. A listener that handles `OnTestPartResult()` is not allowed to generate any |
| 1748 |
failure. |
| 1749 |
|
| 1750 |
When you add listeners to the listener list, you should put listeners that |
| 1751 |
handle `OnTestPartResult()` *before* listeners that can generate failures. This |
| 1752 |
ensures that failures generated by the latter are attributed to the right test |
| 1753 |
by the former. |
| 1754 |
|
| 1755 |
See [sample10_unittest.cc] for an example of a failure-raising listener. |
| 1756 |
|
| 1757 |
[sample10_unittest.cc]: https://github.com/google/googletest/blob/master/googletest/samples/sample10_unittest.cc "Failure-raising listener example" |
| 1758 |
|
| 1759 |
## Running Test Programs: Advanced Options |
| 1760 |
|
| 1761 |
googletest test programs are ordinary executables. Once built, you can run them |
| 1762 |
directly and affect their behavior via the following environment variables |
| 1763 |
and/or command line flags. For the flags to work, your programs must call |
| 1764 |
`::testing::InitGoogleTest()` before calling `RUN_ALL_TESTS()`. |
| 1765 |
|
| 1766 |
To see a list of supported flags and their usage, please run your test program |
| 1767 |
with the `--help` flag. You can also use `-h`, `-?`, or `/?` for short. |
| 1768 |
|
| 1769 |
If an option is specified both by an environment variable and by a flag, the |
| 1770 |
latter takes precedence. |
| 1771 |
|
| 1772 |
### Selecting Tests |
| 1773 |
|
| 1774 |
#### Listing Test Names |
| 1775 |
|
| 1776 |
Sometimes it is necessary to list the available tests in a program before |
| 1777 |
running them so that a filter may be applied if needed. Including the flag |
| 1778 |
`--gtest_list_tests` overrides all other flags and lists tests in the following |
| 1779 |
format: |
| 1780 |
|
| 1781 |
```none |
| 1782 |
TestSuite1. |
| 1783 |
TestName1 |
| 1784 |
TestName2 |
| 1785 |
TestSuite2. |
| 1786 |
TestName |
| 1787 |
``` |
| 1788 |
|
| 1789 |
None of the tests listed are actually run if the flag is provided. There is no |
| 1790 |
corresponding environment variable for this flag. |
| 1791 |
|
| 1792 |
#### Running a Subset of the Tests |
| 1793 |
|
| 1794 |
By default, a googletest program runs all tests the user has defined. Sometimes, |
| 1795 |
you want to run only a subset of the tests (e.g. for debugging or quickly |
| 1796 |
verifying a change). If you set the `GTEST_FILTER` environment variable or the |
| 1797 |
`--gtest_filter` flag to a filter string, googletest will only run the tests |
| 1798 |
whose full names (in the form of `TestSuiteName.TestName`) match the filter. |
| 1799 |
|
| 1800 |
The format of a filter is a '`:`'-separated list of wildcard patterns (called |
| 1801 |
the *positive patterns*) optionally followed by a '`-`' and another |
| 1802 |
'`:`'-separated pattern list (called the *negative patterns*). A test matches |
| 1803 |
the filter if and only if it matches any of the positive patterns but does not |
| 1804 |
match any of the negative patterns. |
| 1805 |
|
| 1806 |
A pattern may contain `'*'` (matches any string) or `'?'` (matches any single |
| 1807 |
character). For convenience, the filter `'*-NegativePatterns'` can be also |
| 1808 |
written as `'-NegativePatterns'`. |
| 1809 |
|
| 1810 |
For example: |
| 1811 |
|
| 1812 |
* `./foo_test` Has no flag, and thus runs all its tests. |
| 1813 |
* `./foo_test --gtest_filter=*` Also runs everything, due to the single |
| 1814 |
match-everything `*` value. |
| 1815 |
* `./foo_test --gtest_filter=FooTest.*` Runs everything in test suite |
| 1816 |
`FooTest` . |
| 1817 |
* `./foo_test --gtest_filter=*Null*:*Constructor*` Runs any test whose full |
| 1818 |
name contains either `"Null"` or `"Constructor"` . |
| 1819 |
* `./foo_test --gtest_filter=-*DeathTest.*` Runs all non-death tests. |
| 1820 |
* `./foo_test --gtest_filter=FooTest.*-FooTest.Bar` Runs everything in test |
| 1821 |
suite `FooTest` except `FooTest.Bar`. |
| 1822 |
* `./foo_test --gtest_filter=FooTest.*:BarTest.*-FooTest.Bar:BarTest.Foo` Runs |
| 1823 |
everything in test suite `FooTest` except `FooTest.Bar` and everything in |
| 1824 |
test suite `BarTest` except `BarTest.Foo`. |
| 1825 |
|
| 1826 |
#### Stop test execution upon first failure |
| 1827 |
|
| 1828 |
By default, a googletest program runs all tests the user has defined. In some |
| 1829 |
cases (e.g. iterative test development & execution) it may be desirable stop |
| 1830 |
test execution upon first failure (trading improved latency for completeness). |
| 1831 |
If `GTEST_FAIL_FAST` environment variable or `--gtest_fail_fast` flag is set, |
| 1832 |
the test runner will stop execution as soon as the first test failure is found. |
| 1833 |
|
| 1834 |
#### Temporarily Disabling Tests |
| 1835 |
|
| 1836 |
If you have a broken test that you cannot fix right away, you can add the |
| 1837 |
`DISABLED_` prefix to its name. This will exclude it from execution. This is |
| 1838 |
better than commenting out the code or using `#if 0`, as disabled tests are |
| 1839 |
still compiled (and thus won't rot). |
| 1840 |
|
| 1841 |
If you need to disable all tests in a test suite, you can either add `DISABLED_` |
| 1842 |
to the front of the name of each test, or alternatively add it to the front of |
| 1843 |
the test suite name. |
| 1844 |
|
| 1845 |
For example, the following tests won't be run by googletest, even though they |
| 1846 |
will still be compiled: |
| 1847 |
|
| 1848 |
```c++ |
| 1849 |
// Tests that Foo does Abc. |
| 1850 |
TEST(FooTest, DISABLED_DoesAbc) { ... } |
| 1851 |
|
| 1852 |
class DISABLED_BarTest : public testing::Test { ... }; |
| 1853 |
|
| 1854 |
// Tests that Bar does Xyz. |
| 1855 |
TEST_F(DISABLED_BarTest, DoesXyz) { ... } |
| 1856 |
``` |
| 1857 |
|
| 1858 |
{: .callout .note} |
| 1859 |
NOTE: This feature should only be used for temporary pain-relief. You still have |
| 1860 |
to fix the disabled tests at a later date. As a reminder, googletest will print |
| 1861 |
a banner warning you if a test program contains any disabled tests. |
| 1862 |
|
| 1863 |
{: .callout .tip} |
| 1864 |
TIP: You can easily count the number of disabled tests you have using |
| 1865 |
`grep`. This number can be used as a metric for |
| 1866 |
improving your test quality. |
| 1867 |
|
| 1868 |
#### Temporarily Enabling Disabled Tests |
| 1869 |
|
| 1870 |
To include disabled tests in test execution, just invoke the test program with |
| 1871 |
the `--gtest_also_run_disabled_tests` flag or set the |
| 1872 |
`GTEST_ALSO_RUN_DISABLED_TESTS` environment variable to a value other than `0`. |
| 1873 |
You can combine this with the `--gtest_filter` flag to further select which |
| 1874 |
disabled tests to run. |
| 1875 |
|
| 1876 |
### Repeating the Tests |
| 1877 |
|
| 1878 |
Once in a while you'll run into a test whose result is hit-or-miss. Perhaps it |
| 1879 |
will fail only 1% of the time, making it rather hard to reproduce the bug under |
| 1880 |
a debugger. This can be a major source of frustration. |
| 1881 |
|
| 1882 |
The `--gtest_repeat` flag allows you to repeat all (or selected) test methods in |
| 1883 |
a program many times. Hopefully, a flaky test will eventually fail and give you |
| 1884 |
a chance to debug. Here's how to use it: |
| 1885 |
|
| 1886 |
```none |
| 1887 |
$ foo_test --gtest_repeat=1000 |
| 1888 |
Repeat foo_test 1000 times and don't stop at failures. |
| 1889 |
|
| 1890 |
$ foo_test --gtest_repeat=-1 |
| 1891 |
A negative count means repeating forever. |
| 1892 |
|
| 1893 |
$ foo_test --gtest_repeat=1000 --gtest_break_on_failure |
| 1894 |
Repeat foo_test 1000 times, stopping at the first failure. This |
| 1895 |
is especially useful when running under a debugger: when the test |
| 1896 |
fails, it will drop into the debugger and you can then inspect |
| 1897 |
variables and stacks. |
| 1898 |
|
| 1899 |
$ foo_test --gtest_repeat=1000 --gtest_filter=FooBar.* |
| 1900 |
Repeat the tests whose name matches the filter 1000 times. |
| 1901 |
``` |
| 1902 |
|
| 1903 |
If your test program contains |
| 1904 |
[global set-up/tear-down](#global-set-up-and-tear-down) code, it will be |
| 1905 |
repeated in each iteration as well, as the flakiness may be in it. You can also |
| 1906 |
specify the repeat count by setting the `GTEST_REPEAT` environment variable. |
| 1907 |
|
| 1908 |
### Shuffling the Tests |
| 1909 |
|
| 1910 |
You can specify the `--gtest_shuffle` flag (or set the `GTEST_SHUFFLE` |
| 1911 |
environment variable to `1`) to run the tests in a program in a random order. |
| 1912 |
This helps to reveal bad dependencies between tests. |
| 1913 |
|
| 1914 |
By default, googletest uses a random seed calculated from the current time. |
| 1915 |
Therefore you'll get a different order every time. The console output includes |
| 1916 |
the random seed value, such that you can reproduce an order-related test failure |
| 1917 |
later. To specify the random seed explicitly, use the `--gtest_random_seed=SEED` |
| 1918 |
flag (or set the `GTEST_RANDOM_SEED` environment variable), where `SEED` is an |
| 1919 |
integer in the range [0, 99999]. The seed value 0 is special: it tells |
| 1920 |
googletest to do the default behavior of calculating the seed from the current |
| 1921 |
time. |
| 1922 |
|
| 1923 |
If you combine this with `--gtest_repeat=N`, googletest will pick a different |
| 1924 |
random seed and re-shuffle the tests in each iteration. |
| 1925 |
|
| 1926 |
### Distributing Test Functions to Multiple Machines |
| 1927 |
|
| 1928 |
If you have more than one machine you can use to run a test program, you might |
| 1929 |
want to run the test functions in parallel and get the result faster. We call |
| 1930 |
this technique *sharding*, where each machine is called a *shard*. |
| 1931 |
|
| 1932 |
GoogleTest is compatible with test sharding. To take advantage of this feature, |
| 1933 |
your test runner (not part of GoogleTest) needs to do the following: |
| 1934 |
|
| 1935 |
1. Allocate a number of machines (shards) to run the tests. |
| 1936 |
1. On each shard, set the `GTEST_TOTAL_SHARDS` environment variable to the total |
| 1937 |
number of shards. It must be the same for all shards. |
| 1938 |
1. On each shard, set the `GTEST_SHARD_INDEX` environment variable to the index |
| 1939 |
of the shard. Different shards must be assigned different indices, which |
| 1940 |
must be in the range `[0, GTEST_TOTAL_SHARDS - 1]`. |
| 1941 |
1. Run the same test program on all shards. When GoogleTest sees the above two |
| 1942 |
environment variables, it will select a subset of the test functions to run. |
| 1943 |
Across all shards, each test function in the program will be run exactly |
| 1944 |
once. |
| 1945 |
1. Wait for all shards to finish, then collect and report the results. |
| 1946 |
|
| 1947 |
Your project may have tests that were written without GoogleTest and thus don't |
| 1948 |
understand this protocol. In order for your test runner to figure out which test |
| 1949 |
supports sharding, it can set the environment variable `GTEST_SHARD_STATUS_FILE` |
| 1950 |
to a non-existent file path. If a test program supports sharding, it will create |
| 1951 |
this file to acknowledge that fact; otherwise it will not create it. The actual |
| 1952 |
contents of the file are not important at this time, although we may put some |
| 1953 |
useful information in it in the future. |
| 1954 |
|
| 1955 |
Here's an example to make it clear. Suppose you have a test program `foo_test` |
| 1956 |
that contains the following 5 test functions: |
| 1957 |
|
| 1958 |
``` |
| 1959 |
TEST(A, V) |
| 1960 |
TEST(A, W) |
| 1961 |
TEST(B, X) |
| 1962 |
TEST(B, Y) |
| 1963 |
TEST(B, Z) |
| 1964 |
``` |
| 1965 |
|
| 1966 |
Suppose you have 3 machines at your disposal. To run the test functions in |
| 1967 |
parallel, you would set `GTEST_TOTAL_SHARDS` to 3 on all machines, and set |
| 1968 |
`GTEST_SHARD_INDEX` to 0, 1, and 2 on the machines respectively. Then you would |
| 1969 |
run the same `foo_test` on each machine. |
| 1970 |
|
| 1971 |
GoogleTest reserves the right to change how the work is distributed across the |
| 1972 |
shards, but here's one possible scenario: |
| 1973 |
|
| 1974 |
* Machine #0 runs `A.V` and `B.X`. |
| 1975 |
* Machine #1 runs `A.W` and `B.Y`. |
| 1976 |
* Machine #2 runs `B.Z`. |
| 1977 |
|
| 1978 |
### Controlling Test Output |
| 1979 |
|
| 1980 |
#### Colored Terminal Output |
| 1981 |
|
| 1982 |
googletest can use colors in its terminal output to make it easier to spot the |
| 1983 |
important information: |
| 1984 |
|
| 1985 |
<pre>... |
| 1986 |
<font color="green">[----------]</font> 1 test from FooTest |
| 1987 |
<font color="green">[ RUN ]</font> FooTest.DoesAbc |
| 1988 |
<font color="green">[ OK ]</font> FooTest.DoesAbc |
| 1989 |
<font color="green">[----------]</font> 2 tests from BarTest |
| 1990 |
<font color="green">[ RUN ]</font> BarTest.HasXyzProperty |
| 1991 |
<font color="green">[ OK ]</font> BarTest.HasXyzProperty |
| 1992 |
<font color="green">[ RUN ]</font> BarTest.ReturnsTrueOnSuccess |
| 1993 |
... some error messages ... |
| 1994 |
<font color="red">[ FAILED ]</font> BarTest.ReturnsTrueOnSuccess |
| 1995 |
... |
| 1996 |
<font color="green">[==========]</font> 30 tests from 14 test suites ran. |
| 1997 |
<font color="green">[ PASSED ]</font> 28 tests. |
| 1998 |
<font color="red">[ FAILED ]</font> 2 tests, listed below: |
| 1999 |
<font color="red">[ FAILED ]</font> BarTest.ReturnsTrueOnSuccess |
| 2000 |
<font color="red">[ FAILED ]</font> AnotherTest.DoesXyz |
| 2001 |
|
| 2002 |
2 FAILED TESTS |
| 2003 |
</pre> |
| 2004 |
|
| 2005 |
You can set the `GTEST_COLOR` environment variable or the `--gtest_color` |
| 2006 |
command line flag to `yes`, `no`, or `auto` (the default) to enable colors, |
| 2007 |
disable colors, or let googletest decide. When the value is `auto`, googletest |
| 2008 |
will use colors if and only if the output goes to a terminal and (on non-Windows |
| 2009 |
platforms) the `TERM` environment variable is set to `xterm` or `xterm-color`. |
| 2010 |
|
| 2011 |
#### Suppressing test passes |
| 2012 |
|
| 2013 |
By default, googletest prints 1 line of output for each test, indicating if it |
| 2014 |
passed or failed. To show only test failures, run the test program with |
| 2015 |
`--gtest_brief=1`, or set the GTEST_BRIEF environment variable to `1`. |
| 2016 |
|
| 2017 |
#### Suppressing the Elapsed Time |
| 2018 |
|
| 2019 |
By default, googletest prints the time it takes to run each test. To disable |
| 2020 |
that, run the test program with the `--gtest_print_time=0` command line flag, or |
| 2021 |
set the GTEST_PRINT_TIME environment variable to `0`. |
| 2022 |
|
| 2023 |
#### Suppressing UTF-8 Text Output |
| 2024 |
|
| 2025 |
In case of assertion failures, googletest prints expected and actual values of |
| 2026 |
type `string` both as hex-encoded strings as well as in readable UTF-8 text if |
| 2027 |
they contain valid non-ASCII UTF-8 characters. If you want to suppress the UTF-8 |
| 2028 |
text because, for example, you don't have an UTF-8 compatible output medium, run |
| 2029 |
the test program with `--gtest_print_utf8=0` or set the `GTEST_PRINT_UTF8` |
| 2030 |
environment variable to `0`. |
| 2031 |
|
| 2032 |
#### Generating an XML Report |
| 2033 |
|
| 2034 |
googletest can emit a detailed XML report to a file in addition to its normal |
| 2035 |
textual output. The report contains the duration of each test, and thus can help |
| 2036 |
you identify slow tests. |
| 2037 |
|
| 2038 |
To generate the XML report, set the `GTEST_OUTPUT` environment variable or the |
| 2039 |
`--gtest_output` flag to the string `"xml:path_to_output_file"`, which will |
| 2040 |
create the file at the given location. You can also just use the string `"xml"`, |
| 2041 |
in which case the output can be found in the `test_detail.xml` file in the |
| 2042 |
current directory. |
| 2043 |
|
| 2044 |
If you specify a directory (for example, `"xml:output/directory/"` on Linux or |
| 2045 |
`"xml:output\directory\"` on Windows), googletest will create the XML file in |
| 2046 |
that directory, named after the test executable (e.g. `foo_test.xml` for test |
| 2047 |
program `foo_test` or `foo_test.exe`). If the file already exists (perhaps left |
| 2048 |
over from a previous run), googletest will pick a different name (e.g. |
| 2049 |
`foo_test_1.xml`) to avoid overwriting it. |
| 2050 |
|
| 2051 |
The report is based on the `junitreport` Ant task. Since that format was |
| 2052 |
originally intended for Java, a little interpretation is required to make it |
| 2053 |
apply to googletest tests, as shown here: |
| 2054 |
|
| 2055 |
```xml |
| 2056 |
<testsuites name="AllTests" ...> |
| 2057 |
<testsuite name="test_case_name" ...> |
| 2058 |
<testcase name="test_name" ...> |
| 2059 |
<failure message="..."/> |
| 2060 |
<failure message="..."/> |
| 2061 |
<failure message="..."/> |
| 2062 |
</testcase> |
| 2063 |
</testsuite> |
| 2064 |
</testsuites> |
| 2065 |
``` |
| 2066 |
|
| 2067 |
* The root `<testsuites>` element corresponds to the entire test program. |
| 2068 |
* `<testsuite>` elements correspond to googletest test suites. |
| 2069 |
* `<testcase>` elements correspond to googletest test functions. |
| 2070 |
|
| 2071 |
For instance, the following program |
| 2072 |
|
| 2073 |
```c++ |
| 2074 |
TEST(MathTest, Addition) { ... } |
| 2075 |
TEST(MathTest, Subtraction) { ... } |
| 2076 |
TEST(LogicTest, NonContradiction) { ... } |
| 2077 |
``` |
| 2078 |
|
| 2079 |
could generate this report: |
| 2080 |
|
| 2081 |
```xml |
| 2082 |
<?xml version="1.0" encoding="UTF-8"?> |
| 2083 |
<testsuites tests="3" failures="1" errors="0" time="0.035" timestamp="2011-10-31T18:52:42" name="AllTests"> |
| 2084 |
<testsuite name="MathTest" tests="2" failures="1" errors="0" time="0.015"> |
| 2085 |
<testcase name="Addition" status="run" time="0.007" classname=""> |
| 2086 |
<failure message="Value of: add(1, 1)
 Actual: 3
Expected: 2" type="">...</failure> |
| 2087 |
<failure message="Value of: add(1, -1)
 Actual: 1
Expected: 0" type="">...</failure> |
| 2088 |
</testcase> |
| 2089 |
<testcase name="Subtraction" status="run" time="0.005" classname=""> |
| 2090 |
</testcase> |
| 2091 |
</testsuite> |
| 2092 |
<testsuite name="LogicTest" tests="1" failures="0" errors="0" time="0.005"> |
| 2093 |
<testcase name="NonContradiction" status="run" time="0.005" classname=""> |
| 2094 |
</testcase> |
| 2095 |
</testsuite> |
| 2096 |
</testsuites> |
| 2097 |
``` |
| 2098 |
|
| 2099 |
Things to note: |
| 2100 |
|
| 2101 |
* The `tests` attribute of a `<testsuites>` or `<testsuite>` element tells how |
| 2102 |
many test functions the googletest program or test suite contains, while the |
| 2103 |
`failures` attribute tells how many of them failed. |
| 2104 |
|
| 2105 |
* The `time` attribute expresses the duration of the test, test suite, or |
| 2106 |
entire test program in seconds. |
| 2107 |
|
| 2108 |
* The `timestamp` attribute records the local date and time of the test |
| 2109 |
execution. |
| 2110 |
|
| 2111 |
* Each `<failure>` element corresponds to a single failed googletest |
| 2112 |
assertion. |
| 2113 |
|
| 2114 |
#### Generating a JSON Report |
| 2115 |
|
| 2116 |
googletest can also emit a JSON report as an alternative format to XML. To |
| 2117 |
generate the JSON report, set the `GTEST_OUTPUT` environment variable or the |
| 2118 |
`--gtest_output` flag to the string `"json:path_to_output_file"`, which will |
| 2119 |
create the file at the given location. You can also just use the string |
| 2120 |
`"json"`, in which case the output can be found in the `test_detail.json` file |
| 2121 |
in the current directory. |
| 2122 |
|
| 2123 |
The report format conforms to the following JSON Schema: |
| 2124 |
|
| 2125 |
```json |
| 2126 |
{ |
| 2127 |
"$schema": "http://json-schema.org/schema#", |
| 2128 |
"type": "object", |
| 2129 |
"definitions": { |
| 2130 |
"TestCase": { |
| 2131 |
"type": "object", |
| 2132 |
"properties": { |
| 2133 |
"name": { "type": "string" }, |
| 2134 |
"tests": { "type": "integer" }, |
| 2135 |
"failures": { "type": "integer" }, |
| 2136 |
"disabled": { "type": "integer" }, |
| 2137 |
"time": { "type": "string" }, |
| 2138 |
"testsuite": { |
| 2139 |
"type": "array", |
| 2140 |
"items": { |
| 2141 |
"$ref": "#/definitions/TestInfo" |
| 2142 |
} |
| 2143 |
} |
| 2144 |
} |
| 2145 |
}, |
| 2146 |
"TestInfo": { |
| 2147 |
"type": "object", |
| 2148 |
"properties": { |
| 2149 |
"name": { "type": "string" }, |
| 2150 |
"status": { |
| 2151 |
"type": "string", |
| 2152 |
"enum": ["RUN", "NOTRUN"] |
| 2153 |
}, |
| 2154 |
"time": { "type": "string" }, |
| 2155 |
"classname": { "type": "string" }, |
| 2156 |
"failures": { |
| 2157 |
"type": "array", |
| 2158 |
"items": { |
| 2159 |
"$ref": "#/definitions/Failure" |
| 2160 |
} |
| 2161 |
} |
| 2162 |
} |
| 2163 |
}, |
| 2164 |
"Failure": { |
| 2165 |
"type": "object", |
| 2166 |
"properties": { |
| 2167 |
"failures": { "type": "string" }, |
| 2168 |
"type": { "type": "string" } |
| 2169 |
} |
| 2170 |
} |
| 2171 |
}, |
| 2172 |
"properties": { |
| 2173 |
"tests": { "type": "integer" }, |
| 2174 |
"failures": { "type": "integer" }, |
| 2175 |
"disabled": { "type": "integer" }, |
| 2176 |
"errors": { "type": "integer" }, |
| 2177 |
"timestamp": { |
| 2178 |
"type": "string", |
| 2179 |
"format": "date-time" |
| 2180 |
}, |
| 2181 |
"time": { "type": "string" }, |
| 2182 |
"name": { "type": "string" }, |
| 2183 |
"testsuites": { |
| 2184 |
"type": "array", |
| 2185 |
"items": { |
| 2186 |
"$ref": "#/definitions/TestCase" |
| 2187 |
} |
| 2188 |
} |
| 2189 |
} |
| 2190 |
} |
| 2191 |
``` |
| 2192 |
|
| 2193 |
The report uses the format that conforms to the following Proto3 using the |
| 2194 |
[JSON encoding](https://developers.google.com/protocol-buffers/docs/proto3#json): |
| 2195 |
|
| 2196 |
```proto |
| 2197 |
syntax = "proto3"; |
| 2198 |
|
| 2199 |
package googletest; |
| 2200 |
|
| 2201 |
import "google/protobuf/timestamp.proto"; |
| 2202 |
import "google/protobuf/duration.proto"; |
| 2203 |
|
| 2204 |
message UnitTest { |
| 2205 |
int32 tests = 1; |
| 2206 |
int32 failures = 2; |
| 2207 |
int32 disabled = 3; |
| 2208 |
int32 errors = 4; |
| 2209 |
google.protobuf.Timestamp timestamp = 5; |
| 2210 |
google.protobuf.Duration time = 6; |
| 2211 |
string name = 7; |
| 2212 |
repeated TestCase testsuites = 8; |
| 2213 |
} |
| 2214 |
|
| 2215 |
message TestCase { |
| 2216 |
string name = 1; |
| 2217 |
int32 tests = 2; |
| 2218 |
int32 failures = 3; |
| 2219 |
int32 disabled = 4; |
| 2220 |
int32 errors = 5; |
| 2221 |
google.protobuf.Duration time = 6; |
| 2222 |
repeated TestInfo testsuite = 7; |
| 2223 |
} |
| 2224 |
|
| 2225 |
message TestInfo { |
| 2226 |
string name = 1; |
| 2227 |
enum Status { |
| 2228 |
RUN = 0; |
| 2229 |
NOTRUN = 1; |
| 2230 |
} |
| 2231 |
Status status = 2; |
| 2232 |
google.protobuf.Duration time = 3; |
| 2233 |
string classname = 4; |
| 2234 |
message Failure { |
| 2235 |
string failures = 1; |
| 2236 |
string type = 2; |
| 2237 |
} |
| 2238 |
repeated Failure failures = 5; |
| 2239 |
} |
| 2240 |
``` |
| 2241 |
|
| 2242 |
For instance, the following program |
| 2243 |
|
| 2244 |
```c++ |
| 2245 |
TEST(MathTest, Addition) { ... } |
| 2246 |
TEST(MathTest, Subtraction) { ... } |
| 2247 |
TEST(LogicTest, NonContradiction) { ... } |
| 2248 |
``` |
| 2249 |
|
| 2250 |
could generate this report: |
| 2251 |
|
| 2252 |
```json |
| 2253 |
{ |
| 2254 |
"tests": 3, |
| 2255 |
"failures": 1, |
| 2256 |
"errors": 0, |
| 2257 |
"time": "0.035s", |
| 2258 |
"timestamp": "2011-10-31T18:52:42Z", |
| 2259 |
"name": "AllTests", |
| 2260 |
"testsuites": [ |
| 2261 |
{ |
| 2262 |
"name": "MathTest", |
| 2263 |
"tests": 2, |
| 2264 |
"failures": 1, |
| 2265 |
"errors": 0, |
| 2266 |
"time": "0.015s", |
| 2267 |
"testsuite": [ |
| 2268 |
{ |
| 2269 |
"name": "Addition", |
| 2270 |
"status": "RUN", |
| 2271 |
"time": "0.007s", |
| 2272 |
"classname": "", |
| 2273 |
"failures": [ |
| 2274 |
{ |
| 2275 |
"message": "Value of: add(1, 1)\n Actual: 3\nExpected: 2", |
| 2276 |
"type": "" |
| 2277 |
}, |
| 2278 |
{ |
| 2279 |
"message": "Value of: add(1, -1)\n Actual: 1\nExpected: 0", |
| 2280 |
"type": "" |
| 2281 |
} |
| 2282 |
] |
| 2283 |
}, |
| 2284 |
{ |
| 2285 |
"name": "Subtraction", |
| 2286 |
"status": "RUN", |
| 2287 |
"time": "0.005s", |
| 2288 |
"classname": "" |
| 2289 |
} |
| 2290 |
] |
| 2291 |
}, |
| 2292 |
{ |
| 2293 |
"name": "LogicTest", |
| 2294 |
"tests": 1, |
| 2295 |
"failures": 0, |
| 2296 |
"errors": 0, |
| 2297 |
"time": "0.005s", |
| 2298 |
"testsuite": [ |
| 2299 |
{ |
| 2300 |
"name": "NonContradiction", |
| 2301 |
"status": "RUN", |
| 2302 |
"time": "0.005s", |
| 2303 |
"classname": "" |
| 2304 |
} |
| 2305 |
] |
| 2306 |
} |
| 2307 |
] |
| 2308 |
} |
| 2309 |
``` |
| 2310 |
|
| 2311 |
{: .callout .important} |
| 2312 |
IMPORTANT: The exact format of the JSON document is subject to change. |
| 2313 |
|
| 2314 |
### Controlling How Failures Are Reported |
| 2315 |
|
| 2316 |
#### Detecting Test Premature Exit |
| 2317 |
|
| 2318 |
Google Test implements the _premature-exit-file_ protocol for test runners to |
| 2319 |
catch any kind of unexpected exits of test programs. Upon start, Google Test |
| 2320 |
creates the file which will be automatically deleted after all work has been |
| 2321 |
finished. Then, the test runner can check if this file exists. In case the file |
| 2322 |
remains undeleted, the inspected test has exited prematurely. |
| 2323 |
|
| 2324 |
This feature is enabled only if the `TEST_PREMATURE_EXIT_FILE` environment |
| 2325 |
variable has been set. |
| 2326 |
|
| 2327 |
#### Turning Assertion Failures into Break-Points |
| 2328 |
|
| 2329 |
When running test programs under a debugger, it's very convenient if the |
| 2330 |
debugger can catch an assertion failure and automatically drop into interactive |
| 2331 |
mode. googletest's *break-on-failure* mode supports this behavior. |
| 2332 |
|
| 2333 |
To enable it, set the `GTEST_BREAK_ON_FAILURE` environment variable to a value |
| 2334 |
other than `0`. Alternatively, you can use the `--gtest_break_on_failure` |
| 2335 |
command line flag. |
| 2336 |
|
| 2337 |
#### Disabling Catching Test-Thrown Exceptions |
| 2338 |
|
| 2339 |
googletest can be used either with or without exceptions enabled. If a test |
| 2340 |
throws a C++ exception or (on Windows) a structured exception (SEH), by default |
| 2341 |
googletest catches it, reports it as a test failure, and continues with the next |
| 2342 |
test method. This maximizes the coverage of a test run. Also, on Windows an |
| 2343 |
uncaught exception will cause a pop-up window, so catching the exceptions allows |
| 2344 |
you to run the tests automatically. |
| 2345 |
|
| 2346 |
When debugging the test failures, however, you may instead want the exceptions |
| 2347 |
to be handled by the debugger, such that you can examine the call stack when an |
| 2348 |
exception is thrown. To achieve that, set the `GTEST_CATCH_EXCEPTIONS` |
| 2349 |
environment variable to `0`, or use the `--gtest_catch_exceptions=0` flag when |
| 2350 |
running the tests. |
| 2351 |
|
| 2352 |
### Sanitizer Integration |
| 2353 |
|
| 2354 |
The |
| 2355 |
[Undefined Behavior Sanitizer](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html), |
| 2356 |
[Address Sanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizer), |
| 2357 |
and |
| 2358 |
[Thread Sanitizer](https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual) |
| 2359 |
all provide weak functions that you can override to trigger explicit failures |
| 2360 |
when they detect sanitizer errors, such as creating a reference from `nullptr`. |
| 2361 |
To override these functions, place definitions for them in a source file that |
| 2362 |
you compile as part of your main binary: |
| 2363 |
|
| 2364 |
``` |
| 2365 |
extern "C" { |
| 2366 |
void __ubsan_on_report() { |
| 2367 |
FAIL() << "Encountered an undefined behavior sanitizer error"; |
| 2368 |
} |
| 2369 |
void __asan_on_error() { |
| 2370 |
FAIL() << "Encountered an address sanitizer error"; |
| 2371 |
} |
| 2372 |
void __tsan_on_report() { |
| 2373 |
FAIL() << "Encountered a thread sanitizer error"; |
| 2374 |
} |
| 2375 |
} // extern "C" |
| 2376 |
``` |
| 2377 |
|
| 2378 |
After compiling your project with one of the sanitizers enabled, if a particular |
| 2379 |
test triggers a sanitizer error, googletest will report that it failed. |