| 1 |
Fuzz Tests for CPython |
| 2 |
====================== |
| 3 |
|
| 4 |
These fuzz tests are designed to be included in Google's `oss-fuzz`_ project. |
| 5 |
|
| 6 |
oss-fuzz works against a library exposing a function of the form |
| 7 |
``int LLVMFuzzerTestOneInput(const uint8_t* data, size_t length)``. We provide |
| 8 |
that library (``fuzzer.c``), and include a ``_fuzz`` module for testing with |
| 9 |
some toy values -- no fuzzing occurs in Python's test suite. |
| 10 |
|
| 11 |
oss-fuzz will regularly pull from CPython, discover all the tests in |
| 12 |
``fuzz_tests.txt``, and run them -- so adding a new test here means it will |
| 13 |
automatically be run in oss-fuzz, while also being smoke-tested as part of |
| 14 |
CPython's test suite. |
| 15 |
|
| 16 |
In addition, the tests are run on GitHub Actions using CIFuzz for PRs to the |
| 17 |
main branch changing relevant files. |
| 18 |
|
| 19 |
Adding a new fuzz test |
| 20 |
---------------------- |
| 21 |
|
| 22 |
Add the test name on a new line in ``fuzz_tests.txt``. |
| 23 |
|
| 24 |
In ``fuzzer.c``, add a function to be run:: |
| 25 |
|
| 26 |
static int $fuzz_test_name(const char* data, size_t size) { |
| 27 |
... |
| 28 |
return 0; |
| 29 |
} |
| 30 |
|
| 31 |
|
| 32 |
And invoke it from ``LLVMFuzzerTestOneInput``:: |
| 33 |
|
| 34 |
#if !defined(_Py_FUZZ_ONE) || defined(_Py_FUZZ_$fuzz_test_name) |
| 35 |
rv |= _run_fuzz(data, size, $fuzz_test_name); |
| 36 |
#endif |
| 37 |
|
| 38 |
Don't forget to replace ``$fuzz_test_name`` with your actual test name. |
| 39 |
|
| 40 |
``LLVMFuzzerTestOneInput`` will run in oss-fuzz, with each test in |
| 41 |
``fuzz_tests.txt`` run separately. |
| 42 |
|
| 43 |
Seed data (corpus) for the test can be provided in a subfolder called |
| 44 |
``<test_name>_corpus`` such as ``fuzz_json_loads_corpus``. A wide variety |
| 45 |
of good input samples allows the fuzzer to more easily explore a diverse |
| 46 |
set of paths and provides a better base to find buggy input from. |
| 47 |
|
| 48 |
Dictionaries of tokens (see oss-fuzz documentation for more details) can |
| 49 |
be placed in the ``dictionaries`` folder with the name of the test. |
| 50 |
For example, ``dictionaries/fuzz_json_loads.dict`` contains JSON tokens |
| 51 |
to guide the fuzzer. |
| 52 |
|
| 53 |
What makes a good fuzz test |
| 54 |
--------------------------- |
| 55 |
|
| 56 |
Libraries written in C that might handle untrusted data are worthwhile. The |
| 57 |
more complex the logic (e.g. parsing), the more likely this is to be a useful |
| 58 |
fuzz test. See the existing examples for reference, and refer to the |
| 59 |
`oss-fuzz`_ docs. |
| 60 |
|
| 61 |
.. _oss-fuzz: https://github.com/google/oss-fuzz |