boringssl/FUZZING.md

Fuzz testing

Modern fuzz testers are very effective and we wish to use them to ensure that no silly bugs creep into BoringSSL.

We primarily use Clang's libFuzzer for fuzz testing and there are a number of fuzz testing functions in fuzz/. They are not built by default because they require libFuzzer at build time.

In order to build the fuzz tests you will need at least Clang 3.7. Pass -DFUZZ=1 on the CMake command line to enable building BoringSSL with coverage and AddressSanitizer, and to build the fuzz test binaries. You'll probably need to set the CC and CXX environment variables too, like this:

CC=clang CXX=clang++ cmake -GNinja -DFUZZ=1 ..

In order for the fuzz tests to link, the linker needs to find libFuzzer. This is not commonly provided and you may need to download the Clang source code and do the following:

svn co http://llvm.org/svn/llvm-project/llvm/trunk/lib/Fuzzer
clang++ -c -g -O2 -std=c++11 Fuzzer/*.cpp -IFuzzer
ar ruv libFuzzer.a Fuzzer*.o

Then copy libFuzzer.a to the top-level of your BoringSSL source directory.

From the build/ directory, you can then run the fuzzers. For example:

./fuzz/cert -max_len=3072 -jobs=32 -workers=32 ../fuzz/cert_corpus/

The arguments to jobs and workers should be the number of cores that you wish to dedicate to fuzzing. By default, libFuzzer uses the largest test in the corpus (or 64 if empty) as the maximum test case length. The max_len argument overrides this.

The recommended values of max_len for each test are:

Test	max_len value
cert	3072
client	20000
pkcs8	2048
privkey	2048
server	4096
spki	1024
read_pem	512
ssl_ctx_api	256

These were determined by rounding up the length of the largest case in the corpus.

There are directories in fuzz/ for each of the fuzzing tests which contain seed files for fuzzing. Some of the seed files were generated manually but many of them are “interesting” results generated by the fuzzing itself. (Where “interesting” means that it triggered a previously unknown path in the code.)

Minimising the corpuses

When a large number of new seeds are available, it's a good idea to minimise the corpus so that different seeds that trigger the same code paths can be deduplicated.

In order to minimise all the corpuses, build for fuzzing and run ./fuzz/minimise_corpuses.sh. Note that minimisation is, oddly, often not idempotent for unknown reasons.

Fuzzer mode

When -DFUZZ=1 is passed into CMake, BoringSSL builds with BORINGSSL_UNSAFE_FUZZER_MODE and BORINGSSL_UNSAFE_DETERMINISTIC_MODE defined. This modifies the library to be more friendly to fuzzers. If BORINGSSL_UNSAFE_DETERMINISTIC_MODE is set, BoringSSL will:

• Replace RAND_bytes with a deterministic PRNG. Call RAND_reset_for_fuzzing() at the start of fuzzers which use RAND_bytes to reset the PRNG state.

• Use a hard-coded time instead of the actual time.

Additionally, if BORINGSSL_UNSAFE_FUZZER_MODE is set, BoringSSL will:

• Modify the TLS stack to perform all signature checks (CertificateVerify and ServerKeyExchange) and the Finished check, but always act as if the check succeeded.

• Treat every cipher as the NULL cipher.

• Tickets are unencrypted and the MAC check is performed but ignored.

This is to prevent the fuzzer from getting stuck at a cryptographic invariant in the protocol.

TLS transcripts

The client and server corpora are seeded from the test suite. The test suite has a -fuzzer flag which mirrors the fuzzer mode changes above and a -deterministic flag which removes all non-determinism on the Go side. Not all tests pass, so ssl/test/runner/fuzzer_mode.json contains the necessary suppressions. To run the tests against a fuzzer-mode bssl_shim, run:

cd ssl/test/runner
go test -fuzzer -deterministic -shim-config fuzzer_mode.json

For a different build directory from build/, pass the appropriate -shim-path flag. If those tests pass, record a set of transcripts with:

go test -fuzzer -deterministic -transcript-dir /tmp/transcripts/

Note the suppressions file is ignored so disabled tests record transcripts too. Then merge into the existing corpora:

cd build/
./fuzz/client -max_len=50000 -merge=1 ../fuzz/client_corpus /tmp/transcripts/tls/client
./fuzz/server -max_len=50000 -merge=1 ../fuzz/server_corpus /tmp/transcripts/tls/server

There are separate corpora, fuzz/client_corpus_no_fuzzer_mode and fuzz/server_corpus_no_fuzzer_mode. These are transcripts for fuzzers with only BORINGSSL_UNSAFE_DETERMINISTIC_MODE defined. To build in this mode, pass -DNO_FUZZER_MODE=1 into CMake. These corpora are updated the same way, but without the -fuzzer and -shim-config flags.