This uses the x86 trap flag and libunwind to test CFI works at each
instruction. For now, it just uses the system one out of pkg-config and
disables unwind tests if unavailable. We'll probably want to stick a
copy into //third_party and perhaps try the LLVM one later.
This tester caught two bugs in P-256 CFI annotations already:
I47b5f9798b3bcee1748e537b21c173d312a14b42 and
I9f576d868850312d6c14d1386f8fbfa85021b347
An earlier design used PTRACE_SINGLESTEP with libunwind's remote
unwinding features. ptrace is a mess around stop signals (see group-stop
discussion in ptrace(2)) and this is 10x faster, so I went with it. The
question of which is more future-proof is complex:
- There are two libunwinds with the same API,
https://www.nongnu.org/libunwind/ and LLVM's. This currently uses the
system nongnu.org for convenience. In future, LLVM's should be easier
to bundle (less complex build) and appears to even support Windows,
but I haven't tested this. Moreover, setting the trap flag keeps the
test single-process, which is less complex on Windows. That suggests
the trap flag design and switching to LLVM later. However...
- Not all architectures have a trap flag settable by userspace. As far
as I can tell, ARMv8's PSTATE.SS can only be set from the kernel. If
we stick with nongnu.org libunwind, we can use PTRACE_SINGLESTEP and
remote unwinding. Or we implement it for LLVM. Another thought is for
the ptracer to bounce SIGTRAP back into the process, to share the
local unwinding code.
- ARMv7 has no trap flag at all and PTRACE_SINGLESTEP fails. Debuggers
single-step by injecting breakpoints instead. However, ARMv8's trap
flag seems to work in both AArch32 and AArch64 modes, so we may be
able to condition it on a 64-bit kernel.
Sadly, neither strategy works with Intel SDE. Adding flags to cpucap
vectors as we do with ARM would help, but it would not emulate CPUs
newer than the host CPU. For now, I've just had SDE tests disable these.
Annoyingly, CMake does not allow object libraries to have dependencies,
so make test_support a proper static library. Rename the target to
test_support_lib to avoid
https://gitlab.kitware.com/cmake/cmake/issues/17785
Update-Note: This adds a new optional test dependency, but it's disabled
by default (define BORINGSSL_HAVE_LIBUNWIND), so consumers do not need
to do anything. We'll probably want to adjust this in the future.
Bug: 181
Change-Id: I817263d7907aff0904a9cee83f8b26747262cc0c
Reviewed-on: https://boringssl-review.googlesource.com/c/33966
Commit-Queue: David Benjamin <davidben@google.com>
Reviewed-by: Adam Langley <agl@google.com>
We can actually use modules now.
Change-Id: I0bd8abaf4e3318069f93fa17e89b4804d03944eb
Reviewed-on: https://boringssl-review.googlesource.com/c/33205
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
Reviewed-by: Adam Langley <agl@google.com>
We test all Intel variants via SDE. For ARM, we can do the next best
thing and tweak with OPENSSL_armcap_P. If the host CPU does not support
the instructions we wish to test, skip it, but print something so we
know whether we need a more featureful test device.
Also fix the "CRASHED" status to "CRASH", to match
https://chromium.googlesource.com/chromium/src/+/master/docs/testing/json_test_results_format.md
(It's unclear if anything actually parses that JSON very carefully...)
Bug: 19
Change-Id: I811cc00a0d210a454287ac79c06f18fbc54f96dd
Reviewed-on: https://boringssl-review.googlesource.com/c/33204
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
Reviewed-by: Adam Langley <agl@google.com>
It's a little hard to read with all those command-lines flying by. Only
print out full commands for failing tests.
Change-Id: I35f2febf7686dbc1ab428fe5d06afee2afa8bcaf
Reviewed-on: https://boringssl-review.googlesource.com/20905
Commit-Queue: Steven Valdez <svaldez@google.com>
Reviewed-by: Steven Valdez <svaldez@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
The changes to the assembly files are synced from upstream's
64d92d74985ebb3d0be58a9718f9e080a14a8e7f. cpu-intel.c is translated to C
from that commit and d84df594404ebbd71d21fec5526178d935e4d88d.
Change-Id: I02c8f83aa4780df301c21f011ef2d8d8300e2f2a
Reviewed-on: https://boringssl-review.googlesource.com/18411
Commit-Queue: Adam Langley <agl@google.com>
Reviewed-by: Adam Langley <agl@google.com>
We lost some parallelism by putting the tests into one binary and have
enough giant test vector files now that this takes some time. Shard them
back up again.
BUG=129
Change-Id: I1d196bd8c4851bf975d6b4f2f0403ae65feac884
Reviewed-on: https://boringssl-review.googlesource.com/16984
Commit-Queue: Adam Langley <agl@google.com>
Reviewed-by: Adam Langley <agl@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
When a test fails, there should be spaces between the argv of the failed
command line.
Change-Id: I5c168a919c1615df34a0eab63a7232453168adb3
Reviewed-on: https://boringssl-review.googlesource.com/15846
Reviewed-by: David Benjamin <davidben@google.com>
This change makes util/all_tests.go run as many test binaries
concurrently as there are cores on the current system. This can be
overridden with -num-workers=1.
Change-Id: Ia3a5e336d208039be9276261a0ac03f7fb774677
Reviewed-on: https://boringssl-review.googlesource.com/14927
Reviewed-by: Adam Langley <agl@google.com>
GTest sends its output to stdout, not stderr. Merge them in the runner
(though eventually we'll teach the bots to run the GTest targets
directly) so we don't lose it.
BUG=129
Change-Id: I7c499cd9572f46f97bd4b7f6c6c9beca057625f2
Reviewed-on: https://boringssl-review.googlesource.com/13624
Reviewed-by: Steven Valdez <svaldez@google.com>
Reviewed-by: David Benjamin <davidben@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
Intel SDE is a tool that can simulate many different Intel chips. This
lets us test whether our CPUID-guarding is correct and would have
caught, for example, this morning's ChaCha20-Poly1305 problem.
Change-Id: I39de2bedb1c29b48b02ba30c51fdce57a5cbe640
Reviewed-on: https://boringssl-review.googlesource.com/13587
Commit-Queue: Adam Langley <alangley@gmail.com>
Commit-Queue: David Benjamin <davidben@google.com>
Reviewed-by: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
For now, this is the laziest conversion possible. The intent is to just
get the build setup ready so that we can get everything working in our
consumers. The intended end state is:
- The standalone build produces three test targets, one per library:
{crypto,ssl,decrepit}_tests.
- Each FOO_test is made up of:
FOO/**/*_test.cc
crypto/test/gtest_main.cc
test_support
- generate_build_files.py emits variables crypto_test_sources and
ssl_test_sources. These variables are populated with FindCFiles,
looking for *_test.cc.
- The consuming file assembles those variables into the two test targets
(plus decrepit) from there. This avoids having generate_build_files.py
emit actual build rules.
- Our standalone builders, Chromium, and Android just run the top-level
test targets using whatever GTest-based reporting story they have.
In transition, we start by converting one of two tests in each library
to populate the three test targets. Those are added to all_tests.json
and all_tests.go hacked to handle them transparently. This keeps our
standalone builder working.
generate_build_files.py, to start with, populates the new source lists
manually and subtracts them out of the old machinery. We emit both for
the time being. When this change rolls in, we'll write all the build
glue needed to build the GTest-based tests and add it to consumers'
continuous builders.
Next, we'll subsume a file-based test and get the consumers working with
that. (I.e. make sure the GTest targets can depend on a data file.)
Once that's all done, we'll be sure all this will work. At that point,
we start subsuming the remaining tests into the GTest targets and,
asynchronously, rewriting tests to use GTest properly rather than
cursory conversion here.
When all non-GTest tests are gone, the old generate_build_files.py hooks
will be removed, consumers updated to not depend on them, and standalone
builders converted to not rely on all_tests.go, which can then be
removed. (Unless bits end up being needed as a malloc test driver. I'm
thinking we'll want to do something with --gtest_filter.)
As part of this CL, I've bumped the CMake requirements (for
target_include_directories) and added a few suppressions for warnings
that GTest doesn't pass.
BUG=129
Change-Id: I881b26b07a8739cc0b52dbb51a30956908e1b71a
Reviewed-on: https://boringssl-review.googlesource.com/13232
Reviewed-by: Adam Langley <agl@google.com>
Passing --quiet makes valgrind only print out errors, so we don't need
to suppress things. Combine that with checking valgrind's dedicated exit
code so we notice errors that happen before the "---DONE---" marker.
This makes that marker unnecessary for valgrind. all_tests.go was not
sensitive to this, but still would do well to have valgrind be silent.
Change-Id: I841edf7de87081137e38990e647e989fd7567295
Reviewed-on: https://boringssl-review.googlesource.com/11128
Reviewed-by: Adam Langley <agl@google.com>
Uses LCOV for C(++) line coverage and Valgrind's Callgrind tool to
generate assembly-level line coverage for the generated assembly
code.
BUG=590332
Change-Id: Ic70300a272c38f4fa6dd615747db568aa0853584
Reviewed-on: https://boringssl-review.googlesource.com/7251
Reviewed-by: David Benjamin <davidben@google.com>
A len(tests) should have been len(testCases), the code never added to the
sync.WaitGroup, and feeding tests to the tests channel blocks on the tests
completing, so with one worker the results didn't stream. (And if the results
channel wasn't large enough, we'd deadlock.)
Change-Id: Iee37507b9706b14cffddd9c1b55fc311ee9b666d
Reviewed-on: https://boringssl-review.googlesource.com/7292
Reviewed-by: Adam Langley <agl@google.com>
Use -num-workers to run multiple workers in parallel when running tests.
Change-Id: Iee5554ee78ec8d77700a0df5a297bd2515d34dca
Reviewed-on: https://boringssl-review.googlesource.com/7285
Reviewed-by: David Benjamin <davidben@google.com>
Currently far from passing and I haven't even tried with a leak checker yet.
Also bn_test is slow.
Change-Id: I4fe2783aa5f7897839ca846062ae7e4a367d2469
Reviewed-on: https://boringssl-review.googlesource.com/4794
Reviewed-by: Adam Langley <agl@google.com>
OpenSSL has traditionally done reference counting with |int|s and the
|CRYPTO_add| function. Unless a special callback is installed (rare),
this is implemented by doing the reference count operations under a
lock.
This change adds infrastructure for handling reference counts and uses
atomic operations when C11 support is available.
Change-Id: Ia023ce432319efd00f77a7340da27d16ee4b63c3
Reviewed-on: https://boringssl-review.googlesource.com/4771
Reviewed-by: Adam Langley <agl@google.com>
This imports the EVP_PKEY test data of upstream's evptests.txt, but
modified to fit our test framework and with a new test driver. The
remainder of the test data will be imported separately into aead_test
and cipher_test.
Some minor changes to the test format were made to account for test
framework differences. One test has different results since we don't
support RSA signatures with omitted (rather than NULL) parameters.
Otherwise, the biggest difference in test format is that the ad-hoc
result strings are replaced with checking ERR_peek_error.
Change-Id: I758869abbeb843f5f2ac6c1cbd87333baec08ec3
Reviewed-on: https://boringssl-review.googlesource.com/4703
Reviewed-by: Adam Langley <agl@google.com>
This matches how upstream imported that test. evp_test will be used for
the subset of upstream's evp_test which land in our crypto/evp layer.
(Some of crypto/evp is in crypto/cipher for us, so those tests will be
in a ported cipher_test.)
Change-Id: Ic899442794b66350e73a706bb7c77a6ff3d2564b
Reviewed-on: https://boringssl-review.googlesource.com/4702
Reviewed-by: Adam Langley <agl@google.com>
This adds a file-based test framework to crypto/test. It knows how to
parse formats similar to either upstream's evp_test and our aead_test.
hmac_test has been converted to that with tests from upstream's
evp_test. Upstream tests it against the deprecated EVP_PKEY_HMAC API,
which will be tested by running evp_test against the same input file, to
avoid having to duplicate the test vectors. hmac_test runs those same
inputs against the supported HMAC_CTX APIs.
Change-Id: I9d2b6adb9be519760d1db282b9d43efd6f9adffb
Reviewed-on: https://boringssl-review.googlesource.com/4701
Reviewed-by: Adam Langley <agl@google.com>
The interface for this is very similar to upstream, but the code is
quite different.
Support for “resuming” (i.e. calling |CMAC_Final| and then computing the
CMAC for an extension of the message) has been dropped. Also, calling
|CMAC_Init| with magic argument to reset it has been replaced with
|CMAC_Reset|.
Lastly, a one-shot function has been added because it can save an
allocation and that's what most callers actually appear to want to do.
Change-Id: I9345220218bdb16ebe6ca356928d7c6f055d83f6
Reviewed-on: https://boringssl-review.googlesource.com/4630
Reviewed-by: David Benjamin <davidben@chromium.org>
Reviewed-by: Adam Langley <agl@google.com>
If the key is missing, it seems the failure is assumed to be expected.
BUG=473924
Change-Id: I62edd9110fa74bee5e6425fd6786badf5398728c
Reviewed-on: https://boringssl-review.googlesource.com/4231
Reviewed-by: Adam Langley <agl@google.com>
Historically, OpenSSL has used callbacks for anything thread related,
but we don't actually have that many threading libraries to worry about:
just pthreads and Windows (I hope).
That suggests that it's quite reasonable to handle threading ourselves,
and eliminate the need for users to remember to install the thread
callbacks.
The first user of this would be ERR, which currently simulates
thread-local storage using a lock around a hash table keyed by the TID.
(Although I suspect that change will need some CMake work in order that
libpthread is automatically included with libcrypto when linking tests
etc, but not on Windows and without lots of ifs.)
Change-Id: I4dd088e3794506747f875c1f3e92b9bc6700fad2
Reviewed-on: https://boringssl-review.googlesource.com/4010
Reviewed-by: Adam Langley <agl@google.com>
Also adds a flag to runner.go to make it more suitable for printing to a pipe.
Change-Id: I26fae21f3e4910028f6b8bfc4821c8c595525504
Reviewed-on: https://boringssl-review.googlesource.com/3490
Reviewed-by: Adam Langley <agl@google.com>
This supports -valgrind as with runner.go. It also works on Windows and
provides a place for implementing Chrome infra's JSON test output format in the
future, as well as whatever magic may be needed for Android.
Change-Id: I26eb68053f95e825561a142dbcdc4fbd84e3687d
Reviewed-on: https://boringssl-review.googlesource.com/3411
Reviewed-by: Adam Langley <agl@google.com>
