When a test fails, there should be spaces between the argv of the failed
command line.
Change-Id: I5c168a919c1615df34a0eab63a7232453168adb3
Reviewed-on: https://boringssl-review.googlesource.com/15846
Reviewed-by: David Benjamin <davidben@google.com>
This change makes util/all_tests.go run as many test binaries
concurrently as there are cores on the current system. This can be
overridden with -num-workers=1.
Change-Id: Ia3a5e336d208039be9276261a0ac03f7fb774677
Reviewed-on: https://boringssl-review.googlesource.com/14927
Reviewed-by: Adam Langley <agl@google.com>
GTest sends its output to stdout, not stderr. Merge them in the runner
(though eventually we'll teach the bots to run the GTest targets
directly) so we don't lose it.
BUG=129
Change-Id: I7c499cd9572f46f97bd4b7f6c6c9beca057625f2
Reviewed-on: https://boringssl-review.googlesource.com/13624
Reviewed-by: Steven Valdez <svaldez@google.com>
Reviewed-by: David Benjamin <davidben@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
Intel SDE is a tool that can simulate many different Intel chips. This
lets us test whether our CPUID-guarding is correct and would have
caught, for example, this morning's ChaCha20-Poly1305 problem.
Change-Id: I39de2bedb1c29b48b02ba30c51fdce57a5cbe640
Reviewed-on: https://boringssl-review.googlesource.com/13587
Commit-Queue: Adam Langley <alangley@gmail.com>
Commit-Queue: David Benjamin <davidben@google.com>
Reviewed-by: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
For now, this is the laziest conversion possible. The intent is to just
get the build setup ready so that we can get everything working in our
consumers. The intended end state is:
- The standalone build produces three test targets, one per library:
{crypto,ssl,decrepit}_tests.
- Each FOO_test is made up of:
FOO/**/*_test.cc
crypto/test/gtest_main.cc
test_support
- generate_build_files.py emits variables crypto_test_sources and
ssl_test_sources. These variables are populated with FindCFiles,
looking for *_test.cc.
- The consuming file assembles those variables into the two test targets
(plus decrepit) from there. This avoids having generate_build_files.py
emit actual build rules.
- Our standalone builders, Chromium, and Android just run the top-level
test targets using whatever GTest-based reporting story they have.
In transition, we start by converting one of two tests in each library
to populate the three test targets. Those are added to all_tests.json
and all_tests.go hacked to handle them transparently. This keeps our
standalone builder working.
generate_build_files.py, to start with, populates the new source lists
manually and subtracts them out of the old machinery. We emit both for
the time being. When this change rolls in, we'll write all the build
glue needed to build the GTest-based tests and add it to consumers'
continuous builders.
Next, we'll subsume a file-based test and get the consumers working with
that. (I.e. make sure the GTest targets can depend on a data file.)
Once that's all done, we'll be sure all this will work. At that point,
we start subsuming the remaining tests into the GTest targets and,
asynchronously, rewriting tests to use GTest properly rather than
cursory conversion here.
When all non-GTest tests are gone, the old generate_build_files.py hooks
will be removed, consumers updated to not depend on them, and standalone
builders converted to not rely on all_tests.go, which can then be
removed. (Unless bits end up being needed as a malloc test driver. I'm
thinking we'll want to do something with --gtest_filter.)
As part of this CL, I've bumped the CMake requirements (for
target_include_directories) and added a few suppressions for warnings
that GTest doesn't pass.
BUG=129
Change-Id: I881b26b07a8739cc0b52dbb51a30956908e1b71a
Reviewed-on: https://boringssl-review.googlesource.com/13232
Reviewed-by: Adam Langley <agl@google.com>
Passing --quiet makes valgrind only print out errors, so we don't need
to suppress things. Combine that with checking valgrind's dedicated exit
code so we notice errors that happen before the "---DONE---" marker.
This makes that marker unnecessary for valgrind. all_tests.go was not
sensitive to this, but still would do well to have valgrind be silent.
Change-Id: I841edf7de87081137e38990e647e989fd7567295
Reviewed-on: https://boringssl-review.googlesource.com/11128
Reviewed-by: Adam Langley <agl@google.com>
Uses LCOV for C(++) line coverage and Valgrind's Callgrind tool to
generate assembly-level line coverage for the generated assembly
code.
BUG=590332
Change-Id: Ic70300a272c38f4fa6dd615747db568aa0853584
Reviewed-on: https://boringssl-review.googlesource.com/7251
Reviewed-by: David Benjamin <davidben@google.com>
A len(tests) should have been len(testCases), the code never added to the
sync.WaitGroup, and feeding tests to the tests channel blocks on the tests
completing, so with one worker the results didn't stream. (And if the results
channel wasn't large enough, we'd deadlock.)
Change-Id: Iee37507b9706b14cffddd9c1b55fc311ee9b666d
Reviewed-on: https://boringssl-review.googlesource.com/7292
Reviewed-by: Adam Langley <agl@google.com>
Use -num-workers to run multiple workers in parallel when running tests.
Change-Id: Iee5554ee78ec8d77700a0df5a297bd2515d34dca
Reviewed-on: https://boringssl-review.googlesource.com/7285
Reviewed-by: David Benjamin <davidben@google.com>
Currently far from passing and I haven't even tried with a leak checker yet.
Also bn_test is slow.
Change-Id: I4fe2783aa5f7897839ca846062ae7e4a367d2469
Reviewed-on: https://boringssl-review.googlesource.com/4794
Reviewed-by: Adam Langley <agl@google.com>
OpenSSL has traditionally done reference counting with |int|s and the
|CRYPTO_add| function. Unless a special callback is installed (rare),
this is implemented by doing the reference count operations under a
lock.
This change adds infrastructure for handling reference counts and uses
atomic operations when C11 support is available.
Change-Id: Ia023ce432319efd00f77a7340da27d16ee4b63c3
Reviewed-on: https://boringssl-review.googlesource.com/4771
Reviewed-by: Adam Langley <agl@google.com>
This imports the EVP_PKEY test data of upstream's evptests.txt, but
modified to fit our test framework and with a new test driver. The
remainder of the test data will be imported separately into aead_test
and cipher_test.
Some minor changes to the test format were made to account for test
framework differences. One test has different results since we don't
support RSA signatures with omitted (rather than NULL) parameters.
Otherwise, the biggest difference in test format is that the ad-hoc
result strings are replaced with checking ERR_peek_error.
Change-Id: I758869abbeb843f5f2ac6c1cbd87333baec08ec3
Reviewed-on: https://boringssl-review.googlesource.com/4703
Reviewed-by: Adam Langley <agl@google.com>
This matches how upstream imported that test. evp_test will be used for
the subset of upstream's evp_test which land in our crypto/evp layer.
(Some of crypto/evp is in crypto/cipher for us, so those tests will be
in a ported cipher_test.)
Change-Id: Ic899442794b66350e73a706bb7c77a6ff3d2564b
Reviewed-on: https://boringssl-review.googlesource.com/4702
Reviewed-by: Adam Langley <agl@google.com>
This adds a file-based test framework to crypto/test. It knows how to
parse formats similar to either upstream's evp_test and our aead_test.
hmac_test has been converted to that with tests from upstream's
evp_test. Upstream tests it against the deprecated EVP_PKEY_HMAC API,
which will be tested by running evp_test against the same input file, to
avoid having to duplicate the test vectors. hmac_test runs those same
inputs against the supported HMAC_CTX APIs.
Change-Id: I9d2b6adb9be519760d1db282b9d43efd6f9adffb
Reviewed-on: https://boringssl-review.googlesource.com/4701
Reviewed-by: Adam Langley <agl@google.com>
The interface for this is very similar to upstream, but the code is
quite different.
Support for “resuming” (i.e. calling |CMAC_Final| and then computing the
CMAC for an extension of the message) has been dropped. Also, calling
|CMAC_Init| with magic argument to reset it has been replaced with
|CMAC_Reset|.
Lastly, a one-shot function has been added because it can save an
allocation and that's what most callers actually appear to want to do.
Change-Id: I9345220218bdb16ebe6ca356928d7c6f055d83f6
Reviewed-on: https://boringssl-review.googlesource.com/4630
Reviewed-by: David Benjamin <davidben@chromium.org>
Reviewed-by: Adam Langley <agl@google.com>
If the key is missing, it seems the failure is assumed to be expected.
BUG=473924
Change-Id: I62edd9110fa74bee5e6425fd6786badf5398728c
Reviewed-on: https://boringssl-review.googlesource.com/4231
Reviewed-by: Adam Langley <agl@google.com>
Historically, OpenSSL has used callbacks for anything thread related,
but we don't actually have that many threading libraries to worry about:
just pthreads and Windows (I hope).
That suggests that it's quite reasonable to handle threading ourselves,
and eliminate the need for users to remember to install the thread
callbacks.
The first user of this would be ERR, which currently simulates
thread-local storage using a lock around a hash table keyed by the TID.
(Although I suspect that change will need some CMake work in order that
libpthread is automatically included with libcrypto when linking tests
etc, but not on Windows and without lots of ifs.)
Change-Id: I4dd088e3794506747f875c1f3e92b9bc6700fad2
Reviewed-on: https://boringssl-review.googlesource.com/4010
Reviewed-by: Adam Langley <agl@google.com>
Also adds a flag to runner.go to make it more suitable for printing to a pipe.
Change-Id: I26fae21f3e4910028f6b8bfc4821c8c595525504
Reviewed-on: https://boringssl-review.googlesource.com/3490
Reviewed-by: Adam Langley <agl@google.com>
This supports -valgrind as with runner.go. It also works on Windows and
provides a place for implementing Chrome infra's JSON test output format in the
future, as well as whatever magic may be needed for Android.
Change-Id: I26eb68053f95e825561a142dbcdc4fbd84e3687d
Reviewed-on: https://boringssl-review.googlesource.com/3411
Reviewed-by: Adam Langley <agl@google.com>
