boringssl/crypto/test/gtest_main.cc

96 lines
2.8 KiB
C++
Raw Normal View History

Do a cursory conversion of a few tests to GTest. For now, this is the laziest conversion possible. The intent is to just get the build setup ready so that we can get everything working in our consumers. The intended end state is: - The standalone build produces three test targets, one per library: {crypto,ssl,decrepit}_tests. - Each FOO_test is made up of: FOO/**/*_test.cc crypto/test/gtest_main.cc test_support - generate_build_files.py emits variables crypto_test_sources and ssl_test_sources. These variables are populated with FindCFiles, looking for *_test.cc. - The consuming file assembles those variables into the two test targets (plus decrepit) from there. This avoids having generate_build_files.py emit actual build rules. - Our standalone builders, Chromium, and Android just run the top-level test targets using whatever GTest-based reporting story they have. In transition, we start by converting one of two tests in each library to populate the three test targets. Those are added to all_tests.json and all_tests.go hacked to handle them transparently. This keeps our standalone builder working. generate_build_files.py, to start with, populates the new source lists manually and subtracts them out of the old machinery. We emit both for the time being. When this change rolls in, we'll write all the build glue needed to build the GTest-based tests and add it to consumers' continuous builders. Next, we'll subsume a file-based test and get the consumers working with that. (I.e. make sure the GTest targets can depend on a data file.) Once that's all done, we'll be sure all this will work. At that point, we start subsuming the remaining tests into the GTest targets and, asynchronously, rewriting tests to use GTest properly rather than cursory conversion here. When all non-GTest tests are gone, the old generate_build_files.py hooks will be removed, consumers updated to not depend on them, and standalone builders converted to not rely on all_tests.go, which can then be removed. (Unless bits end up being needed as a malloc test driver. I'm thinking we'll want to do something with --gtest_filter.) As part of this CL, I've bumped the CMake requirements (for target_include_directories) and added a few suppressions for warnings that GTest doesn't pass. BUG=129 Change-Id: I881b26b07a8739cc0b52dbb51a30956908e1b71a Reviewed-on: https://boringssl-review.googlesource.com/13232 Reviewed-by: Adam Langley <agl@google.com>
2017-01-20 00:05:47 +00:00
/* Copyright (c) 2016, Google Inc.
*
* Permission to use, copy, modify, and/or distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
* SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION
* OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
* CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */
#include <stdio.h>
#include <string.h>
Do a cursory conversion of a few tests to GTest. For now, this is the laziest conversion possible. The intent is to just get the build setup ready so that we can get everything working in our consumers. The intended end state is: - The standalone build produces three test targets, one per library: {crypto,ssl,decrepit}_tests. - Each FOO_test is made up of: FOO/**/*_test.cc crypto/test/gtest_main.cc test_support - generate_build_files.py emits variables crypto_test_sources and ssl_test_sources. These variables are populated with FindCFiles, looking for *_test.cc. - The consuming file assembles those variables into the two test targets (plus decrepit) from there. This avoids having generate_build_files.py emit actual build rules. - Our standalone builders, Chromium, and Android just run the top-level test targets using whatever GTest-based reporting story they have. In transition, we start by converting one of two tests in each library to populate the three test targets. Those are added to all_tests.json and all_tests.go hacked to handle them transparently. This keeps our standalone builder working. generate_build_files.py, to start with, populates the new source lists manually and subtracts them out of the old machinery. We emit both for the time being. When this change rolls in, we'll write all the build glue needed to build the GTest-based tests and add it to consumers' continuous builders. Next, we'll subsume a file-based test and get the consumers working with that. (I.e. make sure the GTest targets can depend on a data file.) Once that's all done, we'll be sure all this will work. At that point, we start subsuming the remaining tests into the GTest targets and, asynchronously, rewriting tests to use GTest properly rather than cursory conversion here. When all non-GTest tests are gone, the old generate_build_files.py hooks will be removed, consumers updated to not depend on them, and standalone builders converted to not rely on all_tests.go, which can then be removed. (Unless bits end up being needed as a malloc test driver. I'm thinking we'll want to do something with --gtest_filter.) As part of this CL, I've bumped the CMake requirements (for target_include_directories) and added a few suppressions for warnings that GTest doesn't pass. BUG=129 Change-Id: I881b26b07a8739cc0b52dbb51a30956908e1b71a Reviewed-on: https://boringssl-review.googlesource.com/13232 Reviewed-by: Adam Langley <agl@google.com>
2017-01-20 00:05:47 +00:00
#include <gtest/gtest.h>
#include <openssl/cpu.h>
#include <openssl/rand.h>
Add an ABI testing framework. Dear reader, I must apologize in advance. This CL contains the following: - A new 256-line perlasm file with non-trivial perl bits and a dual-ABI variadic function caller. - C preprocessor gymnastics, with variadic macros and fun facts about __VA_ARGS__'s behavior on empty argument lists. - C++ template gymnastics, including variadic arguments, template specialization, std::enable_if, and machinery to control template argument deduction. Enjoy. This tests that our assembly functions correctly honor platform ABI conventions. Right now this only tests callee-saved registers, but it should be extendable to SEH/CFI unwind testing with single-step debugging APIs. Register-checking does not involve anything funny and should be compatible with SDE. (The future unwind testing is unlikely to be compatible.) This CL adds support for x86_64 SysV and Win64 ABIs. ARM, AArch64, and x86 can be added in the future. The testing is injected in two places. First, all the assembly tests in p256-x86_64-test.cc are now instrumented. This is the intended workflow and should capture all registers. However, we currently do not unit-test our assembly much directly. We should do that as follow-up work[0] but, in the meantime, I've also wrapped all of the GTest main function in an ABI test. This is imperfect as ABI failures may be masked by other stack frames, but it costs nothing[1] and is pretty reliable at catching Win64 xmm register failures. [0] An alternate strategy would be, in debug builds, unconditionally instrument every assembly call in libcrypto. But the CHECK_ABI macro would be difficult to replicate in pure C, and unwind testing may be too invasive for this. Still, something to consider when we C++ libcrypto. [1] When single-stepped unwind testing exists, it won't cost nothing. The gtest_main.cc call will turn unwind testing off. Change-Id: I6643b26445891fd46abfacac52bc024024c8d7f6 Reviewed-on: https://boringssl-review.googlesource.com/c/33764 Reviewed-by: Adam Langley <agl@google.com> Reviewed-by: Adam Langley <alangley@gmail.com> Commit-Queue: David Benjamin <davidben@google.com>
2018-12-16 00:58:43 +00:00
#include "abi_test.h"
#include "gtest_main.h"
#include "../internal.h"
#if (defined(OPENSSL_ARM) || defined(OPENSSL_AARCH64)) && \
!defined(OPENSSL_STATIC_ARMCAP)
#include <openssl/arm_arch.h>
#define TEST_ARM_CPUS
#endif
Do a cursory conversion of a few tests to GTest. For now, this is the laziest conversion possible. The intent is to just get the build setup ready so that we can get everything working in our consumers. The intended end state is: - The standalone build produces three test targets, one per library: {crypto,ssl,decrepit}_tests. - Each FOO_test is made up of: FOO/**/*_test.cc crypto/test/gtest_main.cc test_support - generate_build_files.py emits variables crypto_test_sources and ssl_test_sources. These variables are populated with FindCFiles, looking for *_test.cc. - The consuming file assembles those variables into the two test targets (plus decrepit) from there. This avoids having generate_build_files.py emit actual build rules. - Our standalone builders, Chromium, and Android just run the top-level test targets using whatever GTest-based reporting story they have. In transition, we start by converting one of two tests in each library to populate the three test targets. Those are added to all_tests.json and all_tests.go hacked to handle them transparently. This keeps our standalone builder working. generate_build_files.py, to start with, populates the new source lists manually and subtracts them out of the old machinery. We emit both for the time being. When this change rolls in, we'll write all the build glue needed to build the GTest-based tests and add it to consumers' continuous builders. Next, we'll subsume a file-based test and get the consumers working with that. (I.e. make sure the GTest targets can depend on a data file.) Once that's all done, we'll be sure all this will work. At that point, we start subsuming the remaining tests into the GTest targets and, asynchronously, rewriting tests to use GTest properly rather than cursory conversion here. When all non-GTest tests are gone, the old generate_build_files.py hooks will be removed, consumers updated to not depend on them, and standalone builders converted to not rely on all_tests.go, which can then be removed. (Unless bits end up being needed as a malloc test driver. I'm thinking we'll want to do something with --gtest_filter.) As part of this CL, I've bumped the CMake requirements (for target_include_directories) and added a few suppressions for warnings that GTest doesn't pass. BUG=129 Change-Id: I881b26b07a8739cc0b52dbb51a30956908e1b71a Reviewed-on: https://boringssl-review.googlesource.com/13232 Reviewed-by: Adam Langley <agl@google.com>
2017-01-20 00:05:47 +00:00
int main(int argc, char **argv) {
testing::InitGoogleTest(&argc, argv);
bssl::SetupGoogleTest();
Add a CFI tester to CHECK_ABI. This uses the x86 trap flag and libunwind to test CFI works at each instruction. For now, it just uses the system one out of pkg-config and disables unwind tests if unavailable. We'll probably want to stick a copy into //third_party and perhaps try the LLVM one later. This tester caught two bugs in P-256 CFI annotations already: I47b5f9798b3bcee1748e537b21c173d312a14b42 and I9f576d868850312d6c14d1386f8fbfa85021b347 An earlier design used PTRACE_SINGLESTEP with libunwind's remote unwinding features. ptrace is a mess around stop signals (see group-stop discussion in ptrace(2)) and this is 10x faster, so I went with it. The question of which is more future-proof is complex: - There are two libunwinds with the same API, https://www.nongnu.org/libunwind/ and LLVM's. This currently uses the system nongnu.org for convenience. In future, LLVM's should be easier to bundle (less complex build) and appears to even support Windows, but I haven't tested this. Moreover, setting the trap flag keeps the test single-process, which is less complex on Windows. That suggests the trap flag design and switching to LLVM later. However... - Not all architectures have a trap flag settable by userspace. As far as I can tell, ARMv8's PSTATE.SS can only be set from the kernel. If we stick with nongnu.org libunwind, we can use PTRACE_SINGLESTEP and remote unwinding. Or we implement it for LLVM. Another thought is for the ptracer to bounce SIGTRAP back into the process, to share the local unwinding code. - ARMv7 has no trap flag at all and PTRACE_SINGLESTEP fails. Debuggers single-step by injecting breakpoints instead. However, ARMv8's trap flag seems to work in both AArch32 and AArch64 modes, so we may be able to condition it on a 64-bit kernel. Sadly, neither strategy works with Intel SDE. Adding flags to cpucap vectors as we do with ARM would help, but it would not emulate CPUs newer than the host CPU. For now, I've just had SDE tests disable these. Annoyingly, CMake does not allow object libraries to have dependencies, so make test_support a proper static library. Rename the target to test_support_lib to avoid https://gitlab.kitware.com/cmake/cmake/issues/17785 Update-Note: This adds a new optional test dependency, but it's disabled by default (define BORINGSSL_HAVE_LIBUNWIND), so consumers do not need to do anything. We'll probably want to adjust this in the future. Bug: 181 Change-Id: I817263d7907aff0904a9cee83f8b26747262cc0c Reviewed-on: https://boringssl-review.googlesource.com/c/33966 Commit-Queue: David Benjamin <davidben@google.com> Reviewed-by: Adam Langley <agl@google.com>
2018-12-21 23:58:36 +00:00
bool unwind_tests = true;
for (int i = 1; i < argc; i++) {
Add a CFI tester to CHECK_ABI. This uses the x86 trap flag and libunwind to test CFI works at each instruction. For now, it just uses the system one out of pkg-config and disables unwind tests if unavailable. We'll probably want to stick a copy into //third_party and perhaps try the LLVM one later. This tester caught two bugs in P-256 CFI annotations already: I47b5f9798b3bcee1748e537b21c173d312a14b42 and I9f576d868850312d6c14d1386f8fbfa85021b347 An earlier design used PTRACE_SINGLESTEP with libunwind's remote unwinding features. ptrace is a mess around stop signals (see group-stop discussion in ptrace(2)) and this is 10x faster, so I went with it. The question of which is more future-proof is complex: - There are two libunwinds with the same API, https://www.nongnu.org/libunwind/ and LLVM's. This currently uses the system nongnu.org for convenience. In future, LLVM's should be easier to bundle (less complex build) and appears to even support Windows, but I haven't tested this. Moreover, setting the trap flag keeps the test single-process, which is less complex on Windows. That suggests the trap flag design and switching to LLVM later. However... - Not all architectures have a trap flag settable by userspace. As far as I can tell, ARMv8's PSTATE.SS can only be set from the kernel. If we stick with nongnu.org libunwind, we can use PTRACE_SINGLESTEP and remote unwinding. Or we implement it for LLVM. Another thought is for the ptracer to bounce SIGTRAP back into the process, to share the local unwinding code. - ARMv7 has no trap flag at all and PTRACE_SINGLESTEP fails. Debuggers single-step by injecting breakpoints instead. However, ARMv8's trap flag seems to work in both AArch32 and AArch64 modes, so we may be able to condition it on a 64-bit kernel. Sadly, neither strategy works with Intel SDE. Adding flags to cpucap vectors as we do with ARM would help, but it would not emulate CPUs newer than the host CPU. For now, I've just had SDE tests disable these. Annoyingly, CMake does not allow object libraries to have dependencies, so make test_support a proper static library. Rename the target to test_support_lib to avoid https://gitlab.kitware.com/cmake/cmake/issues/17785 Update-Note: This adds a new optional test dependency, but it's disabled by default (define BORINGSSL_HAVE_LIBUNWIND), so consumers do not need to do anything. We'll probably want to adjust this in the future. Bug: 181 Change-Id: I817263d7907aff0904a9cee83f8b26747262cc0c Reviewed-on: https://boringssl-review.googlesource.com/c/33966 Commit-Queue: David Benjamin <davidben@google.com> Reviewed-by: Adam Langley <agl@google.com>
2018-12-21 23:58:36 +00:00
#if !defined(OPENSSL_WINDOWS)
if (strcmp(argv[i], "--fork_unsafe_buffering") == 0) {
RAND_enable_fork_unsafe_buffering(-1);
}
#endif
#if defined(TEST_ARM_CPUS)
if (strncmp(argv[i], "--cpu=", 6) == 0) {
const char *cpu = argv[i] + 6;
uint32_t armcap;
if (strcmp(cpu, "none") == 0) {
armcap = 0;
} else if (strcmp(cpu, "neon") == 0) {
armcap = ARMV7_NEON;
} else if (strcmp(cpu, "crypto") == 0) {
armcap = ARMV7_NEON | ARMV8_AES | ARMV8_SHA1 | ARMV8_SHA256 | ARMV8_PMULL;
} else {
fprintf(stderr, "Unknown CPU: %s\n", cpu);
exit(1);
}
uint32_t *armcap_ptr = OPENSSL_get_armcap_pointer_for_test();
if ((armcap & *armcap_ptr) != armcap) {
fprintf(stderr,
"Host CPU does not support features for testing CPU '%s'.\n",
cpu);
exit(89);
}
printf("Simulating CPU '%s'\n", cpu);
*armcap_ptr = armcap;
}
#endif // TEST_ARM_CPUS
Add a CFI tester to CHECK_ABI. This uses the x86 trap flag and libunwind to test CFI works at each instruction. For now, it just uses the system one out of pkg-config and disables unwind tests if unavailable. We'll probably want to stick a copy into //third_party and perhaps try the LLVM one later. This tester caught two bugs in P-256 CFI annotations already: I47b5f9798b3bcee1748e537b21c173d312a14b42 and I9f576d868850312d6c14d1386f8fbfa85021b347 An earlier design used PTRACE_SINGLESTEP with libunwind's remote unwinding features. ptrace is a mess around stop signals (see group-stop discussion in ptrace(2)) and this is 10x faster, so I went with it. The question of which is more future-proof is complex: - There are two libunwinds with the same API, https://www.nongnu.org/libunwind/ and LLVM's. This currently uses the system nongnu.org for convenience. In future, LLVM's should be easier to bundle (less complex build) and appears to even support Windows, but I haven't tested this. Moreover, setting the trap flag keeps the test single-process, which is less complex on Windows. That suggests the trap flag design and switching to LLVM later. However... - Not all architectures have a trap flag settable by userspace. As far as I can tell, ARMv8's PSTATE.SS can only be set from the kernel. If we stick with nongnu.org libunwind, we can use PTRACE_SINGLESTEP and remote unwinding. Or we implement it for LLVM. Another thought is for the ptracer to bounce SIGTRAP back into the process, to share the local unwinding code. - ARMv7 has no trap flag at all and PTRACE_SINGLESTEP fails. Debuggers single-step by injecting breakpoints instead. However, ARMv8's trap flag seems to work in both AArch32 and AArch64 modes, so we may be able to condition it on a 64-bit kernel. Sadly, neither strategy works with Intel SDE. Adding flags to cpucap vectors as we do with ARM would help, but it would not emulate CPUs newer than the host CPU. For now, I've just had SDE tests disable these. Annoyingly, CMake does not allow object libraries to have dependencies, so make test_support a proper static library. Rename the target to test_support_lib to avoid https://gitlab.kitware.com/cmake/cmake/issues/17785 Update-Note: This adds a new optional test dependency, but it's disabled by default (define BORINGSSL_HAVE_LIBUNWIND), so consumers do not need to do anything. We'll probably want to adjust this in the future. Bug: 181 Change-Id: I817263d7907aff0904a9cee83f8b26747262cc0c Reviewed-on: https://boringssl-review.googlesource.com/c/33966 Commit-Queue: David Benjamin <davidben@google.com> Reviewed-by: Adam Langley <agl@google.com>
2018-12-21 23:58:36 +00:00
if (strcmp(argv[i], "--no_unwind_tests") == 0) {
unwind_tests = false;
}
}
if (unwind_tests) {
abi_test::EnableUnwindTests();
}
Add an ABI testing framework. Dear reader, I must apologize in advance. This CL contains the following: - A new 256-line perlasm file with non-trivial perl bits and a dual-ABI variadic function caller. - C preprocessor gymnastics, with variadic macros and fun facts about __VA_ARGS__'s behavior on empty argument lists. - C++ template gymnastics, including variadic arguments, template specialization, std::enable_if, and machinery to control template argument deduction. Enjoy. This tests that our assembly functions correctly honor platform ABI conventions. Right now this only tests callee-saved registers, but it should be extendable to SEH/CFI unwind testing with single-step debugging APIs. Register-checking does not involve anything funny and should be compatible with SDE. (The future unwind testing is unlikely to be compatible.) This CL adds support for x86_64 SysV and Win64 ABIs. ARM, AArch64, and x86 can be added in the future. The testing is injected in two places. First, all the assembly tests in p256-x86_64-test.cc are now instrumented. This is the intended workflow and should capture all registers. However, we currently do not unit-test our assembly much directly. We should do that as follow-up work[0] but, in the meantime, I've also wrapped all of the GTest main function in an ABI test. This is imperfect as ABI failures may be masked by other stack frames, but it costs nothing[1] and is pretty reliable at catching Win64 xmm register failures. [0] An alternate strategy would be, in debug builds, unconditionally instrument every assembly call in libcrypto. But the CHECK_ABI macro would be difficult to replicate in pure C, and unwind testing may be too invasive for this. Still, something to consider when we C++ libcrypto. [1] When single-stepped unwind testing exists, it won't cost nothing. The gtest_main.cc call will turn unwind testing off. Change-Id: I6643b26445891fd46abfacac52bc024024c8d7f6 Reviewed-on: https://boringssl-review.googlesource.com/c/33764 Reviewed-by: Adam Langley <agl@google.com> Reviewed-by: Adam Langley <alangley@gmail.com> Commit-Queue: David Benjamin <davidben@google.com>
2018-12-16 00:58:43 +00:00
// Run the entire test suite under an ABI check. This is less effective than
// testing the individual assembly functions, but will catch issues with
// rarely-used registers.
abi_test::Result abi;
int ret = abi_test::Check(&abi, RUN_ALL_TESTS);
if (!abi.ok()) {
fprintf(stderr, "ABI failure in test suite:\n");
for (const auto &error : abi.errors) {
fprintf(stderr, " %s\n", error.c_str());
}
exit(1);
}
return ret;
Do a cursory conversion of a few tests to GTest. For now, this is the laziest conversion possible. The intent is to just get the build setup ready so that we can get everything working in our consumers. The intended end state is: - The standalone build produces three test targets, one per library: {crypto,ssl,decrepit}_tests. - Each FOO_test is made up of: FOO/**/*_test.cc crypto/test/gtest_main.cc test_support - generate_build_files.py emits variables crypto_test_sources and ssl_test_sources. These variables are populated with FindCFiles, looking for *_test.cc. - The consuming file assembles those variables into the two test targets (plus decrepit) from there. This avoids having generate_build_files.py emit actual build rules. - Our standalone builders, Chromium, and Android just run the top-level test targets using whatever GTest-based reporting story they have. In transition, we start by converting one of two tests in each library to populate the three test targets. Those are added to all_tests.json and all_tests.go hacked to handle them transparently. This keeps our standalone builder working. generate_build_files.py, to start with, populates the new source lists manually and subtracts them out of the old machinery. We emit both for the time being. When this change rolls in, we'll write all the build glue needed to build the GTest-based tests and add it to consumers' continuous builders. Next, we'll subsume a file-based test and get the consumers working with that. (I.e. make sure the GTest targets can depend on a data file.) Once that's all done, we'll be sure all this will work. At that point, we start subsuming the remaining tests into the GTest targets and, asynchronously, rewriting tests to use GTest properly rather than cursory conversion here. When all non-GTest tests are gone, the old generate_build_files.py hooks will be removed, consumers updated to not depend on them, and standalone builders converted to not rely on all_tests.go, which can then be removed. (Unless bits end up being needed as a malloc test driver. I'm thinking we'll want to do something with --gtest_filter.) As part of this CL, I've bumped the CMake requirements (for target_include_directories) and added a few suppressions for warnings that GTest doesn't pass. BUG=129 Change-Id: I881b26b07a8739cc0b52dbb51a30956908e1b71a Reviewed-on: https://boringssl-review.googlesource.com/13232 Reviewed-by: Adam Langley <agl@google.com>
2017-01-20 00:05:47 +00:00
}