Commit Graph

36 Commits

Author SHA1 Message Date
David Benjamin
6c597be1c6 Update tools.
Unfortunately, this requires partially reverting
https://boringssl-review.googlesource.com/31324. This is a mess.

While clang proper includes a fuzzer driver, Chromium doesn't use it.
Chromium builds exclusively with fuzzer-no-link and links to its own
copy of the fuzzer runtime[1]. As of [2], Chromium's clang (which we use
on bots) no longer includes the driver, so we must mimic them.

However, Chromium's setup is somewhat questionable because
fuzzer-no-link pulls in libclang_rt.fuzzer_no_main which still includes
most of libclang_rt.fuzzer, just not the one main function[3]. It
appears Chromium is actually linking two copies of
libclang_rt.fuzzer_no_main. Hopefully this mostly works out as Chromium's
clang and libFuzzer should be relatively aligned, but it's not a good
assumption for our build, which can take other Clangs too.

Thus, if you pass -DFUZZ=1 as-is, we will assume you are using a
"normal" Clang with all its relevant runtimes intact. If, however, you
are using Chromium clang, you must drop the matching libFuzzer where the
bots expected it and build with -DLIBFUZZER_FROM_DEPS=1.

This involves no changes to the bots because we never actually unwound
all the LIBFUZZER_FROM_DEPS bits before.

[1] https://cs.chromium.org/chromium/src/testing/libfuzzer/BUILD.gn?rcl=d21c49585f262e851e2984f96f52905782706325&l=14
[2] c79bf2ea4c
[3] 8ebc3668b0/lib/fuzzer/CMakeLists.txt (L93-L107)
    8ebc3668b0/lib/fuzzer/FuzzerMain.cpp

Change-Id: I946b3c821c3d7e6def7e07f1381f58241611ba3d
Reviewed-on: https://boringssl-review.googlesource.com/c/34184
Commit-Queue: David Benjamin <davidben@google.com>
Reviewed-by: Adam Langley <agl@google.com>
2019-01-07 22:39:55 +00:00
David Benjamin
5f85f2a061 Add DEPS rules to checkout Windows SDE.
Change-Id: Ia2398fa822fef1ac79f2062a8401bdd3ec963727
Reviewed-on: https://boringssl-review.googlesource.com/c/34104
Reviewed-by: Adam Langley <agl@google.com>
Commit-Queue: Adam Langley <agl@google.com>
2019-01-04 00:39:28 +00:00
David Benjamin
8b60cde1ad Update tools.
In particular, pull Go 1.11 to get module support. Also we don't need to
pull in libfuzzer anymore.

Change-Id: I8098f64cef59422d9d43f7eca5bf3ced69eb70c4
Reviewed-on: https://boringssl-review.googlesource.com/31864
Reviewed-by: Adam Langley <agl@google.com>
2018-09-13 17:57:30 +00:00
David Benjamin
e9ae99bba2 Add an option to statically link a custom libc++.
MSan and TSan both require instrumenting everything. Add some machinery so we
can do this on the bots.

Change-Id: I7d2106bc852ee976455d18787d3a20a35373a9e7
Reviewed-on: https://boringssl-review.googlesource.com/30644
Reviewed-by: Adam Langley <agl@google.com>
2018-08-09 21:37:04 +00:00
David Benjamin
1beddac9ce Update tools.
Change-Id: Idb6818ad9b6af2b1cb604e71d936aaca7210aa57
Reviewed-on: https://boringssl-review.googlesource.com/30624
Commit-Queue: Steven Valdez <svaldez@google.com>
Reviewed-by: Steven Valdez <svaldez@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2018-08-09 19:31:22 +00:00
David Benjamin
eaf0a17db8 Add a copy of NASM to util/bot/ in BoringSSL.
This is to transition BoringSSL's Windows build from Yasm to NASM. This
change itself is a no-op for now, but a later change to the BoringSSL
recipes will add a pair of standalone builders here. Then I'll get the
change I have lying around for Chromium moving.

Bug: chromium:766721
Change-Id: I4dca1c299f93bc5c01695983fe0478490c472deb
Reviewed-on: https://boringssl-review.googlesource.com/29324
Reviewed-by: Steven Valdez <svaldez@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2018-06-26 15:47:29 +00:00
David Benjamin
23aa4d228a Update tools.
Change-Id: I4cc9b9fc13a20080d78f9b9bca1e24571f46f117
Reviewed-on: https://boringssl-review.googlesource.com/29146
Commit-Queue: Steven Valdez <svaldez@google.com>
Reviewed-by: Steven Valdez <svaldez@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2018-06-14 13:58:05 +00:00
David Benjamin
81d4a03bb0 Update tools.
Change-Id: I290af4599231040887f41c922550bb87dfb98cb0
Reviewed-on: https://boringssl-review.googlesource.com/28746
Commit-Queue: Steven Valdez <svaldez@google.com>
Reviewed-by: Steven Valdez <svaldez@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2018-05-29 14:24:35 +00:00
David Benjamin
365e48c104 Update tools.
Change-Id: Ibbba2e2fa81b5f8b3a25ebadc50297f50dcb610e
Reviewed-on: https://boringssl-review.googlesource.com/26884
Commit-Queue: Steven Valdez <svaldez@google.com>
Reviewed-by: Steven Valdez <svaldez@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2018-03-30 17:51:56 +00:00
David Benjamin
ec55dc15d3 Update tools.
Also remove the -Wtautological-constant-compare logic. I believe Clang
has since removed that problematic warning from -Wall and that check was
causing problems when we were embedded as a subproject in a project that
didn't set CMP0025.

(In that case, by the time our build file ran, the compiler had already
been detected and the damage done. This unfortunately means the next
Clang version check will hit the same issue, but let's deal with that
when we get there.)

Change-Id: Iea5f262899b74c5b84f707f4cf4ac4b3540c4acb
Reviewed-on: https://boringssl-review.googlesource.com/26375
Reviewed-by: Steven Valdez <svaldez@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2018-03-05 20:49:21 +00:00
Shenghua Zhang
a4f78775b9 [ndk] Change ndk deps in src and relocate to third_party/boringssl
Because the android sdk managed by CIPD (see go/use-adjective-apis),
its deps repo ndk needs to be relocated, as well as hooked by the root
DEPS instead of recursedeps. This CL addes android ndk deps in
util/bot/DEPS.

Bug: chromium:659808
Change-Id: I270053ae56ba1caaf9c67f2240855eafd499b782
Reviewed-on: https://boringssl-review.googlesource.com/24864
Reviewed-by: David Benjamin <davidben@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2018-01-22 21:08:28 +00:00
David Benjamin
8d67f6f0cf Update tools.
This is a reland 9d1f96606c, which should
hopefuly be fine after afd1cd959e. Though
I've also gone ahead and gotten the latest versions of things.
(android_tools and clang updated.)

In particular, get the new NDK. Unfortunately, the new clang picks up
an unfortunate change for clang-cl that we now must work around.

http://llvm.org/viewvc/llvm-project?view=revision&revision=319116

Bug: 109
Change-Id: If19b09c585957fefaffa8c3197a50189402a555a
Reviewed-on: https://boringssl-review.googlesource.com/25025
Reviewed-by: Steven Valdez <svaldez@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2018-01-22 18:30:18 +00:00
David Benjamin
3c92e80d7a Revert "Update tools."
This reverts commit 9d1f96606c.

Reason for revert: aarch64 bots are breaking for some reason.

Original change's description:
> Update tools.
> 
> In particular, get the new NDK. Unfortunately, the new clang picks up
> an unfortunate change for clang-cl that we now must work around.
> 
> http://llvm.org/viewvc/llvm-project?view=revision&revision=319116
> 
> Bug: 109
> Change-Id: I091ca7160683e70cd79b5c2b7a4267fea258ec17
> Reviewed-on: https://boringssl-review.googlesource.com/24644
> Reviewed-by: Steven Valdez <svaldez@google.com>
> Commit-Queue: David Benjamin <davidben@google.com>
> CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>

TBR=davidben@google.com,svaldez@google.com

Change-Id: I98960f295987857c4e42c312059b6d5934bb5e43
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 109
Reviewed-on: https://boringssl-review.googlesource.com/24747
Reviewed-by: David Benjamin <davidben@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2018-01-09 21:45:11 +00:00
David Benjamin
9d1f96606c Update tools.
In particular, get the new NDK. Unfortunately, the new clang picks up
an unfortunate change for clang-cl that we now must work around.

http://llvm.org/viewvc/llvm-project?view=revision&revision=319116

Bug: 109
Change-Id: I091ca7160683e70cd79b5c2b7a4267fea258ec17
Reviewed-on: https://boringssl-review.googlesource.com/24644
Reviewed-by: Steven Valdez <svaldez@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2018-01-09 21:28:00 +00:00
David Benjamin
56d5d7085d Update tools.
The newer clang should hopefully fix the new linux_clang_cfi bot.

Note the VS2017 revision actually went backwards due to
db45606398

Change-Id: Icaca7a57596f063ccca490917d4b78813f2e9537
Reviewed-on: https://boringssl-review.googlesource.com/23824
Reviewed-by: Adam Langley <agl@google.com>
2017-12-04 21:39:54 +00:00
David Benjamin
f7412cb072 Update tools.
Change-Id: Ibdfdc20b280a594f0f876b33ab8e40686d80f9ba
Reviewed-on: https://boringssl-review.googlesource.com/22504
Commit-Queue: Steven Valdez <svaldez@google.com>
Reviewed-by: Steven Valdez <svaldez@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2017-11-02 20:30:39 +00:00
David Benjamin
168fb2e98c Fix DEPS defaults.
Not to land until these two changes are in:
https://chromium-review.googlesource.com/c/chromium/tools/build/+/716263
https://chromium-review.googlesource.com/c/chromium/tools/build/+/719010

Change-Id: I4e6d6fbcb6068405431dd3f9f38071c8af81d8b7
Reviewed-on: https://boringssl-review.googlesource.com/21825
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
Reviewed-by: Steven Valdez <svaldez@google.com>
2017-10-13 18:22:24 +00:00
David Benjamin
24f5b18f93 Update copies of tools.
d3868ac4d6f92e57376924a62e8d52f36d7a326561ec0bbd1d5681759a947134 sde-external-8.9.0-2017-08-06-lin.tar.bz2
9852d57e0d6b4509accb4f9faf862327a79c18b630aac4f35c65ce7a270a9230 strawberry-perl-5.26.1.1-32bit-portable.zip

I've left CMake alone for now because that involves building things and
there's some mess between newer CMakes trying to mess with the Android
NDK, so that needs to be tested out a bit.

Change-Id: I6beea9d4774474ac62e843c8668fcef7f6a6fc04
Reviewed-on: https://boringssl-review.googlesource.com/21824
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
Reviewed-by: Steven Valdez <svaldez@google.com>
2017-10-12 19:31:25 +00:00
David Benjamin
dbf12fc2ce Use new DEPS conditionals.
See
https://groups.google.com/a/chromium.org/d/msg/infra-announce/A6_zQKzCHDo/ZKqSptzeBgAJ.
This allows us to avoid checking out unnecessary things (right now every
Windows bot downloads clang). We also can maintain the SDE bits in DEPS
rather than having to update the recipe for it.

This is the first half of the change which conditions things on
variables but leaves the defaults as they are. This will be followed up
by a change to the recipe to set the variables, then to switch the
defaults.

Change-Id: Iebcc4d0a146d0b0df94f480e539d70cbf4c862d3
Reviewed-on: https://boringssl-review.googlesource.com/21804
Reviewed-by: Steven Valdez <svaldez@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2017-10-12 18:12:06 +00:00
David Benjamin
a93bc1124c Update other bot tools.
Also document what versions of everything we're using as the .sha1 files
don't say.

Change-Id: I2d496c86761f6df6acd20e1af62094b7d89e5c1d
Reviewed-on: https://boringssl-review.googlesource.com/17485
Reviewed-by: Steven Valdez <svaldez@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
2017-06-29 20:12:53 +00:00
David Benjamin
0a20f7860b Update tools from Chromium.
Change-Id: Ied9713decf536c0fd7b25c9156fd8eb167ee57d1
Reviewed-on: https://boringssl-review.googlesource.com/17244
Reviewed-by: Steven Valdez <svaldez@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2017-06-16 16:54:42 +00:00
David Benjamin
1e5cb820de Add an option to build libFuzzer from DEPS.
The bots will need to get libFuzzer externally. As usual, borrow
Chromium's copy.

Change-Id: I5ed879bbc76188838adcf9d51855e76c481ace5b
Reviewed-on: https://boringssl-review.googlesource.com/16146
Reviewed-by: Steven Valdez <svaldez@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
2017-05-10 22:12:32 +00:00
David Benjamin
ebc4de6719 Update tools from Chromium.
Change-Id: I3bf5fcb210c181c3f7131304e41027ba081e3c0e
Reviewed-on: https://boringssl-review.googlesource.com/16145
Reviewed-by: Steven Valdez <svaldez@google.com>
2017-05-10 21:45:33 +00:00
David Benjamin
919d8cf94e Sync util/bot with Chromium.
Change-Id: Iad15c9b824ca48e6c8a0e5f74fb2d296b3c1170e
Reviewed-on: https://boringssl-review.googlesource.com/14032
Commit-Queue: David Benjamin <davidben@google.com>
Commit-Queue: Adam Langley <agl@google.com>
Reviewed-by: Adam Langley <agl@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2017-03-02 18:45:14 +00:00
David Benjamin
0ec5639092 Don't extract archives if unchanged.
This should shave 20% (40 seconds) off our Windows cycle times, going by
the graphs. It's 15% off our Linux ones, but that 15% is only 11
seconds.

Change-Id: I077c3924c722d597f66fc6dec72932ed0c81660a
Reviewed-on: https://boringssl-review.googlesource.com/12562
Reviewed-by: Adam Langley <agl@google.com>
2016-12-02 18:36:32 +00:00
David Benjamin
49b5038b77 Speculative fix to Android bots.
It's recursedeps, not recurse_deps.

Change-Id: I2c5cb293c5928ef5202ee18db5541712e5b012e6
Reviewed-on: https://boringssl-review.googlesource.com/12235
Reviewed-by: David Benjamin <davidben@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
2016-11-13 05:27:28 +00:00
David Benjamin
076ade5218 Update pinned revisions in util/bot.
Change-Id: I0f085aed8bbb430b8d23ba2ac3f7aaa49816d785
Reviewed-on: https://boringssl-review.googlesource.com/12234
Reviewed-by: Steven Valdez <svaldez@google.com>
Reviewed-by: David Benjamin <davidben@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2016-11-13 05:08:52 +00:00
David Benjamin
bc61769e49 Update pinned revisions in util/bot.
Change-Id: Ieee80e5949e7f5cda77a643bae8fb1c521eb3587
Reviewed-on: https://boringssl-review.googlesource.com/10762
Reviewed-by: Steven Valdez <svaldez@google.com>
Reviewed-by: David Benjamin <davidben@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
2016-09-01 19:23:06 +00:00
David Benjamin
3ccf4d6d65 Pull Chromium's android_tools as an android-only dependency.
This will be used by the bots to get adb and the NDK.

BUG=26

Change-Id: Iae07a380c49b4990f0aa7d73c4f0b399924b9784
Reviewed-on: https://boringssl-review.googlesource.com/7986
Reviewed-by: Adam Langley <agl@google.com>
2016-05-19 16:58:13 +00:00
David Benjamin
9e7efdb008 Update the various pinned revisions in util/bot.
See util/bot/UPDATING for where they come from.

Change-Id: Ib2eae6efc737dd8c4e5fb001fd4b478102e0ad6a
Reviewed-on: https://boringssl-review.googlesource.com/7822
Reviewed-by: Steven Valdez <svaldez@google.com>
Reviewed-by: David Benjamin <davidben@google.com>
2016-05-02 17:54:33 +00:00
David Benjamin
659b24d961 Update versions of tools in util/bot.
Update the easy ones here. Also include instructions on how to do this. The
.sha1 files will be updated separately with instructions.

Change-Id: I2a3aba43b8ffbdf930b8a2602dc1460077f6d0e7
Reviewed-on: https://boringssl-review.googlesource.com/7412
Reviewed-by: Steven Valdez <svaldez@google.com>
Reviewed-by: David Benjamin <davidben@google.com>
2016-03-10 16:24:09 +00:00
David Benjamin
4b0afdd220 Add a tool to download Chromium's prebuilt clang.
See tools/clang/scripts/update.sh. This'll be used to run ASan on the bots.

BUG=469928

Change-Id: I6b5093c2db21ad4ed742852944e77a6b32e29e29
Reviewed-on: https://boringssl-review.googlesource.com/4402
Reviewed-by: Adam Langley <agl@google.com>
2015-04-20 18:56:03 +00:00
David Benjamin
05a5beb35c Add support for finding depot_tools' MSVC toolchain on the bots.
- Pull in a trimmed down version of vs_toolchain.py from Chromium.

- Drop in toolchain_vs2013.hash from Chromium to use Chromium's
  current toolchain.

- Add a very hacky vs_env.py to pull in Visual Studio. This is
  loosely based off a handful of lines of Chromium's
  tools/clang/scripts/update.py. This (and vs_toolchain.py) depends
  on gyp which is now pulled in via DEPS.

BUG=430237

Change-Id: Ic29cbb15e19a99616cfe778d0778b9a71c45338a
Reviewed-on: https://boringssl-review.googlesource.com/3900
Reviewed-by: Adam Langley <agl@google.com>
2015-03-25 15:31:55 +00:00
David Benjamin
0d5e080ab9 Drop in prebuilt versions of Windows files.
These are upstream's prebuilt binaries of:
e9493171de0edd8879755aa7229a701010a19561  cmake-3.1.3-win32-x86.zip
ab6e7aee6a915c4d820b86f5227094763b649fce  strawberry-perl-5.20.2.1-32bit-portable.zip
4c4d1951181a610923523cb10d83d9ae9952fbf3  yasm-1.2.0-win32.exe

This is intentionally using yasm 1.2.0 rather than the latest 1.3.0 to match
Chromium's current bundled version. Chromium has additional patches, but they
all seem to be either in 1.2.0 or not relevant for us.

Also update extract.py a little to account for these.

BUG=430237

Change-Id: Iad6687e493900b25390d99882c7ceea62fff8b9b
Reviewed-on: https://boringssl-review.googlesource.com/3710
Reviewed-by: Adam Langley <agl@google.com>
2015-03-02 23:03:38 +00:00
David Benjamin
9a7233cda8 Add a prebuilt CMake for OS X for the bots.
Built from:
45f4d3fa8a2f61cc092ae461aac4cac1bab4ac6706f98274ea7f314dd315c6d0  cmake-3.1.3.tar.gz

We're still waiting on infra before the buildbot master is up, but let's get
this ready for when we do; it should be fairly easy.

BUG=430237

Change-Id: I3a414743d44052e1aa48759fa5f125db4d4913b5
Reviewed-on: https://boringssl-review.googlesource.com/3670
Reviewed-by: Adam Langley <agl@google.com>
2015-02-26 23:49:03 +00:00
David Benjamin
1b5cfb5ba3 Pull in a pre-built tarball of cmake 3.1.3 for the bots.
Built from:
45f4d3fa8a2f61cc092ae461aac4cac1bab4ac6706f98274ea7f314dd315c6d0  cmake-3.1.3.tar.gz

Also drop in an extraction script.

Change-Id: I3487e9d432290a7dbabf854b927412c58c35d12b
Reviewed-on: https://boringssl-review.googlesource.com/3492
Reviewed-by: Adam Langley <agl@google.com>
2015-02-17 23:27:14 +00:00