Now that we have 64-bit C code, courtesy of fiat-crypto, the tradeoff
for carrying the assembly changes:
Assembly:
Did 16000 Curve25519 base-point multiplication operations in 1059932us (15095.3 ops/sec)
Did 16000 Curve25519 arbitrary point multiplication operations in 1060023us (15094.0 ops/sec)
fiat64:
Did 39000 Curve25519 base-point multiplication operations in 1004712us (38817.1 ops/sec)
Did 14000 Curve25519 arbitrary point multiplication operations in 1006827us (13905.1 ops/sec)
The assembly is still about 9% faster than fiat64, but fiat64 gets to
use the Ed25519 tables for the base point multiplication, so overall it
is actually faster to disable the assembly:
>>> 1/(1/15094.0 + 1/15095.3)
7547.324986004976
>>> 1/(1/38817.1 + 1/13905.1)
10237.73016319501
(At the cost of touching a 30kB table.)
The assembly implementation is no longer pulling its weight. Remove it
and use the fiat code in all build configurations.
Change-Id: Id736873177d5568bb16ea06994b9fcb1af104e33
Reviewed-on: https://boringssl-review.googlesource.com/25524
Reviewed-by: Adam Langley <agl@google.com>
Our 64-bit performance was much lower than it could have been, since we
weren't using the 64-bit multipliers. Fortunately, fiat-crypto is
awesome, so this is just a matter of synthesizing new code and
integration work.
Functions without the signature fiat-crypto curly braces were written by
hand and warrant more review. (It's just redistributing some bits.)
These use the donna variants which takes (and proves) some of the
instruction scheduling from donna as that's significantly faster.
Glancing over things, I suspect but have not confirmed the gap is due to
this:
https://github.com/mit-plv/fiat-crypto/pull/295#issuecomment-356892413
Clang without OPENSSL_SMALL (ECDH omitted since that uses assembly and
is unaffected by this CL).
Before:
Did 105149 Ed25519 key generation operations in 5025208us (20924.3 ops/sec)
Did 125000 Ed25519 signing operations in 5024003us (24880.6 ops/sec)
Did 37642 Ed25519 verify operations in 5072539us (7420.7 ops/sec)
After:
Did 206000 Ed25519 key generation operations in 5020547us (41031.4 ops/sec)
Did 227000 Ed25519 signing operations in 5005232us (45352.5 ops/sec)
Did 69840 Ed25519 verify operations in 5004769us (13954.7 ops/sec)
Clang + OPENSSL_SMALL:
Before:
Did 68598 Ed25519 key generation operations in 5024629us (13652.4 ops/sec)
Did 73000 Ed25519 signing operations in 5067837us (14404.6 ops/sec)
Did 36765 Ed25519 verify operations in 5078684us (7239.1 ops/sec)
Did 74000 Curve25519 base-point multiplication operations in 5016465us (14751.4 ops/sec)
Did 45600 Curve25519 arbitrary point multiplication operations in 5034680us (9057.2 ops/sec)
After:
Did 117315 Ed25519 key generation operations in 5021860us (23360.9 ops/sec)
Did 126000 Ed25519 signing operations in 5003521us (25182.3 ops/sec)
Did 64974 Ed25519 verify operations in 5047790us (12871.8 ops/sec)
Did 134000 Curve25519 base-point multiplication operations in 5058946us (26487.7 ops/sec)
Did 86000 Curve25519 arbitrary point multiplication operations in 5050478us (17028.1 ops/sec)
GCC without OPENSSL_SMALL (ECDH omitted since that uses assembly and
is unaffected by this CL).
Before:
Did 35552 Ed25519 key generation operations in 5030756us (7066.9 ops/sec)
Did 38286 Ed25519 signing operations in 5001648us (7654.7 ops/sec)
Did 10584 Ed25519 verify operations in 5068158us (2088.3 ops/sec)
After:
Did 92158 Ed25519 key generation operations in 5024021us (18343.5 ops/sec)
Did 99000 Ed25519 signing operations in 5011908us (19753.0 ops/sec)
Did 31122 Ed25519 verify operations in 5069878us (6138.6 ops/sec)
Change-Id: Ic0c24d50b4ee2bbc408b94965e9d63319936107d
Reviewed-on: https://boringssl-review.googlesource.com/24805
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
Reviewed-by: Adam Langley <agl@google.com>
These date to the old code and have been replaced by the fe and fe_loose
bounds in the header file. Also fix up a comment that the comment
converter didn't manage to convert.
Change-Id: I2e3ea867a8cea2b347d09c304a17e532b2e36545
Reviewed-on: https://boringssl-review.googlesource.com/24525
Commit-Queue: Adam Langley <agl@google.com>
Reviewed-by: Adam Langley <agl@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
Each operation was translated from fiat-crypto output using fiat-crypto
prettyprint.py. For example fe_mul is synthesized in
https://github.com/mit-plv/fiat-crypto/blob/master/src/Specific/X25519/C32/femul.v,
and shown in the last Coq-compatible form at
https://github.com/mit-plv/fiat-crypto/blob/master/src/Specific/X25519/C32/femulDisplay.log.
Benchmarks on Google Cloud's unidentified Intel Xeon with AVX2:
git checkout $VARIANT && ( cd build && rm -rf * && CC=clang CXX=clang++ cmake -GNinja -DCMAKE_TOOLCHAIN_FILE=../util/32-bit-toolchain.cmake -DCMAKE_BUILD_TYPE=Release .. && ninja && ./tool/bssl speed -filter 25519 )
this branch:
Did 11382 Ed25519 key generation operations in 1053046us (10808.6 ops/sec)
Did 11169 Ed25519 signing operations in 1038080us (10759.3 ops/sec)
Did 2925 Ed25519 verify operations in 1001346us (2921.1 ops/sec)
Did 12000 Curve25519 base-point multiplication operations in 1084851us (11061.4 ops/sec)
Did 3850 Curve25519 arbitrary point multiplication operations in 1085565us (3546.5 ops/sec)
Did 11466 Ed25519 key generation operations in 1049821us (10921.9 ops/sec)
Did 11000 Ed25519 signing operations in 1013317us (10855.4 ops/sec)
Did 3047 Ed25519 verify operations in 1043846us (2919.0 ops/sec)
Did 12000 Curve25519 base-point multiplication operations in 1068924us (11226.2 ops/sec)
Did 3850 Curve25519 arbitrary point multiplication operations in 1090598us (3530.2 ops/sec)
Did 10309 Ed25519 key generation operations in 1003320us (10274.9 ops/sec)
Did 11000 Ed25519 signing operations in 1017862us (10807.0 ops/sec)
Did 3135 Ed25519 verify operations in 1098624us (2853.6 ops/sec)
Did 9000 Curve25519 base-point multiplication operations in 1046608us (8599.2 ops/sec)
Did 3132 Curve25519 arbitrary point multiplication operations in 1038963us (3014.5 ops/sec)
master:
Did 11564 Ed25519 key generation operations in 1068762us (10820.0 ops/sec)
Did 11104 Ed25519 signing operations in 1024278us (10840.8 ops/sec)
Did 3206 Ed25519 verify operations in 1049179us (3055.7 ops/sec)
Did 12000 Curve25519 base-point multiplication operations in 1073619us (11177.1 ops/sec)
Did 3550 Curve25519 arbitrary point multiplication operations in 1000279us (3549.0 ops/sec)
andreser@linux-andreser:~/boringssl$ build/tool/bssl speed -filter 25519
Did 11760 Ed25519 key generation operations in 1072495us (10965.1 ops/sec)
Did 10800 Ed25519 signing operations in 1003486us (10762.5 ops/sec)
Did 3245 Ed25519 verify operations in 1080399us (3003.5 ops/sec)
Did 12000 Curve25519 base-point multiplication operations in 1076021us (11152.2 ops/sec)
Did 3570 Curve25519 arbitrary point multiplication operations in 1005087us (3551.9 ops/sec)
andreser@linux-andreser:~/boringssl$ build/tool/bssl speed -filter 25519
Did 11438 Ed25519 key generation operations in 1041115us (10986.3 ops/sec)
Did 11000 Ed25519 signing operations in 1012589us (10863.2 ops/sec)
Did 3312 Ed25519 verify operations in 1082834us (3058.6 ops/sec)
Did 12000 Curve25519 base-point multiplication operations in 1061318us (11306.7 ops/sec)
Did 3580 Curve25519 arbitrary point multiplication operations in 1004923us (3562.5 ops/sec)
squashed: curve25519: convert field constants to unsigned.
import re, sys, math
def weight(i):
return 2**int(math.ceil(25.5*i))
def convert(t):
limbs = [x for x in t.groups() if x.replace('-','').isdigit()]
v = sum(weight(i)*x for (i,x) in enumerate(map(int, limbs))) % (2**255-19)
limbs = [(v % weight(i+1)) // weight(i) for i in range(10)]
assert v == sum(weight(i)*x for (i,x) in enumerate(limbs))
i = 0
ret = ''
for s in t.groups():
if s.replace('-','').isdigit():
ret += str(limbs[i])
i += 1
else:
ret += s
return ret
fe_re = re.compile(r'(\s*,\s*)'.join(r'(-?\d+)' for i in range(10)))
print (re.sub(fe_re, convert, sys.stdin.read()))
Change-Id: Ibd4f7f5c38e5c4d61c9826afb406baebe2be5168
Reviewed-on: https://boringssl-review.googlesource.com/22385
Reviewed-by: Adam Langley <agl@google.com>
Commit-Queue: Adam Langley <agl@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
This change doesn't actually introduce any Fiat code yet. It sets up the
directory structure to make the diffs in the next change clearer.
Change-Id: I38a21fb36b18a08b0907f9d37b7ef5d7d3137ede
Reviewed-on: https://boringssl-review.googlesource.com/22624
Reviewed-by: David Benjamin <davidben@google.com>
Previously, the ed25519 and SPAKE implementations called field element
operations in ways that did not satisfy the preconditions about ranges
of limbs. Furthermore, replacing signed field arithmetic with unsigned field
arithmetic with similar specifications caused tests to fail. This commit
addresses this in three steps:
(1) Split fe into fe and fe_loose, tracking the bounds
(2) Insert carry operations before uses of fe_add/fe_sub/fe_neg whose
input is already within only the loose bounds
(3) Assert that each field element is within the appropriate bounds at
the beginning and end of every field operation.
Throughput diff:
Ed25519 key generation: -2%
Ed25519 signing: -2%
Ed25519 verify: -2%
X25519: roughly unchanged
Detailed benchmarks on Google Cloud's unidentified Intel Xeon with AVX2:
git checkout $VARIANT && ( cd build && rm -rf * && CC=clang CXX=clang++ cmake -GNinja -DCMAKE_TOOLCHAIN_FILE=../util/32-bit-toolchain.cmake -DCMAKE_BUILD_TYPE=Release .. && ninja && ./tool/bssl speed -filter 25519 )
this branch:
Did 11206 Ed25519 key generation operations in 1029462us (10885.3 ops/sec)
Did 11104 Ed25519 signing operations in 1035735us (10720.9 ops/sec)
Did 3278 Ed25519 verify operations in 1087969us (3013.0 ops/sec)
Did 12000 Curve25519 base-point multiplication operations in 1078962us (11121.8 ops/sec)
Did 3610 Curve25519 arbitrary point multiplication operations in 1002767us (3600.0 ops/sec)
Did 11662 Ed25519 key generation operations in 1077690us (10821.3 ops/sec)
Did 10780 Ed25519 signing operations in 1011474us (10657.7 ops/sec)
Did 3289 Ed25519 verify operations in 1083638us (3035.1 ops/sec)
Did 12000 Curve25519 base-point multiplication operations in 1087477us (11034.7 ops/sec)
Did 3610 Curve25519 arbitrary point multiplication operations in 1017023us (3549.6 ops/sec)
Did 11018 Ed25519 key generation operations in 1011606us (10891.6 ops/sec)
Did 11000 Ed25519 signing operations in 1029961us (10680.0 ops/sec)
Did 3124 Ed25519 verify operations in 1045163us (2989.0 ops/sec)
Did 12000 Curve25519 base-point multiplication operations in 1081770us (11092.9 ops/sec)
Did 3610 Curve25519 arbitrary point multiplication operations in 1014503us (3558.4 ops/sec)
master:
Did 11662 Ed25519 key generation operations in 1059449us (11007.6 ops/sec)
Did 10908 Ed25519 signing operations in 1000081us (10907.1 ops/sec)
Did 3333 Ed25519 verify operations in 1078798us (3089.5 ops/sec)
Did 12000 Curve25519 base-point multiplication operations in 1072831us (11185.4 ops/sec)
Did 3850 Curve25519 arbitrary point multiplication operations in 1075821us (3578.7 ops/sec)
Did 11102 Ed25519 key generation operations in 1017540us (10910.6 ops/sec)
Did 11000 Ed25519 signing operations in 1013279us (10855.8 ops/sec)
Did 3311 Ed25519 verify operations in 1066866us (3103.5 ops/sec)
Did 12000 Curve25519 base-point multiplication operations in 1069668us (11218.4 ops/sec)
Did 3905 Curve25519 arbitrary point multiplication operations in 1095501us (3564.6 ops/sec)
Did 11206 Ed25519 key generation operations in 1014127us (11049.9 ops/sec)
Did 10908 Ed25519 signing operations in 1015821us (10738.1 ops/sec)
Did 3344 Ed25519 verify operations in 1100592us (3038.4 ops/sec)
Did 12000 Curve25519 base-point multiplication operations in 1072847us (11185.2 ops/sec)
Did 3570 Curve25519 arbitrary point multiplication operations in 1009373us (3536.8 ops/sec)
Change-Id: Ia014386daf36c913f3ea44c5f9a420b98670e465
Reviewed-on: https://boringssl-review.googlesource.com/22104
Reviewed-by: Adam Langley <agl@google.com>
Commit-Queue: Adam Langley <agl@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
Due to a copy-paste error, the call to |left_shift_3| is missing after
reducing the password scalar in SPAKE2. This means that three bits of
the password leak in Alice's message. (Two in Bob's message as the point
N happens to have order 4l, not 8l.)
The “correct” fix is to put in the missing call to |left_shift_3|, but
that would be a breaking change. In order to fix this in a unilateral
way, we add points of small order to the masking point to bring it into
prime-order subgroup.
BUG=chromium:778101
Change-Id: I440931a3df7f009b324d2a3e3af2d893a101804f
Reviewed-on: https://boringssl-review.googlesource.com/22445
Reviewed-by: Adam Langley <agl@google.com>
Reviewed-by: David Benjamin <davidben@google.com>
Commit-Queue: Adam Langley <agl@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
crypto/{asn1,x509,x509v3,pem} were skipped as they are still OpenSSL
style.
Change-Id: I3cd9a60e1cb483a981aca325041f3fbce294247c
Reviewed-on: https://boringssl-review.googlesource.com/19504
Reviewed-by: Adam Langley <agl@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
The built-in CMake support seems to basically work, though it believes
you want to build a fat binary which doesn't work with how we build
perlasm. (We'd need to stop conditioning on CMAKE_SYSTEM_PROCESSOR at
all, wrap all the generated assembly files in ifdefs, and convince the
build to emit more than one. Probably not worth bothering for now.)
We still, of course, need to actually test the assembly on iOS before
this can be shipped anywhere.
BUG=48
Change-Id: I6ae71d98d706be03142b82f7844d1c9b02a2b832
Reviewed-on: https://boringssl-review.googlesource.com/14645
Commit-Queue: David Benjamin <davidben@google.com>
Commit-Queue: Steven Valdez <svaldez@google.com>
Reviewed-by: Steven Valdez <svaldez@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
This assembly is in gas syntax so is not built on Windows nor when
OPENSSL_SMALL is defined.
Change-Id: I1050cf1b16350fd4b758e4c463261b30a1b65390
Reviewed-on: https://boringssl-review.googlesource.com/6782
Reviewed-by: Adam Langley <agl@google.com>
By doing this the compiler can notice that much of the code is unused in
the case that we know that we can't have a hardware RNG (i.e. ARM).
Change-Id: I72d364a30080364d700f855640e0164c2c62f0de
Reviewed-on: https://boringssl-review.googlesource.com/6001
Reviewed-by: David Benjamin <davidben@chromium.org>
Reviewed-by: Adam Langley <agl@google.com>
Missed a mention of CRYPTO_have_hwrand.
Change-Id: I9756d80105c2fcee487a7badbf4d82f375b5652d
Reviewed-on: https://boringssl-review.googlesource.com/5640
Reviewed-by: Adam Langley <agl@google.com>
Since the caller must check for CRYPTO_hwrand failures anyway, there's not much
point in doing the CRYPTO_have_hwrand check externally.
(As a bonus, CRYPTO_hwrand no longer compiles to abort() on ARM, so linker
deduplicating won't confuse Chrome's crash reporter...)
Change-Id: I2191d835fbda5b70812f14cd9a873a5e35c30c6d
Reviewed-on: https://boringssl-review.googlesource.com/5630
Reviewed-by: Adam Langley <agl@google.com>
I mistakenly believed that only RDSEED could fail. However, the Intel
manuals state that RDRAND can fail too.
I can't actually observe it failing, even with all cores running RDRAND in a
tight loop. In any case, the ChaCha20 masking means that it wouldn't be
a big deal anyway.
Still, this change tests the carry flag after RDRAND and the code falls
back to |CRYPTO_sysrand| if RDRAND has a hiccup. (The Intel manuals
suggest[1] calling RDRAND in a loop, ten times, before considering it to
have failed. But a single failure appears to be such a rare event that
the complexity in the asm code doesn't seem worth it.)
This change also adds an asm function to fill a buffer with random data.
Otherwise the overhead of calling |CRYPTO_rdrand|, and bouncing the data
in and out of memory starts to add up.
Thanks to W. Mark Kubacki, who may have reported this. (There's some
confusion in the bug report.)
Before:
Did 6148000 RNG (16 bytes) operations in 1000080us: 98.4 MB/s
Did 649000 RNG (256 bytes) operations in 1000281us: 166.1 MB/s
Did 22000 RNG (8192 bytes) operations in 1033538us: 174.4 MB/s
After:
Did 6573000 RNG (16 bytes) operations in 1000002us: 105.2 MB/s
Did 693000 RNG (256 bytes) operations in 1000127us: 177.4 MB/s
Did 24000 RNG (8192 bytes) operations in 1028466us: 191.2 MB/s
[1] Intel Reference Manual, section 7.3.17.1.
Change-Id: Iba7f82e844ebacef535472a31f2dd749aad1190a
Reviewed-on: https://boringssl-review.googlesource.com/5180
Reviewed-by: Adam Langley <agl@google.com>
Beyond generally eliminating unnecessary includes, eliminate as many
includes of headers that declare/define particularly error-prone
functionality like strlen, malloc, and free. crypto/err/internal.h was
added to remove the dependency on openssl/thread.h from the public
openssl/err.h header. The include of <stdlib.h> in openssl/mem.h was
retained since it defines OPENSSL_malloc and friends as macros around
the stdlib.h functions. The public x509.h, x509v3.h, and ssl.h headers
were not changed in order to minimize breakage of source compatibility
with external code.
Change-Id: I0d264b73ad0a720587774430b2ab8f8275960329
Reviewed-on: https://boringssl-review.googlesource.com/4220
Reviewed-by: Adam Langley <agl@google.com>
This saves about 6-7k of error data.
Change-Id: Ic28593d4a1f5454f00fb2399d281c351ee57fb14
Reviewed-on: https://boringssl-review.googlesource.com/3385
Reviewed-by: Adam Langley <agl@google.com>
Chromium does not like static initializers, and the CPU logic uses one to
initialize CPU bits. However, the crypto library lacks an explicit
initialization function, which could complicate (no compile-time errors)
porting existing code which uses crypto/, but not ssl/.
Add an explicit CRYPTO_library_init function, but make it a no-op by default.
It only does anything (and is required) if building with
BORINGSSL_NO_STATIC_INITIALIZER.
Change-Id: I6933bdc3447fb382b1f87c788e5b8142d6f3fe39
Reviewed-on: https://boringssl-review.googlesource.com/1770
Reviewed-by: Adam Langley <agl@google.com>
Previously, public headers lived next to the respective code and there
were symlinks from include/openssl to them.
This doesn't work on Windows.
This change moves the headers to live in include/openssl. In cases where
some symlinks pointed to the same header, I've added a file that just
includes the intended target. These cases are all for backwards-compat.
Change-Id: I6e285b74caf621c644b5168a4877db226b07fd92
Reviewed-on: https://boringssl-review.googlesource.com/1180
Reviewed-by: David Benjamin <davidben@chromium.org>
Reviewed-by: Adam Langley <agl@google.com>
David Benjamin
Adam Langley
Andres Erbsen
Arnar Birgisson
Adam Langley
David Benjamin
Adam Langley
Brian Smith