boringssl/include/openssl
David Benjamin 6f0c4db90e Enable upstream's Poly1305 code.
The C implementation is still our existing C implementation, but slightly
tweaked to fit with upstream's init/block/emits convention.

I've tested this by looking at code coverage in kcachegrind and

  valgrind --tool=callgrind --dump-instr=yes --collect-jumps=yes

(NB: valgrind 3.11.0 is needed for AVX2. And even that only does 64-bit AVX2,
so we can't get coverage for the 32-bit code yet. But I had to disable that
anyway.)

This was paired with a hacked up version of poly1305_test that would repeat
tests with different ia32cap and armcap values. This isn't checked in, but we
badly need a story for testing all the different variants.

I'm not happy with upstream's code in either the C/asm boundary or how it
dispatches between different versions, but just debugging the code has been a
significant time investment. I'd hoped to extract the SIMD parts and do the
rest in C, but I think we need to focus on testing first (and use that to
guide what modifications would help). For now, this version seems to work at
least.

The x86 (not x86_64) AVX2 code needs to be disabled because it's broken. It
also seems pretty unnecessary.
https://rt.openssl.org/Ticket/Display.html?id=4346

Otherwise it seems to work and buys us a decent performance improvement.
Notably, my Nexus 6P is finally faster at ChaCha20-Poly1305 than my Nexus 4!

bssl speed numbers follow:

x86
---
Old:
Did 1554000 ChaCha20-Poly1305 (16 bytes) seal operations in 1000536us (1553167.5 ops/sec): 24.9 MB/s
Did 136000 ChaCha20-Poly1305 (1350 bytes) seal operations in 1003947us (135465.3 ops/sec): 182.9 MB/s
Did 30000 ChaCha20-Poly1305 (8192 bytes) seal operations in 1022990us (29325.8 ops/sec): 240.2 MB/s
Did 1888000 ChaCha20-Poly1305-Old (16 bytes) seal operations in 1000206us (1887611.2 ops/sec): 30.2 MB/s
Did 173000 ChaCha20-Poly1305-Old (1350 bytes) seal operations in 1003036us (172476.4 ops/sec): 232.8 MB/s
Did 30000 ChaCha20-Poly1305-Old (8192 bytes) seal operations in 1027759us (29189.7 ops/sec): 239.1 MB/s
New:
Did 2030000 ChaCha20-Poly1305 (16 bytes) seal operations in 1000507us (2028971.3 ops/sec): 32.5 MB/s
Did 404000 ChaCha20-Poly1305 (1350 bytes) seal operations in 1000287us (403884.1 ops/sec): 545.2 MB/s
Did 83000 ChaCha20-Poly1305 (8192 bytes) seal operations in 1001258us (82895.7 ops/sec): 679.1 MB/s
Did 2018000 ChaCha20-Poly1305-Old (16 bytes) seal operations in 1000006us (2017987.9 ops/sec): 32.3 MB/s
Did 360000 ChaCha20-Poly1305-Old (1350 bytes) seal operations in 1001962us (359295.1 ops/sec): 485.0 MB/s
Did 85000 ChaCha20-Poly1305-Old (8192 bytes) seal operations in 1002479us (84789.8 ops/sec): 694.6 MB/s

x86_64, no AVX2
---
Old:
Did 2023000 ChaCha20-Poly1305 (16 bytes) seal operations in 1000258us (2022478.2 ops/sec): 32.4 MB/s
Did 466000 ChaCha20-Poly1305 (1350 bytes) seal operations in 1002619us (464782.7 ops/sec): 627.5 MB/s
Did 90000 ChaCha20-Poly1305 (8192 bytes) seal operations in 1001133us (89898.1 ops/sec): 736.4 MB/s
Did 2238000 ChaCha20-Poly1305-Old (16 bytes) seal operations in 1000175us (2237608.4 ops/sec): 35.8 MB/s
Did 483000 ChaCha20-Poly1305-Old (1350 bytes) seal operations in 1001348us (482349.8 ops/sec): 651.2 MB/s
Did 90000 ChaCha20-Poly1305-Old (8192 bytes) seal operations in 1003141us (89718.2 ops/sec): 735.0 MB/s
New:
Did 2558000 ChaCha20-Poly1305 (16 bytes) seal operations in 1000275us (2557296.7 ops/sec): 40.9 MB/s
Did 510000 ChaCha20-Poly1305 (1350 bytes) seal operations in 1001810us (509078.6 ops/sec): 687.3 MB/s
Did 115000 ChaCha20-Poly1305 (8192 bytes) seal operations in 1006457us (114262.2 ops/sec): 936.0 MB/s
Did 2818000 ChaCha20-Poly1305-Old (16 bytes) seal operations in 1000187us (2817473.1 ops/sec): 45.1 MB/s
Did 418000 ChaCha20-Poly1305-Old (1350 bytes) seal operations in 1001140us (417524.0 ops/sec): 563.7 MB/s
Did 91000 ChaCha20-Poly1305-Old (8192 bytes) seal operations in 1002539us (90769.5 ops/sec): 743.6 MB/s

x86_64, AVX2
---
Old:
Did 2516000 ChaCha20-Poly1305 (16 bytes) seal operations in 1000115us (2515710.7 ops/sec): 40.3 MB/s
Did 774000 ChaCha20-Poly1305 (1350 bytes) seal operations in 1000300us (773767.9 ops/sec): 1044.6 MB/s
Did 171000 ChaCha20-Poly1305 (8192 bytes) seal operations in 1004373us (170255.5 ops/sec): 1394.7 MB/s
Did 2580000 ChaCha20-Poly1305-Old (16 bytes) seal operations in 1000144us (2579628.5 ops/sec): 41.3 MB/s
Did 769000 ChaCha20-Poly1305-Old (1350 bytes) seal operations in 1000472us (768637.2 ops/sec): 1037.7 MB/s
Did 169000 ChaCha20-Poly1305-Old (8192 bytes) seal operations in 1000320us (168945.9 ops/sec): 1384.0 MB/s
New:
Did 3240000 ChaCha20-Poly1305 (16 bytes) seal operations in 1000114us (3239630.7 ops/sec): 51.8 MB/s
Did 932000 ChaCha20-Poly1305 (1350 bytes) seal operations in 1000059us (931945.0 ops/sec): 1258.1 MB/s
Did 217000 ChaCha20-Poly1305 (8192 bytes) seal operations in 1003282us (216290.1 ops/sec): 1771.8 MB/s
Did 3187000 ChaCha20-Poly1305-Old (16 bytes) seal operations in 1000100us (3186681.3 ops/sec): 51.0 MB/s
Did 926000 ChaCha20-Poly1305-Old (1350 bytes) seal operations in 1000071us (925934.3 ops/sec): 1250.0 MB/s
Did 215000 ChaCha20-Poly1305-Old (8192 bytes) seal operations in 1000479us (214897.1 ops/sec): 1760.4 MB/s

arm, Nexus 4
---
Old:
Did 430248 ChaCha20-Poly1305 (16 bytes) seal operations in 1000153us (430182.2 ops/sec): 6.9 MB/s
Did 115250 ChaCha20-Poly1305 (1350 bytes) seal operations in 1000549us (115186.8 ops/sec): 155.5 MB/s
Did 27000 ChaCha20-Poly1305 (8192 bytes) seal operations in 1030124us (26210.4 ops/sec): 214.7 MB/s
Did 451750 ChaCha20-Poly1305-Old (16 bytes) seal operations in 1000549us (451502.1 ops/sec): 7.2 MB/s
Did 118000 ChaCha20-Poly1305-Old (1350 bytes) seal operations in 1001557us (117816.6 ops/sec): 159.1 MB/s
Did 27000 ChaCha20-Poly1305-Old (8192 bytes) seal operations in 1024263us (26360.4 ops/sec): 215.9 MB/s
New:
Did 553644 ChaCha20-Poly1305 (16 bytes) seal operations in 1000183us (553542.7 ops/sec): 8.9 MB/s
Did 126000 ChaCha20-Poly1305 (1350 bytes) seal operations in 1000396us (125950.1 ops/sec): 170.0 MB/s
Did 27000 ChaCha20-Poly1305 (8192 bytes) seal operations in 1000336us (26990.9 ops/sec): 221.1 MB/s
Did 559000 ChaCha20-Poly1305-Old (16 bytes) seal operations in 1001465us (558182.3 ops/sec): 8.9 MB/s
Did 124000 ChaCha20-Poly1305-Old (1350 bytes) seal operations in 1000824us (123897.9 ops/sec): 167.3 MB/s
Did 28000 ChaCha20-Poly1305-Old (8192 bytes) seal operations in 1034854us (27057.0 ops/sec): 221.7 MB/s

aarch64, Nexus 6P
---
Old:
Did 358000 ChaCha20-Poly1305 (16 bytes) seal operations in 1000358us (357871.9 ops/sec): 5.7 MB/s
Did 45000 ChaCha20-Poly1305 (1350 bytes) seal operations in 1022386us (44014.7 ops/sec): 59.4 MB/s
Did 8657 ChaCha20-Poly1305 (8192 bytes) seal operations in 1063722us (8138.4 ops/sec): 66.7 MB/s
Did 350000 ChaCha20-Poly1305-Old (16 bytes) seal operations in 1000074us (349974.1 ops/sec): 5.6 MB/s
Did 44000 ChaCha20-Poly1305-Old (1350 bytes) seal operations in 1007907us (43654.8 ops/sec): 58.9 MB/s
Did 8525 ChaCha20-Poly1305-Old (8192 bytes) seal operations in 1042644us (8176.3 ops/sec): 67.0 MB/s
New:
Did 713000 ChaCha20-Poly1305 (16 bytes) seal operations in 1000190us (712864.6 ops/sec): 11.4 MB/s
Did 180000 ChaCha20-Poly1305 (1350 bytes) seal operations in 1004249us (179238.4 ops/sec): 242.0 MB/s
Did 41000 ChaCha20-Poly1305 (8192 bytes) seal operations in 1005811us (40763.1 ops/sec): 333.9 MB/s
Did 775000 ChaCha20-Poly1305-Old (16 bytes) seal operations in 1000719us (774443.2 ops/sec): 12.4 MB/s
Did 182000 ChaCha20-Poly1305-Old (1350 bytes) seal operations in 1003529us (181360.0 ops/sec): 244.8 MB/s
Did 41000 ChaCha20-Poly1305-Old (8192 bytes) seal operations in 1010576us (40570.9 ops/sec): 332.4 MB/s

Change-Id: Iaa4ab86ac1174b79833077963cc3616cfb08e686
Reviewed-on: https://boringssl-review.googlesource.com/7226
Reviewed-by: Adam Langley <agl@google.com>
2016-02-26 16:05:14 +00:00
..
aead.h Point EVP_aead_chacha20_poly1305 at the standardized version. 2015-12-16 21:22:11 +00:00
aes.h Fix documentation typo. 2015-07-01 21:33:29 +00:00
arm_arch.h Remove CRYPTO_set_NEON_functional. 2016-02-23 23:19:46 +00:00
asn1_mac.h Ditch remaining filename comments from public headers and ssl/ 2015-10-20 18:40:05 +00:00
asn1.h Remove ASN1_R_MALLOC_FAILURE. 2015-12-22 00:12:24 +00:00
asn1t.h Ditch remaining filename comments from public headers and ssl/ 2015-10-20 18:40:05 +00:00
base64.h Deprecate basically the entire base64 implementation. 2015-09-28 21:53:39 +00:00
base.h Rename opensslfeatures.h to opensslconf.h. 2016-02-26 01:32:50 +00:00
bio.h Add BIO_do_connect. 2016-02-26 01:32:21 +00:00
blowfish.h
bn.h Rename the BIGNUM ASN.1 functions. 2016-01-27 22:37:44 +00:00
buf.h Have doc.go parse struct comments. 2016-01-26 23:23:23 +00:00
buffer.h
bytestring.h Parse BER for PKCS#12 more accurately. 2016-02-16 19:32:50 +00:00
cast.h
chacha.h Change |CRYPTO_chacha_20| to use 96-bit nonces, 32-bit counters. 2015-10-26 23:58:46 +00:00
cipher.h Fix up several comments and detect problems in the future. 2015-11-05 20:12:45 +00:00
cmac.h Style: fix some header guards 2015-10-26 18:47:51 +00:00
conf.h Also add a no-op stub for OPENSSL_config. 2016-01-26 15:48:51 +00:00
cpu.h Remove CRYPTO_set_NEON_functional. 2016-02-23 23:19:46 +00:00
crypto.h Tweaks for node.js 2016-01-26 23:23:42 +00:00
curve25519.h Update draft-irtf-cfrg-curves-11 references to RFC 7748. 2016-01-28 00:53:26 +00:00
des.h Fix shared library build on OS X. 2015-10-26 23:39:47 +00:00
dh.h Validate DH public keys for RFC 5114 groups. 2016-02-02 16:44:38 +00:00
digest.h Documentation typo. 2015-09-28 22:18:40 +00:00
dsa.h Reimplement DSA parsing logic with crypto/asn1. 2016-02-17 00:26:01 +00:00
dtls1.h
ec_key.h Rewrite ECPrivateKey serialization. 2016-02-16 23:51:09 +00:00
ec.h Return 0 on error in |EC_POINT_is_on_curve| instead of -1. 2016-02-24 22:00:58 +00:00
ecdh.h Clean up |ECDH_compute_key|. 2015-10-27 17:00:25 +00:00
ecdsa.h Update comments to better document in-place semantics. 2016-01-19 17:01:37 +00:00
engine.h Unwind DH_METHOD and DSA_METHOD. 2015-11-03 22:54:36 +00:00
err.h Have doc.go parse struct comments. 2016-01-26 23:23:23 +00:00
evp.h Implement new PKCS#8 parsers. 2016-02-17 17:24:10 +00:00
ex_data.h Skip free callbacks on empty CRYPTO_EX_DATAs. 2015-12-15 21:32:14 +00:00
hkdf.h Get rid of err function codes. 2015-07-16 02:02:08 +00:00
hmac.h Remove HMAC_CTX_set_flags. 2015-06-02 01:07:07 +00:00
lhash_macros.h
lhash.h Style: fix some header guards 2015-10-26 18:47:51 +00:00
md4.h Store the partial block as uint8_t, not uint32_t. 2015-12-16 19:59:29 +00:00
md5.h Store the partial block as uint8_t, not uint32_t. 2015-12-16 19:59:29 +00:00
mem.h Fix some documentation comments. 2016-01-21 22:12:08 +00:00
obj_mac.h Allocate a NID for X25519. 2015-12-22 18:56:53 +00:00
obj.h Fix up several comments and detect problems in the future. 2015-11-05 20:12:45 +00:00
objects.h
opensslconf.h Rename opensslfeatures.h to opensslconf.h. 2016-02-26 01:32:50 +00:00
opensslv.h Get version-related functions from crypto.h rather than ssl.h. 2015-05-20 22:58:14 +00:00
ossl_typ.h
pem.h Remove param_decode and param_encode EVP_PKEY hooks. 2016-02-17 16:30:29 +00:00
pkcs7.h
pkcs8.h Changes to support node.js's use of PKCS#12. 2016-02-02 19:21:59 +00:00
pkcs12.h
poly1305.h Enable upstream's Poly1305 code. 2016-02-26 16:05:14 +00:00
pqueue.h
rand.h Add a few more no-op stubs for cURL compatibility. 2016-01-26 15:48:41 +00:00
rc4.h
rsa.h Update comments to better document in-place semantics. 2016-01-19 17:01:37 +00:00
safestack.h
sha.h Store the partial block as uint8_t, not uint32_t. 2015-12-16 19:59:29 +00:00
srtp.h Fold srtp.h into ssl.h. 2015-09-14 23:59:37 +00:00
ssl3.h Remove SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER. 2015-12-15 19:14:00 +00:00
ssl.h Add SSL_CIPHER_has_SHA256_HMAC. 2016-02-26 01:33:11 +00:00
stack_macros.h Remove stack macros for nonexistent types. 2015-12-22 00:12:38 +00:00
stack.h Remove stack macros for nonexistent types. 2015-12-22 00:12:38 +00:00
thread.h Readd CRYPTO_{LOCK|UNLOCK|READ|WRITE}. 2015-05-27 15:48:29 -07:00
time_support.h Style: fix some header guards 2015-10-26 18:47:51 +00:00
tls1.h Update references to the extended master secret draft. 2016-02-02 16:37:55 +00:00
type_check.h Try to fix MSVC and __STDC_VERSION__ again. 2015-05-20 13:42:12 -07:00
x509_vfy.h Import “altchains” support. 2016-01-19 17:02:31 +00:00
x509.h Resolve a few old TODOs. 2015-12-22 00:14:35 +00:00
x509v3.h Get rid of err function codes. 2015-07-16 02:02:08 +00:00