This change replace the cmovq scheme with slightly faster SSE2 code.
The SSE2 code was first introduced in Go's curve25519 implementation.
See: https://go-review.googlesource.com/c/39693/
The implementation is basicly copied from the Go assembly.
Change-Id: I25931a421ba141ce33809875699f048b0941c061
Reviewed-on: https://boringssl-review.googlesource.com/16564
Reviewed-by: Adam Langley <agl@google.com>
Commit-Queue: Adam Langley <agl@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
The current X25519 assembly has a 352-byte stack frame and saves the
regsiters at the bottom. This means that the CFI information cannot be
represented in the “compact” form that MacOS seems to want to use (see
linked bug).
The stack frame looked like:
360 CFA
352 return address
⋮
56 (296 bytes of scratch space)
48 saved RBP
40 saved RBX
32 saved R15
24 saved R14
16 saved R13
8 saved R12
0 (hole left from 3f38d80b dropping the superfluous saving of R11)
Now it looks like:
352 CFA
344 return address
336 saved RBP
328 saved RBX
320 saved R15
312 saved R14
304 saved R13
296 saved R12
⋮
0 (296 bytes of scratch space)
The bulk of the changes involve subtracting 56 from all the offsets to
RSP when working in the scratch space. This was done in Vim with:
'<,'>s/\([1-9][0-9]*\)(%rsp)/\=submatch(1)-56."(%rsp)"/
BUG=176
Change-Id: I022830e8f896fe2d877015fa3ecfa1d073207679
Reviewed-on: https://boringssl-review.googlesource.com/13580
Commit-Queue: Adam Langley <agl@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
Reviewed-by: David Benjamin <davidben@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
The Mac ld gets unhappy about "weird" unwind directives:
In chacha20_poly1305_x86_64.pl, $keyp is being pushed on the stack
(according to the comment) because it gets clobbered in the computation
somewhere. $keyp is %r9 which is not callee-saved (it's an argument
register), so we don't need to tag it with .cfi_offset.
In x25519-asm-x86_64.S, x25519_x86_64_mul saves %rdi on the stack.
However it too is not callee-saved (it's an argument register) and
should not have a .cfi_offset. %rdi also does not appear to be written
to anywhere in the function, so there's no need to save it at all.
(This does not resolve the "r15 is saved too far from return address"
errors. Just the non-standard register ones.)
BUG=176
Change-Id: I53f3f7db3d1745384fb47cb52cd6536aabb5065e
Reviewed-on: https://boringssl-review.googlesource.com/13560
Commit-Queue: David Benjamin <davidben@google.com>
Reviewed-by: Adam Langley <agl@google.com>
This change serves to check that all our consumers can process assembly
with CFI directives in it.
For the first change I picked a file that's not perlasm to keep things
slightly simplier, but that might have been a mistake:
DJB's tooling always aligns the stack to 32 bytes and it's not possible
to express this in DWARF format (without using a register to store the
old stack pointer).
Since none of the functions here appear to care about that alignment, I
removed it from each of them. I also trimmed the set of saved registers
where possible and used the redzone for functions that didn't need much
stack.
Overall, this appears to have slightly improved the performance (by
about 0.7%):
Before:
Did 46000 Curve25519 base-point multiplication operations in 3023288us (15215.2 ops/sec)
Did 46000 Curve25519 arbitrary point multiplication operations in 3017315us (15245.3 ops/sec)
Did 46000 Curve25519 base-point multiplication operations in 3015346us (15255.3 ops/sec)
Did 46000 Curve25519 arbitrary point multiplication operations in 3018609us (15238.8 ops/sec)
Did 46000 Curve25519 base-point multiplication operations in 3019004us (15236.8 ops/sec)
Did 46000 Curve25519 arbitrary point multiplication operations in 3013135us (15266.5 ops/sec)
After:
Did 46000 Curve25519 base-point multiplication operations in 3007659us (15294.3 ops/sec)
Did 47000 Curve25519 arbitrary point multiplication operations in 3054202us (15388.6 ops/sec)
Did 46000 Curve25519 base-point multiplication operations in 3008714us (15288.9 ops/sec)
Did 46000 Curve25519 arbitrary point multiplication operations in 3004740us (15309.1 ops/sec)
Did 46000 Curve25519 base-point multiplication operations in 3009140us (15286.8 ops/sec)
Did 47000 Curve25519 arbitrary point multiplication operations in 3057518us (15371.9 ops/sec)
Change-Id: I31df11c45b2ea0bf44dde861d52c27f848331691
Reviewed-on: https://boringssl-review.googlesource.com/13200
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
Reviewed-by: David Benjamin <davidben@google.com>
Reviewed-by: Adam Langley <agl@google.com>
Commit-Queue: Adam Langley <agl@google.com>
Add guards for the architecture and OPENSSL_NO_ASM to
the assembly-language files in crypto/curve25519/asm.
The Dart compilation of BoringSSL includes all files,
because the architecture is not known when gyp is run.
Change-Id: I66f5ae525266b63b0fe3a929012b771d545779b5
Reviewed-on: https://boringssl-review.googlesource.com/7030
Reviewed-by: Adam Langley <agl@google.com>
Some build systems don't like two targets with the same base name and
the curve25519 code had x25519-x86_64.[Sc].
Change-Id: If8382eb84996d7e75b34b28def57829d93019cff
Reviewed-on: https://boringssl-review.googlesource.com/6878
Reviewed-by: Adam Langley <agl@google.com>
