Match the other EVP_DigestSignFinal implementations. Fix the instances in
ssl/t1_enc.c which were not following the EVP_DigestSignFinal contract; on
entry, *out_len should contain the size of the buffer.
Change-Id: Icd44d97a4c98704dea975798c0101d5a37274d17
Reviewed-on: https://boringssl-review.googlesource.com/1130
Reviewed-by: Adam Langley <agl@google.com>
Converting check_autoarg from a macro to a function lost the behavior. Instead,
just move the logic into p_rsa.c which was the only EVP_PKEY implementation
that even needed the flag.
Also document this behavior on each of the functions. Make note of the out =
NULL case only returning the maximum output size, and not necessarily the
actual size.
For testing, update example_sign to determine the signature size using the NULL
behavior rather than querying the RSA key.
Change-Id: Iec6c2862028a5cfdefe8faa0e8c471755070898a
Reviewed-on: https://boringssl-review.googlesource.com/1121
Reviewed-by: Adam Langley <agl@google.com>
Initial fork from f2d678e6e89b6508147086610e985d4e8416e867 (1.0.2 beta).
(This change contains substantial changes from the original and
effectively starts a new history.)
