Since OPENSSL_malloc() and friends report ERR_R_MALLOC_FAILURE, and
at least handle the file name and line number they are called from,
there's no need to report ERR_R_MALLOC_FAILURE where they are called
directly, or when SSLfatal() and RLAYERfatal() is used, the reason
`ERR_R_MALLOC_FAILURE` is changed to `ERR_R_CRYPTO_LIB`.
There were a number of places where `ERR_R_MALLOC_FAILURE` was reported
even though it was a function from a different sub-system that was
called. Those places are changed to report ERR_R_{lib}_LIB, where
{lib} is the name of that sub-system.
Some of them are tricky to get right, as we have a lot of functions
that belong in the ASN1 sub-system, and all the `sk_` calls or from
the CRYPTO sub-system.
Some extra adaptation was necessary where there were custom OPENSSL_malloc()
wrappers, and some bugs are fixed alongside these changes.
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19301)
When a socket connection is done using BIO_new_connect,
the ktls_enable is done too early, and fails with ENOTCONN.
Therefore the KLTS ioctl will fail later with ENOPROTOOPT.
Fix that by doing the ktls_enable after the connection
succeeded, not when the socket is created as that will
always fail.
One example where this happens is doit_localhost in
test/ssl_old_test.c, and therefore, contrary to the expectation
the -client_ktls option did never enable the client KTLS
connection, but this was not noticed, because there was no
diagnostic output, and it was only visible with strace output.
Also enhanced the ssl_old_test -client_ktls/-server_ktls
options together with -v option to print a summary line
if and how KTLS was negotiated in server and client.
While I am already there adjusted the usage info of
the -s_cert, -s_key commands, and allow -time to print the
timings of ktls connections.
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18318)
Richard Levitte
Hugo Landau
Tom Hughes
Bernd Edlinger
Dr. David von Oheimb