On 04/02/2022 15:17, Kamala Ayyar wrote:
Hello,
We are facing a strange handshake failure issue with a test server and
client application using OpenSSL in Windows. We have tried with both
1.1.1g and 3.0.1 versions- same problem. We created a Dll to handle the
OpenSSL functions- where the SSL context, SSL object and certificates
are handled. The certificates are obtained from the Windows store and
converted to cert and key using PKCS12_parse()
The server accepts non secure connection from the client and then passes
the socket to the Dll that calls the TLS_server_method() and creates the
SSL context, SSL object and loads the certificates for use. It however
fails at SSL_accept(m_pSsl). We use a call
back SSL_set_info_callback(m_pSsl, apps_ssl_info_callback) that gave us
the following error information
SSL_accept:Error in before SSL initialization
On the client side the same Dll is called with a client
method TLS_client_method() and the error displayed is SSL_connect:Error
in SSLv3/TLS write client hello
We have confirmed the certificates are good and valid.
The same Dll called from a different heavily threaded application with
over 2000+ clients works well and handshake connections established
without issues on a different port number.
We have also tried to use OpenSSL methods directly without using the Dll
but we get the same failure. This was also used with server and client
on the same machine as well as different machines with the same
outcome. The non secure communication works fine between the server and
the client
What does SSL_get_error() report after SSL_accept() fails?
Also please dump the OpenSSL error stack when it fails, e.g. using
something like ERR_print_errors_fp(stdout);
Matt
