Good catch. You are right, I didn't have fullchain configured. Now I have my cert from LetsEncrypt in SSL_CTX_use_certificate_file along with it's key in SSL_CTX_use_PrivateKey_file. I tried using the fullchain from LetsEncrypt in SSL_CTX_use_certificate_chain_file and it didn't work. I had to find the LetsEncrypt roots for X3 and R3 and add them into the fullchain. Now I get totally different errors in the server logs. The first connection SSL_accept returns: Apr 18 23:32:43 <3.7> RIB wncert[21349]: SSL Error: Accept failed - 0020C142E9550000:error:0A0000C7:SSL routines:tls_process_client_certificate:peer did not return a certificate:/usr/src/crypto/openssl/ssl/statem/statem_srvr.c:3511: Apr 18 23:32:43 <3.7> RIB wncert[21349]: logerr msg=Client did not return a certificate. msg2=(null) The second returns a valid certificate, but never sends any data. It eventually times out and exits. The third connection returns the proper certificate and the appropriate data. If the user makes another request, only one connection is established and it works properly. At this point, I believe, the server issues have been corrected and I am dealing with client issues which I have no control over. Thanks for all your help. I can't imagine just how much time you have put into understanding OpenSSL. The original server code was written back in 98 or so for SSLeay. It had to be brought up to date as the old code used "features" that were eliminated. -- Doug > On Apr 18, 2024, at 18:24, Viktor Dukhovni <openssl-users@xxxxxxxxxxxx> wrote: > > On Thu, Apr 18, 2024 at 04:49:52PM -0700, Doug Hardie wrote: > >>> It should be clear (to a trained eye) from a reported SSL alert >>> error, for example whether the alert was a client to server or a >>> server to client alert. This means not summarising the error >>> messages, but reporting them in full. >> >> Finally got time to get back to this. Here are the server logged >> errors for the first 6 connections. They are all basically identical. >> The first message is the SSL error and the second is the one returned >> by SSL_accept. I do not have access to any client logs or error >> indications. Both of these errors were recorded immediately after the >> SSL_accept call. > > Thanks, the below is much more clear/actionable. > >> Apr 18 16:15:17 <3.7> RIB wncert[20518]: SSL Error: Accept failed - >> 0020A163511D0000:error:0A000416:SSL routines:ssl3_read_bytes: >> sslv3 alert certificate unknown:/usr/src/crypto/openssl/ssl/record/rec_layer_s3.c:1586: >> SSL alert number 46 > > This is a client-to-server alert, initiated by the remote client, to > indicate that the client does not like something about server's > certificate. > > If the server certificate chain is complete (not missing required > intermediate certs), unexpired and issued by a trust-anchor that should > be present on the client, then perhaps the client is actually unhappy > with the CAs in the certificate request, though that seems less likely. > >> These 6 connection attempts occurred prior to the user selecting the >> proper certificate to use. Once that was done, there was only on >> additional connection and it was successful. No error returns and the >> server returned the proper response. > > Therefore, first ensure that clients should be able to validate your > server certificate chain. > >>> But what list of CAs is sent in the certificate request? If it is the >>> CA names from full default system trust store, that could be unhelpful, >>> as well as a security issue, since any of those CAs could impersonate >>> your clients. For client authentication, Postfix only trusts client >>> certificate or public key fingerprints, bypassing the CA trust quagmire, >>> that is typically unhelpful (and a major security risk) for client certs. >> >> Not sure how to answer this. I have set SSL_CTX_load_verify_dir >> (ssl_ctx, "/dev/null"). There is only my root certificate used to >> generate the client certificates in SSL_CTX_load_verify_locations and >> SSL_load_client_CA_file. > > Perhaps some clients don't like requests for certs from CAs they don't > know? > >> My server cert (from letsencrypt) is set in >> SSL_CTX_use_certificate_file along with it's key in >> SSL_CTX_use_PrivateKey_file. > > Are you using "cert.pem" or "fullchain.pem"? The former is not a > complete chain. > >>> Find a way to prevent password sharing, to check password quality, and >>> DO NOT force periodic password rotation, which ensures users choose bad >>> passwords. Then consider various second factor options, TOTP and HOTP >>> perhaps, or SMS, ... Some single-systems also include risk metrics, >>> to bypass second factors when things look "normal". >> >> This is a high intensity, time critical, environment. Management >> doesn't want time wasted trying find/remember passwords. > > That may be so, but client certificates for interactive use by humans > don't have a good usability track record. > > -- > Viktor.