Thanks a lot for this information. I was also just browsing and debugging this exact file, it might not do any harm to understand a little bit more how OpenSSL works..... My traces show that the problem is not coming from the function you are pointing to, but from line 529 :
SSL_CTX_set_verify(_ssl_context,
SSL_VERIFY_PEER,
[](int preverify, X509_STORE_CTX*) -> int { return preverify; });
From my understanding, this function is verifying the certificate on a callback. The part
"[](int preverify, X509_STORE_CTX*) -> int { return preverify; })"
returns 0, which means it failed. That is not really clear to me why, and what does X509_STORE_CTX . I guess that prior to the SSL_CTX_set_verify, I have to do something differently? Like calling SSL_set1_host somewhere before with wss server IP address that matches with SAN from the server certificate? btw, I am using OpenSSL 3.0.7
Thanks a lot!
Le mar. 14 févr. 2023, à 09 h 58, Mark Hack <markhack@xxxxxxxxxxxx> a écrit :
I went and looked at the IX code and this, as we all suspected, has nothing to do with OpenSSL.Here is the offending code in ixwebsocket/IXSocketOpenSSL.cpp which ignores the IP addresses and only checks the DNS name entries:STACK_OF(GENERAL_NAME)* san_names = (STACK_OF(GENERAL_NAME)*) X509_get_ext_d2i((X509*) server_cert, NID_subject_alt_name, NULL, NULL);if (san_names){for (int i = 0; i < sk_GENERAL_NAME_num(san_names); i++){const GENERAL_NAME* sk_name = sk_GENERAL_NAME_value(san_names, i);if (sk_name->type == GEN_DNS){char* name = (char*) ASN1_STRING_data(sk_name->d.dNSName);if ((size_t) ASN1_STRING_length(sk_name->d.dNSName) == strlen(name) &&checkHost(hostname, name)){hostname_verifies_ok = true;break;}}}}In public certificates, IP addresses have been deprecated for a while so several implementations just dont look for them in the certificate extensions.If you can not twist arms to get this corrected, what may work for you is to set the IP addresses as if they were DNS names - uglier than sin - but which should work.RegardsMark HackOn Mon, 2023-02-13 at 16:40 -0500, Pierre-Luc Boily wrote:So, I updated my hosts file. I added a fake url pointing to the server. Then, I created a new certificate with a SAN pointing to the fake URL et voilà, my c++ wss client can connect to my wss server.So, your assumption is correct, the IXWebSocket implementation of openssl cannot use an ip address.Thank you.Le lun. 13 févr. 2023, à 15 h 07, Pierre-Luc Boily <pierreluc.boily@xxxxxxxxx> a écrit :You can see the server cert here : https://pastebin.com/Eb8b9tUfIndeed, server cert shows "localhost", but it also shows the ip address :X509v3 Subject Alternative Name:
DNS:localhost, IP Address:192.168.230.138, IP Address:127.0.0.1By the way, the author of the IXWebSocket c++ library told me :This might be a high level SSL stuff, where you actually need a real hostname and can't use an IP.But on the other hand, I have a _javascript_ websocket client that can connect to my wss server using the same certificate as the c++ client.All of this is for test purposes, eventually, I will use a domain name. But as a workaround, I thought to use a fake domain name that points to the server IP address. Maybe this will work?Thank you.Le lun. 13 févr. 2023, à 10 h 03, Mark Hack <markhack@xxxxxxxxxxxx> a écrit :I have a few ideas what the issue is. Can you start by either attaching the server cert or showing it in text form using the command "openssl x509 ..."Looking at the IX code (and it was a very quick look), I suspect that only the CN is validated. If the server cert shows "localhost" then that is probably the issue.RegardsMark HackOn Fri, 2023-02-10 at 16:13 -0500, Pierre-Luc Boily wrote:Hello,
I have a IXWebSocket c++ wss client connecting to a Node.js wss server(websocket npm package). Everything is fine as long as the client connects to `wss://localhost:8080`. Soon as I use the ip address of the Node.js wss server, I have the error "OpenSSL failed - error:0A000086:SSL routines::certificate verify failed"
## Certificate chain creation ##
I created my own private root ca. I used those commands to generate root ca key/certificate and server key/certificate:
$ openssl genpkey -aes256 -out root-ca/private/ca.private.key -algorithm RSA -pkeyopt rsa_keygen_bits:2048
$ openssl req -config root-ca/root-ca.conf -key root-ca\private\ca.private.key -x509 -days 7500 -sha256 -extensions v3_ca -out root-ca\certs\ca.crt
$ openssl genpkey -out server/private/server.private.key -algorithm RSA -pkeyopt rsa_keygen_bits:2048
$ openssl req -key server\private\server.private.key -new -sha256 -out server\csr\server.csr
$ openssl ca -config root-ca\root-ca.conf -extensions server_cert -days 365 -notext -in server\csr\server.csr -out server\certs\server.crt
The configuration has a `subjectAltName` for both root and server and it looks like this :
See config file : https://pastebin.com/kAcwkp9w
The certificate chain looks valid between my *root ca* and my *server*:
$ openssl verify -CAfile root-ca\certs\ca.crt server\certs\server.crt
server\certs\server.crt: OK
Both `ca.crt` and `server.crt` have a reference to my ip address, so I used the subjectAltName parameter to define it. I thought that my *root ca* would need it (I am not even sure that it makes sense to have a domain on the *root ca*), but it doesn't make any difference.
Code that is not working
My IXWebSocket c++ client :
https://pastebin.com/tLGi3amA
Code that is working
wss _javascript_ client:
I also coded a _javascript_ client (using the same npm package as my server, not ) and this little client can connect using the ip address!!
https://pastebin.com/Huzv59gX
My Node.js server :
https://pastebin.com/QCYg5z1B
Questions :
1. Any idea why my c++ client cannot connect using an ip address to the server, while the _javascript_ client can? (using the same certificate chain)
2. If not, any idea how I could debug this?
3. Would it be possible that the problem is a high level SSL stuff, where you actually need a real hostname and can't use an IP?
