I found out what my problem is! I'm
running it in FIPS mode and this causes the PKCS11 engine to fail
during mutual authentication. I eventually traced the problem to
the following issue:
https://bugzilla.redhat.com/show_bug.cgi?id=1827535
It looks like there is a bug in libp11. Once I made the suggested workaround, it worked.
My original code, which is based on
https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c
worked perfectly after I added in the libp11 fix. :)
Thanks!
George
On 2021-01-11 11:01 a.m., Jan Just Keijser wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1827535
It looks like there is a bug in libp11. Once I made the suggested workaround, it worked.
My original code, which is based on
https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c
worked perfectly after I added in the libp11 fix. :)
Thanks!
George
On 2021-01-11 11:01 a.m., Jan Just Keijser wrote:
Hi,
On 08/01/21 22:35, George wrote:
Hi,
I have been trying to setup mutual authentication using a smart card but I can't seem to get the OpenSSL Engine to send a response back to the server containing client's certificate from the smart card.
I'm using the following to configure the certificate and private key:
ENGINE_ctrl_cmd(engine, "LOAD_CERT_CTRL", 0, &cert_info, NULL, 0);
SSL_CTX_use_certificate(sslContext, cert_info.cert);
EVP_PKEY* privateKey = ENGINE_load_private_key(engine, "2b2586c684d69b670c0a805edf514e720f2b757d8e2faa0b3a7ff23d1ccfc7ba", transfer_pin, &cb_data);
SSL_CTX_use_PrivateKey(sslContext, privateKey);
(I have been using the code in https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c as a guide.)
This seems be successful. However, when I start the mutual authentication with
SSL_connect(ssl)
, the mutual authentications handshake fails. I can see the server requesting the certificate from the client and the client sends back an ACK for this message. However, the client does not send the certificate to the server.
I was looking through the OpenSSL code openssl-1.0.2u\ssl\ssl_rsa.c and noticed something interesting. The comment indicates that the flag RSA_METHOD_FLAG_NO_CHECK should be set for smart cards:
static int ssl_set_pkey(CERT *c, EVP_PKEY *pkey)
{
. . .
#ifndef OPENSSL_NO_RSA
/*
* Don't check the public/private key, this is mostly for smart
* cards.
*/
if ((pkey->type == EVP_PKEY_RSA) &&
(RSA_flags(pkey->pkey.rsa) & RSA_METHOD_FLAG_NO_CHECK)) ;
else
#endif
. . .
}
However, it is not actually set when I use a debugger to inspect the flag. Does it need to be set? If so, how is this done? I could not find anything related to this in
https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c
if you read through the code blob that Michael pointed you to, you will find that this flag needs to be set *under certain circumstances* when using smartcards. It has to do mostly with the situation where
- private key is on the smart card
- the public key/certificate is NOT on the smart card
- you ask OpenSSL to verify the private key without explicitly providing a public key.
I've never run into this issue, but then again, I have not tested very often the case where the certificate was not present on the HSM/smart card but the private key is. YMMV.
As for using pksc11helper versus using libp11: that is just a matter of taste. I used the engine_pkcs11 + libp11 route for the eap-tls code , mostly because it was the first "working" set of tools I found at the time.
You can also take the "pkcs11helper" route, which is what OpenVPN does (see https://github.com/openvpn). Both methods have pro's and con's.
Do you run into problems if you DO not set the RSA_METHOD_FLAG_NO_CHECK flag? All that flag does is to stop OpenSSL from verifying that a public key/cert and private key match/belong together for RSA keys only; if your smartcard supports EC keys then this flag will do you no good.
HTH,
JJK
