I implemented some custom engines and RSA_meth_set_priv_enc seems to map
to other libraries' RSA decrypt operation (NCryptDecrypt ||||on Windows
CNG, Cipher class with Cipher.DECRYPT_MODE on Android). They can do a
TLS connection just fine with a self-signed cert.
I looked at another custom engine and they seem to also use RSA decrypt for
RSA_meth_set_priv_enc:
https://github.com/tpm2-software/tpm2-tss-engine/blob/master/src/tpm2-tss-engine-rsa.c#L163
BoringSSL's (deprecated) rsa_meth_st only has a sign and a decrypt,
having no encrypt operation:
https://commondatastorage.googleapis.com/chromium-boringssl-docs/rsa.h.html#rsa_meth_st
Is this just a naming quirk? I want to put down the nagging feeling I
have a bug somewhere.