On 11/04/2022 16:53, Alon Bar-Lev wrote:
On Mon, Apr 11, 2022 at 11:52 AM Matt Caswell <matt@xxxxxxxxxxx> wrote:
On 10/04/2022 19:18, Alon Bar-Lev wrote:
Hello,
I am trying to migrate to openssl-3.0 API, it seems to be very
complicated to hook primitive private key usage to a custom function.
This is required, for example, to use private keys that reside on
hardware devices or when an application wishes to externalize private
key operations to other programs using IPC.
I hope I am wrong but it seems like an entirely new provider must be
implemented with complete reimplementation of the default providers,
including serialization, padding etc... While in openssl-0/1 it was
quite easy.
You will need to implement a provider, and include a key manager plus an
implementation of whatever operations you want to support, e.g.
signature or asymcipher. Typically where a private key resides on a
hardware device then you don't need to support
serialization/deserialization because the keys can't be
serialized/deserialized anyway. If you do want to support that then the
key manager just needs to be able to import or export keys using the
standard parameters for the algorithm and it will automatically be able
to use the default provider's encoders and decoders. Support for key
generation is also probably optional. You would need to support any
padding that you need - that's considered part of the low level
algorithm implementation.
Thank you Matt,
I am aware I can implement new three providers from scratch.
However, I was hopping you will show me a way to cascade the existing
providers just like we have done in the past with the RSA_METHOD.
It is actually technically possible to cascade providers. There is an
example of this happening in the test code. We have a filter provider
which "wraps" the default provider and modifies its behaviour. See:
https://github.com/openssl/openssl/blob/master/test/filterprov.c
However, I'm not sure that it helps you in this particular instance.
We also have instances of "composite" algorithms, e.g. where a high
level algorithm makes use of some other low level algorithm (consider a
DigestSign operation which uses RSA for signing, but also uses some
digest algorithm as part of its work).
I would like the exact behavior of the existing providers while
overriding the low level RSA operations.
The problem is that we don't currently expose the low level RSA
operations as provider operations. Feasibly we could have two RSA
implementations - a "low" level one and a "high" level one, where the
high level one does all padding etc, and the "low" level one does the
actual raw RSA work. The high level one could then call the low level
one. If we had that then it would probably be feasible for you to hook
in an alternative implementation for just the low level pieces.
That would require some refactoring though.
Are you opened for a discussion for improving this?
Absolutely. I'd suggest you raise an issue on github to discuss it.
I'm sure someone else raised a similar issue some while ago and we had
some similar discussions, but I've gone searching for it and couldn't
find it. I did come across this issue though which touches on some
similar topics:
https://github.com/openssl/openssl/issues/17187
Matt
[1] https://github.com/OpenVPN/openvpn/blob/master/src/openvpn/xkey_provider.c
[2] https://github.com/OpenVPN/openvpn/blob/master/src/openvpn/xkey_helper.c
I wrote a testcase program using openssl-1 APIs[1] which also works
using openssl-3, in this testcase I prepare a new RSA method based on
the default method, hook the private operations and then hook the RSA
object to use the custom method.
I am looking for a way to implement the __hook_evp_pkey function in
openssl-3 api, so that when a private key operation is executed on the
EVP_PKEY or EVP_PKEY_CTX a custom callback will be executed while
public key operation continue to be executed normally.
While looking into the existing RSA providers I can see that the
providers continue to use the deprecated RSA_* functions with the
following comment:
/*
* RSA low level APIs are deprecated for public use, but still ok for
* internal use.
*/
This is exactly what I need... :) To have the RSA low level API be
redirected back to the application so that I can enjoy the default
implementation of signature/rsa_sig.c padding etc while being able to
override the private encrypt. But these low level functions are hidden
from the user.
As the comment says, RSA low level APIs are deprecated. Deprecated does
*not* mean removed. So you can still use them for now, although expect
them to be removed from some future version of OpenSSL.
Marking deprecated APIs is announcing your intentions and gives enough
time for everyone to workout the gaps (if any).
I believe there is a gap which will introduce a great burden for
developers in the existing design, I would like to work with you to
reach similar solution we had in prior openssl versions by leveraging
the current provider approach and create a reference implementation
similar to what I've provided.
If I understand the design correctly the missing bits are the ability
to cascade a provider and access low level primitives, maybe as its
own provider.
Matt
Thanks,
Alon
Can anyone help us to create a testcase of openssl-3? This will help
many applications such as opensc/libp11 opensc/pkcs11-helper openvpn
and probably more.
For your convenience, you may find the program here[1].
Regards,
Alon Bar-Lev
[1] https://github.com/alonbl/openssl-external/blob/master/example.c
---
#include <openssl/err.h>
#include <openssl/evp.h>
#include <openssl/pem.h>
#include <openssl/rsa.h>
#include <string.h>
#include <stdio.h>
static RSA_METHOD *__example_rsa_method;
static int __example_rsa_index;
static int __example_rsa_priv_enc(int flen, const unsigned char *from,
unsigned char *to, RSA *rsa, int padding) {
const RSA_METHOD *rsa_method = NULL;
int ret = -1;
if ((rsa_method = RSA_get_method(rsa)) == NULL) {
goto cleanup;
}
/*
* Do it.
*/
printf("ENCRYPT\n");
memset(to, 0, flen);
ret = 1;
cleanup:
return ret;
}
static int __example_rsa_priv_dec(int flen, const unsigned char *from,
unsigned char *to, RSA *rsa, int padding) {
const RSA_METHOD *rsa_method = NULL;
int ret = -1;
if ((rsa_method = RSA_get_method(rsa)) == NULL) {
goto cleanup;
}
/*
* Do it.
*/
printf("DECRYPT\n");
memset(to, 0, flen);
ret = 1;
cleanup:
return ret;
}
static int __prepare_method(void) {
int ret = 0;
if ((__example_rsa_method =
RSA_meth_dup(RSA_get_default_method())) == NULL) {
goto cleanup;
}
if (!RSA_meth_set1_name(__example_rsa_method, "example")) {
goto cleanup;
}
if (!RSA_meth_set_priv_dec(__example_rsa_method, __example_rsa_priv_dec)) {
goto cleanup;
}
if (!RSA_meth_set_priv_enc(__example_rsa_method, __example_rsa_priv_enc)) {
goto cleanup;
}
if ((__example_rsa_index = RSA_get_ex_new_index(0, "example",
NULL, NULL, NULL)) == -1) {
goto cleanup;
}
ret = 1;
cleanup:
return ret;
}
static int __free_method(void) {
RSA_meth_free(__example_rsa_method);
}
static int __hook_evp_pkey(EVP_PKEY *evp_pkey) {
RSA *rsa = NULL;
int ret = 0;
/*
* Hook private key methods
*/
if (EVP_PKEY_id(evp_pkey) != EVP_PKEY_RSA) {
goto cleanup;
}
if ((rsa = EVP_PKEY_get1_RSA(evp_pkey)) == NULL) {
goto cleanup;
}
if (!RSA_set_method(rsa, __example_rsa_method)) {
goto cleanup;
}
if (!RSA_set_ex_data(rsa, __example_rsa_index, "mystate")) {
goto cleanup;
}
if (EVP_PKEY_set1_RSA(evp_pkey, rsa) != 1) {
goto cleanup;
}
ret = 1;
cleanup:
RSA_free(rsa);
return ret;
}
const static char *pem = (
"-----BEGIN CERTIFICATE-----\n"
"MIIFMDCCBBigAwIBAgISA6sbShb1HQ3TpSVvhSPOS4JJMA0GCSqGSIb3DQEBCwUA\n"
"MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD\n"
"EwJSMzAeFw0yMjAzMTAxNzQ4MDdaFw0yMjA2MDgxNzQ4MDZaMBoxGDAWBgNVBAMT\n"
"D210YS5vcGVuc3NsLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB\n"
"AMZvA0BbvdyVc+06j5e5k6dUr8gqL0KZw0w4xJ0QD6jD/o+czNEMz13YDxuZ5utL\n"
"YGq8uohlK8l2DWqvDfGfm1T4VYQhD2z0Ky0JDTsxDIb5i6kKA+o2j2VPAivfMkBp\n"
"f47rLITa4vqZ8/aro3E0ZVWfbpOOGASteM/g9mLEpRLJQA2/o4uu9xLCsyJkLG8F\n"
"8eTCHUJ8388ZO/3fv8LnN1+/WwciSYcZcZNN44OsrgLNoLh6dzSY+oNZyVGdqxUy\n"
"ZSO2dURx4/28w26RLzXFnGOZinupE6KoVhCHHM0Wqx7YkfudymzwBCPP3+X4Hkab\n"
"1gkZZO9wTpRKrhuW3XtaBMkCAwEAAaOCAlYwggJSMA4GA1UdDwEB/wQEAwIFoDAd\n"
"BgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNV\n"
"HQ4EFgQUW/ht3YVQnVmfAWGArMLkgIyUFNYwHwYDVR0jBBgwFoAUFC6zF7dYVsuu\n"
"UAlA5h+vnYsUwsYwVQYIKwYBBQUHAQEESTBHMCEGCCsGAQUFBzABhhVodHRwOi8v\n"
"cjMuby5sZW5jci5vcmcwIgYIKwYBBQUHMAKGFmh0dHA6Ly9yMy5pLmxlbmNyLm9y\n"
"Zy8wJwYDVR0RBCAwHoIPbXRhLm9wZW5zc2wub3JnggtvcGVuc3NsLm9yZzBMBgNV\n"
"HSAERTBDMAgGBmeBDAECATA3BgsrBgEEAYLfEwEBATAoMCYGCCsGAQUFBwIBFhpo\n"
"dHRwOi8vY3BzLmxldHNlbmNyeXB0Lm9yZzCCAQMGCisGAQQB1nkCBAIEgfQEgfEA\n"
"7wB2AEHIyrHfIkZKEMahOglCh15OMYsbA+vrS8do8JBilgb2AAABf3Uo37wAAAQD\n"
"AEcwRQIhAMDDz1KXMWXblh9maYNLF6vlZOcSXNlp3RgxJhRBYhACAiBB8mU+mqDa\n"
"8RNog7zLQq3426vcfH4r1wufDnQ0su3GyQB1ACl5vvCeOTkh8FZzn2Old+W+V32c\n"
"YAr4+U1dJlwlXceEAAABf3Uo4ZoAAAQDAEYwRAIgVD5+n6KMePTQF2GN4ZKIE8Oz\n"
"lzZPeY90EPY5APu3ZrECIE4HWJ/ZQ/qZ3/7x4Vo+1a1gPoPBM4rsh3d3ormsrkiW\n"
"MA0GCSqGSIb3DQEBCwUAA4IBAQA+TYBjasfMBLlXbwNdYGaVtfbBKyPPhHFHOqi2\n"
"iJfdRnx2Z/KS0gmBisD6SS62dKAjHrUy4wSfRTSpAHAOvo3n7BuYSE+3HIYwyFpB\n"
"P54tJTiEYiAHJvWsPRl8rEqxzYnaR+u0zdKL7Wauk9gJMwGX6fdwhhAgS5WmBe05\n"
"O4mf8jdWgtLQYxS/kvQYrNDTTBA6J+UoNM/JIxXENMh2/6zcFgy0D2ewr0NjAYWU\n"
"Ylf5jVgHjxleRSGnbt19v8dwZcHyBhq+vdndQt0sDQl7aoNEKiCXU2/y0KAtDjGF\n"
"tsFic9a3WMzENWlAUcfACBaGx8Qm9161M9BO396tgHavQLQ8\n"
"-----END CERTIFICATE-----\n"
);
int main(void) {
BIO *bio = NULL;
X509 *x509 = NULL;
EVP_PKEY *evp_pkey = NULL;
EVP_PKEY_CTX *evp_pkey_ctx = NULL;
int ret = 1;
if (__prepare_method() < 1) {
goto cleanup;
}
if ((bio = BIO_new_mem_buf(pem, strlen(pem))) == NULL) {
goto cleanup;
}
if ((x509 = PEM_read_bio_X509(bio, NULL, NULL, NULL)) == NULL) {
goto cleanup;
}
if ((evp_pkey = X509_get_pubkey(x509)) == NULL) {
goto cleanup;
}
if (__hook_evp_pkey(evp_pkey) < 1) {
goto cleanup;
}
#if OPENSSL_VERSION_NUMBER < 0x30000000
if ((evp_pkey_ctx = EVP_PKEY_CTX_new(evp_pkey, NULL)) == NULL) {
goto cleanup;
}
#else
if ((evp_pkey_ctx = EVP_PKEY_CTX_new_from_pkey(NULL, evp_pkey,
NULL)) == NULL) {
goto cleanup;
}
#endif
if (EVP_PKEY_sign_init(evp_pkey_ctx) < 1) {
goto cleanup;
}
{
char buf[1024];
size_t len = sizeof(buf);
if (EVP_PKEY_sign(evp_pkey_ctx, buf, &len, "Test", 4) < 1) {
goto cleanup;
}
}
ret = 0;
cleanup:
ERR_print_errors_fp(stdout);
EVP_PKEY_CTX_free(evp_pkey_ctx);
EVP_PKEY_free(evp_pkey);
X509_free(x509);
BIO_free(bio);
__free_method();
return ret;
}
