On Mon, Apr 11, 2022 at 11:52 AM Matt Caswell <matt@xxxxxxxxxxx> wrote: > > > > On 10/04/2022 19:18, Alon Bar-Lev wrote: > > Hello, > > > > I am trying to migrate to openssl-3.0 API, it seems to be very > > complicated to hook primitive private key usage to a custom function. > > This is required, for example, to use private keys that reside on > > hardware devices or when an application wishes to externalize private > > key operations to other programs using IPC. > > > > I hope I am wrong but it seems like an entirely new provider must be > > implemented with complete reimplementation of the default providers, > > including serialization, padding etc... While in openssl-0/1 it was > > quite easy. > > > > You will need to implement a provider, and include a key manager plus an > implementation of whatever operations you want to support, e.g. > signature or asymcipher. Typically where a private key resides on a > hardware device then you don't need to support > serialization/deserialization because the keys can't be > serialized/deserialized anyway. If you do want to support that then the > key manager just needs to be able to import or export keys using the > standard parameters for the algorithm and it will automatically be able > to use the default provider's encoders and decoders. Support for key > generation is also probably optional. You would need to support any > padding that you need - that's considered part of the low level > algorithm implementation. > Thank you Matt, I am aware I can implement new three providers from scratch. However, I was hopping you will show me a way to cascade the existing providers just like we have done in the past with the RSA_METHOD. I would like the exact behavior of the existing providers while overriding the low level RSA operations. Even if I would implement an entirely new provider I guess I need to keep using deprecated low level RSA_* functions for the public key part. OpenVPN had brute forced this[1][2] in about 1500 lines of code of what used to be about 120 lines of code. I would like to raise my concern that this openssl-3.0 provider interface may need some improvement to allow easier integration, similar to what we had in openssl since about ever. Are you opened for a discussion for improving this? [1] https://github.com/OpenVPN/openvpn/blob/master/src/openvpn/xkey_provider.c [2] https://github.com/OpenVPN/openvpn/blob/master/src/openvpn/xkey_helper.c > > > I wrote a testcase program using openssl-1 APIs[1] which also works > > using openssl-3, in this testcase I prepare a new RSA method based on > > the default method, hook the private operations and then hook the RSA > > object to use the custom method. > > > > I am looking for a way to implement the __hook_evp_pkey function in > > openssl-3 api, so that when a private key operation is executed on the > > EVP_PKEY or EVP_PKEY_CTX a custom callback will be executed while > > public key operation continue to be executed normally. > > > > While looking into the existing RSA providers I can see that the > > providers continue to use the deprecated RSA_* functions with the > > following comment: > > > > /* > > * RSA low level APIs are deprecated for public use, but still ok for > > * internal use. > > */ > > > > This is exactly what I need... :) To have the RSA low level API be > > redirected back to the application so that I can enjoy the default > > implementation of signature/rsa_sig.c padding etc while being able to > > override the private encrypt. But these low level functions are hidden > > from the user. > > As the comment says, RSA low level APIs are deprecated. Deprecated does > *not* mean removed. So you can still use them for now, although expect > them to be removed from some future version of OpenSSL. Marking deprecated APIs is announcing your intentions and gives enough time for everyone to workout the gaps (if any). I believe there is a gap which will introduce a great burden for developers in the existing design, I would like to work with you to reach similar solution we had in prior openssl versions by leveraging the current provider approach and create a reference implementation similar to what I've provided. If I understand the design correctly the missing bits are the ability to cascade a provider and access low level primitives, maybe as its own provider. > Matt > Thanks, Alon > > > > Can anyone help us to create a testcase of openssl-3? This will help > > many applications such as opensc/libp11 opensc/pkcs11-helper openvpn > > and probably more. > > > > For your convenience, you may find the program here[1]. > > > > Regards, > > Alon Bar-Lev > > > > [1] https://github.com/alonbl/openssl-external/blob/master/example.c > > > > --- > > > > #include <openssl/err.h> > > #include <openssl/evp.h> > > #include <openssl/pem.h> > > #include <openssl/rsa.h> > > #include <string.h> > > #include <stdio.h> > > > > static RSA_METHOD *__example_rsa_method; > > static int __example_rsa_index; > > > > static int __example_rsa_priv_enc(int flen, const unsigned char *from, > > unsigned char *to, RSA *rsa, int padding) { > > const RSA_METHOD *rsa_method = NULL; > > int ret = -1; > > > > if ((rsa_method = RSA_get_method(rsa)) == NULL) { > > goto cleanup; > > } > > > > /* > > * Do it. > > */ > > printf("ENCRYPT\n"); > > memset(to, 0, flen); > > ret = 1; > > > > cleanup: > > > > return ret; > > } > > > > static int __example_rsa_priv_dec(int flen, const unsigned char *from, > > unsigned char *to, RSA *rsa, int padding) { > > const RSA_METHOD *rsa_method = NULL; > > int ret = -1; > > > > if ((rsa_method = RSA_get_method(rsa)) == NULL) { > > goto cleanup; > > } > > > > /* > > * Do it. > > */ > > printf("DECRYPT\n"); > > memset(to, 0, flen); > > ret = 1; > > > > cleanup: > > > > return ret; > > } > > > > > > static int __prepare_method(void) { > > int ret = 0; > > > > if ((__example_rsa_method = > > RSA_meth_dup(RSA_get_default_method())) == NULL) { > > goto cleanup; > > } > > > > if (!RSA_meth_set1_name(__example_rsa_method, "example")) { > > goto cleanup; > > } > > > > if (!RSA_meth_set_priv_dec(__example_rsa_method, __example_rsa_priv_dec)) { > > goto cleanup; > > } > > > > if (!RSA_meth_set_priv_enc(__example_rsa_method, __example_rsa_priv_enc)) { > > goto cleanup; > > } > > > > if ((__example_rsa_index = RSA_get_ex_new_index(0, "example", > > NULL, NULL, NULL)) == -1) { > > goto cleanup; > > } > > > > ret = 1; > > > > cleanup: > > > > return ret; > > } > > > > static int __free_method(void) { > > RSA_meth_free(__example_rsa_method); > > } > > > > static int __hook_evp_pkey(EVP_PKEY *evp_pkey) { > > > > RSA *rsa = NULL; > > int ret = 0; > > > > /* > > * Hook private key methods > > */ > > > > if (EVP_PKEY_id(evp_pkey) != EVP_PKEY_RSA) { > > goto cleanup; > > } > > > > if ((rsa = EVP_PKEY_get1_RSA(evp_pkey)) == NULL) { > > goto cleanup; > > } > > > > if (!RSA_set_method(rsa, __example_rsa_method)) { > > goto cleanup; > > } > > > > if (!RSA_set_ex_data(rsa, __example_rsa_index, "mystate")) { > > goto cleanup; > > } > > > > if (EVP_PKEY_set1_RSA(evp_pkey, rsa) != 1) { > > goto cleanup; > > } > > > > ret = 1; > > > > cleanup: > > > > RSA_free(rsa); > > > > return ret; > > } > > > > const static char *pem = ( > > "-----BEGIN CERTIFICATE-----\n" > > "MIIFMDCCBBigAwIBAgISA6sbShb1HQ3TpSVvhSPOS4JJMA0GCSqGSIb3DQEBCwUA\n" > > "MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD\n" > > "EwJSMzAeFw0yMjAzMTAxNzQ4MDdaFw0yMjA2MDgxNzQ4MDZaMBoxGDAWBgNVBAMT\n" > > "D210YS5vcGVuc3NsLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB\n" > > "AMZvA0BbvdyVc+06j5e5k6dUr8gqL0KZw0w4xJ0QD6jD/o+czNEMz13YDxuZ5utL\n" > > "YGq8uohlK8l2DWqvDfGfm1T4VYQhD2z0Ky0JDTsxDIb5i6kKA+o2j2VPAivfMkBp\n" > > "f47rLITa4vqZ8/aro3E0ZVWfbpOOGASteM/g9mLEpRLJQA2/o4uu9xLCsyJkLG8F\n" > > "8eTCHUJ8388ZO/3fv8LnN1+/WwciSYcZcZNN44OsrgLNoLh6dzSY+oNZyVGdqxUy\n" > > "ZSO2dURx4/28w26RLzXFnGOZinupE6KoVhCHHM0Wqx7YkfudymzwBCPP3+X4Hkab\n" > > "1gkZZO9wTpRKrhuW3XtaBMkCAwEAAaOCAlYwggJSMA4GA1UdDwEB/wQEAwIFoDAd\n" > > "BgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNV\n" > > "HQ4EFgQUW/ht3YVQnVmfAWGArMLkgIyUFNYwHwYDVR0jBBgwFoAUFC6zF7dYVsuu\n" > > "UAlA5h+vnYsUwsYwVQYIKwYBBQUHAQEESTBHMCEGCCsGAQUFBzABhhVodHRwOi8v\n" > > "cjMuby5sZW5jci5vcmcwIgYIKwYBBQUHMAKGFmh0dHA6Ly9yMy5pLmxlbmNyLm9y\n" > > "Zy8wJwYDVR0RBCAwHoIPbXRhLm9wZW5zc2wub3JnggtvcGVuc3NsLm9yZzBMBgNV\n" > > "HSAERTBDMAgGBmeBDAECATA3BgsrBgEEAYLfEwEBATAoMCYGCCsGAQUFBwIBFhpo\n" > > "dHRwOi8vY3BzLmxldHNlbmNyeXB0Lm9yZzCCAQMGCisGAQQB1nkCBAIEgfQEgfEA\n" > > "7wB2AEHIyrHfIkZKEMahOglCh15OMYsbA+vrS8do8JBilgb2AAABf3Uo37wAAAQD\n" > > "AEcwRQIhAMDDz1KXMWXblh9maYNLF6vlZOcSXNlp3RgxJhRBYhACAiBB8mU+mqDa\n" > > "8RNog7zLQq3426vcfH4r1wufDnQ0su3GyQB1ACl5vvCeOTkh8FZzn2Old+W+V32c\n" > > "YAr4+U1dJlwlXceEAAABf3Uo4ZoAAAQDAEYwRAIgVD5+n6KMePTQF2GN4ZKIE8Oz\n" > > "lzZPeY90EPY5APu3ZrECIE4HWJ/ZQ/qZ3/7x4Vo+1a1gPoPBM4rsh3d3ormsrkiW\n" > > "MA0GCSqGSIb3DQEBCwUAA4IBAQA+TYBjasfMBLlXbwNdYGaVtfbBKyPPhHFHOqi2\n" > > "iJfdRnx2Z/KS0gmBisD6SS62dKAjHrUy4wSfRTSpAHAOvo3n7BuYSE+3HIYwyFpB\n" > > "P54tJTiEYiAHJvWsPRl8rEqxzYnaR+u0zdKL7Wauk9gJMwGX6fdwhhAgS5WmBe05\n" > > "O4mf8jdWgtLQYxS/kvQYrNDTTBA6J+UoNM/JIxXENMh2/6zcFgy0D2ewr0NjAYWU\n" > > "Ylf5jVgHjxleRSGnbt19v8dwZcHyBhq+vdndQt0sDQl7aoNEKiCXU2/y0KAtDjGF\n" > > "tsFic9a3WMzENWlAUcfACBaGx8Qm9161M9BO396tgHavQLQ8\n" > > "-----END CERTIFICATE-----\n" > > ); > > > > int main(void) { > > BIO *bio = NULL; > > X509 *x509 = NULL; > > EVP_PKEY *evp_pkey = NULL; > > EVP_PKEY_CTX *evp_pkey_ctx = NULL; > > int ret = 1; > > > > if (__prepare_method() < 1) { > > goto cleanup; > > } > > > > if ((bio = BIO_new_mem_buf(pem, strlen(pem))) == NULL) { > > goto cleanup; > > } > > > > if ((x509 = PEM_read_bio_X509(bio, NULL, NULL, NULL)) == NULL) { > > goto cleanup; > > } > > > > if ((evp_pkey = X509_get_pubkey(x509)) == NULL) { > > goto cleanup; > > } > > > > if (__hook_evp_pkey(evp_pkey) < 1) { > > goto cleanup; > > } > > > > #if OPENSSL_VERSION_NUMBER < 0x30000000 > > if ((evp_pkey_ctx = EVP_PKEY_CTX_new(evp_pkey, NULL)) == NULL) { > > goto cleanup; > > } > > #else > > if ((evp_pkey_ctx = EVP_PKEY_CTX_new_from_pkey(NULL, evp_pkey, > > NULL)) == NULL) { > > goto cleanup; > > } > > #endif > > > > if (EVP_PKEY_sign_init(evp_pkey_ctx) < 1) { > > goto cleanup; > > } > > > > { > > char buf[1024]; > > size_t len = sizeof(buf); > > if (EVP_PKEY_sign(evp_pkey_ctx, buf, &len, "Test", 4) < 1) { > > goto cleanup; > > } > > } > > > > ret = 0; > > > > cleanup: > > > > ERR_print_errors_fp(stdout); > > > > EVP_PKEY_CTX_free(evp_pkey_ctx); > > EVP_PKEY_free(evp_pkey); > > X509_free(x509); > > BIO_free(bio); > > > > __free_method(); > > > > return ret; > > } > >
