Hi Matt, All,
thanks for the replies, I really appreciate the help :) Further
responses inline...
On 22/08/2023 06:20, Dr. Pala wrote:
Hello OpenSSL Community,
We are in the process of upgrading the LibPKI project to use the OSSL 3.x instead of the OSSL 1.1.1x that we are using today ... and we can use some help. Specifically, we noticed [...] falcon one (e.g., EVP_PKEY_type() vs. EVP_PKEY_EC, EVP_PKEY_RSA, etc.)? Example Code: https://github.com/openca/libpki/blob/28c510193735e6a68671bfb4cb4e66589a1fd9d0/src/drivers/hsm_main.c#L530
We use EVP_PKEY_is_a() extensively internally to check whether a pkey is a particular type.
https://www.openssl.org/docs/man3.1/man3/EVP_PKEY_is_a.html
Thanks! I guess that internally uses the same approach? How do
you know what strings to use for matching different algorithms? It
would have probably been useful to give these IDs some form of
structure... maybe they could be OIDs, instead of just "names" ?
This would make things a lot easier, I think, when you have
multiple providers... I was really bummed by the fact that we lost
the ultimate NID/resource that was quite unique and neat :) With
this system that uses text we will have to start "guessing" what
is picked up... I fear it might lead to some difficult-to-debug
situations... :( I guess you went through all these considerations
in the past...
I also saw that there is a EVP_PKEY_get0_type_name() that
provides the "first key type name that is found" - is that
preferred instead?
* *OBJ_add_sigid() and OBJ_find_sigid_by_algs()* do not seem to work
as usual. Specifically, with the above configuration (i.e., with the
[...]
apparent reason. Example Code:
https://github.com/openca/libpki/blob/28c510193735e6a68671bfb4cb4e66589a1fd9d0/src/openssl/pki_oid_defs.c#L622
This sounds like it should work, so I don't know why you are seeing odd behaviour. Perhaps raise a github issue for this?
Ouch! I really hoped there was a new mechanism for registering
IDs. The interesting part is that the same code compiled against
1.1.1x works correctly (i.e., the new sigid are found after being
added), so I am leaning toward a possible issue with the 3.x
approach... I will try to put together some code that reproduces
the issue and submit it to the repo... would it be ok if the
example uses the oqsprovider? As I understand, the issue should be
there independently of the providers, you can always register OIDs
and add them to the stack of signature IDs.
One thing that does spring to mind is that I know that oqsprovider already registers various sigid OIDs - so plausibly there is some kind of issue caused by that:
https://github.com/open-quantum-safe/oqs-provider/blob/b6da1e97b7180a779981d1b0bfa0ab2cd723066d/oqsprov/oqsprov.c#L592-L619
https://github.com/open-quantum-safe/oqs-provider/blob/b6da1e97b7180a779981d1b0bfa0ab2cd723066d/oqsprov/oqsprov.c#L45-L71
Thanks for the links - I was wondering how they replaced the use
of the objects database to updating the OIDs internally. One thing
that they seem to do that I do not is the step with the assigning
of the ID ... and it seems that is needed to avoid spending too
much time in lookup... is it something that providers have to do
or are encouraged to do? This seems something that could
potentially be done within the providers' "glue" code?
* *How to replace PKEY/ASN1_PKEY methods.* In our library, we
currently provide a new type of algorithm (i.e., Composite Crypto)
[...] can we inject it programmatically? Example Code:
https://github.com/openca/libpki/blob/28c510193735e6a68671bfb4cb4e66589a1fd9d0/src/openssl/composite/composite_init.c#L326
Yes, you would need a new provider to add a new algorithm (although the old approach should still *work* - even though it is deprecated). You can just add a "built-in" one if you want (so no need to create a separate .so file for it - it can just live in your application code)
https://www.openssl.org/docs/man3.1/man3/OSSL_PROVIDER_add_builtin.html
Thanks, that is exactly what I was looking for...
Start by looking at the page for general information for provider authors:
https://www.openssl.org/docs/man3.1/man7/provider.html
Richard Levitte has a "toy" provider example which you can look at here:
https://github.com/provider-corner/vigenere/blob/main/vigenere.c
Great! That is really useful - learning by example :)
To implement an algorithm yourself you need to first determine what kind of algorithm it is (e.g. cipher, signature, key-exchange, etc) and then implement the appropriate provider functions for that algorithm. To implement a new signature algorithm you would need to implement the signature functions as well as the key management functions. E.g. see
https://www.openssl.org/docs/man3.1/man7/provider-signature.html
and
https://www.openssl.org/docs/man3.1/man7/provider-keymgmt.html
I will definitely start to look into that... thanks for the
links. One question I have is about "double dipping", so to speak.
Would it be easy to get the code from the BUILTIN version into a
stand-alone provider? I am thinking about this to see if it would
be worth trying to provide support for Composite Crypto in plain
OpenSSL instead of having to link the full LibPKI library if you
need just the hybrid functionality ...
* *EVP_PKEY_get0() fails when called from within the pkey_bits()
function of the EVP_PKEY_ASN1_METHOD*. Specifically, when the
pkey_bits() function is called with the parameter of type `const
EVP_PKEY *pk`, the EVP_PKEY_get0(pk) function always returns NULL.
What are we supposed to use to retrieve the internal key structure?
Example Code:
https://github.com/openca/libpki/blob/28c510193735e6a68671bfb4cb4e66589a1fd9d0/src/openssl/composite/composite_ameth.c#L963
EVP_PKEY_get0() returns a pointer to the legacy key structure (e.g. RSA, DSA, DH, EC_KEY, etc). It won't work if you call it on a provider based EVP_PKEY (returns NULL) but you can call the typed equivalents, e.g. EVP_PKEY_get0_RSA(). In that case it gives you a cached *copy* of the legacy key (the real key is held in the provider and is inaccessibly directly to the application).
Actually, this is related to the PKEY_METH/ASN1_PKEY_METH that we develop to support Composite Crypto. In there, we need to get the reference to the internal data structure of the key to retrieve the stack of keys - for example, when we want to call the EVP_PKEY_bits(), which, in turn, calls the ASN1_PKEY_METH's pkey_bits() function. In there, we use the EVP_PKEY_get0() function to retrieve the internal key structure (COMPOSITE_KEY *). The problem is that EVP_PKEY_get0() returns NULL and, therefore, fails. The key is loaded by the method and the internal structure is assigned with the EVP_PKEY_assign_COMPOSITE() function which, in turn, calls the EVP_PKEY_assign() - see the code here:
It's unclear to me from your question whether you are talking about your *own* EVP_PKEY type, i.e. one you've defined yourself that does not use a provider. If so then this should still work as it did before, i.e. as long as data has been assigned to the key via EVP_PKEY_assign() then it should be available via a subsequent call to EVP_PKEY_get0().
Well... I guess it should work, but it does not :(
I am not sure what is going on, but there are definitely
differences with how the code works in 1.1.1x - I hoped for some
additional procedures we might needed to take... but it seems we
are heading towards multiple "beautiful" debugging session(s)...
maybe the issue is still related to OIDs/IDs... ? We'll see... if
I understand what's going on I will submit an Issue in GitHub...
but I hope the issue is in our code :)
Thanks again Matt (and the community), I hope these considerations and questions might help other people who are in the middle of migrating. Thanks!
Cheers,
Max
OpenCA Labs Director
