Solved - Re: provider not loaded on MacOS - Re: How to access keys on HW tokens via PKCS11 Provider?

[David von Oheimb <it@xxxxxxxxxxxxx>]

 

If you try to strace the command without -provider pkcs11 do you
see any attempt to load the provider shared module?

Good thought to try 'strace' instead of the hard-to-use and in this case not very useful OPENSSL_TRACE=CONF.
 Yet I cannot reproduce the problem on Linux, while on MacOS, strace is not available.
 Would be really good if the OpenSSL config module loader provided better tracing.

 I had manually added some printfs to crypto/provider_core.c to find out that, for some reason,
 provider_activate() and provider_init() only get called for "default", but not for "pkcs11":

OPENSSL_TRACE=CONF apps/openssl x509 -noout -text -in "pkcs11:object=...;type=cert"

TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Configuration in section openssl_init
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Adding config module 'alg_section'
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Adding config module 'providers'
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Adding config module 'random'
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Loading providers module: section provider_sect
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Configuring provider pkcs1
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider command: module = /usr/local/lib/ossl-modules/pkcs11.dylib
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider command: pkcs11-module-path = /Library/OpenSC/lib/onepin-opensc-pkcs11.so
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider params: start section pkcs11_sect
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider params: module = /usr/local/lib/ossl-modules/pkcs11.dylib
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider params: pkcs11-module-path = /Library/OpenSC/lib/onepin-opensc-pkcs11.so
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider params: finish section pkcs11_sect
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Configuring provider default
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider command: activate = 1
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider params: start section default_sect
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider params: activate = 1
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider params: finish section
default_sect
provider_activate name = default
provider_init name = default
TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Running module providers
(provider_sect) returned 1

Yet when I add  -provider pkcs11  to the command line, this outputgets extended by:

provider_activate name = pkcs11
provider_init name = pkcs11
module_path = (null)
merged_path = /Users/david/openssl/providers/pkcs11.dylib

and the provider loading works, making use, e.g, of 
pkcs11-module-path = /Library/OpenSC/lib/onepin-opensc-pkcs11.so

Maybe this related to the annoying fact that LD_LIBRARY_PATH does not
work with MacOS, while DYLD_LIBRARY_PATH is a kind of replacement.

    David

Not sure how LD_LIBRARY_PATH is related. It is not used when loading the provider modules. 

      I see.

> It is suspicious that there is no provider command activate=1 trace
> line for the pkcs11 provider.
>       
      Oh - I had originally copied that line

activate = 1

but at some point I must have accidentally deleted it for my pkcs11_section on MacOS - very sorry for my confusion!
Now, after re-adding it, the provider does get loaded automatically also without  "-provider pkcs11" given on the command line 🙂
So it actually works now, like it already did for me on Linux (where the "activate = 1" has been present all the time).

Is the provider module path correct in the TRACE above? Could you try this tracing on Linux to compare?

When using the config file, /usr/local/lib/ossl-modules/pkcs11.dylib is correct in the sense that does contain the lib for MacOS,
but when "activate = 1" is missing in that section, the "module = ..." config line is not is really used, and this fact is not given in the trace.
This is just revealed by the printf("merged_path = %s\n", merged_path) output I added to provider_init() in provider_core.c.

Instead, like when not using the config file at all, -provider pkcs11 silently gets translated to loading /Users/david/openssl/providers/pkcs11.dylib
according to my setting OPENSSL_MODULES=/Users/david/openssl/providers and a copy of the provider lib is also there.

On the other hand, somewhat confusingly to me, the config line pkcs11-module-path = /Library/OpenSC/lib/onepin-opensc-pkcs11.so does take effect also without "activate = 1" (unless overridden by, e.g. PKCS11_PROVIDER_MODULE=/Library/OpenSC/lib/onepin-opensc-pkcs11.so).

David