[SOLVED] Re: OpenSSL 3.0 hangs at exit with FIPS provider

[Thomas Dwyer III <tomiii@xxxxxxxxxx>]

It turns out the problem was caused by a misinterpretation of the phrase "add the following lines near the beginning" in section 7.1 of the documentation at https://wiki.openssl.org/index.php/OpenSSL_3.0 for enabling FIPS support. I added these lines to the very top of the file:

openssl_conf = openssl_init

.include /usr/local/ssl/fipsmodule.cnf

[openssl_init]
providers = provider_sect

[provider_sect]
fips = fips_sect

This caused the existing default section to now become part of the [provider_sect] section. Apparently any name=value line in that particular section where no [value] section exists causes OpenSSL to hang at exit when the FIPS provider is used. I consider this a bug, of course, but at least now I know what's causing it and how to work around it.

Regarding how to confirm which provider is actually providing a given algorithm, I found that EVP_MD_provider() returns NULL for any EVP_MD obtained via EVP_get_digestbyname() (even after it's used successfully by EVP_DigestInit_ex()) but it returns a valid OSSL_PROVIDER for any EVP_MD obtained via EVP_MD_fetch(). Is this intentional?

Tom.III

On Wed, Jul 15, 2020 at 10:20 AM Thomas Dwyer III <tomiii@xxxxxxxxxx> wrote:

Platform: Linux x86_64

I understand this is still alpha but how complete is the FIPS provider right now? I'm following the documentation at https://wiki.openssl.org/index.php/OpenSSL_3.0 but I'm having a problem where my application hangs during exit() when I use the "fips" provider. I reduced my code down to this minimal snippet that reproduces the problem:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <openssl/evp.h>
#include <openssl/crypto.h>
#include <openssl/err.h>
#include <openssl/provider.h>

int
main(int argc, char **argv)
{
        OSSL_PROVIDER *pvdr = NULL;
        EVP_MD_CTX *ctx;
        const EVP_MD *md;
        char *alg = "sha1";
        int rc = 0;

        pvdr = OSSL_PROVIDER_load(NULL, "fips");
        if (pvdr == NULL) {
                fprintf(stderr, "Error loading FIPS provider\n");
                exit(1);
        }

        md = EVP_get_digestbyname(alg);
        if (!md) {
                fprintf(stderr, "unknown digest '%s'\n", alg);
                exit(1);
        }

        ctx = EVP_MD_CTX_create();

        if (EVP_DigestInit_ex(ctx, md, NULL) != 1) {
                long err = ERR_get_error();
                char *msg = ERR_error_string(err, NULL);
                fprintf(stderr, "EVP_DigestInit_ex() failed: %s\n", msg);
                exit(1);
        }

        EVP_MD_CTX_destroy(ctx);

        rc = OSSL_PROVIDER_unload(pvdr);
        if (rc != 1) {
                fprintf(stderr, "Error unloading FIPS provider\n");
                exit(1);
        }

        printf("finished!\n");

        exit(0);
}

When I run this it prints "finished!" and then hangs in some kind of spin loop consuming 100% CPU. Skipping the call to EVP_DigestInit_ex() allows it to exit successfully, as does inserting a call to OPENSSL_init_crypto() at the very top with the OPENSSL_INIT_NO_ATEXIT flag. Passing "default" instead of "fips" to OSSL_PROVIDER_load() also seems to work fine. What am I missing?

Also, per section 7.8 of the wiki referenced above, I'm unable to confirm that the digest algorithm I want to use is being provided by the FIPS module. EVP_MD_provider(md) returns NULL even though the actual digest is computed correctly.

Thanks,

Tom.III