I found a solution to this problem. Just in case if someone finds this
later, I am using EVP_PKEY_verify_recover, d2i_X509_SIG and
X509_SIG_get0 to retrieve the Hash algorithm from the signature.
My test code:
int
do_pk1_verifyrecover(EVP_PKEY * pkey, unsigned char * sig, size_t siglen,
unsigned char * m, size_t mlen)
{
int ret;
EVP_PKEY_CTX * ctx;
ctx = EVP_PKEY_CTX_new(pkey, NULL);
EVP_PKEY_verify_recover_init(ctx);
EVP_PKEY_CTX_set_rsa_padding(ctx, RSA_PKCS1_PADDING);
ret = EVP_PKEY_verify_recover(ctx, m, &mlen, sig, siglen);
// If the signature was not PKCS1 padded, an error should happen here
if (ret <= 0) {
printf("Sign failed with error %s\n",
ERR_error_string(ERR_get_error(), NULL));
} else {
_print_buf("Recovered Message", m, mlen);
}
// Cleanup
if (ctx)
EVP_PKEY_CTX_free(ctx);
return ret;
}
int
get_sig_algo_from_pkcs1_enc(const unsigned char * encoded_msg, size_t msglen)
{
int ret;
X509_SIG * sig;
const X509_ALGOR * algo = NULL;
const ASN1_OCTET_STRING * digest = NULL;
sig = d2i_X509_SIG(NULL, &encoded_msg, msglen);
X509_SIG_get0(sig, &algo, &digest);
ret = OBJ_obj2nid(algo->algorithm);
if (ret == NID_undef) {
printf("Error getting algo NID from algo object");
ret = -1;
}
_print_algo_name(algo);
// Cleanup
if (sig)
X509_SIG_free(sig);
return ret;
}
main()
...
// 3. Recover message
// After decryption, data will be == or < siglen
// Allocate siglen bytes to be safe
encoded_msg = malloc(siglen);
msglen = siglen;
if (do_pk1_verifyrecover(pkey, sig, siglen, encoded_msg, msglen) < 0){
ret = -1;
goto cleanup;
}
// 4. Retrieve Hash Algo from the recovered PKCS1 encoded message
// Nvm about whether signature was pkcs1 or not because
// if not previous setup would have generated an error.
sig_algo_nid = get_sig_algo_from_pkcs1_enc(encoded_msg, msglen);
md = EVP_get_digestbynid(sig_algo_nid);
if (!md) {
ret = -1;
printf("Error: Unrecognized md in signature\n");
goto cleanup;
}
// 5. Verify using the retrieved hash algo in previous step
if (do_verify(pkey, md, sig, siglen) < 0){
ret = -1;
goto cleanup;
}
On Mon, Apr 24, 2023 at 3:46 PM Atul Singh <singh.atulks@xxxxxxxxx> wrote:
>
> RSASSA-PKCS1-v1_5 signature scheme encodes the AlgorithmIdentifier of
> Hash Function in the message data before passing it through the
> encryption[1]. So, technically one doesn't need to know the Hash
> Function beforehand for verification purposes -- it can be decoded
> right out of the decrypted octets.
>
> Does openssl provide a way to perform verification without the
> application supplying the Hash Function for RSASSA-PKCS1-v1_5
> signatures? I have played around with EVP_DigestVerifyInit and
> friends, but it doesn't seem to be possible -- The Hash Function is
> required at the init context stage itself and there doesn't seem to be
> any way around it[2].
>
> I also did not see anything obvious in rsa.h that could just return
> the AlgorithmIdentifier that can be used for EVP_DigestVerifyInit
> later.
>
> The reason I am asking this question is that IKEv2 leaves it to the
> implementation to choose their Authentication method unannounced[3].
> If an IKE peer chooses RSA Signature as their authentication method,
> it can then go on to use any of the Hash Functions supported by RSA
> (RFC5996 only recommends making SHA1 default). This is causing a lot
> of incompatibility issues between vendors. Technically, one could try
> all possible algorithms one by one but that means doing 4 passes for
> SHA1, SHA256, SHA384 and SHA512 (assuming MD2 and MD5 are not in use
> anymore).
>
> Regards,
> Atul
>
> [1] https://datatracker.ietf.org/doc/html/rfc3447#section-9.2
> [2] https://www.openssl.org/docs/man1.1.1/man3/EVP_DigestInit.html
> [2] https://datatracker.ietf.org/doc/html/rfc5996#section-3.8
