Help with Error: data too large for modulus

Gautam Bhat <mindentropy@xxxxxxxxx> · Fri, 14 Aug 2020 01:33:30 +0530

Hi,

I am trying to do a walkthrough of verifying a certificate signing.

1) I have pulled the signature as follows:
   openssl asn1parse -in cert.pem -out cert.sig -noout -strparse 638

The offset of 638 is because asn1parse of the cert.pem file produces:
  625:d=2  hl=2 l=   9 prim:   OBJECT            :sha256WithRSAEncryption
  636:d=2  hl=2 l=   0 prim:   NULL
  638:d=1  hl=4 l= 257 prim:  BIT STRING

2) I have pulled the public key of the CA certificate as follows:
    openssl x509 -in ca_cert.pem -pubkey -noout > ca_cert.pubkey

3) I am trying to decrypt the signature file to get the hash as follows:
    openssl rsautl -verify -pubin -inkey ca_cert.pubkey -in cert.sig -asn1parse

Unfortunately I get an error in the above step as:
140155781719872:error:04067084:rsa
routines:rsa_ossl_public_decrypt:data too large for
modulus:crypto/rsa/rsa_ossl.c:548:

The size of the cert.sig file is 256 bytes. I am not sure where I am
going wrong and would need some assistance.

Thanks,
Gautam.