On an AlmaLinux 8.6 VM hosted in Proxmox:
[root@certs ~]# openssl verify -CAfile
/etc/ssl/certs/local/DOMAIN.wildcards.pem
/etc/ssl/certs/local/DOMAIN.wildcards.pem
C = US, O = Let's Encrypt, CN = R3
error 2 at 1 depth lookup: unable to get issuer certificate
error /etc/ssl/certs/local/DOMAIN.wildcards.pem: verification failed
[root@certs ~]# openssl version
OpenSSL 1.1.1k FIPS 25 Mar 2021
The VM has CPUs of type host. It did the same with a CPU type of
kvm64. I switched to host because kvm64 did not pass the aes CPU flag
through.
If I copy the PEM file to a bare metal system running Ubuntu Server
20.04, it verifies:
elyograg@bilbo:~$ openssl verify -CAfile DOMAIN.wildcards.pem
DOMAIN.wildcards.pem
DOMAIN.wildcards.pem: OK
elyograg@bilbo:~$ openssl version
OpenSSL 1.1.1f 31 Mar 2020
Other bare metal systems and their results with the same PEM file:
Verifies on Proxmox (the one running the VM) with openssl 1.1.1n
Verifies on Ubuntu 22.04 with openssl 3.0.2
Fails on CentOS 7.5.1804 with openssl 1.0.2k-fips
----
I also have the quictls fork of openssl on the VM, built from source,
and it fails with exactly the same error message:
[root@certs ~]# /usr/local/bin/qssl verify -CAfile
/etc/ssl/certs/local/DOMAIN.wildcards.pem
/etc/ssl/certs/local/DOMAIN.wildcards.pem
C = US, O = Let's Encrypt, CN = R3
error 2 at 1 depth lookup: unable to get issuer certificate
error /etc/ssl/certs/local/DOMAIN.wildcards.pem: verification failed
[root@certs ~]# /usr/local/bin/qssl version
OpenSSL 3.0.5+quic 5 Jul 2022 (Library: OpenSSL 3.0.5+quic 5 Jul 2022)
---
I have redacted the domain name from the filename in what I pasted
above, but everything else is untouched. The PEM file contains the
server cert, the letsencrypt issuing cert, the private key, and
generated dhparams. It works for most software that can handle PEM
files for TLS. The only software I am sure about that utilizes the
dhparams is haproxy. In case it matters, the server cert has a 4096 bit
key. The certbot program is functioning correctly.
Does anyone have any idea why this would fail in this way? Is there
some information I can gather that would help with troubleshooting? The
little evidence I have says it is failing on RPM distros and passing on
DEB distros. But the sample size is way too small to adequately support
that hypothesis.
In the unlikely event this is an XY problem, here is the X: I am trying
to set up a letsencrypt certificate creation/renewal system on the VM
for work that I have running on my own server. One of the things that I
have my script doing is validating the certificate file that it produces
before it declares success. I would like the VM to do the same, but
right now I can't because of this issue.
Thanks,
Shawn
