Thanks for the detailed answer.
From strace I can see that I'm using /lib/x86_64-linux-gnu/libssl.so.1.1
When I use the eventmachine lib that uses the wrong cert chain I can see with strace :
openat(AT_FDCWD, "/usr/lib/ssl/cert.pem", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/usr/lib/ssl/certs/2e5ac55d.0", {st_mode=S_IFREG|0644, st_size=1200, ...}) = 0
openat(AT_FDCWD, "/usr/lib/ssl/certs/2e5ac55d.0", O_RDONLY) = 8
stat("/usr/lib/ssl/certs/2e5ac55d.1", 0x7ffe1a8e5d80) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/ssl/cert.pem", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/usr/lib/ssl/certs/2e5ac55d.0", {st_mode=S_IFREG|0644, st_size=1200, ...}) = 0
openat(AT_FDCWD, "/usr/lib/ssl/certs/2e5ac55d.0", O_RDONLY) = 8
stat("/usr/lib/ssl/certs/2e5ac55d.1", 0x7ffe1a8e5d80) = -1 ENOENT (No such file or directory)
When I use another lib that uses the correct cert chain I can see with strace :
openat(AT_FDCWD, "/usr/lib/ssl/cert.pem", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/ssl/cert.pem", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/usr/lib/ssl/certs/8d33f237.0", 0x7fff1b4b0f90) = -1 ENOENT (No such file or directory)
stat("/usr/lib/ssl/certs/4042bcee.0", {st_mode=S_IFREG|0644, st_size=1939, ...}) = 0
openat(AT_FDCWD, "/usr/lib/ssl/certs/4042bcee.0", O_RDONLY) = 8
openat(AT_FDCWD, "/usr/lib/ssl/cert.pem", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/ssl/cert.pem", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/usr/lib/ssl/certs/8d33f237.0", 0x7fff1b4b0f90) = -1 ENOENT (No such file or directory)
stat("/usr/lib/ssl/certs/4042bcee.0", {st_mode=S_IFREG|0644, st_size=1939, ...}) = 0
openat(AT_FDCWD, "/usr/lib/ssl/certs/4042bcee.0", O_RDONLY) = 8
In the second case I can see it tries to load the R3 certificate ( 8d33f237.0 ). I wonder why in the first case it does not try to find each certificate in the chain from the trust store at all. I wonder if it can be related to the fact I do not see any call to SSL_CTX_set_cert_store in the lib. On the other hand I suppose if we do not call this it would pick the "default" trust store from the system which seems to be the case here because it can find /usr/lib/ssl/certs/2e5ac55d.0 .
This looks like to be an issue in the eventmachine lib itself. I need to brush up my C skills and have a deeper look at this :).
Thanks
On Sat, Oct 2, 2021 at 8:11 PM Viktor Dukhovni <openssl-users@xxxxxxxxxxxx> wrote:
On Sat, Oct 02, 2021 at 06:21:00PM +0100, Angus Robertson - Magenta Systems Ltd wrote:
> > Yes. To make things even more complex, a few sites also have an
> > older version of R3 that is directly signed by the DST root:
> >
> > - leaf <- R3 <- DST Root CA X3 (self-signed)
> >
> > but that's far from common at this point.
>
> That old R3 [CA] was issued last winter and got installed in Windows
> Server 2018 intermediate stores then, and was still being sent out on
> 29th and 30th, despite expiring on 29th.
Not just Windows, at least one Unix system running Postfix is still
vending a chain with the R3 signed by DST that expired on 2021-09-29:
issuer=O = Digital Signature Trust Co., CN = DST Root CA X3
subject=C = US, O = Let's Encrypt, CN = R3
notBefore=Oct 7 19:21:40 2020 GMT
notAfter=Sep 29 19:21:40 2021 GMT
SHA256 Fingerprint=73:0C:1B:DC:D8:5F:57:CE:5D:C0:BB:A7:33:E5:F1:BA:5A:92:5B:2A:77:1D:64:0A:26:F7:A4:54:22:4D:AD:3B
-----BEGIN CERTIFICATE-----
MIIEZTCCA02gAwIBAgIQQAF1BIMUpMghjISpDBbN3zANBgkqhkiG9w0BAQsFADA/
MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
DkRTVCBSb290IENBIFgzMB4XDTIwMTAwNzE5MjE0MFoXDTIxMDkyOTE5MjE0MFow
MjELMAkGA1UEBhMCVVMxFjAUBgNVBAoTDUxldCdzIEVuY3J5cHQxCzAJBgNVBAMT
AlIzMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAuwIVKMz2oJTTDxLs
jVWSw/iC8ZmmekKIp10mqrUrucVMsa+Oa/l1yKPXD0eUFFU1V4yeqKI5GfWCPEKp
Tm71O8Mu243AsFzzWTjn7c9p8FoLG77AlCQlh/o3cbMT5xys4Zvv2+Q7RVJFlqnB
U840yFLuta7tj95gcOKlVKu2bQ6XpUA0ayvTvGbrZjR8+muLj1cpmfgwF126cm/7
gcWt0oZYPRfH5wm78Sv3htzB2nFd1EbjzK0lwYi8YGd1ZrPxGPeiXOZT/zqItkel
/xMY6pgJdz+dU/nPAeX1pnAXFK9jpP+Zs5Od3FOnBv5IhR2haa4ldbsTzFID9e1R
oYvbFQIDAQABo4IBaDCCAWQwEgYDVR0TAQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8E
BAMCAYYwSwYIKwYBBQUHAQEEPzA9MDsGCCsGAQUFBzAChi9odHRwOi8vYXBwcy5p
ZGVudHJ1c3QuY29tL3Jvb3RzL2RzdHJvb3RjYXgzLnA3YzAfBgNVHSMEGDAWgBTE
p7Gkeyxx+tvhS5B1/8QVYIWJEDBUBgNVHSAETTBLMAgGBmeBDAECATA/BgsrBgEE
AYLfEwEBATAwMC4GCCsGAQUFBwIBFiJodHRwOi8vY3BzLnJvb3QteDEubGV0c2Vu
Y3J5cHQub3JnMDwGA1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9jcmwuaWRlbnRydXN0
LmNvbS9EU1RST09UQ0FYM0NSTC5jcmwwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYf
r52LFMLGMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjANBgkqhkiG9w0B
AQsFAAOCAQEA2UzgyfWEiDcx27sT4rP8i2tiEmxYt0l+PAK3qB8oYevO4C5z70kH
ejWEHx2taPDY/laBL21/WKZuNTYQHHPD5b1tXgHXbnL7KqC401dk5VvCadTQsvd8
S8MXjohyc9z9/G2948kLjmE6Flh9dDYrVYA9x2O+hEPGOaEOa1eePynBgPayvUfL
qjBstzLhWVQLGAkXXmNs+5ZnPBxzDJOLxhF2JIbeQAcH5H0tZrUlo5ZYyOqA7s9p
O5b85o3AM/OJ+CktFBQtfvBhcJVd9wvlwPsk+uyOy2HI7mNxKKgsBTt375teA2Tw
UdHkhVNcsAKX1H7GNNLOEADksd86wuoXvg==
-----END CERTIFICATE-----
The EE (depth 0 server) certificate is not expired, and yet somehow the
server is building a chain with a fresh leaf cert, and a rather stale
issuer CA.
It verifies via the DANE implementation in OpenSSL, because its "2 1 1"
record with a fresh RRSIG specifies the R3 CA as trusted, and its
expiration date is not in scope since it was signed by an entity outside
the effective trust chain.
Validation would fail for the same chain with WebPKI, unless there's a
new improved R3 in the trust store (not just the roots).
My DANE survey scan engine checks trust-anchor cert expiration date,
even when an intermediate CA, mostly because the implementation is
done that way, but I can retroactively justify it because it makes
sense to be more strict in tools that look for potential issues.
Implementations other than OpenSSL might similarly reject such a
suboptimal chain.
--
Viktor.
