> On 19 Nov 2020, at 20:34, Graham Leggett <minfrin@xxxxxxxx> wrote: > > On 13 Nov 2019, at 12:29, Graham Leggett <minfrin@xxxxxxxx> wrote: > >> On 13 Nov 2019, at 01:37, William Brown <wbrown@xxxxxxx> wrote: >> >>>> Does anyone know why 389ds would suddenly stop sending the full certificate chain while replicating? >>>> >>>> It also looks like the error handling in 389ds SSL is broken - if the slave sent “unknown CA" to the master, the master needs to log that fact, and not report the error as “success”. >>> >>> We'll need to see the output of certutil -L -d /etc/dirsrv/slapd-<instance>/ from both the master and replica servers please. >>> >>> In a TLS auth process the client doesn't send it's CA - if you get unknown CA it's most likely the replica has either had the CA and it's chain members expire, or they are not marked as trusted for client auth. So that's why I'd like to see the certutil output please. >> >> I discovered the same problem had been reported in OpenLDAP: https://www.centos.org/forums/viewtopic.php?t=67042 >> >> This in turn is caused by a regression in NSS, where it is no longer sufficient to have a trusted root certificate, you now need all intermediate certificates marked as trusted as well. >> >> Making the following change to the intermediate certs fixed the problem: >> >> [root@ldap01 ~]# certutil -L -d /etc/dirsrv/slapd-hg >> >> Certificate Nickname Trust Attributes >> SSL,S/MIME,JAR/XPI >> >> intermediateB ,, >> intermediateA ,, >> rootrootroot CT,C,C >> ldap01 u,u,u >> [root@ldap01 ~]# certutil -M -d /etc/dirsrv/slapd-hg -t "CT,C,C" -n "intermediateA" >> [root@ldap01 ~]# certutil -M -d /etc/dirsrv/slapd-hg -t "CT,C,C" -n "intermediateB" >> [root@ldap01 ~]# certutil -L -d /etc/dirsrv/slapd-hg >> >> Certificate Nickname Trust Attributes >> SSL,S/MIME,JAR/XPI >> >> intermediateA CT,C,C >> intermediateB CT,C,C >> rootrootroot CT,C,C >> ldap01 u,u,u >> >> Raised the bug here: https://bugzilla.redhat.com/show_bug.cgi?id=1771979 > > Coming back to this one - got to the bottom of this while investigating something else that wasn’t working. > > This wasn’t a regression in NSS, but rather a regression in the openldap libraries shipped by RHEL7.5 and above. > > For reasons that I haven’t found, there was an architecture change made half way through the RHEL7 lifecycle where openldap was linked to openssl instead of NSS. > > Openldap's NSS support and openldap’s openssl support differ in a fundamental way - with NSS, when openldap makes an SSL connection intermediate certificates are filled in by the client side as normal. With openssl, when openldap makes an SSL connection intermediate certificates are ignored, and the connection breaks. > > The hack workaround above fixes this because openldap’s openssl support expects you to place intermediate certs in your trusted certificate store. As soon as you mark the intermediates as trusted in NSS, the hack workaround in 389ds that makes replication sort-of work bound to two different crypto libraries exports trusted certs across into the ca certificate list passed to openldap. Openldap then finds the intermediates and things work. > > Fundamentally there are two bugs: > > - https://bugzilla.redhat.com/show_bug.cgi?id=1898924 > > - An architectural change half way through the lifecycle of what is supposed to be a stable OS. I seem to remember this change (this was at a time I worked at RH). If memory serves correctly, OpenLDAP upstream removed/deprecated their NSS support. This was making it much much harder to apply fixes for issues both stability and technical, so moving to OpenSSL was the "best move" for customer support. Even internally to 389-ds which has to link to OpenLDAP for some outbound client operations, it internally swapped from NSS to OpenSSL for this as well, which involves extracting some certificates into temporary stores for OpenLDAP client to use. It's quite fun to put it mildly. There are very good reasons why those decisions were made, and that was very carefully managed too. It was a lot of work and ultimately, it did make the OpenLDAP client library better for our maintainers and many consumers, but as you have noticed, these are complex systems, designed and built in a time that preceded deep testing of things. I believe our client TLS auth tests were made *after* the OpenLDAP to OpenSSL switch was made, which could be why this was not noticed. Anyway, I hope that this gives some more context to why this happened. — Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs, Australia _______________________________________________ 389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx
