For the benefit of anyone who finds this thread via a search, the issue turns out to be this change to SSSD which introduced parallel requests: https://sssd.io/release-notes/sssd-2.7.3.html But then the rpc.mountd "client" threads would send 8 requests (rpc.mountd threads) to SSSD which then all get sent over the wire to your LDAP/AD server whereas before it was a single request. New fixes from Redhat: https://bugzilla.redhat.com/show_bug.cgi?id=2234829 https://github.com/SSSD/sssd/issues/6911 Daire On Thu, 24 Aug 2023 at 18:39, Daire Byrne <daire@xxxxxxxx> wrote: > > Hi, > > We have lots of Linux storage servers running combinations of RHEL7, > RHEL8 and more recently RHEL9. We also use "manage-gids" and have lots > of groups of users and apply permissions to directories on the > exported filesystems. > > We also use sssd and AD/LDAP on these storage servers to resolve the > groups and do the user lookups. This setup has worked great for our > needs for many years but we have noticed a change in RHEL9 which > results in many more uid/gid lookups hitting our LDAP servers. > > It seems like with RHEL7 & 8 era kernels and nfs-utils, sssd/nss would > receive a single request from rpc.mountd whereas with RHEL9 we now get > duplicated requests for each rpc.mountd thread (8 by default) even for > a single client mount. So 8 uid/gid requests hit sssd at the same > time, and because it's not in cache, all those 8 requests go out over > the wire to our AD server. > > So for lookups not in the cache, we have 8 times more requests hitting > our LDAP servers. Not to mention that sssd sometimes crashes or loses > connectivity with the LDAP server with this increased load. > > I had a look through the changes for linux-nfs but nothing jumped out > at me in that time frame (other than code to make exportd > multi-threaded). Does anyone have any ideas where this change of > behaviour might be coming from? > > RHEL9: nfs-utils-2.5.4 > RHEL8: nfs-utils-2.3.3 > > Cheers, > > Daire
